Estimate signed OOF-residual offsets

Reconstruct projected OOF ensemble residuals with oofEnsemble() and write their cell-pooled sample quantiles. The resulting offsets can be added to a point prediction by predictModel(). They are descriptive residual-offset estimates: this helper does not construct a predictive distribution or establish conditional, marginal, conformal, or two-sided interval coverage.

Usage

auditQuantiles(.path.model, .path.train, .path.summary,
  .path.datasets = file.path("data", "datasets"), .min = -Inf, .max = Inf,
  .response_pattern = NULL, .sort_keys = character(0), verbose = TRUE)

Arguments

.path.model, .path.train, .path.datasets, .min, .max, .response_pattern, .sort_keys: Passed unchanged to oofEnsemble(). See that reference for the model, split, identifier, projection, regular-expression, and ordering contracts.
.path.summary: Path to the output directory. It is created recursively when absent. Assigned CSV files are overwritten.
verbose: A non-missing logical value. If true, invoke debug reporting. Messages are emitted only when options(aR.verbose = TRUE) and options(aR.quiet = FALSE) are both in effect.

Value

Invisibly, a named list with residuals, the merged row-level OOF table, and quantiles, the six-column cell summary.

Estimator

For cell $(r,d)$ and each retained reconstructed cast row $h=(\mathtt{rowIndex},\mathtt{obs})$, let $e_{hrd}=y_{hrd}-\hat y_{hrd}^{OOF}$ be the signed residual returned by oofEnsemble(). A row index can therefore contribute more than once when methods supplied different saved observations; each returned row is a separate quantile observation. For $p\in\{0.50,0.90,0.95\}$, the helper computes R's type-7 sample quantile. With the $n$ retained residual rows sorted as $e_{(1)}\le\cdots\le e_{(n)}$, define $h=(n-1)p+1$, $j=\lfloor h\rfloor$, and $\gamma=h-j$; then $$q_{p,rd}=(1-\gamma)e_{(j)}+\gamma e_{(j+1)},$$ with R's endpoint convention. The values q_50, q_90, and q_95 are signed and have the response's units. They are computed separately for every response–dataset cell; no centering, symmetrization, covariate conditioning, or finite-sample calibration is performed.

Files, partial runs, and return

The helper first writes residuals_oof.csv with columns response, dataset, rowIndex, SampleID, y_obs, y_hat_oof, and residual. If a compatible file already exists, every prior row whose response occurs in the current result is removed before the current rows are appended. Replacement is by response, not by response–dataset cell: an omitted dataset for a current response is not retained. Rows for other responses remain.

Quantiles are recomputed from that complete merged residual table and prediction_quantiles.csv is overwritten with columns response, dataset, n, q_50, q_90, and q_95. Thus a partial response run carries prior other-response residuals and their recomputed summaries forward. Existing quantile rows are never merged independently. Row order is not a public sorting guarantee.

The return is an invisible list named residuals and quantiles, containing exactly the two tables written in the current call. If OOF reconstruction returns no row, the function stops after creating .path.summary and before writing either assigned CSV. A failure after the residual write can leave that file updated while the quantile file remains from an earlier call or absent. Existing-file schema, file, quantile, data.table, and upstream reconstruction errors propagate. The helper does not fit models or alter the random-number state.

References

Hyndman, R. J., and Fan, Y. (1996). Sample quantiles in statistical packages. The American Statistician, 50(4), 361–365. doi:10.1080/00031305.1996.10473566 .

Barber, R. F., Candes, E. J., Ramdas, A., and Tibshirani, R. J. (2021). Predictive inference with the jackknife+. The Annals of Statistics, 49(1), 486–507. doi:10.1214/20-AOS1965 . This reference distinguishes coverage procedures using fold- or leave-one-out-specific test predictions from the descriptive pooled offset computed here.

Examples

root <- tempfile("ssel-offsets-")
model_dir <- file.path(root, "model")
train_dir <- file.path(root, "train")
summary_dir <- file.path(root, "summary")
dir.create(model_dir, recursive = TRUE)
dir.create(train_dir)

data.table::fwrite(
  data.table::data.table(y = c(1, 3)),
  file.path(train_dir, "target_demo_TRAIN.csv")
)
data.table::fwrite(
  data.table::data.table(SampleID = c("S1", "S2")),
  file.path(train_dir, "target_demo_TRAIN_ids.csv")
)
fit <- list(
  pred = data.frame(
    tune = 1L, pred = c(0, 2), obs = c(1, 3), rowIndex = 1:2
  ),
  bestTune = data.frame(tune = 1L)
)
saveRDS(fit, file.path(model_dir, "lm_target_demo.Rds"))

offsets <- auditQuantiles(
  .path.model = model_dir,
  .path.train = train_dir,
  .path.summary = summary_dir,
  .response_pattern = "target",
  verbose = FALSE
)
offsets$quantiles
#>    response dataset     n  q_50  q_90  q_95
#>      <char>  <char> <int> <num> <num> <num>
#> 1:   target    demo     2     1     1     1
# Both signed residuals equal 1, so all three offsets equal 1.

unlink(root, recursive = TRUE)