Reconstruct a projected OOF ensemble

Recover the selected-tuning out-of-fold (OOF) predictions stored in fitted caret models and reconstruct ssel's inverse-RMSE ensemble for every response–dataset cell. Use this helper to inspect training-row ensemble residuals. It recomputes cell-local weights from the saved OOF predictions; it does not reuse the unseen-row weight state described by trainRegressionModel().

Usage

oofEnsemble(.path.model, .path.train,
  .path.datasets = file.path("data", "datasets"), .min = -Inf, .max = Inf,
  .response_pattern = NULL, .sort_keys = character(0))

Arguments

.path.model: Path to an existing directory containing model files named method_response_dataset.Rds. The method identifier must be a non-empty string without an underscore. Matching files must contain an object with caret-compatible pred and bestTune components.
.path.train: Path to an existing directory containing response_dataset_TRAIN.csv and, preferably, response_dataset_TRAIN_ids.csv files. A TRAIN file must exist for a model to be eligible. Its values are read but are not used to replace the observed responses stored in the model.
.path.datasets: Path to assembled dataset.csv files used only by the compatibility mapping when a TRAIN-ID sidecar is absent.
.min, .max: Numeric lower and upper response bounds, in response units. Predictions are projected with pmax(.min, pmin(.max, z)). The function does not validate finiteness or .min <= .max.
.response_pattern: A non-empty character regular expression inserted into the model-filename parser to identify responses. Use noncapturing groups when grouping is needed: additional capturing groups change the parser's field positions. The default NULL deliberately fails.
.sort_keys: Character vector of columns that must already define the assembled-dataset row order when the compatibility ID mapping is used. An empty vector disables the order check.

Value

A data.table with one retained row per reconstructed cell and distinct (rowIndex, obs) cast key. Columns are response, dataset, integer rowIndex, character SampleID, y_obs, y_hat_oof, and residual. The final three columns have the response's units. Row indices and sample IDs can repeat when saved observations disagree across methods. If no row is reconstructed, returns an empty zero-column table.

OOF reconstruction

For a matching model, the function inner-joins model$pred to model$bestTune by every best-tuning column. It converts pred, obs, and rowIndex to numeric or integer values and retains only rows where all three are finite. Predictions are then projected onto the configured bounds. Repeated rows for one method, response, dataset, and row index are reduced by taking separate arithmetic means of obs and the projected prediction.

Let $\Pi_{[a,b]}(z)=\min\{b,\max\{a,z\}\}$ be interval projection, let $y_{mi}$ be the saved observed value, and let $\tilde y_{mi}$ be method $m$'s averaged projected OOF prediction at row $i$. The observed value is method-specific at this stage; equality across methods is not checked. In cell $(r,d)$, the helper computes $$\widetilde{\mathrm{RMSE}}_{mrd} =\left[n_m^{-1}\sum_i(y_{mi}-\tilde y_{mi})^2\right]^{1/2}$$ and defines the package weights $$\tilde w_{mrd} =\frac{1/\widetilde{\mathrm{RMSE}}_{mrd}} {\sum_j(1/\widetilde{\mathrm{RMSE}}_{jrd})}.$$ These weights are a package reconstruction policy, not an optimality claim. A cell is skipped when any method-level RMSE is non-finite or zero. A one-method cell is retained with weight one.

The wide reconstruction is keyed jointly by rowIndex and the averaged saved obs. Let $h=(i,o)$ denote one distinct joint key and let $A_h$ be the methods present at that key. The reconstructed value is $$\hat y_h^{OOF} =\Pi_{[a,b]}\left( \frac{\sum_{m\in A_h}\tilde w_m\tilde y_{mh}} {\sum_{m\in A_h}\tilde w_m}\right).$$ For a complete key, $A_h$ contains every cell method and the denominator is one. A key with no method prediction is excluded. If methods supply different averaged observed values for the same row index, they create separate keys, each is aggregated with its available methods, and the result can contain repeated rowIndex and SampleID values. The function does not validate a common observed response across methods. The returned signed residual is $e_h=o-\hat y_h^{OOF}$, in response units.

Sample identifiers

The preferred mapping reads the SampleID column from response_dataset_TRAIN_ids.csv; row $i$ supplies the identifier for caret rowIndex = i. No sidecar length or uniqueness check is made before indexing, but an OOF index beyond its length stops the call.

If the sidecar is absent, the compatibility path reads dataset.csv. That file must contain SampleID, the response, and every .sort_keys column. When sort keys are supplied, the file must already equal its ordering by those keys. Identifiers are then taken only from rows whose response is non-missing. This older mapping is safe only when TRAIN rows are exactly those assembled rows, in unchanged order, with no additional removal or reordering. Feature complete-case filtering, outlier removal, or any other extra filter can silently shift identifiers even when the maximum-index check passes; the sidecar is required in those cases. A missing dataset, missing required column, broken ordering invariant, or out-of-range OOF index stops the call.

Skips, ordering, and side effects

Nonmatching filenames, absent TRAIN files, unreadable models, failed best-tune merges, empty saved predictions, and models with no finite saved row are skipped silently. A cell can also disappear under the RMSE and row-availability rules above. If nothing remains, the function returns an empty zero-column data.table. Result order is an implementation consequence and is not a sorting guarantee.

The helper reads files but writes none, does not fit models, and does not alter the random-number state. Base R, data.table, and file-reading errors not listed above propagate.

References

Kuhn, M. (2008). Building predictive models in R using the caret package. Journal of Statistical Software, 28(5), 1–26. doi:10.18637/jss.v028.i05 .

Parikh, N., and Boyd, S. (2014). Proximal Algorithms. Foundations and Trends in Optimization, 1(3), 127–239. doi:10.1561/2400000003 .

Examples

root <- tempfile("ssel-oof-")
model_dir <- file.path(root, "model")
train_dir <- file.path(root, "train")
dir.create(model_dir, recursive = TRUE)
dir.create(train_dir)

data.table::fwrite(
  data.table::data.table(y = c(1, 3)),
  file.path(train_dir, "target_demo_TRAIN.csv")
)
data.table::fwrite(
  data.table::data.table(SampleID = c("S1", "S2")),
  file.path(train_dir, "target_demo_TRAIN_ids.csv")
)
save_oof <- function(method, prediction) {
  fit <- list(
    pred = data.frame(
      tune = 1L, pred = prediction, obs = c(1, 3), rowIndex = 1:2
    ),
    bestTune = data.frame(tune = 1L)
  )
  saveRDS(fit, file.path(model_dir, paste0(method, "_target_demo.Rds")))
}
save_oof("lower", c(0, 2))
save_oof("mixedCase", c(0.5, 2.5))

oof <- oofEnsemble(
  .path.model = model_dir,
  .path.train = train_dir,
  .response_pattern = "target"
)
oof[, .(SampleID, y_obs, y_hat_oof, residual)]
#>    SampleID y_obs y_hat_oof  residual
#>      <char> <num>     <num>     <num>
#> 1:       S1     1 0.3333333 0.6666667
#> 2:       S2     3 2.3333333 0.6666667
# Both learners have RMSE 1 and 0.5, so their weights are 1/3 and 2/3.

unlink(root, recursive = TRUE)