Build per-response training and prediction files

Read assembled dataset CSV files, choose response and predictor columns, and write the per-response files consumed by the model-fitting helpers. Use this low-level helper when the assembled CSVs already exist and their row roles and columns are known.

Usage

buildDataset(.path.input, .path.train, .file.index = NULL,
  Y.PATTERN = NULL, KEY = NULL, COLS.id = character(0),
  features = NULL, OUTLIER_REMOVAL = FALSE, SET_COL = NULL,
  ITER_SUFFIX = NULL, .shadow = FALSE)

Arguments

.path.input: Path to an existing directory. Each non-recursive file whose name ends in .csv is treated as one assembled dataset.
.path.train: Path to an existing writable output directory. The function does not create it.
.file.index: Reserved for compatibility; currently ignored.
Y.PATTERN: A non-empty regular expression used case-insensitively to select response column names. Required despite the NULL default.
KEY: A single non-empty name of the row-identifier column. Required despite the NULL default.
COLS.id: A character vector of additional identifier columns to exclude from inferred predictors. SET_COL is always excluded as well. Names absent from a given CSV are silently ignored.
features: NULL, or a character vector giving the exact base predictors in output order. NULL infers predictors as every column that is not a response, KEY, or an identifier. Exact names must exist and cannot name responses or identifiers.
OUTLIER_REMOVAL: A non-missing logical scalar. If TRUE, detectOutliers() is applied to each complete training split and rows it flags are removed.
SET_COL: A single non-empty name of the row-role column. The exact value "train" marks training rows; other non-missing values mark prediction rows. The column must contain no missing values. This precondition is not proactively validated. Required despite the NULL default.
ITER_SUFFIX: NULL, or a suffix identifying prior-iteration response columns. Available matching columns are added to an exact features set; each response's own suffixed column is excluded from its split.
.shadow: A non-missing logical scalar. If TRUE, add a standard-normal column named shadow using the fixed seed 12345. It is inferred as a predictor when features = NULL; with exact features it must be named.

Value

Invisibly returns NULL. The result is the set of CSV files described in Files written.

Input CSV schema

The input filename without .csv becomes the dataset identifier used in output filenames. Each CSV must contain KEY, SET_COL, the columns matched by Y.PATTERN, and any exact features. Response columns are eligible only when numeric after cleaning. Predictor and response units are not converted or scaled; CSV unit metadata is neither read nor written, so callers must supply consistent units.

Before splitting, the helper reads "", "N/I", "N/A", and "-" as missing, removes a column named *, removes duplicate rows, and deletes ASCII spaces from every character value. It passes each character column through toNumeric() and replaces the whole column only when every non-missing value converts successfully. A partially convertible column remains character.

Row and predictor filtering

Predictors with at most one distinct non-missing training value are removed. Training rows incomplete in any retained predictor or the current response are omitted. Prediction rows are omitted when a retained predictor is missing or a character predictor has a value not observed in training; one warning reports the number omitted. No imputation or unit conversion is performed.

Files written

For every numeric response response in dataset dataset, three CSV files are written:

response_dataset_TRAIN.csv: retained predictors followed by the response renamed y;
response_dataset_TEST.csv: retained predictors followed by the original KEY column;
response_dataset_TRAIN_ids.csv: one column named SampleID, containing the training identifiers aligned with the filtered TRAIN rows.

Existing files with those names are overwritten. Other files in .path.train are not removed. The function writes no files for a CSV with no numeric matched response, and an input directory with no matching CSV files is a no-op.

Validation and side effects

Missing required selectors, required columns, or output directories produce errors. Exact features also error when they contain missing, empty, duplicated, response, or identifier names. SET_COL values must be non-missing, but this is not checked before splitting; a missing row role can therefore produce an indirect downstream error. COLS.id names are not validated, and absent names are silently ignored. KEY uniqueness and missingness are not checked. Shadow generation restores an existing global random-number state; if no state existed, the call initializes one.

Examples

root <- tempfile("ssel-buildDataset-")
input_dir <- file.path(root, "assembled")
output_dir <- file.path(root, "splits")
dir.create(input_dir, recursive = TRUE)
dir.create(output_dir)

assembled <- data.frame(
  SampleID = c("S1", "S2", "P1", "P2"),
  set = c("train", "train", "predict", "predict"),
  feature_a = c(1, 2, 3, 4),
  feature_b = c("1,5", "2,5", "3,5", "4,5"),
  target = c(10, 20, NA, NA)
)
utils::write.csv(assembled, file.path(input_dir, "demo.csv"),
                 row.names = FALSE, na = "")

buildDataset(
  .path.input = input_dir,
  .path.train = output_dir,
  Y.PATTERN = "^target$",
  KEY = "SampleID",
  SET_COL = "set"
)

sort(basename(list.files(output_dir)))
#> [1] "target_demo_TEST.csv"      "target_demo_TRAIN.csv"    
#> [3] "target_demo_TRAIN_ids.csv"
utils::read.csv(file.path(output_dir, "target_demo_TRAIN.csv"))
#>   feature_a feature_b  y
#> 1         1       1.5 10
#> 2         2       2.5 20
utils::read.csv(file.path(output_dir, "target_demo_TEST.csv"))
#>   feature_a feature_b SampleID
#> 1         3       3.5       P1
#> 2         4       4.5       P2

unlink(root, recursive = TRUE)