Pipeline API

H2MLPipeline

h2ml.pipeline.pipeline.H2MLPipeline

Orchestrates the full h2ml 4-step workflow.

Step 1 — CV all models (× all y-transforms) on all features → select best (model [, transform]) Step 2 — SHAP importance + correlation-based feature reduction using winning model/transform Step 3 — CV all models on reduced features (winning transform only) → select best stage Step 4 — Hyperparameter optimization + final CV → select overall best

Y-transform sweep

Pass a list of transform names to run() to sweep transforms within each step rather than running the full pipeline independently per transform.

result = pipeline.run(store, transforms=["log", "sqrt", "count"])

Partial runs

run_step1_only() — step 1 only, quick model screening run_step1_to_step2() — steps 1-2, inspect SHAP importance / reduced features run_step1_to_step3() — steps 1-3, full selection without HPO run_from_step3() — resume from a pre-reduced result (steps 3-4)

Parameters:

Name	Type	Description	Default
config	PipelineConfig	Pipeline configuration.	required
models	Optional[list[ModelWrapper]]	Override the default model registry list. When None, build_models() constructs a task-appropriate set from the registry.	None
metadata	Optional[RunMetadata]	Optional experiment labels (schema, target, batch) that appear as columns in fold/agg DataFrames. When None, only stage is set.	None

Raises:

Type	Description
ValueError	If the resolved model list is empty.

Example

pipeline = H2MLPipeline(config=PipelineConfig()) result = pipeline.run(store) result.summary()

run(store, transforms=None)

Execute the full 4-step pipeline.

Parameters:

Name	Type	Description	Default
store	PipelineData	Input features and target.	required
transforms	Optional[Iterable[str]]	Optional sequence of y-transform names to sweep (regression only). When provided, step 1 runs all models × all transforms and selects the best (model, transform). Step 3 re-runs only the winning transform on the reduced feature set. Defaults to None (no transform sweep).	None

Returns:

Type	Description
PipelineResult	PipelineResult with all four steps completed.

run_from_step3(result, transform_stores=None)

Resume from step 3 using a PipelineResult from run_step1_to_step2(). Requires result.features, result.features_reduced and result.best_model_name to be set.

Parameters:

Name	Type	Description	Default
result	PipelineResult	PipelineResult carrying steps 1-2 state.	required
transform_stores	Optional[dict[str, PipelineData]]	Pre-built y-transform stores; rebuilt from result.features when None.	None

Returns:

Type	Description
PipelineResult	PipelineResult with steps 3-4 completed.

Raises:

Type	Description
ValueError	If features, features_reduced, best_model_name, or selector is missing from result.

run_step1_only(store, transforms=None)

CV all models on all features and select the best — quick model screening. After: result.best_model_name, result.step1_agg_df.

Returns:

Type	Description
PipelineResult	PipelineResult with step 1 completed.

run_step1_to_step2(store, transforms=None)

Run steps 1-2. After: result.selector.importance_summary() / result.features_reduced.feature_names

Returns:

Type	Description
PipelineResult	PipelineResult with steps 1-2 completed.

run_step1_to_step3(store, transforms=None)

Run steps 1-3 (model selection on full and reduced features) without HPO. Useful for inspecting best_stage and comparing models before committing to step 4's compute cost. After: result.best_model_name, result.best_stage, result.step3_agg_df

Returns:

Type	Description
PipelineResult	PipelineResult with steps 1-3 completed (no HPO).

run_step4_only(result, transform_stores=None)

Run only step 4 (HPO + final CV) on a result that already has steps 1–3 complete.

Typical use: load a saved result, then call this to add or redo optimisation without repeating CV on all models.

features, features_reduced, selector, best_model_name, best_stage,

best_model_value.

Parameters:

Name	Type	Description	Default
result	PipelineResult	PipelineResult with steps 1-3 complete.	required
transform_stores	Optional[dict[str, PipelineData]]	Pre-built y-transform stores; rebuilt from result.features when None.	None

Returns:

Type	Description
PipelineResult	PipelineResult with step 4 completed.

Raises:

Type	Description
ValueError	If any of the required steps 1-3 fields are missing.

PipelineConfig

h2ml.pipeline.pipeline.PipelineConfig dataclass

Configuration for H2MLPipeline.

Attributes:

Name	Type	Description
task_type	TaskType | str	Task to run. Accepts the string "classification" or "regression" (case-insensitive) or a TaskType member; normalised to TaskType in post_init.
metric	str	Short metric name used for model selection (steps 1–3) and HPO (step 4). Minimisation direction and sort order are derived automatically — no need to set a separate flag. Classification: "AUC" (default), "AUC_PR", "F1", "LogLoss", "Brier". Regression: "R2" (default), "MAE", "RMSE".
n_splits	int	CV folds used in steps 1–3 for model screening and feature selection. More folds = more reliable estimates, higher cost. Default: 5.
random_state	int	Seed for fold splitting and model initialisation. Default: 42.
verbose	bool	Log step-by-step progress. Default: False.
corr_threshold	float	(Step 2) Drop a feature when its correlation with any already-retained feature exceeds this value in Pearson, Spearman, or Kendall. Range (0, 1]. Default: 0.7.
min_features	int	(Step 2) Hard lower bound on features kept after the correlation filter; prevents the selector from removing everything. Default: 1.
n_trials	int	(Step 4) Total Optuna trials budget. Each trial evaluates one hyperparameter configuration using opt_n_splits-fold CV, so total model fits = n_trials × opt_n_splits (plus a final n_splits-fold CV on the best params). Default: 50.
opt_n_splits	int	(Step 4) CV folds inside each Optuna trial. Fewer folds make each trial faster at the cost of a noisier score estimate. Should be ≥ n_splits // 2 to avoid unreliable HPO scores; setting it equal to n_splits gives unbiased estimates at higher compute cost. Default: 3.
n_hpo_repeats	int	(Step 4) Number of independent HPO runs with different fold seeds. The repeat with the highest best_value is kept. Trials are divided evenly across repeats: trials_per_repeat = max(1, n_trials // n_hpo_repeats), so the total fit count stays constant. Default: 1.
spatial_cv_method	str	Splitting strategy when spatial coordinates are provided (all spatial-CV fields are ignored when store.coords is None). "block" — quantile-grid blocking; fast, no clustering. "spcv" — Agglomerative Hierarchical Clustering + cluster ensemble; more spatially coherent folds, slower. Default: "spcv".
spatial_cv_metric	SpatialMetric	Distance metric used by both splitters. "euclidean" — projected coordinates (metres). "haversine" — geographic lat/lon in decimal degrees. Default: "euclidean".
n_blocks_per_fold	int	Number of spatial blocks assigned to the test set per fold in the block splitter. Default: 5.
time_bin_resolution	str	Temporal granularity for binning dates in spatial CV and local conformal calibration. "month" — 12 bins (Jan…Dec). "season" — 4 bins (DJF, MAM, JJA, SON). Default: "month".
ahc_threshold	Optional[float]	Distance threshold for cutting the AHC dendrogram in SPCVSplitter. Derived automatically from the data when None. Default: None.
pca_components	float	Variance fraction retained by PCA applied to block covariates before AHC clustering in SPCVSplitter. Default: 0.95.
exact_max_samples	int	Sample count below which exact scipy AHC is used; above this an approximate sklearn AHC (k-NN graph) is used instead. Default: 5 000.
knn_neighbors	int	k for the k-NN connectivity graph in approximate AHC. Default: 15.
handle_imbalance	bool	(Classification only) Inject class_weight="balanced" into every model whose registry entry has supports_class_weight=True. No effect on regression tasks. Default: False.

metric_col property

Full agg DataFrame column name, e.g. 'AUC' → 'AUC_Test_Mean'.

minimize_metric property

True for error metrics (LogLoss, Brier, MAE, RMSE), False for score metrics.

spatial_cv property

Bundle the spatial-CV parameters passed to the splitter and optimizer.

PipelineResult

h2ml.pipeline.pipeline.PipelineResult dataclass

Container for all artifacts produced by H2MLPipeline.

Fields are populated progressively as steps complete; check completed_steps to see which steps have run. Use summary() to compare all stages at once and build_final_model() to refit on the full training set.

Attributes:

Name	Type	Description
features	Optional[PipelineData]	Input PipelineData (full feature set).
step1_fold_df	Optional[DataFrame]	Per-fold metrics from step 1 (all models × transforms).
step1_agg_df	Optional[DataFrame]	Aggregated (mean ± std) step-1 metrics per model.
best_model_name	Optional[str]	Winning model name after step 1.
best_model_value	Optional[float]	Best model's score on the selection metric.
best_model_std	Optional[float]	Fold std of the best model's selection metric.
features_reduced	Optional[PipelineData]	PipelineData after step-2 feature reduction.
selector	Optional[FeatureSelector]	Fitted FeatureSelector (SHAP + correlation filter).
step3_fold_df	Optional[DataFrame]	Per-fold metrics from step 3 (reduced features).
step3_agg_df	Optional[DataFrame]	Aggregated step-3 metrics per model.
best_stage	Optional[str]	Winning stage after step 3: "default" or "reduced".
best_feature_stage	Optional[str]	Feature stage step 4 is built on — "default" or "reduced", never "optimized".
step3_reduced_stores	Optional[dict[str, 'PipelineData']]	Cached reduced PipelineDatas keyed by transform name ("" for no transform). Not persisted.
best_params	Optional[dict]	Best hyperparameters found in step 4 (None = defaults).
step4_fold_df	Optional[DataFrame]	Per-fold metrics from the step-4 final CV.
step4_agg_df	Optional[DataFrame]	Aggregated step-4 metrics.
step1_cv_result	Optional[list[CVResult]]	Raw step-1 CVResults — preserved for plots/persistence.
step3_cv_result	Optional[list[CVResult]]	Raw step-3 CVResults.
step4_cv_result	Optional[CVResult]	Raw step-4 CVResult.
y_transform	Optional[str]	Winning y-transform name (set when run() sweeps transforms).
cv_type	str	"spatial" when store.coords was set, else "random".
spatial_cv_metric	str	Distance metric forwarded from PipelineConfig; used by build_final_model to build LocalConformalCalibration.
time_bin_resolution	str	Temporal bin resolution forwarded from PipelineConfig.
cv_warnings	list[str]	Warnings for models with ≥1 failed CV fold (steps 1 & 3).
metric	Optional[str]	Short selection metric name, e.g. "AUC" or "R2".
splitter	Any	Splitter built once in step 1 and reused in steps 3-4. Not persisted.

best_cv_result property

CVResult for the final winning model. Returns step4_cv_result when HPO ran, otherwise the best model's CVResult from step3 (used when opt_enabled=False).

completed_steps property

List of completed step numbers, e.g. [1, 2, 3] after run_step1_to_step3().

oof_labels property

True labels paired with oof_predictions.

oof_predictions property

Out-of-fold predictions for the best model's CV result. Classification: positive-class probability (binary) or probability matrix (multiclass). Regression: predicted values. None if no CV result is available.

build_final_model()

Fit the best model on the full training dataset and return a FinalModel.

The FinalModel handles inference (predict / predict_proba), scaling, and can be saved independently of this result.

Returns:

Type	Description
'FinalModel'	FinalModel ready for prediction on new data.

load(path) classmethod

Reload a result previously saved with result.save(path).

save(path)

Persist this result to path. Reload with PipelineResult.load(path).

summary(metric=None, ascending=False)

Combined agg DataFrame across all completed stages.

Parameters:

Name	Type	Description	Default
metric	Optional[str]	Column name to sort by (e.g. "AUC_Test_Mean"). When None, rows are returned in stage order (default → reduced → optimized).	None
ascending	bool	Sort direction; set True for error metrics (RMSE, MAE).	False

Returns:

Type	Description
DataFrame	DataFrame with a "Stage" column prepended. Empty if no steps have run.

Raises:

Type	Description
ValueError	If metric is given but not present in the summary columns.