EDA API

Exploratory data analysis helpers for inspecting a dataset before it enters the pipeline. Import them from h2ml.utils:

from h2ml.utils import (
    profile_dataset,
    flag_outliers,
    target_correlations,
    check_pipeline_readiness,
    correlated_feature_pairs,
)

See the Exploratory Data Analysis guide for a worked walkthrough.

profile_dataset

h2ml.utils.eda.profile_dataset(df)

Print a one-screen profile of a DataFrame.

Reports overall shape and memory footprint, then a per-column table of dtype, null count, null percentage, distinct-value count, and the first row's value as a sample.

Parameters:

Name	Type	Description	Default
df	DataFrame	DataFrame to profile.	required

flag_outliers

h2ml.utils.eda.flag_outliers(df, multiplier=1.5)

Summarise IQR-based outliers for each numeric column (Tukey's fences).

A value is an outlier when it falls below Q1 - multiplierIQR or above Q3 + multiplierIQR. multiplier=1.5 is the conventional "mild outlier" threshold; 3.0 flags only extreme outliers. NaNs are ignored — quantiles skip them and NaN never satisfies the fence comparisons.

Parameters:

Name	Type	Description	Default
df	DataFrame	DataFrame to scan. Only numeric columns are considered.	required
multiplier	float	IQR multiplier setting the fence width. Default 1.5.	1.5

Returns:

Type	Description
DataFrame	One row per numeric column with lower/upper bounds, outlier count, and
DataFrame	outlier percentage. Empty (with the expected columns) when df has no rows
DataFrame	or no numeric columns.

target_correlations

h2ml.utils.eda.target_correlations(df, target, n_features=None, sort_by='pearson')

Rank numeric features by their correlation with the target.

Computes Pearson (linear), Spearman, and Kendall (both monotonic) coefficients between each numeric feature and the target, giving a quick read on the relationships to expect during modelling. Correlations use pairwise-complete observations, so NaNs are dropped per feature rather than poisoning the result.

Parameters:

Name	Type	Description	Default
df	DataFrame	DataFrame containing the target and feature columns.	required
target	str	Name of the (numeric) target column.	required
n_features	int | None	Number of top features to return. None returns all.	None
sort_by	str	Coefficient to rank by — "pearson", "spearman", or "kendall". Rows are ordered by descending absolute value, so the strongest relationships (positive or negative) come first; NaNs sink last.	'pearson'

Returns:

Type	Description
DataFrame	DataFrame with columns [feature, pearson, spearman, kendall], one row per
DataFrame	numeric feature, sorted strongest-first. Empty (with those columns) when no
DataFrame	numeric features are present.

Raises:

Type	Description
ValueError	If target is missing, non-numeric, or sort_by is invalid.

check_pipeline_readiness

h2ml.utils.eda.check_pipeline_readiness(df, target=None, task=None, n_splits=5, max_classes=2)

Report issues that would stop (or degrade) an H2MLPipeline run, before you run it.

Mirrors the checks in H2MLPipeline._validate_store, but reports them all at once instead of raising on the first, so you can fix everything in one pass. Numeric columns are what become the model matrix X, so the finite/variance checks target those.

This is deliberately a blockers-only verdict — for descriptive per-column stats use profile_dataset, and for outliers use flag_outliers.

Parameters:

Name	Type	Description	Default
df	DataFrame	DataFrame of features (and optionally the target).	required
target	str | None	Target column name. When given, target-specific checks run.	None
task	str | None	"classification" enables class-count / cardinality / imbalance checks. None or "regression" skips them.	None
n_splits	int	Planned CV fold count, checked against the row count.	5
max_classes	int	For classification, a numeric target with more than this many distinct values is flagged as likely count/continuous data mistakenly typed as classification (the pipeline would treat each value as its own class). Default 2 — only a binary 0/1 target passes; raise it if you genuinely have a numeric multiclass label.	2

Returns:

Type	Description
DataFrame	DataFrame [check, severity, columns, detail], one row per detected issue
DataFrame	(severity "error" = will break the run, or "warning"). Empty when ready.

correlated_feature_pairs

h2ml.utils.eda.correlated_feature_pairs(df, threshold=0.7, method='pearson', target=None)

List pairs of numeric features whose absolute correlation meets a threshold.

Surfaces multicollinearity / redundancy between features — a preview of what the pipeline's step-2 correlation filter (corr_threshold, default 0.7) will prune. The target is excluded so this stays feature↔feature; for feature↔target relationships use target_correlations.

Note: step 2 drops a feature when it exceeds corr_threshold in any of Pearson, Spearman, or Kendall; this helper inspects one method at a time for clarity.

Parameters:

Name	Type	Description	Default
df	DataFrame	DataFrame of features (and optionally the target).	required
threshold	float	Minimum |correlation| for a pair to be reported. Range [0, 1].	0.7
method	Literal['pearson', 'spearman', 'kendall']	"pearson", "spearman", or "kendall".	'pearson'
target	str | None	Target column to exclude from the feature set (optional).	None

Returns:

Type	Description
DataFrame	DataFrame [feature_1, feature_2, corr] for pairs with |corr| >= threshold,
DataFrame	strongest first. Empty (with those columns) when fewer than two numeric
DataFrame	features are present or none clear the threshold.

Raises:

Type	Description
ValueError	If method is invalid or threshold is outside [0, 1].