openquant.hyperparameter_tuning

Pure Python. Signatures are as written in the source; descriptions are the docstrings. All Python modules.

Hyperparameter search on purged k-fold splits (AFML chapter 9), for any Python model.

With scikit-learn, pass purged splits as cv; the leakage control is in the splits::

    from sklearn.model_selection import GridSearchCV
    from openquant.cross_validation import purged_kfold_splits

    splits = purged_kfold_splits(t0, t1, n_splits=5, pct_embargo=0.01)
    search = GridSearchCV(model, param_grid, cv=splits, scoring="neg_log_loss")
    search.fit(X, y, sample_weight=w)

scikit-learn fits with sample_weight but scores unweighted (AFML §9.2 and Snippet 7.4). :func:purged_search scores each fold with :func:classification_score, which applies the test fold's weights, and evaluates candidates from :func:expand_param_grid or :func:sample_param_sets in the same order as the Rust grid_search and randomized_search.

Functions

classification_score

classification_score(
    y_true: Any,
    probabilities: Any,
    sample_weight: Any = None,
    scoring: str = 'neg_log_loss',
) -> float

Score probabilities of y == 1 against 0/1 labels, weighting samples.

scoring is "neg_log_loss" (higher is better), "accuracy" or "balanced_accuracy" (mean recall over the classes present).

expand_param_grid

expand_param_grid(param_grid: Mapping[str, Sequence[Any]]) -> list[dict[str, Any]]

Every combination of a grid of int, float or bool values, keys in sorted order.

The same candidates, in the same order, as the Rust grid_search evaluates.

purged_search

purged_search(
    make_estimator: Callable[[dict[str, Any]], Any],
    param_sets: Sequence[Mapping[str, Any]],
    X: Any,
    y: Any,
    t0: Any,
    t1: Any,
    *,
    n_splits: int = 5,
    pct_embargo: float = 0.01,
    scoring: str = 'neg_log_loss',
    sample_weight: Any = None,
) -> dict[str, Any]

Cross-validate each parameter set on purged folds and pick the best mean score.

make_estimator(params) returns an unfitted model with fit(X, y, sample_weight=...) and predict_proba(X). Each fold fits a fresh model on the training samples with their weights and scores the test fold with :func:classification_score and the test weights, as the Rust grid_search does. Returns best_params, best_score and trials (params, fold_scores, mean_score per candidate). On a tie the later candidate wins, as in Rust.

sample_param_sets

sample_param_sets(
    param_space: Mapping[str, tuple],
    n_iter: int,
    seed: int,
) -> list[dict[str, Any]]

n_iter random draws from param_space, seeded, as the Rust randomized_search.

Each value of param_space is one of ("choice", [values]), ("uniform", low, high), ("log_uniform", low, high) (AFML §9.3: sample scale parameters such as C uniformly in their logarithm) or ("int", low, high) with high included. The same seed always gives the same draws.