filters

Concept Overview

Instead of sampling at fixed intervals, AFML Chapter 2 uses structural event filters to detect when something meaningful happens in the price process. This produces training examples that correspond to real market inflection points rather than arbitrary calendar dates.

The CUSUM filter tracks a cumulative sum of returns (or price changes). It resets to zero when the cumulative deviation exceeds a threshold h, and the reset point becomes an event. This captures points where the price has moved “enough” since the last event. The filter is directional: it tracks both positive and negative cumulative deviations separately.

The z-score filter standardizes the current value against a rolling mean and standard deviation, firing when the z-score exceeds a threshold. This is useful for mean-reverting signals where you want events when the price deviates significantly from its recent average.

Both filters replace the naive approach of labeling every bar, which creates highly correlated and redundant training examples.

When to Use

Apply event filters immediately after bar construction and before labeling. They bridge raw bars to the labeling module: bars go in, event timestamps come out.

Prerequisites: A price series (close prices from bars), and optionally timestamps.

Alternatives: Fixed-interval sampling (simpler but creates redundant events), or custom event logic for strategy-specific triggers.

Mathematical Foundations

Symmetric CUSUM Filter

$S_t^{+}=\max\!\left(0,\,S_{t-1}^{+}+r_t\right),\qquad S_t^{-}=\min\!\left(0,\,S_{t-1}^{-}+r_t\right),\qquad \text{event at }t\iff S_t^{+}>h_t\;\lor\;S_t^{-}<-h_t$

where $r_t=\ln(p_t/p_{t-1})$ is the log return and $h_t$ the threshold — a constant for Threshold::Scalar, a per-bar series for Threshold::Dynamic. Both arms are needed: $S^{+}$ alone only ever detects upward runs. Whichever arm breaches is reset to $0$ and the bar is emitted as an event, so the filter measures runs away from the last event rather than a cumulative level.

Z-score Filter

$z_t=\frac{x_t-\mu_t}{\sigma_t},\qquad \text{event at }t\iff|z_t|>h$

where $\mu_t$ and $\sigma_t$ are the rolling mean and standard deviation over the lookback window ending at $t$.

Key Parameters

Parameter	Type	Description	Default
close	list[float]	Input price series (close prices)	—
threshold	float	CUSUM trigger level; controls event frequency (Python: scalar only)	—
threshold	Threshold	CUSUM trigger: Threshold::Scalar(f64) or Threshold::Dynamic(Vec) (Rust)	—
mean_window	int	Rolling mean lookback for z-score filter	—
std_window	int	Rolling std lookback for z-score filter	—
timestamps	list[str]	Optional timestamps; use _timestamps variants to get event times instead of indices	—

Usage Examples

Python

CUSUM and z-score event detection

import openquant

close = [100.0, 100.1, 99.9, 100.2, 100.05, 100.3, 99.7, 100.1]
# The filters bindings parse "%Y-%m-%d %H:%M:%S" — a space, not an ISO "T".
timestamps = [
    "2024-01-02 09:30:00", "2024-01-02 09:31:00",
    "2024-01-02 09:32:00", "2024-01-02 09:33:00",
    "2024-01-02 09:34:00", "2024-01-02 09:35:00",
    "2024-01-02 09:36:00", "2024-01-02 09:37:00",
]

# CUSUM filter: fires when cumulative deviation exceeds threshold
event_indices = openquant.filters.cusum_filter_indices(close, 0.02)

# With timestamps: returns event timestamps directly
event_ts = openquant.filters.cusum_filter_timestamps(close, timestamps, 0.02)

# Z-score filter: fires when z-score exceeds threshold
z_indices = openquant.filters.z_score_filter_indices(close, mean_window=20, std_window=20, threshold=2.0)
z_ts = openquant.filters.z_score_filter_timestamps(close, timestamps, mean_window=20, std_window=20, threshold=2.0)

Rust

CUSUM with static and dynamic thresholds

use openquant::filters::{cusum_filter_indices, cusum_filter_indices_checked, Threshold};

let close = vec![100.0, 100.1, 99.9, 100.2];

// Static threshold
let idx = cusum_filter_indices(&close, Threshold::Scalar(0.02));

// Dynamic threshold (e.g. volatility-scaled per bar)
let dynamic_h = vec![0.02, 0.025, 0.018, 0.022];
let idx = cusum_filter_indices_checked(&close, Threshold::Dynamic(dynamic_h)).unwrap();

Common Pitfalls

• Setting the CUSUM threshold too tight in volatile regimes — you get too many events and labels become noisy. Scale h by recent volatility.
• Using different thresholds in training vs live inference — the event distribution shifts and the model sees a different regime.
• Applying CUSUM to non-stationary raw prices instead of returns or log-returns — the filter becomes meaningless as the price drifts.
• Python bindings only support scalar thresholds — use the Rust API directly if you need dynamic (per-bar) thresholds.

API Reference

Python API

• filters.cusum_filter_indices
• filters.cusum_filter_timestamps
• filters.z_score_filter_indices
• filters.z_score_filter_timestamps

Rust API

• cusum_filter_indices
• cusum_filter_timestamps
• cusum_filter_indices_checked
• cusum_filter_timestamps_checked
• z_score_filter_indices
• z_score_filter_timestamps
• z_score_filter_timestamps_checked
• Threshold
• FilterError

Risk Notes and Caveats

• Calibrate thresholds to target event frequency, not just sensitivity.
• Use identical filtering in train and live pipelines.
• Rust API supports dynamic (per-bar) thresholds via Threshold::Dynamic; Python bindings accept only a scalar threshold.
• Rust _checked variants return Result<…, FilterError> for input validation; Python raises exceptions.

Related Modules

• data-structures
• labeling
• sample-weights