hrp

Mean-variance optimisation needs the inverse of the covariance matrix, and the inverse is least stable exactly when it matters most. The more correlated the assets, the closer the matrix is to singular, the more its inverse amplifies estimation error, and the more diversification is needed: AFML calls this Markowitz’s curse (§16.3). The optimiser responds to small changes in the inputs with large changes in the weights, and out of sample it often loses to equal weighting.

Hierarchical Risk Parity (López de Prado, 2016) avoids the inversion altogether. It uses the covariance matrix twice, once to decide which assets are alike and once to read variances off it, and never solves a linear system. It needs no expected returns. The result is not optimal in sample by any criterion, and in the chapter’s Monte Carlo experiments (§16.6) it has lower out-of-sample variance than the minimum-variance portfolio that is.

Three steps

Tree clustering (§16.4.1). Turn correlations into distances, $d_{ij}=\sqrt{\tfrac12(1-\rho_{ij})}$ . Then, as the book’s second step, measure how far apart two assets’ whole rows of distances are,

\tilde d_{ij} \;=\; \sqrt{\textstyle\sum_{n}\,(d_{ni}-d_{nj})^2},

so two assets are close when they sit at similar distances from every other asset, not just from each other. Build a single-linkage tree on $\tilde d$ : repeatedly merge the two closest clusters, where the distance between clusters is that of their closest members. This is the default, distance="distance_of_distances" (see Which distance is clustered).

Quasi-diagonalisation (§16.4.2). Read the leaves of the tree from left to right. Reordering the covariance matrix that way puts similar assets next to each other, so the large covariances lie along the diagonal. No values change; this is a permutation. ordered_indices holds it.

Recursive bisection (§16.4.3). Start with every weight at 1. Split the ordered list in half. Give each half the variance it would have under inverse-variance weights, $\tilde V=\tilde w^\top \Sigma\,\tilde w$ with $\tilde w_i\propto 1/\Sigma_{ii}$ , and scale the two halves’ weights by

\alpha \;=\; 1-\frac{\tilde V_{\text{left}}}{\tilde V_{\text{left}}+\tilde V_{\text{right}}} \qquad\text{and}\qquad 1-\alpha

so the riskier half gets less. Recurse into each half until every piece is a single asset. Weights are positive and sum to 1 by construction.

What the hierarchy buys

The example gives nine assets the same volatility, so any difference between methods comes from correlation alone. Six are equities sharing one factor, two are bonds sharing another, and one is a commodity on its own.

import random

from openquant import hrp

# Nine assets with the same volatility: six equities that share a factor, two bonds that share
# another, and one commodity on its own. Correlation within a group is about 0.8.
rng = random.Random(2)
sizes = {"equity": 6, "bond": 2, "commodity": 1}
names = [f"{g}_{i}" for g, k in sizes.items() for i in range(k)]
returns = []
for _ in range(750):
    row = []
    for k in sizes.values():
        factor = rng.gauss(0, 0.01)
        row += [0.9 * factor + 0.45 * rng.gauss(0, 0.01) for _ in range(k)]
    returns.append(row)

weights, order = hrp.allocate_hrp(names, asset_returns=returns)
print("leaf order:", " ".join(names[i] for i in order))

var = [sum(r[j] ** 2 for r in returns) / len(returns) for j in range(9)]
ivp = [(1 / v) / sum(1 / u for u in var) for v in var]
print("group        inverse-variance      HRP")
for g in sizes:
    pick = [i for i, n in enumerate(names) if n.startswith(g)]
    print(f"{g:10s}   {sum(ivp[i] for i in pick):16.3f}   {sum(weights[i] for i in pick):6.3f}")

leaf order: equity_1 equity_5 equity_4 equity_0 equity_2 equity_3 commodity_0 bond_0 bond_1
group        inverse-variance      HRP
equity                  0.679    0.552
bond                    0.214    0.235
commodity               0.107    0.213

Inverse-variance weighting sees nine equally risky assets and gives each a ninth, which puts 68% of the portfolio on one factor because that factor happens to have six tickers. HRP sees that the six are one bet and cuts it to 55%. A minimum-variance optimiser would go further, and would do so by inverting a matrix in which six columns are nearly collinear.

The leaf order also shows the method’s known weakness. Bisection cuts the list in half, not the tree: nine leaves split four and five, which puts equity_2 and equity_3 in the second half with the commodity and the bonds, away from their own cluster. That is why equities end up at 55% rather than the third that three equal clusters would suggest. (With distance="correlation" the leaves come out in another order, the split falls elsewhere, and equities get 48%: the tree matters to the weights through where the halves are cut.) hcaa and López de Prado’s later nested clustered optimisation exist to address this.

From Rust

use nalgebra::DMatrix;
use openquant::hrp::{HierarchicalRiskParity, HrpError};

// Assets 0 and 1 are nearly the same bet; asset 2 is independent. All have variance 0.04.
let covariance = DMatrix::from_row_slice(3, 3, &[
    0.040, 0.036, 0.000,
    0.036, 0.040, 0.000,
    0.000, 0.000, 0.040,
]);
let names: Vec<String> = ["a", "b", "c"].map(String::from).to_vec();

let mut model = HierarchicalRiskParity::new();
model.allocate(&names, None, None, Some(&covariance), None, false)?;

// The tree joins a and b first, so they are adjacent in the leaf order.
assert_eq!(model.clusters[0], [0, 1]);
let position = |asset: usize| model.ordered_indices.iter().position(|&i| i == asset).unwrap();
assert_eq!(position(0).abs_diff(position(1)), 1);

assert!((model.weights.iter().sum::<f64>() - 1.0).abs() < 1e-12);
assert!(model.weights.iter().all(|w| *w > 0.0));
// The independent asset gets more than either member of the correlated pair.
assert!(model.weights[2] > model.weights[0] && model.weights[2] > model.weights[1]);

assert_eq!(model.allocate(&names, None, None, None, None, false), Err(HrpError::NoData));

allocate takes whichever of prices, returns or a covariance matrix you have. If returns are given, prices are ignored; a supplied covariance matrix is always used as is. After it returns, weights, ordered_indices, clusters (the merge list, scipy-style: ids below $n$ are assets, $n+k$ is the cluster made by merge $k$ ), and the seriated correlation and distance matrices are public fields. HierarchicalRiskParity::new() clusters on the default distance; HierarchicalRiskParity::with_distance(HrpDistance::Correlation) (Python: allocate_hrp(..., distance="correlation")) picks the other.

Which distance is clustered

AFML’s Snippet 16.4 calls sch.linkage(dist, 'single') with the square matrix $d$ . scipy reads a square array as one observation per row, so the book’s tree is built on the Euclidean distance between rows of $d$ , which is $\tilde d$ above and the second step §16.4.1 describes. mlfinlab, and this library before #167, pass $d$ as pairwise distances instead. The two trees usually differ: they do on all four cases of the 23-ETF test fixture, and in the book’s 10-asset simulation the leaf order differs at 97% of rebalances.

`distance=` (Python)	`HrpDistance::` (Rust)	Tree built on
`"distance_of_distances"` (default)	`DistanceOfDistances` (default)	$\tilde d_{ij}$ , as Snippet 16.4
`"correlation"`	`Correlation`	$d_{ij}$ , as mlfinlab

The default was chosen by measurement. On the book’s own Monte Carlo (§16.5, Snippets 16.4 and 16.5: 10 assets, 260-day window, monthly rebalance, 10,000 runs, seeds [51, k]; the HRP runbook is the code), SYNTHETIC data:

	$\tilde d$ (default)	$d$ (`"correlation"`)
HRP mean OOS variance ×1e4	3.59	3.81
IVP / HRP, mean OOS variance	1.380	1.300
CLA / HRP, mean OOS variance	1.419	1.337
runs where HRP beats IVP	82.9%	76.8%
runs where HRP beats CLA	65.4%	62.2%
HRP turnover per rebalance	0.122	0.216
HRP effective number of assets	7.27	7.26

Clustering as the book does lowers HRP’s variance (paired log-ratio $t=4.4$ against the pairwise tree), brings the IVP ratio to the book’s figure (about 1.38, quoted from memory), and nearly halves turnover. A plausible reason for the lower turnover, not tested separately: $\tilde d$ compares whole rows of distances, so one noisy correlation moves it less, and the leaf order is steadier from one window to the next. Pass distance="correlation" to reproduce mlfinlab or pre-#167 results.

What to watch for

The default tree changed in #167. Weights from earlier versions were built on pairwise distances; pass distance="correlation" (HrpDistance::Correlation) to get them back.
use_shrinkage is a fixed 10% shrink of the off-diagonal terms, not a Ledoit–Wolf or OAS estimator. For a real shrinkage estimate, compute the covariance yourself and pass it in.
Returns are simple returns and the covariance is the sample covariance, unannualised. HRP weights do not depend on the scale, so annualisation is immaterial here.
resample_by keeps every 5th ("W") or 21st ("M") row. It is positional, not calendar-aware. Before #93 was fixed this path scrambled the matrix; weights from older versions with resampling are wrong.
plot_clusters does not return a dendrogram. ivl and leaves are the real leaf order; icoord and dcoord are placeholders with every link drawn at height 1. To draw the tree, use clusters with the seriated distances.
Single linkage chains. One asset moderately close to two groups can glue them together early, and the rest of the tree inherits it. Look at seriated_correlations before trusting the weights; a blocky diagonal means the ordering worked.
Long-only, fully invested, no constraints. There is no way to bound a weight or target a volatility. For that see portfolio-optimization.

hcaa — allocation down the tree’s own branches (Ward linkage by default) with other allocation metrics.
onc — choose the number of clusters instead of bisecting blindly.
codependence — distances other than correlation.
cla, portfolio-optimization — the convex optimisers HRP is an answer to.

API reference

Python

Signatures, parameters, return values and errors for every function: openquant.hrp.

hrp.allocate_hrp — Hierarchical Risk Parity weights (AFML chapter 16, Snippets 16.1-16.4).

Rust

Rustdoc for this commit, with every public item of openquant::hrp (after a crates.io release the same pages are on docs.rs):

hrp