Measuring Asset Dependence with Mutual Information and Variation of Information
Summary
This reference explains how information theory can measure dependence between variables, including asset returns. It introduces entropy as uncertainty, then defines mutual information as the reduction in uncertainty about one variable from observing another. Mutual information is symmetric and nonnegative but is not a metric because it does not satisfy the triangle inequality. Variation of information combines the two conditional entropies and is presented as a true metric.
For continuous data, the document describes discretizing observations into bins to estimate entropy and notes that estimates depend on bin selection. It summarizes optimal binning research and outlines a copula-based alternative: transform observations to normalized ranks, whose compact uniform margins can reduce difficulties from unbounded supports and arbitrary marginal bins. Illustrative examples compare linear, nonlinear, absolute-value, and independent relationships across several dependence measures. They demonstrate intended use, but no numerical findings are reported in the text, and the estimates remain sensitive to sample and estimation choices. The material is methodological background rather than a trading strategy.
Key ideas
- Entropy represents uncertainty in a variable's probability distribution.
- Mutual information measures the uncertainty reduction in one variable when another is observed, but it is not a metric.
- Variation of information is a metric based on the conditional uncertainty in both directions.
- Continuous observations can be discretized for estimation, though bin choices affect the results.
- Copula methods use normalized ranks to address some difficulties with unbounded marginals and binning.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.