Mutual Information and Distance Correlation for Nonlinear Dependence

# Most natural generalization of covariance/correlation to model dependence of extreme events


# Most natural generalization of covariance/correlation to model dependence of extreme events












One of the most serious shortcomings of covariance/correlation are the assumptions of linearity and normality.

What is the most natural generalization of these measures of dependence when you want to model the dependence structure of extreme events using heavy-tailed distributions, e.g. the Generalized extreme value distribution?

With "most natural generalization" I mean that the classical covariance/correlation is included as a special case when the usual assumptions hold.

(Disclosure: This question was posted at Cross Validated nearly two weeks ago, yet didn't receive any answers)

## Answer by Amelio Vazquez-Reina (score 4, accepted)

https://quant.stackexchange.com/a/4825

Mutual information measures how much knowing one variable reduces uncertainty about another variable. It considers any type of dependency (linear or non-linear), it's measured in bits, and it is widely used in machine learning, computer vision NLP and other fields.

## Answer by vonjd (score 3)

https://quant.stackexchange.com/a/4855

The answer of user27915816 led me into the right direction, yet I think I found an even better generalization: Distance Correlation (dCor)

There are several reasons for that:

- It generalizes classical (i.e. linear) correlation in the sense that linearity is a special case. It gives identical readings for linear dependence.

- There are analogs for variance, covariance and standard deviation, so these identities hold: $$\operatorname{dVar}^2_n(X) := \operatorname{dCov}^2_n(X,X)$$ and $$\operatorname{dCor}(X,Y) = \frac{\operatorname{dCov}(X,Y)}{\sqrt{\operatorname{dVar}(X)\,\operatorname{dVar}(Y)}}$$

- $dCor=0$ implies true independence, all other readings imply linear or non-linear dependence - Compare the following readings, first linear correlation (source):

and distance correlation (source):

Beware, oversimplification ahead: The reason it shows this behavior is basically that it is the correlation of the characteristic functions of the random variables, i.e. the Fourier transforms of the probability density functions, i.e. a rotation from the time into the frequency domain. Therefore not only linear dependence is being tested but basically all functional dependencies which can be represented by the (periodic) complex exponential function. To get an intuition read also this article: Here.

There are implementations in Excel and R.