Interpreting Random Matrix Theory Eigenvalue Cleansing
Summary
The document discusses fitting a Marchenko–Pastur (MP) eigenvalue distribution to a sample correlation matrix built from S&P 500 stock returns. It notes that a smoothed density plot can make the empirical and theoretical peak heights appear quite different even when their shapes and spectral cutoffs seem similar. The answer recommends using a discrete bar chart to inspect eigenvalue separation, especially the large eigenvalue associated with the market factor.
For cleansing, the described approach replaces eigenvalues below the upper noise-band threshold with their average. This also replaces values below the lower threshold, since those are included in the set beneath the upper cutoff. The response presents averaging as one possible cleansing method, not a uniquely required procedure. It does not provide a detailed derivation of MP calibration, quantify the impact of density smoothing, or compare alternative cleansing methods.
Key ideas
- A smoothed density plot can obscure the discrete distribution of sample eigenvalues.
- A bar chart can make separation between the bulk and large factor eigenvalues easier to inspect.
- The described RMT cleansing method averages eigenvalues below the upper noise-band cutoff.
- That procedure includes eigenvalues below the lower cutoff as well.
Tags
Full text
# RMT (Random Matrix Theory) issue with callibrating MP distribution - # RMT (Random Matrix Theory) issue with callibrating MP distribution - I am seeing an issue when callibrating an MP distribution. Assume a log return series for the SP500 with the following dimensions dim(xts.sp500.ret.stocksonly) ==> [1] 1133 478 ``` sp500.cor <- cor.empirical(xts.sp500.ret.stocksonly) sp500.eigens <- eigen(sp500.cor)$values sp500.eigen.density <- density(sp500.eigens,n=5000) plot(sp500.eigen.density,xlim=c(0,4),main="sp500 returns eigenvalue density") ``` I assume my 'Q' value is 1133/478 = Problem: Even though the 'shape' and cutoffs seem OK -- the density value (y axis) seem vastly OFF. Peak of 1.5 for the real series - 5 or so for the theoretical, (note I am truncating the plot so the market eigenvalues are not shown, they are huge around 200). Question: 1) Is this expected? 2) How does this affect callibration? Should I trust the results and simply look at the cutoffs? 3) Also when 'cleansing' the matrix I see most code (e.g. filter.RMT in tawny) simply replaces values below Lambda+ with the average, what about Lambda- though? thanks much! ## Answer by Ram Ahluwalia (score 3, accepted) https://quant.stackexchange.com/a/2811 - The plot function is smoothing the plot. You should show the distribution of eigenvalues via a bar chart. Because a bar chart is discrete you can better discern the separation of the top-most eigenvaules. The top-most eigenvalue (representing the market factor) should be substantially greater than bulk of the eigenvalue distribution. - I assume by "calibration" you mean "eigenvalue cleansing". The approach would be to apply the RMT cleansing procedure on eigenvalues beneath the upper noise-band (lambda+) - The procedure of replacing eigenvalues below Lambda+ with the average will by definition also replace values below Lambda- as well. Replacing the eigenvalues by the average is one of many possible cleansing procedures.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.