Handling Singular Covariance Matrices in High Dimensions

# Answer by user6430 (score 2)


# How to treat large (5K-10K) non-positive-definite (particularly near-singular) covariance matrices for Cholesky decomposition?












I have a very large covariance matrix (around 10000x10000) of returns, which is constructed using a sample size of 1000 for 10000 variables. My goal is to perform a (good-looking) Cholesky decomposition of this matrix. However, as expected, this matrix is near-singular with very small ( < 10^-10 ) eigenvalues (around 5K-6K out of 10K).

I tried a naive approach; doing an eigenvalue decomposition, and setting all the eigenvalues that are less than 10^-10 to 10^-10. In other words, if eigenvalue < 10^-10, then eigenvalue = 10^-10. After that, I reconstructed the matrix with the modified eigenvalue matrix. However, although I can perform Cholesky decomposition, it is very unstable.

What is the best way to handle this PD approximation for large matrices?

## Answer by user6430 (score 2)

https://quant.stackexchange.com/a/9664

Let $t$ be the number of days (time periods), and let $p$ be the number of assets. You have $t=1000$ and $p=10000$. For any given dataset, it is assumed that the sample covariance matrix $\mathbf{C}$ accurately represents the population covariance matrix $\boldsymbol{\Sigma}$, however, as $p \rightarrow t$ or if $p > t$ (as in your case), the eigenvalues become unreliable and can also take on a value of zero, resulting in lack of positive definiteness. With high-dimensional datasets becoming more popular, there is greater potential for the number of dimensions to approach the sample size ($p \rightarrow t$), leading to biased eigenvalues of $\mathbf{C}$ and $\mathbf{R}$. Certainly, there will be $p-t$ zero eigenvalues whenever $p>t$ and one zero eigenvalue whenever $p=t$.

You can use singular value decomposition (SVD), which will extract the singular values (eigenvalues) along with the remaining singular values. If $\mathbf{X}$ is your return matrix ($t$ rows, $p$ columns) then use the R syntax below to look at the eigenvalues ("eigvals") from eigendecomposition versus the singular values ("s") from SVD:

```
R=cor(X);

p <- ncol(X);

t <- nrow(X);

lambdae <- eigen(R);

eigvals <- as.vector(lambdae$values);

E<-as.matrix(lambdae$vectors);

s<-svd(R)

s$d
```

Last, if you are going to do anything with your data, you might perform dimension reduction by using the eigenvectors to represent your data for dimensions that have non-zero eigenvalues, as they are uncorrelated. You could also use PCA after extracting the singular-values, and ignore the zero eigenvalues. The loadings with the principal components will represent correlation between the original 10000 assets and the reduced orthogonal (non-correlated) dimensions.

## Answer by user12348 (score 2)

https://quant.stackexchange.com/a/11240

Your issue demonstrated in R with interesting solution Equity Risk Model Using PCA. Another useful link in Matlab by Nick Higham himself NCM implementation by Nick Higham written for Matlab. Another good discussion on shrinkage and other aspects of Correlation Adjustment

## Answer by Richi Wa (score 1)

https://quant.stackexchange.com/a/9628

I think that your problem can be solves by using another estimator for your covariance matrix. A so called shrinkage estimator leads to covariance matrix that is non-singular. Then a Cholesky decomposition should work (maybe there is even a short-cut in the shrinkage world, I will check alter on).

The R package corpcor contains functions to perform shrinkage estimation. More information can be found on the webpage of the developer.

## Answer by Marco Breitig (score 1)

https://quant.stackexchange.com/a/11220

From LEP's answer:

> there will be p−t zero eigenvalues whenever p>t and one zero eigenvalue whenever p=t.

This is the main reason, your true covariance matrix will have p-t eigenvalues exactly 0. With computer arithmetic you'll have lots of eigenvalues around machine precision, usually about 10^-15. So there should not only be around 5K-6K zero eigenvalues, but about 9K.

You also might want to have a look at Computing the nearest correlation matrix—a problem from finance by Nicholas J. Higham.