Why Covariance Shrinkage Improves Portfolio Risk Estimates

# Why shrink the covariance matrix?


# Why shrink the covariance matrix?












I'm trying to understand why it's useful to shrink the covariance matrix for portfolio construction or in fact general. Think I missing something. I know if you have 5,000 stocks it's a lot of calculations but if we assume that computing power is not a problem.

## Answer by vonjd (score 18, accepted)

https://quant.stackexchange.com/a/11570

Have a look at this classic paper:

Honey, I Shrunk the Sample Covariance Matrix by O. Ledoit and M. Wolf

The abstract answers your question already:

> The central message of this article is that no one should use the sample covariance matrix for portfolio optimization. It is subject to estimation error of the kind most likely to perturb a mean-variance optimizer. Instead, a matrix can be obtained from the sample covariance matrix through a transformation called shrinkage. This tends to pull the most extreme coefficients toward more central values, systematically reducing estimation error when it matters most. Statistically, the challenge is to know the optimal shrinkage intensity. Shrinkage reduces portfolio tracking error relative to a benchmark index, and substantially raises the manager's realized information ratio.

N.B.: Shrinking in this context does not mean use fewer values!

If you want to perform a few experiments yourself, then the Python pyportfolioopt package is for you.

## Answer by RRL (score 10)

https://quant.stackexchange.com/a/11567

The estimation of a covariance matrix is unstable unless the number of historical observations $T$ is greater than the number of securities $N$ (5000 in your example). Consider that 10 years of data represents only 120 monthly observations and about 2500 daily observations.

Depending on the application, using data dating farther back than 10 years may be impractical and undesirable for many reasons -- de-listed stocks, regime changes, etc. In fact, risk management applications often require covariance estimations over recent periods of time (1-3 years).

Computational applications ranging from portfolio construction to Monte Carlo simulation generally require that the estimated covariance matrix is non-singular and positive definite. If N is greater than T, then the estimated covariance matrix will be singular. Furthermore a variety of small sample problems persist until the number of observations is an order of magnitude larger than the number of securities.

## Answer by Phil H (score 7)

https://quant.stackexchange.com/a/11566

- Transaction costs - even for banks, funds etc, every trade has an associated cost, so if you would be buying a small number of shares, it's probably cheaper to carry the risk and not make those small trades.

- The source data is imperfect, and contains noise. A lot of the smaller components are simply artefacts of that noise so it would be both an unnecessary expense and an additional unwanted risk to include them all.

When you come to rebalance the portfolio, the fewer items in the basket, the fewer trades are needed to rebalance the whole thing. Also, many of the small components will have the greatest variance, so you will end up dumping all of the holding and buying something else. By comparison, the largest components should be the most stable, and so theoretically can be left unchanged within some bound.

## Answer by MichaelJ (score 5)

https://quant.stackexchange.com/a/12611

Go ahead and compute a sample covariance matrix with 5,000 stocks on a few years (or less) of daily or monthly returns data. This can be done almost instantly on a modern computer. There is a very good chance that this matrix will not be a covariance matrix. You can check by inspecting the eigenvalues. If any are negative then you don't have a covariance matrix, as a covariance matrix must be positive semi-definite. In theory, a sample covariance matrix is always positive semi-definite, but when it is computed with finite precision that is often not the case. Most portfolio construction techniques, in particular those based on convex quadratic programming, further require that the supplied covariance matrix is positive definite.

Thus, the sample covariance matrix isn't really a viable option for a lot of portfolio construction methods. There are several ways to get a positive definite covariance matrix. All of these can considered shrinkage methods. In general, you want to use whatever method gives the most accurate estimates while guaranteeing positive definiteness.