Interpreting PCA Eigenvectors and Projections in Risk Models
Summary
The document explains how principal component analysis can be interpreted in portfolio risk modeling. For a covariance matrix of asset returns, the first eigenvector is a unit-length portfolio direction with the greatest variance; each subsequent eigenvector maximizes variance among directions orthogonal to those already selected. The eigenvectors therefore define a ranked basis of portfolio exposures.
It also distinguishes projection from regression, noting that the operations can produce different results and that the choice may matter for statistical inference as well as practical modeling. Scaling choices affect the covariance matrix and can change the resulting basis; the response does not fully explain how to interpret rescaled loadings or when to standardize and restore asset scales. The discussion gives a conceptual starting point, but leaves several implementation questions open, including the exact factor-return and residual construction for a chosen risk model.
Key ideas
- The leading covariance eigenvector defines the unit portfolio direction with maximum variance.
- Later eigenvectors maximize variance subject to orthogonality to earlier directions.
- Projection and regression are distinct operations and can yield different results.
- Scaling returns before PCA can change the eigenvectors and resulting factor basis.
Tags
Full text
# PCA risk modelling # PCA risk modelling Been doing loads of reading about PCA, FA and SVD but still fail to understand the fundamentals of how PCA links with factor analysis in the context of risk modelling. Here is where I'm stuck: Given a TxN matrix, each row is a time period, column is an asset. This is returns in the basis of assets. We center each column and find the eigenvectors of the cov matrix. Then regress out the eigenvectors. The regression coefficients are thought of as factor returns and an eigenvector (if endowed with variance via sqrt(eigenvalue)) give exposures of each asset to the factor (portfolio) associated w that eigenvector. In factor modelling, we say each factor is a portfolio (weighted combination of assets), its returns are the returns of the assets weighted by the weights which define the portfolio. Mathematically, the eigenvectors are a basis. What variance is it that they maximise here? Is it that the eigenvector with the highest eigenvalue includes exposures of assets to a portfolio with the highest possible risk - dispersion of returns over time ? Next, why do I need regression? Can't I just project my original variables (assets) onto the subspace spanned by SOME of the factors, and keep only that (or remove it to get residual returns). Scale: I've seen many answers about "endowing the eigenvectors with scale to get loadings" by multiplying them by the sqrt(eigenvector), so that their squared norm gives the variance. What role does this play in the factor model exactly? Finally, I've seen legacy code at work where they standardise X first (divide each column by std) and at the end rescale it back up. I've also seen examples where they DON'T scale it back up in the end, what is the purpose of that and the difference between the two? ## Answer by Rylan (score 1) https://quant.stackexchange.com/a/79814 A few comments: - Suppose you have eigenvectors of a covariance matrix. One way you can think of the first eigenvector is the vector of length $1$ where, if we held a portfolio whose weights were given by that vector, the variance would be maximized (ie no other vector of length $1$ would give a greater variance.) The second eigenvector is the unit-length vector orthogonal to the first eigenvector that gives the greatest variance. And so on. - Projection is a slightly different operation from regression. Here's a link that I think has a nice visual. https://blogs.sas.com/content/iml/2020/02/05/visualize-residuals-for-various-regression-methods.html The first graph in the three is classic "regression" while the third is classic "projection". (The middle one I guess is a different kind of regression but not one that I see used!) I think the distinction is important if you are trying to do statistical inference, in practice it depends on the use but you might find they give results that are different but not completely disagreeing. - I'm not the best person to answer this, I will say for now that scaling in the covariance matrix will impact the vectors that come out, meaning you'll end up with a different basis. Different scalings can have different uses.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.