How Scaling Changes PCA Factors and Their Explanatory Power
Summary
This discussion explains why applying principal component analysis to standardized asset returns can produce different factor returns and regression results from applying it to raw returns. Standardizing gives each asset equal variance before PCA, so the resulting components are orthogonal in that standardized space. If those component weights are then applied to unstandardized returns, the reconstructed factor series need not remain uncorrelated. PCA on raw returns instead emphasizes high-variance assets, so its components are uncorrelated under the covariance-based decomposition but can be dominated by a few assets.
The reported comparison is an empirical observation: raw-return factors gave better explanatory power for the assets in the question, while the post-standardization factor returns were correlated. The answer attributes this to changing the weighting and interpretation of the assets. This is a conceptual explanation rather than a general performance result; outcomes depend on how factors are constructed, transformed, and used in regressions. Standardizing is not universally preferable or inferior, and factor orthogonality depends on using the same data representation throughout the analysis.
Key ideas
- Standardizing returns makes PCA weight assets by correlation structure rather than their original variance.
- PCA components are orthogonal in the space on which PCA is performed.
- Applying component weights to differently scaled returns can yield correlated factor series.
- PCA on raw returns emphasizes high-variance assets, which may make factors resemble those assets.
- Factor explanatory power depends on consistent scaling between PCA and subsequent regression analysis.
Tags
Full text
# PCA factors not uncorrelated # PCA factors not uncorrelated I ran into an interesting case recently. I am trying to construct a set of uncorrelated factors for a statistical factor model. I have started with picking a certain amount of assets (indices) which I believe would fully capture the different dynamics of the US market. Naturally the set of indices are not uncorrelated so after making the data stationary (i.e taking the time series of daily returns) I perform PCA to extract the common independent factors. After choosing an appropriate number of components I can then use the matrix with factor loadings of the different PCs and the matrix of the time series of the different assets to obtain historical factor returns for my new PC factors. Here is where it gets interesting and unexpected for me. I tried performing the procedure first by standard scaling the returns of each asset (mean 0 and unit variance). Interestingly enough doing it this way yielded PC factors whose returns are quite correlated. I then tried directly performing the PCA on the returns without scaling them and then obtained factor returns which as expected are uncorrelated. Furthermore I was particularly surprised that the factors obtained via the non-scaled returns had on average better explanatory power when running regressions for different assets I am trying to explain with the factor model. What is the deal here? I have always thought that it is better to scale data when performing PCA while in this case my empirical results show the opposite. The only explanation I can come up with so far is that by taking the returns of the price series in the first place I have already 'scaled' and 'centered' the data in some sense and that the subsequent scaling simply washes away important information from my data. Any further ideas? ## Answer by Arshdeep (score 2) https://quant.stackexchange.com/a/78957 If $X1$ and $X2$ are almost perfectly correlated (but variance of x1 is a million times variance of x2) so that the first (normalised) pca factor is (1/sqrt(2),1/sqrt(2)) and second is (1/sqrt(2),-1/sqrt(2)), they are orthogonal. However if you replace the normalised assets in eigenvectors by NON normalised assets, they are now correlated because x2 is insignificant in comparison to x1. Infact they will now be perfectly correlated. There is absolutely nothing funny going on, it's just how the data is. If you create factors out of non scaled returns, you are (mostly) ranking assets in terms of their variances. Thus your factors are (mostly) assets themselves. If you take k of these, k of your regressions will have perfect R squares. Again this is not surprising. In the other case, you changed your PCA factors to un-normalized, and they no longer reflect the "PCA" decomposition of your real returns. So poorer performance is expected, as these are not the factors that govern real data. In other words, by rescaling the eigen vectors, you drowned out the assets with small variances, and no wonder you will not be able to explain the real returns of these assets.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.