Handling Collinear Factor Loadings in Return Estimation
Summary
The note considers cross-sectional estimation of factor returns when asset-loading columns are linearly dependent. Ordinary least squares then has no unique solution because the loading matrix’s cross-product is singular. A market factor combined with industry indicators is a practical example: a common return can be attributed either to the market or to every industry.
One suggested remedy is a ridge penalty on squared factor returns, which makes the estimation system invertible and shrinks estimated factor returns as the penalty grows. The answer recommends an identifying constraint instead when exposures encode groups, such as requiring industry-factor returns to sum to zero. That assigns common movement to the market factor and leaves industry factors to capture relative returns. Ridge regularization can help limit overfitting with many factors, but the note gives no universal penalty choice or empirical comparison and says it is not the preferred fix for collinear exposures.
Key ideas
- Linearly dependent factor exposures make ordinary least-squares factor returns non-unique.
- A market factor plus exhaustive group indicators cannot distinguish common movement without an identifying convention.
- Constraining group-factor returns, such as requiring their sum to be zero, allocates common return to the market factor.
- A ridge penalty makes the system solvable and shrinks factor estimates, but is not the preferred treatment of collinearity.
Tags
Full text
# Estimating factor returns with linearly dependent loadings
# Estimating factor returns with linearly dependent loadings
Given an $n\times 1$ vector of asset returns $r$, and a $n\times k$ matrix of factor loadings $X$, we can express the asset returns in terms of as-yet-unknown factors $f$ using
$$ r = Xf + \epsilon $$
and then estimate the factor returns with a cross-sectional regression
$$ \hat{f} = (X^TX)^{-1} X^Tr $$
which is equivalent to minimizing the cross-sectional stock-specific return $\|{\epsilon} \|^2$. However, this relies on the loading matrix having linearly independent columns, so that $X^TX$ is invertible.
There are sensible reasons for choosing $X$ to not have linearly independent columns. For example, with 4 stocks from two distinct industries, we may want to consider an overall market factor, and two industry factors -
$$ X = \left[\matrix{1 & 1 & 0 \\ 1 & 1 & 0 \\ 1 & 0 & 1 \\ 1 & 0 & 1}\right] $$
Whilst any one of these columns can be expressed in terms of the others, it seems undesirable to remove the market factor, and unsymmetric to remove either of the industry factors.
Taking a cue from ridge regression we could consider minimizing a combination of the idiosyncratic return variance, and the factor return variance -
$$ L(\epsilon,f) = \|\epsilon\|^2 + \lambda \|f\|^2 $$
leading to factor returns
$$ \hat{f} = ( X^T X + \lambda I )^{-1} X^T r $$
where the inverse is defined for all $\lambda > 0$. My questions are -
- Is there any theoretical basis for this approach (beyond the fact that it allows you to solve the problem)
- What is the effect of $\lambda$ on the estimated factor returns? Clearly, increasing $\lambda$ reduces the explained idiosyncratic return, but can we say more than this?
- Is there an obvious "best" choice for $\lambda$ or does it need to be chosen in an ad-hoc manner?
## Answer by Chris Taylor (score 0, accepted)
https://quant.stackexchange.com/a/81641
In the case where you have linearly dependent factor exposures, you would typically put a constraint on the factor returns in the regression. This commonly occurs when your exposures represent group membership, for example industries, country exposure, or currency exposure.
The reason for the constraint is that a factor model containing a market factor plus e.g. industry membership is underspecified. You cannot tell the difference between an additional 1% market return, and an additional 1% return in every industry.
By specifying a constraint that the sum of the factor returns representing industries is zero, you ensure that any "common" factor return ends up in the market factor, and only the industry-specific return ends up in the industry factors.
Adding a ridge-like penalty on squared factor returns can also be done, and indeed may commonly be done, especially when there are a large number of factors, to prevent the factor model from "overfitting" and explaining too much of the stock idiosyncratic return, but it is not the best way to handle co-linear factor exposures.Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.