Correlation-Based Portfolio Diversification with Weight Regularization
Summary
The document considers whether removing highly correlated indexes from an equal-weight portfolio can improve diversification, and whether negatively correlated assets should also be excluded. The suggested alternative is to choose portfolio weights by minimizing the portfolio’s aggregate correlation-based risk, subject to weights being nonnegative and summing to one. This optimizes across the full correlation matrix rather than filtering assets one pair at a time.
It also proposes adding an L2 penalty on the weights. The answer argues this can spread allocations more evenly among correlated groups and reduce overfitting, potentially helping out-of-sample performance. The document provides no empirical comparison, return estimates, or implementation details, so it does not establish that this method improves Sharpe ratio. Correlation alone also does not capture all dimensions of portfolio risk, and the proposal’s effectiveness depends on the data and constraints used.
Key ideas
- Pairwise correlation screening is one possible diversification heuristic, but it does not directly optimize portfolio risk.
- The proposed objective minimizes portfolio risk using the full asset correlation matrix.
- The weights are constrained to be nonnegative and to sum to one.
- An L2 penalty is suggested to regularize weights and temper concentration within correlated groups.
- The document offers no empirical evidence that the approach improves realized performance.
Tags
Full text
# how to construct a diversified portfolio based on correlation
# how to construct a diversified portfolio based on correlation
I have a porfolio of indexes and I built up a python model based on spearman correlation (I used a spearman and not a pearson because, after running some test on outliers and normality checks, I have some outliers and normality checks failed) to eliminate all the indexes with correlation coefficient higher than 0.8 expecting to reduce the risk and improve sharpe ratio. I have no target variable obviously, so the reason for eliminating feature is not to run a machine learning algorithm for prediction but just to improve the performance of the portfolio. It is a portfolio construction problem as I assume for now, equal weights, no optimisation at this point.
My first question is: is this a good approach to diversify? or there is a better way? I thought of PCA but I do not want to lose interpretability Second question, are indexes with negative correlations to be removed as well? looking at the definition of a portfolio volatility, the risk should reduce with neg correlation however, returns have opposite effect on the trend and they should cancel each other out.
## Answer by develarist (score 2)
https://quant.stackexchange.com/a/57397
The maximum decorrelation portfolio can ensure your portfolio is not so correlated in one general asset class:
min $\mathbf{w^{T} C w} $
subject to constraints that weights sum to 1 and are non-negative, where $\mathbf{C}$ is the correlation matrix of multivariate asset returns. If you also regularize the portfolio weights with an L2-norm by adding $\| \mathbf{w} \|^2$ to the objective function, this should also alleviate multicollinearity by assigning equal weights within each correlated group of assets and improve out-of-sample performance by reducing overfittingShown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.