Selecting a Diversified Stock Subset with Covariance and Correlation

# How do I find the most diversified portfolio, or least correlated subset, of stocks?


# How do I find the most diversified portfolio, or least correlated subset, of stocks?












I have a trading system that chooses top 10 stocks in Nasdaq 100 ranked on relative strength and some other factors. However, I'd like to take positions in only 5 of these 10 stocks based on how minimally correlated these are to others for diversification effect. How do I resolve this? I do have the correlation/covariance matrices computed. Literature seems to indicate applying weights to reduce correlations but I felt there should be a simpler solution. That said, the stocks doesn't need to be equally weighted if it is easier to compute these weights.

A computationally easier solution is preferred even if it is not completely accurate since I need to implement this in Amibroker trading software.

## Answer by Tal Fishman (score 15)

https://quant.stackexchange.com/a/1708

One simple method, based on the principles of mean-variance optimization, is to set the weights proportional to the product of the inverse of the covariance matrix and a vector of standard deviations. This implicitly assumes that the normalized expected return of each stock is equal. If you wish, you can take only the top 5 weights and set the others to zero. The actual problem you face, of selecting just 5 stocks, can be solved rigorously with an optimizer, but since it is not a quadratic program, may be difficult to solve.

Update

A more sophisticated but very interesting additional possibility is to find the "Maximum Diversification Portfolio (MDP)", as defined in Toward Maximum Diversification (free version, hat tip vonjd). The MDP is defined as the portfolio that maximizes the Diversification Ratio (DR), which in turn is defined as the ratio of the portfolio’s weighted average volatility to its overall volatility. A follow-up paper investigates the properties of this portfolio. From the paper:

> This measure [DR] embodies the very nature of diversification whereby the volatility of a long-only portfolio of assets is less than or equal to the weighted sum of the assets' volatilities. As such, the DR of a long-only portfolio is greater than or equal to one, and equals unity for a single asset portfolio. Consider for example an equal-weighted portfolio of two independent assets with the same volatility: its DR is equal to $\sqrt{2}$, and to $\sqrt{N}$ for $N$ independent assets.

$DR(\bf{w})=\frac {\sum_i{\it{w}_i\sigma_i}} {\sigma(\bf{w})}$

## Answer by Fabio (score 9)

https://quant.stackexchange.com/a/2052

The problem of the selecting the best portfolio (according to some risk measure) with a limited number of assets can be formulated as a mixed integer linear or quadratic program and is reviewed in the recent paper "Portfolio selection problems in practice: a comparison between linear and quadratic optimization models". It can be solved for reasonable sizes by several of the best optimizers like CPLEX or XPRESS. However, in the case of 5 stocks out of 10 there are only 252 possible possible different subsets (namely 10 choose 5) and they could be all exaustively explored with repect to the risk measure of preference by any personal computer.

## Answer by MichaelJ (score 4)

https://quant.stackexchange.com/a/12596

If you only need to pick 5 out of 10 and want equal weights then just enumerate all 252 possibilities (as pointed out above) and compute the portfolio volatility

$(\textbf{1}'K^{(i)}\textbf{1})^{1/2} = \left( \sum_{ij}K^{(i)}_{ij} \right)^{1/2}$,

where $K^{(i)}$ is the covariance matrix for the $i$th subset. Then use whatever subset gives the lowest portfolio volatility. Here you are minimizing portfolio volatility so you will be biased towards lower volatility stocks. If you don't care about volatility per se and just want to minimize the contribution to portfolio risk related to correlation (somewhat loosely defined) then you can use the Most Diversified Portfolio (MDP) method. This method aims to minimize the diversification ratio

$\frac{w'\sigma^{(i)}}{\left(w'K^{(i)}w\right)^{1/2}} =\frac{\sum_j\sigma_j}{ \left( \sum_{ij}K^{(i)}_{ij} \right)^{1/2}}$

Again, just plug in values for each subset and use whatever gives the largest value.

Personally, I would argue that a few aspects of what your doing are inefficient.

- Why equal weights? If you have a covariance matrix then you can almost always find less risky portfolios. Each stock has a different volatility so equal weights tends to take too much risk in more volatile stocks.

- Why consider only your top 10? It is possible your best 5-stock portfolio includes stocks outside of your top-10 rankings due to correlations.

- Instead, consider attempting to generate expected returns for your stocks. You can do this by running a simple linear regression using your sorting metric.

As has been pointed out, the full mean-variance optimization is hard to solve when you have a cardinality constraint and a large number of stocks to consider. A common approach is to employ $l_1$ norm based methods. The gist of it is instead of solving the standard mean-variance QP

$\min_w \{ \lambda w'Kw - r'w \}, w \geq 0, \sum_i w_i = 1$,

drop the budget constraint and add an $l_1$ penalty, i.e.

$\min_w \{ \lambda w'Kw - r'w + \gamma ||w||_1 \}, w \geq 0$.

As you slowly increase $\gamma$ the $w$ vector will get sparser. Stop once you only have 5 non-zero values. Afterwards, re-scale the weights to sum to one. This version of the problem is a convex relaxation of the actual cardinality constrained problem. The $l_1$-norm penalty can also be motivated as the solution to a robust portfolio optimization problem where returns are uncertain, but satisfy a box constraint.

## Answer by FJ Hsu (score 0)

https://quant.stackexchange.com/a/85520

Several of the answers focus on assigning portfolio weights (mean-variance optimization, diversification ratio, etc.). Those approaches are appropriate if the goal is to determine weights for a fixed universe.

However, your question is slightly different: you want to choose a subset of 5 stocks out of 10 whose return streams are as independent as possible.

One natural way to formulate this is directly on the correlation matrix. Suppose you compute the correlation matrix $C$ of the candidate stocks. For any subset $S$ of size $k$, consider the submatrix $C_S$. A useful diversification objective is to maximize the determinant of this submatrix:

$\max_{S:|S|=k} \det(C_S)$

(or in practice $\log \det(C_S)$ for numerical stability).

Intuitively, the determinant measures how much “volume” the selected assets span in return space. If two assets are highly redundant, the determinant shrinks. If the assets capture independent directions of variation, the determinant increases. In that sense it captures joint diversification, not just low pairwise correlations.

In your specific case the problem is actually very manageable computationally. As another has already mentioned, there are only

$\binom{10}{5} = 252$

possible subsets, so you can simply evaluate the objective for all subsets and pick the one with the largest value. This can be implemented with only a few lines of code once the correlation matrix is available.

For larger universes brute force quickly becomes infeasible, and heuristic search methods (such as greedy selection or beam search) become useful. I wrote a longer explanation of this approach and the search methods involved in another answer here, along with a small Python package called Diversifind that implements these ideas.