How Universe Size Affects Portfolio Optimization and Realized Performance
Summary
The discussion compares optimizing over the S&P 500 with optimizing over the broader S&P 1500. Because the smaller index is contained within the larger one, the larger-universe optimization can reproduce any feasible S&P 500 portfolio by assigning zero exposure to the additional stocks. Under the same objective and constraints, its theoretical optimum therefore cannot be worse. That result concerns the optimization objective, not realized returns.
The answers explain why actual performance and implementation costs remain uncertain. Solver limits, heuristic methods, model error, changing market conditions, liquidity differences, diversification, constituent turnover, and the cost of matching an index can all affect outcomes. One response illustrates changing weights with a simple constituent-selection backtest, while explicitly noting that it is not a proper study and omits index rebalancing. The discussion provides no robust evidence that either universe will outperform in practice; conclusions depend on assumptions, investment horizon, and whether the model has predictive validity.
Key ideas
- The S&P 500 is a subset of the S&P 1500, so the larger universe contains all portfolios feasible in the smaller one if zero exposures are allowed.
- A larger feasible set cannot reduce the theoretical optimum under identical objectives and constraints.
- Solver limits and heuristic optimization can make a smaller-universe solution appear better in practice.
- Realized performance depends on model validity, nonstationarity, liquidity, diversification, turnover, and implementation costs.
- The illustrative weight-change backtest is limited and does not establish that either universe will outperform.
Tags
Full text
# Portfolio Optimization using S&P Universes
# Portfolio Optimization using S&P Universes
Assuming a set portfolio optimization problem, if all optimization inputs are kept constant, what would you expect, in terms of results, if you run the same optimization using the S&P500 as opposed to the S&P 1500, as a universe?
Intuitivelty, would you expect the strategy based on the S&P500 or S&P1500 to outperform?
## Answer by arodrisa (score 2, accepted)
https://quant.stackexchange.com/a/15274
Well, you are asking something very subjective. In addition it should be mentioned that S&P500 are the companies with higher capitalization of S&P1500. Therefore a huge weight of S&P1500 is set by S&P500. In fact, as it can be seen in 2008 both went down a 37%, in the other hand S&P500 has 80% of the total of the US equity Market.
After taking into account you should ask yourself a couple questions: What do I expect to perform better big caps, mid caps or small caps? And more important, is the 20% represented by mid and small caps going to out perform big caps?
If you buy a fund which tracks S&P1500 you can understand this investment as something similar to core-satellite investment, in which you invest 80% in the index and 20% in trying to outperform the index. If you consider that the risk of investing on this market is worth your expected growth, go for it.
In my opinion, regarding the actual economic situation, I won't take this option until,at least 2 years,until things change. I don't think that the economy will grow that much in 1-2 years time.
## Answer by David Nehme (score 2)
https://quant.stackexchange.com/a/15281
You are asking two different questions: what would be the model result, and what would be the actual performance of an actual portfolio.
The optimal model results with the S&P 1500 will be at least as good as the model results with the S&P 500. The S&P is a proper subset of the S&P 1500, so you can get the results of the S&P 500 model by solving the S&P 1500 model with the added constraints that all non-S&P 500 shares have zero exposure.
More explicitly if you partition the exposure variables into $x_{500}$ and $x_{1500}$ (where $x_{1500}$ represent exposures to S&P 1500 \ S&P 500) and let $y$ be a vector of all other variables in your model, the two optimizations are $$\eqalign z_{1500} = {\rm maximize} f(x) \hspace{0.2in} \\ \mbox{subject to:} {\hspace 1.5 in} \\ g_j(x_{500}, x_{1500}, y) = b_j {\hspace 0.15in} \forall j$$ and, $$\eqalign z_{1500} = {\rm maximize} f(x) \hspace{0.2in} \\ \mbox{subject to:} {\hspace 1.5 in} \\ g_j(x_{500}, x_{1500}, y) = b_j {\hspace 0.15in} \forall j \\ x_{1500} = 0 {\hspace 1.38in} $$ For optimal solutions to both problems $z_{500} \le z_{1500}$.
If you are using a heuristic to solve both models instead of using a MIP solver like gurobi or CPLEX, then it is theoretically possible that your S&P 500 model will produce better results. In fact, that might happen if, for example, you solver is running with a restrictive time limit, you have cardinality and minimum investment constraints and most of the optimal exposures happen to be in the S&P 500. Even with a MIP solver, you are solving to a tolerance. So if you are using the default MIP tolerances $z_{500} \le z_{1500} \cdot (1-{\rm MIPGap})$.
As far as actual performance of the two portfolios, if your model has some validity to the real world (which is questionable), then you would expect the S&P 1500 portfolio to at least match either the return or variability of the S&P 500 portfolio. I say expect because even portfolio optimization theory doesn't say that a given sub-optimal portfolio will under-perform an optimal portfolio over any single time period.
## Answer by Drew (score 1)
https://quant.stackexchange.com/a/15270
On the same time period or different ones? It's difficult to say for a different time period. It's actually difficult to say for the same time period, because dynamics are non-stationary.
Let's think about it like this: say you perform mean-variance on the S&P1500, with a short time period: this implies that your estimate of the covariance matrix is probably terrible. But, the nonstationarity of the problem might mean this is a better estimate than the classical one. Hence the estimate on the S&P1500 performs better than the one for the S&P500, since in fact having a large sample with a process which is not ergodic doesn't really help.
## Answer by RndmSymbl (score 0)
https://quant.stackexchange.com/a/15282
Intuitively, I would be thinking about how SP500 and SP1500 are different. A few points that come to my mind:
- Different liquidity of constituents
- Number and frequency of rebalancing/constituents changes
- Different levels of diversification
- Different cost of matching the index
Since, again intuitively (EDIT: and a little less so since I have added few links), it seems that the liquidity premium, due to inclusion of lower-cap firms (Bessembinder 2002), on the SP1500 is unlikely to be entirely compensated by higher average returns and diversification (compare S&P data, Statman 1987 and Goetzmann & Kumar 2008). Therefore the operational cost will most likely account for the difference. Since the index is cap-weighted one would, on average, trade more often for a SP1500 setup and that would stand a good chance to pull down the after fee returns.
EDIT: To further illustrate that the larger index likely incurs higher costs a simple backtest of a 10 vs 100 constituents selection of the SP500 is shown below.
Figure 1: Portfolio weight changes of first 10 alphabetically sorted symbols in SP500
Figure 2: Portfolio weight changes of first 100 alphabetically sorted symbols in SP500
This is not, by any means, a proper study. It's just the first items of the index when symbols are alphabetically sorted and rebalancing for index changes is not even considered. However one can observe, keeping all optimization parameters constant, the number and magnitude of portfolio weight changes differ.
As has been pointed out by other answers different optimisation assumptions, especially with regard to the time horizon, the investment objectives and market expectations affect the outcome.Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.