In-Sample and Out-of-Sample Evaluation in Mean-Variance Optimization
Summary
The document explains how in-sample and out-of-sample estimates apply to mean-variance optimization (MVO). In-sample results evaluate portfolio weights on the same historical data used to estimate or optimize them, while out-of-sample results evaluate performance on data not used in that process. The distinction matters because an investor can only use today's estimates to make future investments; historical fit alone does not show how a portfolio will perform later.
The responses also discuss limits to applying these terms in MVO. Each period's estimated efficient portfolio can differ, and MVO relies on historical volatility and expected returns that may not describe future conditions. The discussion highlights that validation results depend on how data are divided: excluding market openings, closings, or special events can make a strategy appear stronger than it is. One answer suggests random or dynamically chosen splits, but the document does not establish a preferred validation procedure or resolve how best to preserve time structure in financial data.
Key ideas
- In-sample performance measures a strategy on data used to estimate its parameters or portfolio weights.
- Out-of-sample evaluation tests the resulting strategy on a separate period or dataset.
- For asset allocation, future performance is more relevant to investors than fit on historical data.
- MVO portfolios can change across estimation periods, complicating comparisons between out-of-sample results.
- Data-splitting choices and omitted market conditions can materially affect validation results.
Tags
Full text
# What do the terms in-sample and out-of-sample estimates mean in MVO? # What do the terms in-sample and out-of-sample estimates mean in MVO? How do the in-sample estimates and out-of-sample estimates I so often hear authors refer to in emperical analysis of MVO differ? ## Answer by SRKX (score 8) https://quant.stackexchange.com/a/7144 The term in sample and out of sample are commonly used in any kind of optimization or fitting methods (MVO is just a particular case). When you make the optimization, you compute optimal parameters (usually the weights of the optimal portfolio in asset allocation) over a given data sample, for example, the returns of the securities of the portfolio for the past 5 years. The thing is, if you run your strategy over the given data set (over the last five years), you do it in-sample i.e you evaluate your result over the sample you used to fit it. This should technically give you the best possible result. If you evaluate the strategy over the next two months, then you will do it out of sample, i.e you evaluate on a period which is different from the one you optimized the strategy on. In asset allocation, it is important to use out of sample backtesting because you will only be able to invest today the result of an optimization made today. Hence you will earn tomorrow's strategy performance; not yesterday's. ## Answer by Matt Wolf (score 2) https://quant.stackexchange.com/a/7061 I do not think they are directly applicable to MVO because inherently you always model the efficient frontier or asset selection on in-sample data and the result is measured out-of-sample. You can't say, "hey I model it in-sample over 2005 data and then I measure the performance of the portfolio over 2006 data and compare that with results derived from 2010 data." 2010 returns may be derived from the "most efficient" portfolio that may be completely different from the "most efficient" portfolio you built in 2005. Actually one inherent weakness of MVO is that it "treats return as a future expectation and uses volatility as a proxy for risk, the flaw being that volatility is a historical parameter and you cannot assume that today’s prices provide an accurate forecast for the future." (Mean–Variance Optimization: A Primer). Obviously there is connection between in-sample and out-of-sample data as indicated in above quote but as I mentioned I would not consider its usage in the same way as in-sample and out-of-simple data are used, for example in those optimization where the optimization is done once, in-sample, and results are comparable across different out-of-sample time frames. ## Answer by Vortex (score 1) https://quant.stackexchange.com/a/30339 In practical terms, it means your strategy will perform similarly as measured by say Sharpe ratio on the sample of data it has not seen before. Note, there is plenty of ambiguity in the my previous sentence. For example, if you tested on a set of equity data from 10AM to 3PM, removing market openings and closings and struck out special events, you might think you have done a good job optimizing. Then you throw market openings, closing and special events back in and your Sharpe ratio goes negative. What most don't discuss is how to divide your set of data into in sample and out of sample. I tend to think it should be random which is the closest it gets to fair. Then you could use first set for optimization and second for validation. Even better, during optimization choose the sets dynamically.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.