Handling Crisis Periods in Value-Momentum Backtests

Summary

The document raises a sample-selection question for a value-momentum asset allocation strategy. Its in-sample period includes the global financial crisis, and the strategy optimizes a value cutoff for allocating across asset classes. The concern is that unusually low value readings during the crisis could pull the chosen cutoff toward that episode, potentially making the optimized rule less representative of later conditions. The author questions whether one crisis year could dominate a longer estimation period and whether removing it would undermine the purpose of testing across varied scenarios.

No optimization results or replies are included, so the document does not establish how influential the crisis observations actually are. It frames a general backtesting tradeoff: extreme episodes can distort parameter selection, but excluding them can leave a strategy unprepared for severe market conditions. Resolving the issue would require examining the objective function, sensitivity of the cutoff to crisis observations, and performance across distinct regimes, while avoiding selective exclusion based only on outcomes.

Key ideas

• Extreme crisis observations can influence optimized strategy parameters.
• The effect of a crisis year depends on its contribution to the objective function.
• Removing outliers may improve apparent consistency while weakening stress robustness.
• Parameter sensitivity and performance across market regimes can help assess the concern.
• The document poses the issue but reports no empirical analysis or conclusion.

Tags

# Crisis in in-sample period


# Crisis in in-sample period












I am backtesting a value momentum asset allocation strategy and my in sample period is from 2003 to 2011 and out sample from 2012 to 2019. I am optimising a cutoff for value on in sample to allocate on various asset classes.

We were having a discussion regarding this and someone pointed out that presence of crisis in in sample will skew our optimisation towards it as value was very low(overvalued) during that period and essentially optimisation would be biased towards that(give a higher cutoff). That high cutoff will be less likely to happen again because crisis on that scale hasn't happened after 2008

I have 2 objections to this school of thought:

1) My sample consists of 8 other years than just the crisis year 2008, so would optimisation would that greatly be affected by just one year? 2) Is it right to remove outliers specifically from the in sample period just for sake of consistency? Doesn't that beat the point of incorporating different scenarios in in-sample so our model is ready for it. This way we should remove every outlier then, why just 2008, every month return has fallen by >10% is essentially an outlier.

I would like to know thoughts of this community regarding this

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.