Avoiding Survivorship Bias in Historical Index Backtests
Summary
The document explains how to backtest a strategy using a changing stock index over a historical period. Its central recommendation is to use the constituents that belonged to the index at each point in time, rather than freezing the starting membership or applying today’s membership to the full sample. Historical membership changes, company exits and entries, splits, and other corporate actions all affect the data needed for a realistic test.
It also notes that changing constituents require portfolio rebalancing and can create transition costs, which should be represented alongside trading rules such as stops and limits. Testing only the index return does not answer how a strategy trading its individual members would have performed. The advice aims to reduce sampling bias and match the backtest to the strategy’s actual universe. The document offers general guidance but no worked example, data source, or method for estimating costs.
Key ideas
- Use historical index membership as it existed at each date in the backtest.
- Account for constituent entries, removals, splits, and other corporate actions.
- Model rebalancing and transition costs when index composition changes.
- Index-level returns alone may not represent a strategy that trades individual constituents.
Tags
Full text
# Backtesting with Stock Indices, how does one deal with it? # Backtesting with Stock Indices, how does one deal with it? I was wondering how to backtest using stock indices. For example, the FTSE100 has had changes in its components over time. How does one go about testing a time period from i.e. 2000 - 2018, even if there have been changes in the FTSE100 components in between the time? I.e., would it make sense to use the initial components of the FTSE100 (say at 2000) and continue using those components even if they have been removed from the FTSE later on? ## Answer by Vitomir (score 2, accepted) https://quant.stackexchange.com/a/46542 You need to use the proper index constituents each point in time, recording index entrance, exit, splits, corporate actions etc. otherwise you will end up with sampling bias. If you are backtesting a trading strategy consider that your equity line will be affected by all those movements and you will need to rebalance as index constituents change. This will also likely materialize in transition costs, which should be included in backtesting together with stop loss and limits. This also explains why a backtest just considering the index returns is useless. To summarize, the data for the backtest should be the closest possible to the strategy. ## Answer by Dhruv Mahajan (score 0) https://quant.stackexchange.com/a/46539 Why would you want to use the components that were removed at an earlier point.Indices are supposed to be a representative of a market or a particular security type. If some company due to loss in market share or other reasons gets removed from the index, it means that the company is not a broad representative of the market, you would not want to keep it in present terms.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.