Avoiding Survivorship Bias in Historical Index Constituent Research

# Index reconstruction


# Index reconstruction












This is quite a rookie question. I've searched for solutions but have no luck finding the exact same question...

I am going to do some research on historical prices/returns of the underlying stocks of an index, e.g. S&P 500/Russell 3000. However, the indices are always reconstructed at a point of time, for example ticker A was in the index but then kicked out. How should I structure/modify the dataset? Does it make sense to include all companies that have appeared in the index and just assign returns to be 0s when they are not? What is the industry convention on this?

Edits: Please note I'm not trying to replicate the index return, but to test strategies using the constituents' returns.

Thanks!

## Answer by Richard at NorgateData (score 2)

https://quant.stackexchange.com/a/38931

Perhaps you just need to treat the index like a simulated portfolio.

Simplistically, if a constituent enters the index, you add the it to your portfolio. If it exits the index, you sell it.

Prior (or post) a stock being in the index is irrelevant. If you did incorporate such periods, this would introduce anomalies such as survivorship bias and look-ahead bias.

If you're not holding it, there is no exposure and therefore no return.

Indexes are somewhat more complex than this though - they are weighted on various other factors including market cap, liquidity, free-float, cap limits etc. The index "re-weights" itself every day on the close.

## Answer by David Addison (score 1)

https://quant.stackexchange.com/a/38939

The replicating methodology depends on the index you are trying to replicate. In most instances, investors try to replicate some variation of a float adjusted capitalization weighted (i.e., float weighted) total return index (i.e., most S&P indices).

In this case, you can very closely replicate the total return of the index if you know for all times a) what the constituents are; b) what the float adjusted capitalizations are; c) the ADJUSTED price changes of the constituents. If you are using adjusted prices, you do not have to keep a tally of dividends and splits since this is reflected. Also, you do not need to keep track of former or future constituents since the only thing which matters to the return is the float weighted price change.

To illustrate, S&P indices typically define the change in the index by a Laspeyres index:

$\frac{I + \Delta I}{I} = \frac{\sum_i P_{i,1}*Q_{i,0}}{\sum P_{i,0}*Q_{i,0}} \,; \forall i \in I$

where: $I$ is the index level; $P_i$ is the price of asset $i$; and, $Q_i$ is the float adjusted share count of asset $i$.

Please reference this following S&P document for a more robust definition: http://us.spindices.com/documents/methodologies/methodology-index-math.pdf

Total Returns Indices are further defined as follows:

$\frac{I_{TR,t}}{I_{TR,t-1}} = \frac{I_{t-1} + \Delta I_t + \sum_{i,t} (D_{i,t}*Q_{i,t})}{I_{TR,t-1}}$

where: $I_{TR} $ is the total return index level; and, $D_{i,t}$ is the dividend for asset $i$ on dividend ex-date $t$.

So, to answer your question, the ability to replicate the index depends on having the right data. If, for example, you cannot infer float adjusted weights, you will not be able to accurately replicate the S&P 500 or similar index.