Point-in-Time and Survivorship Bias in P/E Ratio Backtests
Summary
The document explains why a monthly strategy that ranks stocks by price-to-earnings ratios and takes opposite positions in the top and bottom groups can show misleading backtest performance. Historical prices and earnings are not enough to reconstruct the information investors actually had: earnings are released after the periods they describe, and figures may later be amended. A credible test therefore needs point-in-time records of both publication and revision dates to avoid look-ahead bias.
The response also highlights survivorship bias: some current-company databases omit firms that have gone out of business, making historical results reflect only survivors. It cautions that a data source designed for current company research may not support historical testing without additional work. The note identifies key data pitfalls but does not quantify their impact or evaluate the strategy after correcting for them.
Key ideas
- Historical earnings should enter a backtest only after they were publicly released.
- Later earnings revisions can introduce look-ahead bias if a database shows only the latest figures.
- Point-in-time data should record what information was available at each historical date.
- A universe that excludes failed or delisted companies creates survivorship bias.
Tags
Full text
# Why is it wrong to rank stocks by P/E ratio, sell the top quartile, and buy the bottom quartile? # Why is it wrong to rank stocks by P/E ratio, sell the top quartile, and buy the bottom quartile? I am reading Advances in Financial Machine Learning by Marcos López de Prado. In chapter 11 The Dangers of Backtesting, exercise 11.5 asks: > We download P/E ratios from Bloomberg, rank stocks every month, sell the top quartile, and buy the long quartile. Performance is amazing. What’s the sin? Indeed, what is the problem? I don't see anything wrong with the method. ## Answer by nbbo2 (score 6, accepted) https://quant.stackexchange.com/a/66197 Proper backtesting is difficult, because of various biases that easily slip into the results if you are not careful. For example how do you compute historical P/E's. Well, you have historical E's and historical P's so you can divide P by E (as Dimitri Vulis suggests it is better to divide E by P, as most academic studies do). However beware of look ahead bias (using information in the backtest that was not available to investors at the time). The earnings of ACME Corp. for the year ending Q2 2016 are not known on the last day of the second quarter of 2016. They become public with a lag. You need a database that has info about when the earnings were released and also when the earnings were amended/revised (if they were). Most db only have the latest info and the period to which it is related. Instead you need "point in time" information, i.e. what information was available at various past points. Also Bloomberg removes companies that have gone out of business. Another bias (survival bias, i.e. you are testing only the companies that survived). You need a database that includes dead companies. In summmary Bloomberg is great for looking up the most recent info on current companies, it is not necessarily suited for historical backtests (at least without considerable additional work).
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.