Skip to content
All library documents

Why Point-in-Time Data Matters for Strategy Backtests

Article Quant Q&A · Author: Triplusfin
Read the original

Summary

The document examines why market data providers may offer datasets with survivorship or point-in-time biases. Its answer argues that many customers use data for current analysis rather than historical strategy testing, so they may not need complete histories of securities that later disappeared or accurate records of what was known at each past date. This helps explain why biased data remains available in the market.

For researchers, the practical suggestion is to look for providers that explicitly emphasize backtesting and point-in-time or survivorship-bias-free data. The rationale is that customers testing historical strategies depend on those properties. The response offers a general selection heuristic, not an independent audit of any provider or proof that a provider’s data is unbiased. It also does not explain how to measure residual bias or validate a dataset before using it.

Key ideas

  • Survivorship bias can make historical results look better by excluding securities that later disappeared.
  • Point-in-time data aims to represent the information and securities available at each historical date.
  • Many data customers focus on current analysis and may not require backtest-quality historical records.
  • A provider’s emphasis on backtesting can be a useful clue when evaluating its data offering.
  • Researchers still need to assess the dataset’s actual coverage and historical accuracy.

Tags

BacktestingEquities
Full text
# How can market data providers survive selling point-in-time-biased datasets?


# How can market data providers survive selling point-in-time-biased datasets?












I’ve been interested in algorithmic trading for several years now, but I’m going to give up on my dream because of the market data requirements. I started with a very cheap, no-name data provider, and the data was terrible. Then I switched to a very expensive, no-name provider. The data quality was okay, but the data was riddled with survivorship and point-in-time biases. In the end, I even went so far as to get a .com domain, pretending to be a company, and obtained a license from an institutional provider (at least the homepage claims that some well-known financial institutions are clients). And even this data wasn’t 100% point-in-time. As soon as I confronted them about it, I at least got my money back. That is also why I am not allowed to publicly disclose the provider's name.

Of course, I understand why the providers do this. If they left all the errors in, anyone could easily analyze the data quality and realize that you could never generate any alpha using the real unbiased data. The only question that keeps nagging at me for a long time is why this system works and why these companies haven’t gone bankrupt long ago. Because as soon as you realize what they’re doing, the data is completely worthless, and you might as well just use Brownian motion as market data. At least this way you know the underlying probability distribution.

Is it simply the case that their clients are mostly private individuals with no professional experience in the field—and who simply don’t realize they’re being taken advantage of? Or am I missing something, and are there use cases in algorithmic trading where biased data can actually be useful?

## Answer by Brian from QuantRocket (score 1)

https://quant.stackexchange.com/a/85572

Most traders are not algorithmic traders and therefore do not need or demand survivorship-bias-free data. The prevalence of survivorship bias among data providers reflects this basic fact. If you are not backtesting your strategies, you will not benefit from survivorship-bias-free data.

Because most datasets have survivorship bias, providers that offer point-in-time data tend to emphasize that fact strongly. If a data provider doesn't mention its data being point-in-time, it is reasonable to assume that it is not. Look for providers that emphasize backtesting, not just real-time analysis. If a provider emphasizes backtesting, it is more likely to offer higher-quality data, because otherwise it would not stay in business.

QuantRocket offers a suite of survivorship-bias-free data integrations suitable for backtesting.

Disclosure: I am affiliated with QuantRocket.

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.