A backtest returned 20% annualized. It used 8 years of data, 1,640 trades, and a clean walk-forward split. Then we replaced the vendor’s current history with the values recorded at the time, and the edge fell to 3%.
The surprising number wasn’t the original return. It was the 17 percentage points that depended on revised data. That kind of hindsight can hide in economic releases, company fundamentals, index membership, and even supposedly simple market fields. Historical data tells you what the source says now. A point-in-time dataset tells you what your strategy could have known then.
Why can historical data change after the event?
Some values are estimates first and corrections later. An economic statistic released on Friday might be revised the next month. A company can restate an earnings figure. A data vendor can repair a bad print, fill a missing interval, or replace a preliminary value with a final one. A database that stores only the latest value makes all of those changes look as if they had always been true.
That matters when a strategy ranks companies by reported revenue or trades a market after a scheduled data release. If it sees the corrected number months early, the backtest has given it information that was not in the market at decision time. The signal calculation can be flawless; its input is still from the wrong version of history.
There’s an easy trap here: a field called date often describes the period a value covers, not when it became available. A quarterly revenue row marked March 31 might not have been published until May. The label is about the business period, not the strategy’s information set.
Which timestamp should a backtest use?
Track the value’s effective period and its availability time separately. When possible, keep the publication time and ingestion time too. A strategy could only consume a value after it was published and received by the system; if those times differ, use the later one for a conservative replay.
| Timestamp | What it describes | How it helps |
|---|---|---|
| Effective time | The period or event the value refers to | Groups observations by the thing being measured |
| Publication time | When the source released this version | Prevents a release from appearing before it existed |
| Ingestion time | When your system received the version | Captures delivery delays and late backfills |
For example, suppose a company’s quarterly sales cover January through March, are reported on May 8 at 16:15 Eastern, and arrive in your research store at 16:17. A strategy that decides at the May 8 close cannot use that report if its close-time decision happened at 16:00. A strategy deciding the next morning can. The quarter-end date alone can’t answer either question.
How do I test whether revisions are driving the result?
Run the same strategy twice: once with the data version available at each historical decision time, and once with the latest revised history. Keep execution assumptions, universe, and code fixed. The difference is a revision-sensitivity result, not a new strategy comparison.
Then inspect where the gap comes from. Does performance change on release days? Are the biggest differences concentrated in a few restated companies or corrected observations? Does the strategy trade before the relevant publication time? A small event-level audit often explains more than another chart of cumulative returns.
Those numbers are an illustration, not a universal penalty. The useful output is a paired result for your own strategy, with the same trades and costs wherever possible. If the revised version wins by a wide margin, find out which fields and dates account for the difference before trusting the headline backtest.
What should a point-in-time data audit record?
Start with the fields your strategy actually consumes. For each one, identify the source, timestamp meaning, revision policy, and whether prior versions can be retrieved. Ask the vendor how corrections are represented: overwritten rows, new version rows, or a separate audit log. “Historical” on a product page doesn’t settle this.
- Store each version with its publication or first-available timestamp.
- Join features to decisions using availability time, not period labels.
- Keep a frozen snapshot or source version for reproducible runs.
- Flag values that arrive late, are revised, or have no reliable history.
And keep the scope practical. If your strategy uses only exchange trades and bars, economic-data revisions may be irrelevant. But bar corrections and vendor backfills can still matter, especially when a signal depends on a thin market or a precise event window.
When point-in-time history isn’t available
Don’t quietly treat today’s value as yesterday’s. Mark the field as a limitation, exclude it from claims that depend on historical availability, or run a conservative delay and show how much the result changes. A delayed proxy can be useful for research, provided the delay is explicit and the limitation travels with the result.
The backtest’s job is to reconstruct a decision using the information set that existed at the time. If your data can only show the latest version, the honest result has a boundary: it can test the strategy against today’s reconstructed history, but it cannot establish exactly what a historical agent could have seen.
← All posts
