Your macro backtest traded the revised jobs report

Dear builder of the payroll-filter strategy,

You’ve built a simple rule: after the US jobs report, your strategy holds an equity ETF for five sessions if monthly payroll growth exceeds 175,000. Otherwise, it stays in cash. You’ve scheduled entry after the equity market opens, included trading costs, and kept the threshold fixed. Then you downloaded a historical payroll series and used it to reconstruct every signal.

Your remaining problem is the download. A historical economic series can contain revised estimates that weren’t available on the dates your strategy supposedly traded. To backtest a macro signal honestly, you need the version available at each decision time. Moving the entry forward one bar won’t repair a number published two months later.

Your January observation has several birthdays

Take this invented release history. These dates and payroll changes illustrate the mechanics; they aren’t reported economic results.

If your download carries January as +210,000, your replay enters the ETF in February. Your actual rule would have stayed in cash. Every price, order timestamp and commission can be correct while that entire trade is fictitious.

You can’t assume this contamination makes performance better. Revisions can create winning trades, create losing trades, or remove either. The defect is that your simulation answers a question your strategy couldn’t have asked at the time.

And January is only the period being measured. It isn’t the date you learned the measurement. A row labeled January 1 doesn’t give you permission to trade it on January 1.

Give each value an availability history

Your research table needs more than a month and a number. Keep the reference period, value, release timestamp, vintage identifier and source. For an ongoing collection, also record when your system received the release. Preserve old versions instead of updating their values in place.

At a decision timestamp, select the newest eligible version of each observation whose availability timestamp is no later than that decision. Then calculate your features from that reconstructed snapshot.

Your five-session holding period doesn’t make this bookkeeping optional. It gives you more room to choose a conservative entry time; it doesn’t give you early access to revisions.

For older research, you may have evidence of public release time without any record of your own receipt time. Keep that distinction explicit. You can model access after a documented publication timestamp with a stated delay. You can’t describe that assumption as a measured historical delivery.

You’ll also want a timezone-aware timestamp. Store the release’s documented local time and convert it correctly; a fixed UTC offset for New York will fail across daylight saving changes. Give your future self that small kindness. September-you shouldn’t have to decipher March-you’s column named date_actual_final2.

Your rolling features need the whole vintage

Suppose you replace the fixed threshold with “payroll growth exceeds its trailing twelve-month average.” You now need the prior observations as they stood at that decision time, including revisions already published by then.

Using the first release of every month forever defines a different feature. That can be a legitimate design if you explicitly want a history of initial announcements. It doesn’t reconstruct the economic history visible on a particular morning, because that morning’s information set may already include revisions to earlier months.

If you derive monthly payroll changes from employment levels, reconstruct the level series for the relevant vintage before differencing. Mixing a newly released level with an older vintage’s previous-month level can manufacture a change that no published snapshot contained.

You therefore need to specify what your feature means: initial announcements, the latest available economic picture, or revisions themselves. “Payroll growth” leaves too much undecided.

Repair one release before rerunning ten years

You can start with ALFRED, which provides vintage histories for many economic series. Check coverage for your exact series and period. A vintage date alone doesn’t establish intraday availability; pair it with documented release timing before using it for a same-day signal.

For your first audit, choose one release and reconstruct it by hand:

1. Find the archived release and record its publication timestamp, reference month and initial value.
2. Rebuild the input snapshot your strategy would have received before entry.
3. Calculate the signal manually and compare it with your replay.
4. Add a later revision to the data store and verify that the earlier decision stays unchanged.

That last check is especially useful in an automated research pipeline. Give your research agent the snapshot cutoff and selected vintage identifiers alongside the feature values. You need enough evidence to trace a trade back to a particular release, even after the underlying database has grown.

Once that single decision reproduces, rerun the history and compare signal disagreements before comparing returns. Count entries created, removed or moved by the repair. You’ll learn more from those changed decisions than from a lone before-and-after Sharpe.

Your February trade has to survive on February’s information. Leave April’s revision in April.