An options backtest can change without a line of strategy code moving. A vendor corrects a bad quote, adds a late trade, revises contract metadata, or changes its filter for crossed markets. You rerun the same dates and get a different fill, a different position, maybe a different Sharpe. That’s not necessarily a bug in the strategy. It is a change in the evidence the strategy was tested against.
To make an options backtest reproducible, treat market data as a versioned input. Record which snapshot you used, preserve the raw observations that drove each decision, and compare changed results at the contract and timestamp level before trusting the new aggregate.
Why did my options backtest change when the code stayed the same?
Options records are unusually sensitive to corrections because a contract’s apparent tradability depends on several fields moving together: bid, ask, sizes, underlying price, expiration, strike, and the contract’s multiplier or deliverable. A corrected quote can turn an apparent opportunity into a crossed market. A metadata fix can change the meaning of a position’s multiplier.
Imagine a strategy that buys a call when the ask is below $2.00. On the first data load, the 10:14 quote is $1.95 ask, $1.90 bid, with size 5. A later vendor revision changes the ask to $2.05 after reconciling a delayed exchange message. One observation flips the signal. If that contract was the strategy’s only trade that day, a modest data correction can dominate the day’s return.
Quote filtering matters too. Vendors may drop stale quotes, remove crossed markets, or normalize timestamps differently. Those policies can be sensible, but they are part of the dataset’s identity. “SPY options, January through March” does not specify a reproducible history.
What is an options data vintage?
A data vintage is the exact version of the historical observations available to a particular research run. Keep the vendor and dataset name, retrieval time, date range, schema version, and any cleaning or filtering rules. If the vendor exposes snapshot IDs or revision timestamps, save those as well.
For every simulated decision, retain enough lineage to answer: which contract was considered, which quote was selected, when that quote was published or received, and what rule made it eligible? The raw record and the derived record both matter. A cleaned NBBO without its source timestamps can conceal whether a quote was stale or arrived after the simulated decision.
| Record to preserve | Why it matters | Example |
|---|---|---|
| Source snapshot | Identifies the history used by the run | Vendor revision ID and retrieval timestamp |
| Quote provenance | Explains the selected market | Exchange, event time, receive time, bid/ask and sizes |
| Contract terms | Defines what the option represented | Strike, expiration, multiplier and deliverable |
| Cleaning policy | Shows which observations were excluded | Staleness threshold and crossed-quote handling |
How do I check whether a vendor revision changed my results?
Run the same strategy and configuration against both snapshots. First compare the number of eligible observations and contracts by date. Then compare decisions and fills: which signals appeared or disappeared, which quote changed, and whether the new record changed the modeled execution price.
Don’t stop at total return. Track changes in trade count, entry and exit prices, rejected signals, and the proportion of P&L coming from trades that exist in only one vintage. A run can keep a similar Sharpe while individual trades are being replaced; the headline statistic won’t tell you that the history underneath it moved.
For a useful audit, hold the strategy, fees, and execution assumptions fixed. If several things change at once, you won’t know whether the divergence came from the vendor correction or your own experiment.
Should I always use the newest options history?
Use the newest history when the goal is to work from the vendor’s best current reconstruction. Keep the old snapshot when you need to reproduce a prior result or understand what your research system knew at the time. Those answer different questions.
For point-in-time research, a corrected record can be more accurate now and still unavailable to the strategy then. Preserve publication and receive times where possible; otherwise, label the limitation plainly. A modern reconstruction is not automatically a historical information set.
When an options backtest changes, the first question should be “Which observations changed?” The question “Which run is right?” comes after you can answer that.
What should I save with every backtest?
Save a compact manifest beside the results: data provider and snapshot, retrieval time, schema and cleaning versions, contract metadata source, strategy commit, execution assumptions, and run configuration. Keep a link from each trade to the quote records that justified it.
That small record turns a mysterious rerun into a diff you can inspect. Options history will keep getting corrected. Your research process should make each correction visible, so a better archive improves the evidence without quietly rewriting the experiment.
← All posts
