A backtest can look more convincing after you make its data worse. Drop the ugly prints, fill the gaps with tidy prices, and the Sharpe may rise. That improvement can be a warning: the cleaning rule may have erased the exact market conditions your strategy needs to survive.
Data quality matters. A crossed quote or a duplicated trade can create a fill no one could have received. But a filter that makes inconvenient observations disappear is also a model choice. Treat it with the same suspicion you’d give a new signal feature.
When is a bad price actually bad data?
Start by asking what the observation represents. A trade far from the prevailing market could be a corrupt record, a genuine sweep through thin liquidity, or a venue-specific event. A one-minute candle with a zero-volume bar might mean no trades occurred, while a missing candle could mean the feed failed. Those cases deserve different handling.
Consider a simple mean-reversion strategy that buys when price falls 1% below a short moving average and exits at the average. During a thin overnight period, one trade prints 1.2% below the prior price, then the market snaps back. If the print is genuine, a strategy with a resting limit order might have filled. If it is a bad tick that no executable quote supported, counting the fill invents an edge.
Deleting the bar is not automatically safer. The strategy may still have had an order resting in the market, and the next real trade could fill it. A defensible replay needs to model what the strategy could observe and what its order could plausibly execute against.
What does a cleaning rule do to the result?
Run the strategy under explicit data treatments and compare the changes. Keep the raw source and record each exclusion or repair with a reason. Then report how much of the result depends on those decisions.
| Data treatment | Potential benefit | Risk to the backtest |
|---|---|---|
| Remove a print outside a documented exchange error range | Prevents a known invalid record from setting an extreme | The rule may misclassify legitimate trades during a fast market |
| Carry the last price across a missing interval | Keeps indicators from failing on gaps | Creates a price at which no trade may have occurred |
| Use consolidated quotes instead of last trades | Gives a better view of displayed execution prices | Quote size, venue access, and queue position still matter |
| Drop all outlier bars by a statistical threshold | Reduces sensitivity to extreme values | Can remove real stress periods where the strategy loses most |
Suppose an unfiltered run has a Sharpe of 0.9, while a run excluding 0.8% of bars rises to 1.4. That is not a data-quality victory by itself. Check whether the removed bars cluster around volatility spikes, whether they account for a meaningful share of P&L, and whether the exclusion rule was defined before you saw the result.
Keep a data exception ledger: source, timestamp, affected fields, reason for the action, and whether the decision was made before or after reviewing strategy performance.
How do I test whether cleaning changed the edge?
Freeze the strategy and compare several plausible treatments over the same dates. Include the raw replay where possible, a documented correction set, and a conservative execution case for observations that remain ambiguous. For each run, report trade count, turnover, drawdown, and the P&L contribution from affected intervals alongside Sharpe.
Then inspect the trades that change. If the strategy's top winners vanish when a small group of suspect prints is excluded, read those records individually. If the losses vanish instead, apply the same scrutiny. A rule that only gets careful attention when it harms the backtest is already doing research work.
For a reproducible study, version the vendor files or their checksums, the cleaning code, and the exception list. A vendor may revise history; your next run should make it possible to tell whether the strategy changed or the inputs did.
But isn’t cleaning necessary?
Yes. Keeping an obviously corrupt record can manufacture a trade, distort volatility, or break an indicator. And nobody should pretend a raw feed is ground truth. The practical point is narrower: cleaning deserves evidence, a frozen rule, and a sensitivity check because it can alter which market the backtest claims to represent.
When a data fix improves performance, ask what observation it removed and whether you would have made the same decision without seeing the equity curve. If the answer is uncertain, keep both versions in the research record. The difference between them is part of the result.
← All posts
