A useful backtest test has nothing to do with finding a better parameter: stop the process halfway through, restore it, and finish the same replay. With identical inputs and a controlled execution simulator, its decisions, orders and equity should match an uninterrupted run.
If they don't, you've found a state-management problem. The strategy depends on something you didn't save or couldn't reconstruct. That dependency matters whenever research runs are resumed, workers are replaced, or a paper-trading service deploys new code.
I like this test because the expected answer is unusually clear. There is no argument about whether the market changed. Both runs receive the same market.
Here are three wrong ways to restart. The numbers are illustrative; each failure can happen in an otherwise deterministic system.
1. Reload a few bars and call the indicators warm
Suppose a strategy uses a 100-period exponential moving average. Its update is:
alpha = 2 / 101
ema_next = alpha * close + (1 - alpha) * ema_previousThe uninterrupted process carries the accumulated EMA forward. The restarted process fetches 100 bars, seeds the EMA with the first close, and assumes that a 100-period indicator needs 100 observations.
That assumption confuses the indicator's smoothing parameter with a finite memory window. An EMA retains a decaying contribution from its initial state. If two versions begin with an EMA difference of 10 price units, identical subsequent prices shrink that difference as follows:
| Updates since initialization | Remaining difference | Fraction of initial error |
|---|---|---|
| 100 | 1.353 | 13.53% |
| 250 | 0.0674 | 0.674% |
| 500 | 0.000454 | 0.00454% |
The calculation is 10 * (99 / 101)^k. Fetching 100 bars gives you only 99 updates if the first observation supplies the seed.
The wreckage usually appears near a decision boundary. One run sees price above the EMA; the other sees it below. A small numerical difference creates a whole extra trade. Once that happens, cooldowns, available cash and later decisions can diverge too.
Save the recursive indicator state, its initialization status and the last processed event. Alternatively, replay from a known starting state. A longer warm-up can produce an acceptable approximation, but choose its length from an explicit error tolerance and check whether that tolerance can change decisions. “Five times the period” is a convention, not a proof.
And indicators aren't the whole history. A rolling percentile needs its window. An online model may need its optimizer state. A rule that waits three bars after a loss needs to remember the loss and the counter.
2. Save positions and forget the orders in flight
Your target position is 10 units. A buy order for 10 has filled 4, leaving 6 outstanding. You checkpoint the position as 4, restart, and submit another buy for the missing 6.
If the original remainder and the replacement both fill, you own 16.
The backtest version of this bug often stays hidden because restarting its fill engine silently erases working orders. In paper trading, the simulator or external service may retain them. The same recovery code then produces different exposure depending on which component survived.
| At restart | Actual state | Position-only recovery sees |
|---|---|---|
| Target position | 10 | 10 |
| Filled position | 4 | 4 |
| Outstanding buy quantity | 6 | 0 |
| Additional quantity needed | 0 | 6 |
The wreckage is an unexplained burst of orders immediately after recovery. Sometimes it doubles exposure. Sometimes it closes a position whose protective order is still working, leaving that order capable of opening a new position later.
A checkpoint needs order identity and lifecycle state alongside positions. Recovery must reconcile those records with the execution system before generating fresh actions. An order with an unknown outcome needs investigation; treating “no acknowledgement saved” as “never submitted” is how duplicate orders are born.
Stable client order identifiers help you look up what happened. They prevent duplicates only when the receiving system actually enforces the required uniqueness or idempotency rules. Persist processed execution identifiers too, so a replayed fill doesn't increase the position twice.
I have a soft spot for the boring order-status screen. On restart day, its little rows suddenly become the most interesting interface in the building.
3. Restore the position and start a fresh P&L ledger
Consider an unlevered spot example with no fees. Start with $10,000 cash, buy 10 units at $100, and checkpoint when the mark reaches $110.
The correct state is $9,000 cash plus a position worth $1,100: equity of $10,100. If recovery restores the 10 units but resets cash to the original $10,000, it reports $11,100. You've manufactured $1,000 by restarting a process.
Other versions are less spectacular. Recovery keeps equity intact but resets the entry price to $110. Total equity can remain correct while realized-versus-unrealized attribution changes. If a stop or exit condition references entry price, the accounting shortcut now changes trading behavior.
Or the system forgets the previous equity high. Suppose equity peaked at $10,600 before falling to $10,100. Its drawdown is about 4.72%. Reset the high-water mark on recovery and the strategy suddenly believes its drawdown is zero. Any drawdown-based risk control has just received an unauthorized reset.
The wreckage can therefore be a discontinuity in equity, a suspiciously improved drawdown, or a risk rule that stops firing after deployments. Preserve the ledger and the strategy's accounting-dependent state: cash movements, positions, applicable cost basis, accrued charges, and risk-control memory. Reconcile restored equity against the ledger at the same valuation timestamp.
A checkpoint needs a consistent boundary. Saving cash after a fill and position quantity before that fill produces a state that never existed. Commit related state together, or record a durable event sequence from which it can be rebuilt. Store the event cursor with that state so recovery neither skips nor applies the fill twice.
The test I would keep in the research harness runs one uninterrupted reference replay, then restarts a second run at deliberately awkward points: during indicator initialization, after a partial fill, and while a risk limit is active. Use the same event order and preserve any simulator random state. Compare the first decision after recovery, the order and fill records, and the equity path. A matching final balance alone can hide offsetting errors.
For a crash after submission but before acknowledgement, the harness also needs to preserve the execution service's state independently of the strategy process. Otherwise it deletes the very uncertainty you're trying to test.
A strategy's specification includes what it remembers. Make that memory explicit enough that you can kill the process halfway through a replay and show exactly how it gets back to work.
← All posts
