You told me your paper-trading process crashed overnight. You restarted it before the next decision, saw no errors, and assumed it had picked up where it left off. Then it submitted a buy for an asset you already owned. You want to know whether the strategy has a bug.
It might. But first, check whether the restarted process knows the same things as the paper account. A restart is a state-recovery problem: positions, cash, open orders, and any strategy memory that affects the next decision all have to agree. Getting the code running again is the easy part.
What did the strategy believe before it crashed?
Write down the last decision the process made before stopping. For each traded instrument, capture the intended position, the account’s actual position, outstanding orders, and the signal state that will affect the next action. If those records don’t exist, you can’t tell whether the restart restored state or merely initialized a fresh strategy.
Say your hourly strategy buys one unit when its trend signal turns positive, then holds until it turns negative. The process submits the buy at 14:00, but crashes before recording that the order was accepted. The paper venue fills it. At restart, the strategy sees no recorded position and a still-positive signal, so it submits another buy. The signal is behaving consistently. Its memory of what the account owns is stale.
That’s why the account’s current position must be reconciled against persisted strategy state before new orders are allowed. A local snapshot can be useful, but the account or venue is the authority for what actually filled.
Which state has to survive a restart?
Start with the state that can change the next order. Keep a durable record with a timestamp and a version for each update; make a copy before changing your recovery logic so you can replay the failure later.
| State | Why it matters | Recovery check |
|---|---|---|
| Positions and cash | They determine exposure and available buying power | Compare persisted values with the paper account |
| Open orders | An order may have filled while the process was down | Query order status and reconcile partial fills |
| Signal memory | Crossings, cooldowns, and holding periods may span decisions | Restore the last committed decision inputs |
| Last processed event | It controls where the strategy resumes consuming data | Replay events after that point without applying them twice |
Signal memory is easy to overlook. A strategy that trades on a crossover may store yesterday’s indicator value to detect a new crossing. If it starts with an empty value, it can mistake an existing condition for a fresh event. A cooldown timer has the same problem: restarting must not silently reset a rule that was meant to persist.
How can you make recovery repeatable?
Give every order a stable client ID derived from the strategy run and decision it came from. If the process retries after a timeout, it can check whether that decision already created an order instead of placing a duplicate. Record state changes only after confirming the corresponding account event, and keep the order and fill IDs that explain the change.
Then test the boundary that caused your incident. Run the strategy through a decision, save state, stop it, and restore it with the paper account’s positions and order history. Compare the next decision and resulting orders with an uninterrupted run. Repeat with a partially filled order and with a crash between submission and acknowledgement. The resumed run should not invent a new position or repeat a completed action.
Keep a recovery log: last processed event, account snapshot time, open-order statuses, restored strategy version, and the first decision after restart. It turns “it came back up” into something you can audit.
What should you trust after the restart?
Trust the process only once the account and strategy state reconcile, pending orders have known statuses, and the next decision matches the decision you would expect from a continuous run. If you can’t explain a mismatch, pause paper order submission and inspect the event trail. A running process is not proof of a recovered strategy.
You’ve already found the useful part of the crash: it exposed an assumption your normal run never had to confront. Make the restart a repeatable scenario, and you’ll know what your strategy remembers before you ask it to trade again.
← All posts
