Speeding Up Repeated Backtests by Separating Estimation from Decisions
Summary
The document discusses ways to reduce the time needed for repeated strategy backtests. It suggests identifying the data the strategy actually uses and maintaining a buffer of relevant observations as new events arrive. For computationally intensive logic, it proposes isolating that work in a C++ library called from the existing trading platform, rather than rebuilding the entire platform workflow.
Its main optimization is to separate model estimation from decisions that use the estimates. When repeated tests share the same expensive estimation step, compute and save reusable outputs during an initial run, then apply different decision parameters in later runs. This can avoid rereading data and recalculating an unchanged model. The approach depends on whether the saved outputs truly remain valid across the scenarios being tested; the document gives an illustrative workflow, not benchmarks or guidance on preventing look-ahead bias in walk-forward evaluation.
Key ideas
- Limit inputs to the market data required by the strategy.
- Keep a buffer of relevant observations that updates as trading events occur.
- Move computational bottlenecks into a compiled library callable from the existing platform.
- Separate expensive model estimation from decision rules when estimates can be reused.
- Save reusable estimation outputs to reduce repeated computation, while checking that they remain valid for each test scenario.
Tags
Full text
# Faster way to backtest/Walkforward # Faster way to backtest/Walkforward I am currently using Ninja Trader to program and test my strategies and the forward testing in very time intensive. I am thinking of writing my own code in either c++ or c#. The question I have is my logic correct in what I would need to programme? I am thinking I would need code to :- 1) read my comma separated tick data and put it into an array. 2) code that then reads through the data in order and applies my model to it. any input would be appreciated. ## Answer by Malick (score 1) https://quant.stackexchange.com/a/23246 Don't need to re-invent the wheel, I suggest you to isolate the time-consuming part of your algorithm in a c++ dll and to call it directly from Ninja trader or whatever platforms. Regarding the data here three advices: - Identify exactly the data you need (ex: if your strategy is based on bar data, you may not need higher/lower prices...) - Always keep a large data buffer that you update at every events you are susceptible to trade. - The most important trick: if your algorithm is going through the same data several times, try to separate the estimation part from the decision part of your algorithm. In doing so, you only need to estimate your model once (during the first test), and then you can base your next backtests on the first estimation. If it is not clear a simple example: First test: you have a very complex model you estimate every 5 minutes. Run it once and save the outputs in a txt file. Next tests, you do not need to re-estimate your models, just take the outputs of the first test (from the .txt file) and apply your new parameters on it (ex: new confidence interval...). Obviously you'll save a lot of time but you need to design your algorithm in such a way you are obtaining all the relevant parameters during the first loop. This is not so easy to obtain but it worth to do it because you will be able to run a bunch of backtests in a few seconds (just require to read a txt file). Also in doing so you only need to read the data once (during the first test). In summary, your first estimation should produce outputs wich are not linked to a particular scenario but some kinds of "pre-inputs" to a decision model.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.