Accounting for Live Order Impact in Strategy Backtests

Summary

The document raises a backtesting problem for strategies whose orders make up a meaningful share of a thin market’s order book. When a trader tests a small strategy change, historical market data may reflect price and liquidity conditions shaped by the live strategy’s existing orders. A backtest that ignores those orders may therefore compare the change against a market path that would not have occurred under the live trading setup.

The text frames the issue but does not propose a method, present an experiment, or report evidence for a particular adjustment. It leaves open how to model the existing orders’ effect on the market and how to distinguish that effect from ordinary market movement. Any useful approach would need to account for order size and market impact, while recognizing that reconstructing the counterfactual market without those orders is difficult, especially in thin markets.

Key ideas

• Large live orders can change the market conditions used by a strategy backtest.
• A test of a strategy change may be biased if it ignores the live strategy’s concurrent orders.
• The document poses the counterfactual market impact problem but does not give a solution.

Tags

# Adjusting for your own orders in future backtests


# Adjusting for your own orders in future backtests












I was asked this question in an interview and despite thinking about it for a while, I haven't been able to come up with a good answer.

Suppose you have a strategy you are running where at certain points your orders are a significant percentage of the order book (e.g., perhaps because you are trading a very thin market). Now you have made a small change to the strategy and want to backtest your change. How can you compensate in your backtest for the fact that your (live) strategy has orders in the market around the same time that your backtest has orders in the market? Presumably your live orders impact the market and the market would be different if your live orders were not there.

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.