Aligning Backtests with Live Trading and Fill Assumptions
Summary
The document considers whether one framework can support backtesting, paper trading, and live execution across multiple asset types through a broker such as Interactive Brokers. Its central recommendation is to choose the intended execution platform first, then adapt a backtesting framework to match observed live behavior. A facade or adapter can make backtest and execution interfaces consistent without requiring strategy code to be rewritten for each environment.
The answers stress that backtests depend on assumptions about latency, adverse selection, market impact, and fills, and that these assumptions can be hidden or unrealistic. An example framework describes bar-based fills: market orders use the next bar’s open, while limit and stop-limit orders are evaluated using the next bar’s open, high, low, and close. It does not use volume and assumes sufficiently liquid markets. These conventions are simplified models, not evidence that simulated fills will match live execution; live comparison and calibration remain necessary.
Key ideas
- Backtests rely on assumptions about latency, market impact, adverse selection, and order fills.
- Choose an execution platform and calibrate the backtest to behavior observed in live trading.
- An adapter can provide a shared interface between backtesting and execution systems.
- Bar-based fill rules approximate execution using later bar prices and may not reflect actual fills.
- A framework’s supported assets and broker connections do not by themselves guarantee realistic simulation.
Tags
Full text
# Are there any integrated framework that I can back-test and paper/live trading in one place? # Are there any integrated framework that I can back-test and paper/live trading in one place? I'm trying to start working on a fully automated algorithmic trading system, and I'm a little struggling with the framework to use. The requirements I have in my mind are: - Needs to support back-testing. - Needs to be able to do paper/live trading with Interactive Brokers, without rewriting the code with their API. - Needs to support most security types, like stock, option, future, Forex, etc. For 1 and 2 I know there's quantopian. For 2 and 3 I know there are frameworks like pyalgotrade. But I can't find anything that support all the three. If there is one solution can you give me some recommendations? Or if there's no such thing exists, can you tell me how you handle these problems, with minimum efforts? Thanks a lot! ## Answer by Thomas Johnson (score 3, accepted) https://quant.stackexchange.com/a/25307 Remember that all back testing is full of lies assumptions. Latency (both line latency and latency internal to the exchanges), adverse selection, market impact (yes, even you have market impact), etc, are all based on assumptions. These assumptions are educated guesses at best, but more often terrible models are used (you always get filled at at mid!) and they're totally hidden from you. The best choice, if you have the capability, is to choose your execution platform and then tweak an existing backtesting framework to match the behavior you observe in live trading (or build a backtesting framework from scratch if necessary). If you're worried about API compatibility, use a facade or adapter to make the backtesting and execution APIs look the same.. ## Answer by mementum (score 2) https://quant.stackexchange.com/a/25508 Edit (2016-06-21): Now with live data/trading integration with Interactive Brokers. It has taken a while but it has finally arrived. `backtrader` (https://github.com/mementum/backtrader) can do `1` and `3` and is in the process of getting `2` ironed out. A live data feed from IB will make it into the next release (due in the next few days) and it will then be down to mapping of orders. On the project page you can see a list of other similar (some more, some less) python projects and may prove to be closer to be your cup of tea, or maybe not. Disclaimer: I am the author of `backtrader` Filling: Quick explanation as Thomas Johnson is right in the need to make assumptions. - Volume is not used. I only trade in markets and assets which are liquid enough - Pricewise ... it depends on the order type with the following assumption in place: the bar which the system is evaluating is gone ... no match can be done against it `Market`: The order gets filled with the opening price of the next bar `Close`: meant for intraday systems and gets matched with the last bar of the session at the close price `Limit`: starting with next bar the system looks at the open/high/low/close prices to try to give a "better" price (example: open is better than the limit, open is chosen) or limit `StopLimit`: similar to limit, but looking again at the 4 prices components to see if the stop trigger price and limit prices can be executed in the same bar.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.