When Event-Driven Backtesting Helps Strategy Research
Summary
The document compares vectorized and event-driven backtesting. Vectorized methods apply signals and returns across arrays, which can make them convenient for quickly testing research ideas. The response notes that vectorized computation is still implemented through iteration internally, so it is not inherently less correct than processing observations one event at a time.
Event-driven systems process data as it becomes available, which can make event ordering easier to reason about and reduce opportunities for look-ahead errors. They also more closely resemble live trading systems that receive streaming data, and can naturally incorporate transaction costs, liquidity limits, and market impact. One answer describes a research framework with execution, accounting, and performance components that can be adapted for live use. These are practical advantages, not guarantees: an event-driven design can still contain errors, and simplified vectorized research can be appropriate when its timing assumptions and costs are handled carefully.
Key ideas
- Vectorized backtests are useful for quickly testing signals against return series.
- The choice between vectorized and event-driven computation is not inherently a correctness distinction.
- Processing data event by event makes data timing and signal ordering easier to inspect.
- Event-driven simulations can model execution costs, liquidity constraints, and market impact.
- A streaming architecture can ease the transition from historical simulation to live operation.
Tags
Full text
# Why do we need event-driven backtesters? # Why do we need event-driven backtesters? I am reading this article at quantstart regarding event-driven backtesters. It seems to me that the main advantage of using an event-driven backtesters is that it avoids look-ahead bias. Usually I download stock price data at yahoo finance, which contains datetime index on pandas. But in Python's statsmodels, particularly time series forecasting models such as ARIMA and GARCH, they fit on data nicely. > Question: Why do we need event-driven backtesters? ## Answer by madilyn (score 2) https://quant.stackexchange.com/a/46792 You don't need an event-driven backtester. To establish some convention, a function or method that takes a vector as an argument (e.g. a MATLAB function, `statsmodels` API methods) is sometimes interchangeably and confusingly referred to as a vectorized function. This doesn't necessarily mean that it uses SIMD vectorization, although quite often it is perfectly suited to use SIMD vector extensions given the input data layout - and a modern compiler will exploit that. Behind the scenes, the said vectorized function is simply passing the elements through a loop, just like an event-driven backtester in that sense. So there's no correctness reason for preferring one approach over another. However, the main upside of wrapping this loop around an event-driven backtester rather than a vectorized function is that the former is identical to how you'll implement it for production data (which comes streaming in 1 event at a time) whereas the latter needs to be modified to handle streaming, realtime data. There's also a small upside in that vectorized functions tend to make it hard to reason around the sequence of events and avoid lookahead bias - chances are that all you need is an off-by-one indexing error to cause severe harm to the accuracy of your backtest, whereas it's quite impossible to make this mistake if you are processing the elements in an event-driven manner. ## Answer by Jacques Joubert (score 2) https://quant.stackexchange.com/a/46794 The two types of backtesters have slightly different purposes. The vectorised backtest is a rather crude way to quickly test a strategy. You do it by multiplying the signal vector with the returns vector and the result is the equity curve. The event-driven backtester is a more well thought out simulation. By making use of an event driven backtester we can stop look ahead bias to a large extent by only feeding in the data as it becomes available. This also very closely matches how your trading will take place in real life via an execution system. We also have the advantage of building in transaction costs, liquidity constraints, and market impact. This is not something you can do with the vectorized method. (You could add transaction costs after the fact). The event driven system you refer to from quantstart has the added advantage that you can swap out your backtester for a live model rather easily by just changing a parameter. It already creates a blotter, accounting system, pre and post performance metrics. Event driven is the way to go if you want to build out an institutional grade infrastructure. Vectorised backtesters are for quick research ideas but if you have an event driven one, then you can forget about vectorised... Oh and I know of funds that have implemented the event-driven architecture and I have used it personally. It's the way to go.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.