Preparing and Validating Event Data for High-Frequency Backtests
Summary
This documentation explains the event record format used by a high-frequency backtesting system and how to validate timestamps in market data. Each record stores an event type, exchange and local timestamps, price, quantity, and optional order or extra values. An example converts raw cryptocurrency depth updates and trades into normalized events, illustrating how feed fields map into the structured representation.
The main data-quality issue is that exchange event order can differ from local receipt order. The system uses separate flags to preserve chronological ordering by exchange time and by local time, and offers routines to detect and correct ordering problems. It also requires positive feed latency, meaning local receipt follows the exchange event. Clock synchronization errors can violate that assumption; the document suggests improving synchronization or applying a latency offset. These are data handling requirements rather than a trading signal, and the examples do not assess strategy performance or quantify the effects of timestamp corrections.
Key ideas
- Market events are stored with event flags, exchange and local timestamps, price, quantity, and optional fields.
- Exchange event order and local receipt order can differ when feeds arrive with latency.
- Separate flags preserve chronological ordering in each timestamp domain.
- Validation and correction routines help identify and repair event-order problems.
- Clock synchronization errors can produce negative latency and undermine data validity.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.