Basic Tick Data Cleaning for More Reliable Backtests
Summary
The article recommends cleaning tick data before using it in strategy research or backtests, since duplicate records, implausible prices, out-of-order timestamps, and missing fields can distort results. Its example workflow sorts records by timestamp, removes duplicates, drops rows lacking price or volume, and filters out nonpositive prices and volumes.
These steps are presented as a basic starting point rather than a complete quality-control system. The author says filtering thresholds should be adapted to the instrument and its trading rules. The article offers practical experience as motivation, but no measured comparison of cleaned and uncleaned backtests, and it does not describe methods for detecting subtler errors or validating the source data.
Key ideas
- Tick records can contain duplicates, implausible prices, disordered timestamps, or missing fields.
- A basic cleaning pass sorts by time, removes duplicates, and drops records without price or volume.
- Filtering nonpositive price and volume values can remove some invalid records.
- Cleaning rules and anomaly thresholds need to reflect the instrument and its trading rules.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.