Compressing Tick Data for Faster Mathematical Backtests
Summary
This document describes a MetaTrader workflow that records historical real ticks once, then replays them in mathematical mode for strategy testing. Sharing a tick file among optimizer agents is intended to avoid repeated data writes and reduce the time spent preparing optimization runs. The workflow covers saving data, replaying a selected period, and building yearly archives for repeated tests.
The author compares storage formats and replay speed using a 2023 BTCUSDT tick sample, reporting compression statistics and matching strategy calculation results against the MetaTrader tester. The examples show that retaining only bid and ask uses less space and can replay faster than formats that add volumes or ZIP compression. Reconstructed symbol properties are needed for calculations, and virtual trading requires separate setup. The comparison is implementation-specific: replay speed depends on the saved fields and compression choice, while commission calculations may differ from the standard tester. Historical tick counts can also vary slightly at the first tick when restoring archives.
Key ideas
- Recording real ticks once lets multiple optimization agents replay a shared file in mathematical mode.
- The storage format trades off tick detail, file size, compression, and replay speed.
- The document compares replay results with MetaTrader’s real-tick tester using a BTCUSDT sample.
- Strategy code must use symbol properties restored from the tick file for accurate calculations.
- Virtual trading and commission handling have limitations that can affect comparisons.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.