Compressing Tick Data to Reduce Memory Use in Trading Systems

Summary

The article describes replacing the general-purpose MQL tick structure with smaller representations tailored to instrument type. For Forex and CFDs, a compact structure stores bid, ask, millisecond time, and flags; an exchange-oriented version also retains last price and volume. The suggested reductions come from omitting redundant or unused fields, storing passive prices as floats, and narrowing the flags field. The article reports that these structures occupy one-third and one-half the space of the standard tick structure, respectively.

It presents historical tick-array virtual testing and compact file storage as use cases. A testing library can decompress each stored tick as it is processed, preserving the existing internal logic while reducing the in-memory input array. These savings depend on instrument data and the assumption that reduced-precision prices are suitable for storage; the article does not provide benchmarks beyond the stated structure sizes or discuss precision error in detail.

Key ideas

• Compact tick structures omit fields considered redundant or unused for each market type.\nThe Forex and CFD representation stores bid and ask as floats with millisecond time and flags.\nAn exchange representation also preserves last price and volume.\nThe article reports memory sizes of one-third and one-half of the standard structure.\nDecompression during virtual testing can let existing logic process compact historical tick arrays.

Tags

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.