Measuring Market Structure with Lempel-Ziv Complexity and SAX
Summary
This article explains how to estimate structure in financial returns by converting rolling windows into symbols with Symbolic Aggregate Approximation (SAX), then counting distinct phrases with the Lempel-Ziv 1976 algorithm. The normalized count approaches one for random symbol sequences and falls when the sequence contains repeatable patterns. The accompanying toolkit includes an indicator, distance measure, tests, and market-scanning scripts.
It emphasizes that the phrase-count formula has no tuning parameters, but the full measurement pipeline still depends on choices such as window length, alphabet size, aggregation, and whether to use returns or price levels. Tests include fixed examples, behavior checks, and comparison with an independent Python implementation. Market scans of EURUSD and gold are described as finding efficiency-like readings on H1 and mild microstructure on M5, without a directional edge. Fat-tailed returns can lower complexity even without predictability, and steady drift can disappear during normalization. A low reading alone therefore does not establish a tradable opportunity.
Key ideas
- Lempel-Ziv complexity estimates how many new phrases are needed to represent a symbolic sequence.
- SAX maps normalized return windows into symbols while retaining information about relative move sizes.
- The normalized complexity is intended to approach one for random sequences and decline when patterns repeat.
- Pipeline choices affect the reading even though the Lempel-Ziv count itself has no tuning parameters.
- Fat tails can reduce measured complexity without creating a predictable or tradable signal.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.