Using Relative Entropy to Assess and Transform Indicator Data
Summary
The article presents entropy as a way to assess how much information an indicator's observed values contain before using them as predictors in machine learning. Because the usual entropy calculation applies to discrete outcomes, it recommends dividing a continuous indicator's range into equal-width intervals, counting observations per interval, and calculating relative entropy by comparing measured entropy with the theoretical maximum. Values closer to that maximum are treated as more evenly distributed and potentially more informative, though this is only a screening measure.
The described scripts analyze built-in or custom indicator buffers over a selected history sample and can display raw and transformed value distributions. The discussion considers transformations, including a logarithmic transform and an extreme transform, to address skew and heavy tails. The article offers a practical preprocessing workflow rather than evidence that high-entropy indicators predict returns or improve trading. Results depend on choices such as sample, interval count, and transformation; arrow indicators with missing values are unsuitable for the described scripts and may need other encodings.
Key ideas
- Entropy can summarize how indicator observations are distributed across possible values.
- Continuous indicator values must be discretized into intervals for the method described.
- Relative entropy compares observed entropy with the maximum possible entropy for the chosen intervals.
- Distribution plots and transformations can help identify skew and heavy tails before model training.
- Entropy is a screening aid, not proof that an indicator predicts markets or improves a strategy.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.