Incremental Mean and Variance Updates for Trading Data Streams
Summary
The document explains recursive methods for tracking a sample mean and variance as observations arrive, then extends the discussion to exponentially weighted statistics. These updates keep the current estimates rather than the full history, which can reduce storage and computation for streaming market data. It also presents Python code for maintaining an exponentially weighted mean and variance.
The article compares simple and exponential averages, derives an approximate correspondence between an EMA’s smoothing factor and an SMA window by matching their centroids, and explains how to adjust the smoothing factor when the update interval changes. It gives an illustrative conversion example, but no empirical trading results. The methods are presented as tools for calculating indicators and other statistics, not as a tested trading strategy. The discussion does not address initialization bias, missing observations, or the precise variance convention, so implementations should check those details for their intended use.
Key ideas
- Recursive sample statistics can be updated using the prior mean and variance plus the newest observation.
- An exponentially weighted mean retains part of its prior estimate and incorporates a fraction of the latest value.
- An exponentially weighted variance can be updated alongside the mean without storing the entire data history.
- Matching the centroids of an SMA and EMA provides an approximate way to choose an EMA smoothing factor.
- Changing the data update interval requires adjusting the EMA weight to preserve a similar decay rate.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.