Exponential Smoothing for Recursive Covariance Updates
Summary
The document asks how to update a sample covariance matrix when a new observation arrives while reducing the influence of older data. It presents exponential smoothing as a recursive approach: combine the current covariance estimate with a newly calculated estimate using a weighting parameter. A parameter closer to one gives more weight to the existing estimate, so observations fade more slowly.
The response highlights that the update rule alone does not settle how to construct the new estimate. One option is to exponentially weight the observations used to calculate covariance; another is to use a rolling window that discards observations before a cutoff. The answer offers these as alternatives and leaves the choice of weighting and estimator design to the practitioner. It does not provide a derivation, guidance for selecting the parameter, or empirical comparisons, so it serves as a concise starting point rather than a complete implementation method.
Key ideas
- A recursive covariance estimate can blend the current estimate with a newly calculated one.
- A higher smoothing weight preserves more of the existing estimate and slows adaptation.
- The new covariance input can come from exponentially weighted observations or a truncated rolling sample.
- The document gives no rule for choosing the smoothing parameter or comparing the alternatives.
Tags
Full text
# Update sample covariance matrix
# Update sample covariance matrix
I would like to update a covariance matrix $\mathbf{R}_T$ with a new incoming sample at time $T+1$, i.e. I would like a rank-1 update of the form $\frac{1}{T+1} [T \mathbf{R}_T + \mathbf{x}_{T+1}\mathbf{x}_{T+1}^{\top}]$. However I want a weighted average by forgetting past observations.
That is, I would like something of the form:
$$\mathbf{R}_{T+1}= \frac{1}{T+1} \big[\sum_{i=1}^{T-1} \alpha^{T-i}\mathbf{x}_{i}\mathbf{x}_{i}^{\top} + \alpha^0\mathbf{x}_{T+1}\mathbf{x}_{T+1}^{\top}\big] $$,
subject to $\sum \alpha=1$. But I would like to express $\mathbf{R}_{T+1}$ as function of $\mathbf{R}_{T}$, because I already have it. So I want to forget the data while retaining the covariance information. Could you please tell me where to search something related to it? I have read someting about the EWMA model, but not sure whether it's what I am searching.
Thanks.
## Answer by mark leeds (score 1)
https://quant.stackexchange.com/a/58754
Hi: Exponential smoothing weights observations by taking a weighted combination of the old estimate and the new. So, if you denote your original matrix ( or current covariance matrix ) as $R_t$ and your new one as $R^{*}_t$, then exponential smoothing does
$R_{t+1} = \lambda R_{t} + (1- \lambda) R^{*}_t $.
But there are two issues with doing this update.
- The value of $\lambda$. The closer it is to 1.0, the more weight is being put on the old ( current ) estimate.
- How to calculate the $R^{*}_t$ ? You may want to exponentially smooth the values that go into the calculation of the covariance matrix or just use a covariance matrix that cuts off the raw observations before some $t = t^{*}$.
So, exponential smoothing, in this case, is part art and part science.Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.