Using Cook’s Distance to Explain Rolling Regression Changes
Summary
The document asks how to identify which observations explain changes in sector risk contributions between consecutive rolling regressions. The two windows share most of their daily data but drop one date and add another, making the change in the reported coefficients difficult to attribute by comparing the windows alone.
The proposed approach is to measure each observation’s influence, starting with Cook’s distance, which assesses how regression results change when a point is omitted. A direct alternative is to delete observations one at a time and recalculate the statistics, a procedure closely related to jackknife resampling. The response recommends beginning with a sample containing both dates, then testing the effect of removing each separately. This isolates their individual influence more clearly. The advice identifies diagnostic methods but gives no calculations or evidence for the example, and influence diagnostics show statistical impact rather than establishing a qualitative market cause.
Key ideas
- Comparing adjacent rolling windows does not by itself identify which changed observation drove a regression result.
- Cook’s distance measures how much an individual observation influences a fitted regression.
- Deleting observations one at a time and recomputing statistics is a jackknife-like diagnostic.
- Including both dates in a common sample helps isolate the effect of removing either one.
Tags
Full text
# Comparing two regressions that differ by a few data points # Comparing two regressions that differ by a few data points I have built a model that explains how much risk of the stock market (S&P 500 index) is attributable to each sector, where each sector is independent from each other (correlation coefficients among the sectors are all 0). For example, I have the following data: ``` Period Consumer Discretionary Consumer Staples Energy Financials Health Care Industrials Information Tech Materials Telecommunication Utilities 2018-04-26 to 2018-05-25 10.32% 7.13% 7.75% 12.61% 15.21% 7.34% 27.62% 7.79% 1.76% 2.45% 2018-04-27 to 2018-05-29 7.90% 5.22% 4.96% 20.67% 12.27% 10.97% 21.85% 9.72% 5.41% 1.01% ``` where I am using rolling-regression (using daily return, 31 days rolling-period) to calculate the values. It is interpretated that, during 2018-04-26 and 2018-05-25, 10.32% of the total market risk is explained by the Consumer Discretionary sector, independent from other sectors. Likewise, during 2018-04-27 to 2018-05-29, 21.85% of the total market risk is explained by the Information Tech sector, independent from other sectors. The input data for those two regressions are exactly the same, except the first one contains 2018-04-26 and does not contain 2018-05-29 while the second one does not contain 2018-04-26 but does contain 2018-05-29. So any two consecutive rows differ by two data points. My question is, using the regressions output and input data, is it possible to identify which input data are responsible for changes in the values? For example, you can see that the risk contribution of Financials increases from 12.61% to 20.67% between the two rows. Is it because something happened on 2018-05-29, which is present in the second regression but not the first regression? Or is it because of 2018-04-26, which is present in the first regression but not the second regression? If neither, does it mean that those two data points (2018-04-26 and 2018-05-29) are not that important in explaining the change? I am interested in this analysis because, given the output data, I want to know what actually took place in the market and explain it qualitatively. What is the name of this kind of analysis? Thank you very much. ## Answer by Tim Wilding (score 3, accepted) https://quant.stackexchange.com/a/41272 Given your current setup, it would be hard to tell whether 2018-04-26 or 2018-05-29 is the cause of the change. There are a variety of regression diagnostics that can be used to determine which point is significant. I would recommend starting with Cook's Distance. This is a measure of the influence of an individual point on the final regression. It works by calculating the effect of deleting an individual observation from the regression. If you don't have access to a library for calculating Cook's Distance, then you could simply delete each observation from your regression and recalculate your stats. This is very close to a technique called JackKnife Resampling. I would also recommend starting from a regression sample that includes both days, so that you can isolate the effect of removing one or other of the days when using these techniques. BTW, I also want to echo that you might have more luck on Cross Validated or Stats (see, e.g., https://stats.stackexchange.com/questions/8344/influence-functions-and-ols).
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.