Why Linear Interpolation Changes Correlation and Regression Statistics

Summary

The document asks why correlations, regression coefficients, and R-squared change when monthly observations are expanded into daily observations using linear interpolation. Its central point is that the computed correlation depends on the observations included in the calculation, so interpolated daily values can change the result. The response refers to the sample correlation formula, which is calculated from the paired observations and their sums and products.

The discussion is brief and does not quantify the effect or explain the specific role of serial dependence created by interpolation. Interpolated observations are derived from the original monthly data rather than independent new measurements, so treating the enlarged sample as if it contained more independent information can mislead inference. The example therefore raises a useful sampling-frequency and data-construction issue, but it does not provide a full statistical treatment or evidence about any particular dataset.

Key ideas

• Correlation is calculated from the observations included in the sample.
• Linearly interpolated daily values can change correlations and regression statistics computed from monthly data.
• Interpolated points are derived from the original observations and do not represent new independent measurements.
• A larger row count after interpolation does not by itself establish that estimates are more reliable.

Tags

# does oversampling affect the correlation?


# does oversampling affect the correlation?












I have a dataset of monthly data. One column is my target variable and all the other are my feature. I have computed correlation between my target and all the other feature and then I made linear regression and got my betas and R2.

Now my question is more theoretical. if I oversample to daily data (I used a linear interpolation) and compute again correlation, betas and R2, they have changed a lot. Can anybody explain me why that happens? is correlation affected by oversampling? I might expect my betas to change because I have much more data after oversampling and so the R2, but not really the correlation if the size of my monthly data was already quite large. Thanks

## Answer by Tosh (score 2)

https://quant.stackexchange.com/a/55688

When you carry out correlation coefficient between target variable (denoted as x) and feature variable (denoted as y), the correlation coefficient is a function of sample size:

$ r = \frac{n \Sigma xy - (\Sigma x \Sigma y)}{\sqrt{(n\Sigma x^2 - \bar{x}^2 )(n\Sigma y^2 - \bar{y}^2 )}}$

So daily data will impact on correlation.

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.