Calculating Rolling Regression Slopes for Market Data
Summary
The document addresses calculating rolling slope features for a time series stored in a pandas data frame. The original attempt fits a line to slices of volume and volume-price-trend observations, then writes results at positions beyond the current frame, producing an indexing error. The proposed approach instead applies a slope function to each rolling window of the VPT series and uses the window position as the independent variable. Separate rolling calculations produce short and longer window features, with incomplete windows left unset through the minimum-observation setting.
Two implementations are shown: one uses NumPy’s polynomial fit, and another uses SciPy’s linear regression function. The latter passes arrays directly into the rolling callback, which the response notes can improve performance. The examples demonstrate mechanics, but do not discuss scaling the series, interpreting slope units, choosing window lengths, or preventing look-ahead bias when using the resulting features in a trading model.
Key ideas
- Apply a slope function within each rolling window instead of assigning results beyond the data frame’s bounds.
- Regress each window’s series values against their positions to obtain its slope.
- Require a complete window to avoid calculating slopes from partial initial observations.
- NumPy polynomial fitting and SciPy linear regression are both presented as implementations.
- The example does not address feature scaling or look-ahead control in a strategy.
Tags
Full text
# Rolling Calculation of Slope in Python
# Rolling Calculation of Slope in Python
I am trying to calculate Slope for the rolling window of 5 and 20 periods and append it to the existing data frame. The length of the total dataset would be let's say 30 days. I have two columns "Volume" and "Vpt", I have tried sklearn (linregress) and numpy (polyfit) but in both the scenario, I am getting an error message "IndexError: iloc cannot enlarge its target object". Please see below code and help to resolve this issue.
```
for j in range(len(temp)):
x = np.array(temp['Volume'][j:j+5])
y = np.array(temp['vpt'][j:j+5])
slope = np.polyfit(x,y,1)
# slope, intercept, r_value, p_value, std_err = linregress(x, y)
temp['slope_5'].iloc[j+5] = slope[0]
a = np.array(temp['Volume'][j:j+20])
b = np.array(temp['vpt'][j:j+20])
slope_1 = np.polyfit(a,b,1)
# slope, intercept, r_value, p_value, std_err = linregress(a, b)
temp['slope_20'].iloc[j+20] = slope_1[0]
```
## Answer by Add (score 2)
https://quant.stackexchange.com/a/69312
Actually, I found an answer to my question above.
```
def np_slope(data):
return np.polyfit(np.arange(data.shape[0]),data.values,1)[0]
temp.index = temp['Volume']
temp['slope_5'] = temp['vpt'].rolling(5,min_periods=5).apply(np_slope,raw=False)
temp['slope_20']= temp['vpt'].rolling(20,min_periods=20).apply(np_slope,raw=False)
```
## Answer by gnzsnz (score 0)
https://quant.stackexchange.com/a/79635
this is a similar solution, but using scipy.stats.linregress
```
from scipy.stats import linregress
def slope(data):
return linregress(np.arange(data.shape[0]),data).slope
temp.index = temp['Volume']
temp['slope_5'] = temp['vpt'].rolling(5,min_periods=5).apply(slope,raw=True)
temp['slope_20']= temp['vpt'].rolling(20,min_periods=20).apply(slope,raw=True)
```
as per pandas apply documentation, i'm passing `raw=True` in order to pass a numpy array, which can improve performance, specially on reduction functionsShown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.