Correcting the Price Shift When Calculating Log Returns
Summary
The document addresses an apparent stationarity and ARIMA fitting problem that arises while calculating log returns from a stock price series. Its central diagnosis is that the price series is shifted in the wrong direction: the displayed calculation divides each price by a later observation, whereas conventional chronological log returns compare the current price with the preceding one. Correcting the shift aligns each return with the change from the previous observation to the current observation.
The response recommends ensuring that prices are numeric before calculating the log ratio. The question reports a strongly negative Augmented Dickey–Fuller statistic, along with errors involving Hessian inversion and noninvertible initial moving-average coefficients for higher-order ARIMA models. The answer does not demonstrate that the corrected returns resolve those estimation errors, nor does it discuss stationarity test assumptions, data ordering, or model selection. It therefore provides a useful data-construction correction, but not a full diagnosis of the ARIMA fitting failures.
Key ideas
- A return series should compare each price with the preceding observation in chronological order.
- Shifting prices toward future observations reverses the intended return alignment.
- Prices should be converted to numeric values before calculating log returns.
- The proposed correction does not by itself explain every ARIMA estimation warning.
Tags
Full text
# Issues making series stationary
# Issues making series stationary
I am trying to run some ARIMA forecasts and I switched recently from R to Python.
I am struggling for some reason to make this series stationary .
I try to take the log returns of stock prices as such :
$$ ret = \ln{\frac{P_{i}}{P_{i-1}}}$$
I am using this line in Python:
```
[x for x in np.log(df.price/df.price.shift(-1)) if str(x) != 'nan']
```
see the first observations in the data
```
date price logret fcast
0 2020-08-03 11823.690000 0.041427 0.0
1 2020-08-02 11343.880000 0.020389 0.0
2 2020-08-01 11114.930000 0.001104 0.0
3 2020-07-31 11102.670000 0.015222 0.0
```
the adFuller test returns :
```
adfuller(df.logret)
(-17.0273775639977, 8.372792842473242e-30, 1, 578, {'1%': -3.441714324024304, '5%': -2.8665533998436215, '10%': -2.5694399997605393}, -1971.579460294467)
```
from my understanding this series should be stationary.
But when I run a for loop to generate the ARIMA grid I get a weird error telling me that series is not stationary.
AR/MA(5,5)
```
ERROR Example:
HessianInversionWarning: Inverting hessian failed, no bse or cov_params available
warn('Inverting hessian failed, no bse or cov_params '
```
AR/MA(10,10)
```
Another ERROR :
ValueError: The computed initial MA coefficients are not invertible
You should induce invertibility, choose a different model order, or you can
pass your own start_params.
```
## Answer by Quantoisseur (score 3)
https://quant.stackexchange.com/a/57101
Your shift is in the wrong direction.
Do this:
```
df.price = pd.to_numeric(df.price)
df['logret'] = np.log(df.price/df.price.shift(1))
```Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.