Why Daily Price Forecasts Revert to the Last Price and When to Model Returns
Summary
The note describes an attempt to forecast the S&P 500’s next daily closing price with a Temporal Convolutional Network. The model uses daily closes, minimizes squared error, and is regularized with methods such as early stopping and dropout. Adding volatility and interest-rate inputs, or changing the input window, did not prevent converged models from predicting close to the latest observed price.
The replies explain that the last price is a strong forecast target under squared error and recommend modeling returns instead of price levels, since a return target cannot simply reproduce the current price. One reply also refers to research finding that multiple predictors did not consistently outperform the historical average return, and suggests examining a lower frequency. The discussion provides no reported out-of-sample test of the proposed changes. It offers a modeling intuition, not evidence that a neural network or return forecast will produce profitable signals.
Key ideas
- With a price-level target, squared-error training can favor predictions close to the latest observed price.
- Modeling returns changes the prediction target and avoids simply reproducing the current price level.
- Adding features and changing the input window did not resolve the behavior in the author’s experiments.
- The replies cite evidence that predictors have not consistently beaten the historical average return.
- The proposed return target and lower-frequency analysis are suggestions, not validated results in this discussion.
Tags
Full text
# One-day-ahead prediction of S&P500 with Temporal Convolutional Networks
# One-day-ahead prediction of S&P500 with Temporal Convolutional Networks
I'm trying to predict the one-day ahead movement of the S&P 500 with Temporal Convolutional Networks 1 to capture some "memory".
I use daily close data with the loss function $\mathrm{MSE}(f(x_0, \ldots, x_T), y_T)$ where $f(x_0, \ldots, x_T) = \hat y_T := x_{T+1}$ is the output of the neural network. I've tried countless of hyperparameters with this very rudimentary model. But almost all models that do converge close enough converge to the naive estimation of the future closing price by the last known price. I've tried adding more features like VIX and interest rates to no avail.
Otherwise I'm employing early stopping, drop out for regularization, weight normalization for normalization etc and have tried long (almost a year) and short (week) input sequences.
I realize that this is a very rudimentary model and I did not expect anything ground breaking. I want to understand why things behave like they do.
Question:
- Why is it that this network, with the bare minimum of information, convergs to the naive estimation of the last known price?
- How can one make small alterations, perhaps to the loss function or somthing else so it doesn't get "stuck" converging to the naive estimator?
## Answer by phdstudent (score 7, accepted)
https://quant.stackexchange.com/a/74850
As @Bob Jansen says above the last price is actually an excellent predictor, but you should do it in the return space.
Goyal and Welch (2007) try to do this with multiple predictors and find that nothing consistently beats the naive average historical return. It would be very interesting if your model converges to the same answer. Also why do this at a daily frequency? What happens if you do monthly?
## Answer by Bob Jansen (score 4)
https://quant.stackexchange.com/a/74849
- Because the last known price is an excellent predictor.
- I would try modelling returns instead of prices. Now the model can't converge to last price.Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.