Benchmarking Financial Forecasts Against a Random Walk
Summary
The document raises a model-evaluation problem: a recurrent neural network appears to forecast familiar nonfinancial datasets well, yet forecasts of macroeconomic series and stock-index measures do not beat the persistence forecast that carries the latest observation forward. It proposes comparing the neural model with standard methods such as ARIMA and VAR, and asks which financial or economic series might provide a useful forecasting benchmark because they depart from random-walk-like behavior.
The evidence is the author’s reported experience with GDP, unemployment, index opening prices, and realized volatility, where apparently strong fit metrics were surpassed by the random-walk baseline. The document does not provide a systematic experiment, dataset definitions, forecast horizons, or a proposed alternative series, so it should be read as a question about benchmark design rather than proof that these variables are unpredictable. Its practical lesson is to include a simple persistence benchmark before interpreting model fit or tuned forecasts.
Key ideas
- Forecasting models should be compared with a persistence or random-walk baseline.
- High R-squared or low RMSE alone does not show that a model improves on a simple benchmark.
- The author reports that ARIMA and recurrent neural networks failed to beat the baseline on the cited macro and market series.
- The document asks for financial or economic series with more exploitable structure but does not identify one.
Tags
Full text
# Good (non-random walk) financial time series to perform forecasting on # Good (non-random walk) financial time series to perform forecasting on I would like to start with a brief caveat, namely that I am by no means a domain expert in financial markets. Therefore the question I am asking may sound silly to a practitioner but I am asking it since I have not been able to find any satisfying answer online so far. I have developed a recurrent neural network (`RNN`) model for time-series forecasting. I now want to test its performance against more standard statistical/econometric models such as `ARIMA` or `VAR`. The model outperforms ARIMA in a few of the typical datasets used for model testing (shampoo sales, minimum temperatures...). However, what I am really interested in is finding out how the model performs on macroeconomic and financial time series (both univariate and multivariate). Here is where the problems start. When I apply the model to macroeconomic indicators (GDP, unemployment...) or stock price indices (opening prices, realized volatility), forecasts appear to be very good at first (high R-squared, low RMSE). However, once I compare them with a baseline random walk model (i.e. `y_t=y_t-1`), I discover that this always gives the best forecast possible. In other words, both ARIMA and RNN models approximate the random walk forecast but always remain below it (this becomes especially clear after parameter tuning where the best estimates are always selected with a lag never greater than one). This has led me to believe that the series I am considering all more or less exhibit a random walk behaviour. Therefore, I would like to know if anyone could point me to any financial indicator which has been shown to not exhibit this kind of behaviour (perhaps one with strong seasonal components which can be learned by both `ARIMA` and `RNN` models). Ideally this would have daily or monthly observations in order to guarantee as much data as possible, but even quarterly or annually series will do if the time span is long enough.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.