Building a Walk-Forward Loop for Out-of-Sample Forecasting

Summary

The document describes a walk-forward approach to out-of-sample forecasting in R. At each step, a fixed-length moving training window drops its oldest observation and includes the latest one. A model is fitted on that window, used to produce a forecast, and the forecast and its date are stored before the process advances. This procedure lets a researcher evaluate forecasts sequentially on observations outside each fitting window.

The reported issue is that the result matrix contains only the first and last loop outputs. The answer identifies misuse of R’s `range` function in the loop sequence: it returns the smallest and largest values of a vector, rather than generating every integer from zero through the endpoint. The answer therefore points to the loop’s iteration sequence as the problem. The note does not provide a complete corrected function or examine other details, such as indexing boundaries, forecast horizons, or how results should be initialized, so those parts still require care in an implementation.

Key ideas

• A walk-forward evaluation refits the model as a fixed-length training window advances through time.
• Each fitted model generates a forecast for an observation outside its training window.
• Store each forecast with its associated date to build an out-of-sample series.
• In R, `range` returns the minimum and maximum values; it does not generate a sequence of loop indices.

Tags

# How to build a loop function for out-of-sample backtesting?


# How to build a loop function for out-of-sample backtesting?












Many statistical libraries in R offer the possibility to fit a model and then use the results of optimization to predict values some periods ahead. However, many do not have the possibility to backtest the results out-of-sample.

Therefore, I want to build an R function that allows me to (walk forward approach):

- Define a training set using a moving window (each looping time, remove oldest observation & add most recent)

- Run optimizer thus calibrating the model

- Use the calibrated model to generate n step ahead forecast

- Store the new forecast in a vector of out-of-sample predicted values (together with the date of forecast)

- Loop through 1-4 I tried the following (x is the length of the out-of-sample set, n the fixed length of the training set):

```
for (j in range (0:x)){
    append <- vector()
    forecast <- vector()
    set <- train [j+1:n+j,]
    fit <- fit(data = set, model) 
    forecast <- predict(fit, ahead = 1) 
    append <- cbind(lubridate::as_date(ts_date[n+j+1]), forecast)
    forc <- rbind(forc, append)
}
```

However, the matrix forc contains only the first and the last result of the loop.

Can anyone spot a mistake here?

## Answer by Bob Jansen (score 2, accepted)

https://quant.stackexchange.com/a/48892

You need to remove the call to range. In Python it’s necessary but here it just returns the smallest and largest element of the vector.

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.