Choosing Historical Data for Time Series Forecasting
Summary
This note considers how much historical data to use when forecasting financial time series, such as predicting the next daily market close. It cautions against choosing a lookback window by searching for the interval whose forecast most closely matches a recent observed price, or by combining forecasting with reversed-data backcasting without a clear rationale.
Instead, data requirements should follow the research objective and the dynamics the model is intended to capture. The answer depends on whether the task is high-frequency trading or a slower strategy, whether the target behavior involves market cycles, and what data are being analyzed, such as ticks, trades, bid and ask quotes, or order-book activity. The note offers a framework for selecting data frequency and history, not a tested optimal-window rule. It gives no comparative forecast results, and the appropriate sample remains dependent on the model, market, and intended use.
Key ideas
- Choose historical data to match the research objective and intended trading horizon.
- Separate the sample used to specify a model from the data used to backtest it.
- Include enough history to represent the market cycles the model aims to capture.
- Select data frequency and type to fit the dynamics being studied.
Tags
Full text
# Selecting timeframe for time series analysis # Selecting timeframe for time series analysis In technical analysis, we may use confluence of direction for 3 timeframes to roughly gauge bias of market now. Similarly, if we use time series forecasting methods to predict(say daily data-whether S&P is going higher tomorrow), how much historical daily data would be optimal(bet 2 weeks-1month-3 months)? Too much or too little past data does not give accurate prediction. (1) generate results in 5 days intervals(within 3 months) until you get the best interval prediction that is closest to yesterday's closing value...Then use this interval for predicting tomorrow's close? (2) combine 3 months forecasting and backcasting(reverse data) until there is a result that coincides...then use this day as starting reference point for forecasting? http://www.spiderfinancial.com/support/documentation/numxl/tips-and-tricks/backward-forecast http://pakaccountants.com/what-is-backcasting-and-difference-forecasting/ Other suggestions? ## Answer by Matt Wolf (score 2) https://quant.stackexchange.com/a/6995 You should never start out asking how much data you should incorporate in your research effort. You should start with the following points: - Make sure you understand the difference between sampling size to specify your model and data used to run back tests over. Those are entirely different animals. - First, think about your end goal, what are you trying to achieve. Do you look to develop a high frequency trading model? If yes it makes no sense to incorporate price intelligence from 1 month ago. - What dynamics are you trying to capture? If you look to trade market cycles then you want to incorporate the amount of data that covers different market cycles. - What type of data are you trying to analyze? Tick, compressed, bid/ask vs. trades, order book dynamics? When you answer those question you should most likely have a very good idea how much data you need, at what frequency.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.