Forecasting Sparse Short Time Series with Trend and Mean-Reversion Checks
Summary
The document considers how to project a sparse monthly series over a one-year horizon when only two years of history are available. The measure is described as a person’s skill, but the suggested modeling considerations also apply to short time series more generally. Before choosing a forecasting method, the answer recommends clarifying whether the variable is continuous or discrete, its possible bounds, how it may evolve, and whether related explanatory variables are available.
As a provisional approach, it suggests regressing the observed measure on a constant, its lag, and a time trend, while plotting the data to inspect for trend or mean reversion. If the series appears to decay, a regression of log values on time may help estimate the decay rate. These are initial suggestions rather than a validated forecast: the response provides no data analysis or performance evidence, and emphasizes that suitable dynamics cannot be determined without more information about the variable and its context.
Key ideas
- Clarify the variable’s measurement scale, bounds, and expected behavior before selecting a forecasting model.
- A regression with a constant, lagged value, and time trend is offered as a provisional baseline.
- Plotting the observations can reveal possible trend or mean-reverting behavior.
- A log-linear regression may help describe decay when the measure appears to decline.
- The short and sparse history limits confidence in forecasts, and no results are reported.
Tags
Full text
# Techniques for forecasting short-frame data? # Techniques for forecasting short-frame data? I'm having a problem in which a time series of 24 data points is given to forecast the next 12 data points. This 24 data points might be sparse (many are missing). Do you have any suggestion on what technique can be used? Thanks, ADDED: The series is a transformed data from tabulated dataset which shows a measure of one's `skill' over time. The dataset given is only capture in the previous 2 years on a monthly basis, I need to find either an estimation of his/her skill over the next 12 months, or some average measure is also aceptable. ## Answer by Tal Fishman (score 2) https://quant.stackexchange.com/a/1775 In order to properly answer this question, I would ask that you also tell us: - Is the "skill" variable continuous or discrete? How is it actually measured? - Can you put theoretical bounds on the skill? Is it, say, between 0 and 1, or in some well-defined range? - Do you expect skill to change over the next 12 months? Is the person being measured continuing to put effort into improving his skill over time? Does skill tend to mean-revert? - Do you have any other variables which may reasonably be correlated with skill? Until you have answered these questions, my provisional answer is that you should regress the skill on a constant, a lag, and a time trend. I also agree with @QuantGuy's recommendation that you plot the data to see if there are any obvious patterns such as trending or mean-reversion. You may, for example, see that skill appears to be deteriorating, in which case you may want to estimate the rate of decay using a regression of log skill on time. Proper estimation of the dynamics of the process will help you determine both what the skill will be in the immediate future and what it will be in one year's time.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.