TEMPO: Decomposing Time Series for GPT-Based Forecasting

Summary

The article reviews TEMPO, a forecasting architecture that adapts a pretrained GPT-style model to time series. TEMPO first separates a series into trend, seasonal, and residual components, maps them into embeddings, and combines the components with learned soft prompts that encode temporal structure. It also uses normalization, reconstruction loss, and segmented tokens with positional information. The article discusses the motivation for explicit decomposition, arguing that attention alone may not separate interacting trend and seasonal patterns reliably.

The implementation section describes an MQL5 adaptation for financial data. It substitutes piecewise linear representation for the trend estimate and explores using attention across component spectra to identify shared frequencies for seasonality. The author says a pretrained language model was unavailable, so the implementation could not evaluate transfer from pretrained representations. The article presents architecture and implementation choices, but defers model results on historical data to a later installment; it therefore supplies no evidence here of forecasting accuracy or trading returns.

Key ideas

• TEMPO decomposes time series into trend, seasonal, and residual components before passing representations to a GPT-style model.
• Learned prompts are associated with each component to provide temporal structure for forecasting.
• Normalization, reconstruction loss, and segmented tokens support model training and representation learning.
• The MQL5 adaptation uses piecewise linear representation for trend and explores shared spectral frequencies for seasonality.
• The implementation lacks a pretrained language model and reports no forecasting or trading results in this article.

Tags

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.