U-Shaped Transformer Forecasting

Article MQL5 articles

Summary

This article explains a U-shaped Transformer approach for time-series modeling and its implementation for trading research. The architecture groups Transformer layers and uses patch merging and splitting to represent the series at different scales. Skip connections carry high-frequency detail from the encoder toward the decoder, while attention processes broader context. The described training procedure first pre-trains the model to reconstruct masked patches across mixed datasets, then freezes the representation and tunes a task-specific output head on a smaller dataset. Weighted sampling and mini-batch normalization are also presented as ways to address data imbalance and training instability.

The article describes a MQL5 implementation with trainable positional encoding and reports testing on historical market data, characterized as encouraging. It also notes that the experiment covered only a one-month interval and does not demonstrate stable performance over longer periods. The implementation is presented as a technology demonstration, not a ready-to-trade system. More broadly, the article acknowledges that simpler multilayer perceptrons can outperform Transformer models on some time-series benchmarks, so this architecture is not assured to be the best choice for a given forecasting task.

Key ideas

The model uses an encoder-decoder structure with skip connections to preserve fine-grained temporal information across scales.
Patch merging and splitting let attention process representations at different temporal resolutions.
Pre-training reconstructs masked patches before a task-specific head is fine-tuned with the main representation frozen.
Mixed datasets, weighted sampling, and mini-batch normalization are used to improve generalization and training stability.
The reported market test is short and does not establish durable trading performance.

U-Shaped Transformers for Multiscale Time-Series Forecasting

Summary

Key ideas

Tags