Integrating ST-Expert Attention Modules into Extralonger
Summary
The article describes a proposed integration of ST-Expert, a graph-based mixture-of-experts approach, with Extralonger, a modular Transformer framework for time-series forecasting. It frames market data as both sequences and networks: assets or time points are nodes, and changing relationships are edges. ST-Expert is intended to adapt those connections and shift emphasis among specialist blocks as market drivers change.
The discussion maps the integration onto Extralonger’s temporal, spatial, and mixed processing routes. For the temporal route, it interprets self-attention logits as a graph of relationships between sequence elements and suggests replacing conventional logit generation with ST-Expert graph-based dependencies. The surviving implementation discussion mentions value blocks, dependency graphs, attention distribution, and Sparse SoftMax for local attention. This is an architectural and implementation installment, not a trading evaluation: it provides no forecast-accuracy or profitability results, and says those assessments will follow in a later article.
Key ideas
- ST-Expert combines temporal modeling with graph representations of relationships among assets or time points.
- A mixture of expert blocks is intended to reweight analysis as market relationships and drivers shift.
- Extralonger separates forecasting into temporal, spatial, and mixed processing routes.
- The article proposes using graph-derived dependencies in place of standard self-attention logits.
- Sparse SoftMax is discussed as a way to focus local attention, but performance results are deferred.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.