Using Transformer Self-Attention to Classify Sequential Price Changes

Summary

The article explains a simplified, decoder-only transformer applied to sequential security price changes in an MQL5 signal class. It outlines positional encoding with sine waves, then describes causal self-attention: project inputs into query, key, and value vectors, scale query-key dot products, normalize them with SoftMax, and combine values into an output. A feed-forward network processes that output, and the article frames attention weights as a way to assess the relative importance of earlier observations.

The implementation is presented as an exploratory framework with one stack and one thread, adjustable for more stacks. It does not report forecast or trading performance, and it is not a complete account of ChatGPT’s architecture; the article explicitly says the deployed system is not publicly confirmed. Its category-theory analogies are conceptual, and claims about better efficiency or resilience are not demonstrated with results here. The proposed application is a preliminary signal classification exercise rather than a validated trading strategy.

Key ideas

• Causal self-attention compares each input with itself and earlier inputs to model sequence dependencies.
• Scaled query-key dot products are converted to normalized weights that combine value vectors.
• Sine-based positional encodings are added to price-change inputs to retain sequence information.
• The MQL5 example is an exploratory single-stack decoder, with no performance evidence provided.
• The article’s analogies between attention relationships and category theory are interpretive rather than validated trading claims.

Tags

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.