Relational Reinforcement Learning with Self-Attention for Trading Patterns

Summary

The article explains how relational reinforcement learning can represent objects and events as nodes in a graph, with weighted links expressing their dependencies. It motivates this approach with a channel breakout example: formation, boundary breaks, and possible pullbacks are distinct events whose sequence matters. The article argues that convolutional models may detect individual patterns but can struggle to express relationships among them, especially when patterns vary in duration or shape.

It describes self-attention as a way to estimate pairwise dependencies among sequence elements. Query and Key embeddings produce normalized attention weights, which are applied to Value embeddings; multiple attention heads can capture different relationships. The model combines this relational block with an Intrinsic Curiosity Module for reinforcement learning. The text reports a backtest profit factor of 1.31 and recovery factor of 2.85, while cautioning that the EA is for evaluation and requires substantial refinement and testing before live trading. Results from a single reported test do not establish robustness across markets or conditions.

Key ideas

• Relational models encode objects and events as graph nodes and their dependencies as weighted links.
• Self-attention uses Query and Key embeddings to estimate directional relationships between sequence elements.
• Value embeddings carry information weighted by the relationships learned through attention.
• The described reinforcement learning system adds a relational block and trains it with an Intrinsic Curiosity Module.
• The reported backtest metrics are limited evidence and do not establish readiness for live trading.

Tags

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.