Skip to content
All library documents

MacroHFT: Context-Aware Reinforcement Learning for Crypto Trading

Article arXiv papers · Author: Chuqiao Zong et al.
Read the original

Summary

MacroHFT is a reinforcement-learning approach for minute-level cryptocurrency trading. It addresses two stated challenges in existing methods: agents may overfit and fail to adapt their policies to financial context, while decisions from a single agent can be biased when market conditions change quickly. The method uses market indicators, including trend and volatility, to organize data and train multiple specialized sub-agents, each with an adapter that adjusts behavior to current conditions.

A second training phase adds a hyper-agent that combines the sub-agents’ decisions. A memory mechanism supports this higher-level decision process as it responds to market fluctuations. Experiments across cryptocurrency markets are reported to show state-of-the-art performance on minute-level tasks. The summary does not specify the assets, evaluation period, baselines, costs, or robustness tests, so the result cannot by itself establish live profitability or generalization.

Key ideas

  • MacroHFT uses multiple specialized reinforcement-learning agents for cryptocurrency trading.
  • Market trend and volatility inform how the training data and agent roles are organized.
  • Conditional adapters let sub-agents adjust policies to market conditions.
  • A memory-equipped hyper-agent combines sub-agent decisions.
  • The reported experiments concern minute-level tasks, with limited evaluation details in the description.

Tags

CryptoHigh-frequency tradingMachine learningVolatilityMomentum
Full text
# MacroHFT: Memory Augmented Context-aware Reinforcement Learning On High Frequency Trading


# MacroHFT: Memory Augmented Context-aware Reinforcement Learning On High Frequency Trading









High-frequency trading (HFT) that executes algorithmic trading in short time scales, has recently occupied the majority of cryptocurrency market. Besides traditional quantitative trading methods, reinforcement learning (RL) has become another appealing approach for HFT due to its terrific ability of handling high-dimensional financial data and solving sophisticated sequential decision-making problems, \emph{e.g.,} hierarchical reinforcement learning (HRL) has shown its promising performance on second-level HFT by training a router to select only one sub-agent from the agent pool to execute the current transaction. However, existing RL methods for HFT still have some defects: 1) standard RL-based trading agents suffer from the overfitting issue, preventing them from making effective policy adjustments based on financial context; 2) due to the rapid changes in market conditions, investment decisions made by an individual agent are usually one-sided and highly biased, which might lead to significant loss in extreme markets. To tackle these problems, we propose a novel Memory Augmented Context-aware Reinforcement learning method On HFT, \emph{a.k.a.} MacroHFT, which consists of two training phases: 1) we first train multiple types of sub-agents with the market data decomposed according to various financial indicators, specifically market trend and volatility, where each agent owns a conditional adapter to adjust its trading policy according to market conditions; 2) then we train a hyper-agent to mix the decisions from these sub-agents and output a consistently profitable meta-policy to handle rapid market fluctuations, equipped with a memory mechanism to enhance the capability of decision-making. Extensive experiments on various cryptocurrency markets demonstrate that MacroHFT can achieve state-of-the-art performance on minute-level trading tasks.

Shown in full with attribution under the source's licence. Licence: abstract CC0

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.