Skip to content
All library documents

Mixture-of-Experts for Crypto Order Execution: Stability and Tail Risk

Article arXiv papers · Author: Alexander Ardaiz et al.
Read the original

Summary

This study compares Double Deep Q-Learning with K-means-partitioned mixtures of experts and parameter-matched dense networks for BTC/USDT order execution. The evaluation uses five-minute mean-aggregated Binance limit order book data and focuses on implementation shortfall, training-seed variation, policy collapse, and within-policy tail risk. Results are compared with time-weighted average price and immediate liquidation benchmarks in a replay environment where early liquidation is nearly costless because of the stated reward and penalty design.

No learned configuration significantly improves mean shortfall over the baseline learner, and each has higher mean shortfall than both simple benchmarks in this specification. Repeated-seed and ablation analyses associate collapse prevention with annealed exploration rather than an intrinsic mixture-of-experts benefit; combining changes to exploration and reward can bring collapse back. The largest expert configuration shows the lowest across-seed dispersion, but this is not robust to multiple-comparison adjustment, while within-policy tail risk worsens as expert count grows. These findings are specific to the tested environment and specifications, and the changing attribution across seed counts highlights evaluation uncertainty.

Key ideas

  • The tested mixture-of-experts configurations do not significantly lower mean shortfall compared with vanilla Double Deep Q-Learning.
  • In the specified replay environment, time-weighted average price and immediate liquidation outperform the learned configurations on mean shortfall.
  • Annealed exploration suppresses observed policy collapses without requiring expert partitioning.
  • Across-seed dispersion and within-policy tail risk move differently as expert count increases.
  • Attribution of failure modes can change with the number of training seeds, so repeated-seed evaluation matters.

Tags

CryptoExecutionMachine learningMarket microstructureRisk management
Full text
# Mixture-of-Experts for Cryptocurrency Order Execution: Training Stability, Tail Risk, and Failure Modes


# Mixture-of-Experts for Cryptocurrency Order Execution: Training Stability, Tail Risk, and Failure Modes









Deep reinforcement-learning policies for order execution can vary substantially across training seeds, so apparent architectural gains may reflect favourable training realisations rather than reproducible properties of the architecture. We evaluate vanilla Double Deep Q-Learning (DDQL), K-means-partitioned mixtures of DDQL experts at $K \in \{2, 4, 8\}$, and dense networks parameter-matched to the $K{=}4$ and $K{=}8$ expert budgets on 5-minute mean-aggregated BTC/USDT limit order book data from Binance. No learned configuration significantly improves mean implementation shortfall over DDQL. Under the reported specification, all have higher mean shortfall than TWAP (0.39 bps) and immediate liquidation (0.21 bps) in an environment whose frictionless replay and terminal-urgency penalty make early liquidation nearly costless; 11/100 vanilla-DDQL runs, versus none in either MoE $K{\geq}4$ arm, converge to a policy that waits until forced liquidation. We then decompose this specification on a device-matched baseline. Annealed exploration alone eliminates observed collapses (12/100 to 0/100; exact McNemar $p{=}4.9{\times}10^{-4}$), matching the elimination under expert partitioning. Combining annealed exploration with the aligned reward restores collapse in 19/30 runs; with all three specification changes, it rises to 48/100. In this environment, expert partitioning is unnecessary to suppress collapse and appears to mask a training-specification failure rather than confer an intrinsic performance benefit. No MoE $K{=}8$ run collapses under any of the six specifications tested. Across-seed dispersion is lowest at $K{=}8$ but non-monotone and not robust to family-wise adjustment, while within-policy tail risk worsens monotonically with $K$. The apparent attribution of the failure mode reverses between 30 and 100 seeds, illustrating the importance of repeated-seed evaluation.

Shown in full with attribution under the source's licence. Licence: abstract CC0

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.