Comparing DQN and PPO for Reinforcement Learning in Forex Trading

Summary

This study applies deep reinforcement learning to foreign exchange trading, framing trade choice as a sequence of decisions rather than relying on direct price forecasts. It adapts a Sure-Fire statistical arbitrage policy into three possible actions and converts continuous price histories into Gramian Angular Field images. The authors compare Deep Q Learning with Proximal Policy Optimization using EUR/USD, GBP/USD, and AUD/USD data at four-hour intervals.

Training uses data from 1 August through 30 November 2018, while the test period is December 2018. The authors report favorable investment performance for models able to represent complex, random market behavior and states that adequately describe the trading environment. The supplied description gives no return figures, risk metrics, fee assumptions, or detailed model settings, so the strength and comparability of the results cannot be assessed here. Its evidence is a feasibility test on three currency pairs over a brief historical split; it does not establish that either algorithm will perform reliably in other periods or under live execution conditions.

Key ideas

• The approach treats forex trading as a sequential decision problem using reinforcement learning.
• Three trading actions are defined within an adapted statistical arbitrage policy.
• Price windows are encoded as Gramian Angular Field images for model input.
• Deep Q Learning and Proximal Policy Optimization are compared on three currency pairs.
• The brief historical evaluation reports favorable performance but omits detailed metrics and trading cost assumptions.

Tags

# Deep Reinforcement Learning for Foreign Exchange Trading


# Deep Reinforcement Learning for Foreign Exchange Trading









Reinforcement learning can interact with the environment and is suitable for applications in decision control systems. Therefore, we used the reinforcement learning method to establish a foreign exchange transaction, avoiding the long-standing problem of unstable trends in deep learning predictions. In the system design, we optimized the Sure-Fire statistical arbitrage policy, set three different actions, encoded the continuous price over a period of time into a heat-map view of the Gramian Angular Field (GAF) and compared the Deep Q Learning (DQN) and Proximal Policy Optimization (PPO) algorithms. To test feasibility, we analyzed three currency pairs, namely EUR/USD, GBP/USD, and AUD/USD. We trained the data in units of four hours from 1 August 2018 to 30 November 2018 and tested model performance using data between 1 December 2018 and 31 December 2018. The test results of the various models indicated that favorable investment performance was achieved as long as the model was able to handle complex and random processes and the state was able to describe the environment, validating the feasibility of reinforcement learning in the development of trading strategies.

Shown in full with attribution under the source's licence. Licence: abstract CC0

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.