Skip to content
All library documents

Using Reinforcement Learning to Recover Market Equilibria

Article Quant Q&A · Author: Stéphane
Read the original

Summary

The document describes a proposed research approach: define learning environments from market microstructure models with known solutions, then use reinforcement learning to see whether agents recover features of those equilibria. As an initial example, the author discretizes Kyle’s single-auction model and trains an agent with Q-learning and an epsilon-greedy policy to mimic the insider’s behavior.

The author reports that preliminary results appear encouraging, but supplies no quantitative evidence, implementation details, or conclusions about how closely the learned behavior matches the equilibrium. The main request is for finance-specific reinforcement-learning examples, especially problems such as trade execution and portfolio selection, ideally with code. It is a research direction and a request for references, not a tutorial or validated strategy.

Key ideas

  • Known microstructure equilibria can provide benchmarks for reinforcement-learning agents.
  • The author applies tabular Q-learning with an epsilon-greedy policy to a discretized Kyle auction.
  • The stated preliminary outcome is encouraging, but no evaluation details are provided.
  • Potential financial applications mentioned include optimal execution and portfolio selection.

Tags

Machine learningMarket microstructureExecutionPortfolio construction
Full text
# Looking for references on reinforcement learning in finance


# Looking for references on reinforcement learning in finance












I plan on using reinforcement learning for a research project. To be specific, I plan to define learning environments using market microstructure models whose solutions are well known and see if I can recover some attributes of equilibrium solutions using reinforcement learning.

I already went through the Q-Learning (and Deep-Q Learning) and Deep-Learning series available for free at https://pythonprogramming.net/. For fun, I wrote a script that discretizes Kyle's single auction model and teaches an agent how to behave like the Kyle insider using Q learning and an epsilon-greedy algorithm. The preliminary results seem encouraging, but I'd like to have more examples of codes and problems to get a better feel for it so I can eventually work on more complicated cases.

So, if you have references where I could find problems in finance like optimal trade execution, portfolio selection, etc. where reinforcement learning is used, I would greatly appreciate it. Ideally, this would come with the code in python as my goal would be to get better acquainted with how people tackle these problems. Note that I am also fairly familiar with R and MATLAB, if ever. I ask the question specifically here because I would really like these problems to be financial problems and I suspect at least some users of quant.stackexchange worked on these things. It's also because people who study or work in quantitative finance are likelier to know what sorts of financial problems can be viewed as games -- it's really the field-specific eample that I'm looking for.

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.