Offline Reinforcement Learning for Price-Impact-Aware Liquidation

Summary

The document studies how a trader can liquidate a risky asset when trades create temporary price impact and the impact kernel is unknown. It proposes estimating the kernel, or propagator, nonparametrically from static data containing correlated price paths, trading signals, and metaorders. Estimation accuracy is measured with a metric that explicitly depends on the available dataset.

A greedy execution strategy based only on the estimated kernel can have higher costs than intended. The authors attribute this to spurious correlation between the strategy and estimator, as well as uncertainty tied to a biased cost function. Their proposed offline reinforcement learning method uses a pessimistic loss that accounts for estimation uncertainty and an optimizer designed to remove that correlation. They derive an asymptotically optimal bound on execution costs without requiring precise knowledge of the true kernel, and report numerical experiments supporting the estimator and strategy. The excerpt does not give experiment details or quantify practical performance under live trading conditions.

Key ideas

• The method estimates a transient price-impact kernel from static data with correlated trajectories and trading activity.
• A greedy strategy based on the estimate can be suboptimal because the strategy and estimator are spuriously correlated.
• A pessimistic loss incorporates uncertainty in the estimated impact kernel.
• The proposed offline reinforcement learning approach derives an asymptotic execution-cost bound without precise kernel knowledge.
• Numerical experiments are reported, but the excerpt provides no live-trading results.

Tags

# An Offline Learning Approach to Propagator Models


# An Offline Learning Approach to Propagator Models









We consider an offline learning problem for an agent who first estimates an unknown price impact kernel from a static dataset, and then designs strategies to liquidate a risky asset while creating transient price impact. We propose a novel approach for a nonparametric estimation of the propagator from a dataset containing correlated price trajectories, trading signals and metaorders. We quantify the accuracy of the estimated propagator using a metric which depends explicitly on the dataset. We show that a trader who tries to minimise her execution costs by using a greedy strategy purely based on the estimated propagator will encounter suboptimality due to so-called spurious correlation between the trading strategy and the estimator and due to intrinsic uncertainty resulting from a biased cost functional. By adopting an offline reinforcement learning approach, we introduce a pessimistic loss functional taking the uncertainty of the estimated propagator into account, with an optimiser which eliminates the spurious correlation, and derive an asymptotically optimal bound on the execution costs even without precise information on the true propagator. Numerical experiments are included to demonstrate the effectiveness of the proposed propagator estimator and the pessimistic trading strategy.

Shown in full with attribution under the source's licence. Licence: abstract CC0

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.