Preventing Duplicate Market Orders with Client-Side Risk Controls

# Ensuring market order creation idempotency


# Ensuring market order creation idempotency












I am trying to find a way of placing market orders on crypto exchanges in an idempotent way. Specifically, I want to ensure that retrying the same request does not cause another order to be placed. For example, Stripe offers its API users to specify an idempotency header (https://stripe.com/docs/api/idempotent_requests). This ensures that issuing the same request does not result in a repeated operation to be performed.

I have looked at the API docs of a few major crypto exchanges and failed to find any similar mechanisms. For instance, I have tried to use the `ClOrdId` on Okex and it appears the exchange allows me to place as many market orders as I like with the same `ClOrdId`.

How is this problem normally solved?

Apologies if this question is outside the scope of this StackExchange site.

## Answer by databento (score 5)

https://quant.stackexchange.com/a/64127

I can't speak for cryptocurrency markets, however in conventional market venues, I have never seen it done on market-side. What you're thinking of is usually implemented on the client side as a pre-trade risk check.

#### Approach 1: Strict idempotency (for any two calls with the same parameters, return the same position state)

This is the most naive solution. You can store the entire history of all new orders placed as an array of structs containing order parameters and a corresponding array of achieved position states. When a new order is placed, iterate through this to check for any matching past order - and if so, compare achieved past position state with theoretical would-be position state.

On the contrary, such strict idempotency is not a pragmatic solution, because in practice, there's many types of strategies that actually intend to place two identical, consecutive order requests to double down on a position.

Here's a naive example: You have a momentum signal that triggers for your strategy to cross the spread for N lots. You get filled for N, and subsequently 10 other trades follow you to deplete the level to some M < N size. All else equal, your momentum signal has probably gotten stronger because of the autocorrelated trades, and moreover, there is now aggressive queue priority you can get by crossing again for N at the same price so the residual quantity (N-M) of your order becomes the new top.

Another issue with a strict idempotency is that it is expensive for a pre-trade check to compare your order against all previous calls with the same parameters - and you don't want a pre-trade check to slow down the critical path of an outgoing order.

One other issue is that your new order being evaluated could have slippage, which means you have no way of knowing for the certain the theoretical state achieved when that order is released.

#### Approach 2: Order throttling

More likely, what you want is simply to avoid an errant strategy from blowing up by sending duplicate orders in a tight loop. This can be achieved without strict idempotency. One practical solution is to define some rate limits. Here's some examples with decreasing generality:

- Order actions per second

- New order actions per second

- Position-increasing order actions per second

- New order actions per second per (symbol, price)

- New order actions per second per (symbol, price, size)

The first two types are very commonly used.

The third has the benefit that it won't throttle situations where you're rushing to derisk your inventory, but is perhaps less common because it tends to be subsumed by other liquidation logic.

The last two have the benefit of being better approximations of "idempotency", since you're still making some effort to check parameters (symbol, price, size) that make an order unique. However, because of the large space of possible combinations of order parameters, a full comparison check is expensive and impractical.

For the implementation of these rate limits, two common heuristics are that (i) there are usually market-level or session-level rate limits that you should treat as looser limits than your own, (ii) you want to estimate the rates in an efficient, amortized O(1) algorithm. There's many variations of such algorithms, some using naive time bucketing, others approximating the rate with an exponential decay adjustment.

#### Approach 3: Check for any two consecutive, duplicate orders

Checking for any two consecutive, duplicate orders has the advantage of being simpler to compute than an actual rate limit while assuring constant time performance.

There's a naive example where this check fails: some errant strategies will place duplicate orders in separate cycles, e.g.: (AAPL, +100), (SBUX, +100), (AAPL, +100), (SBUX, +100), ...

#### Approach 4: Check for any two duplicate orders over some user-defined interval

To resolve the issue of separate cycles while maintaining reasonable performance, you could define some maximum time apart to check for two duplicate orders. This has a few implementation advantages: (i) it lets you bound the runtime of the risk check since you don't have to check two orders that are placed a long duration apart, (ii) it is easy to intuit, since markets usually have predictable order-to-ack times.

The downside of this approach is that it has linear memory profile.

#### Approach 5: Hashing and partial comparisons

A problem with approaches 3 and 4 is that a bitwise comparison of all of the order parameters of a window of past orders can be too slow for your use case. We've also not addressed the problem of memory use for approach 4.

You could improve on approaches 3 and 4 by only comparing the most significant bits of the parameters, or hashing your order parameters and only comparing the hashes within the time window.

The downside of this approach is that you could end up with false positives. However, this may not be a problem for certain classes of higher frequency strategies that are robust to a handful of order rejects every few days.