Four numbers, all measured off our own paper-trading stack, one strategy, 1-minute BTCUSDT perpetual signals, 31 days and 1,418 orders. Median time from bar close to order acknowledged by the exchange: 195 ms. 99th percentile: 2,970 ms. Median on the 1% of minutes with the highest realized volatility: 840 ms. And the latency constant that sat in the backtest config for most of the preceding year: 0.
Three of those numbers are boring. Everyone knows the zero is wrong, everyone eventually replaces it with something, and the something is almost always a constant somebody picked by feel. 100 ms, 250 ms, "let's be conservative, 500." The number that actually matters is 840, because it says your latency is a function of the market state, and the market states where it blows out are the same market states where your signal thinks it has found something.
Where the 195 milliseconds goes
Here's the full budget, measured stage by stage with monotonic-clock timestamps written at each hop. Box is an AWS instance in ap-northeast-1, same region as the venue's endpoints, which is about as good as a non-colocated retail-shaped setup gets.
| Stage | p50 | p95 | p99 |
|---|---|---|---|
| Bar close → WS kline-closed event received | 128 ms | 402 ms | 1,940 ms |
| Parse + feature state update | 3 ms | 9 ms | 28 ms |
| Decision: model eval + risk checks | 11 ms | 22 ms | 60 ms |
| Order submit → exchange ack | 47 ms | 180 ms | 1,100 ms |
| End-to-end (measured, not summed) | 195 ms | 690 ms | 2,970 ms |
Two things to note before the interesting part. First, the medians don't add: 128 + 3 + 11 + 47 = 189, and the measured end-to-end median is 195. Percentiles are not additive, and if you build your latency estimate by summing per-stage p99s you'll get 3,128 ms when the real p99 is 2,970. Close enough here, wildly wrong when stages are correlated in the other direction. Measure the whole path.
Second, look at the top row. Two-thirds of the median budget is spent waiting for the exchange to tell you the bar closed. Not network, not your code, not order routing. The venue aggregates trades into a kline, decides the minute is over, and pushes. That push has a queue in front of it, and the queue is shared with every other subscriber on that stream.
The 840
Bucket every one of those 1,418 orders by the realized volatility of the minute that produced the signal, and the distribution comes apart.
| Signal-bar volatility bucket | Orders | p50 end-to-end | p99 end-to-end |
|---|---|---|---|
| Bottom quartile | 201 | 141 ms | 488 ms |
| Middle two quartiles | 796 | 183 ms | 1,210 ms |
| Top quartile | 407 | 312 ms | 2,880 ms |
| Top 1% of all minutes | 14 | 840 ms | 4,100 ms |
Every stage degrades together, which is why it compounds. The venue's aggregation and broadcast queue is fed by matching-engine load, so a minute with 11,000 trades in it pushes later than a minute with 600. Your own process is handling more depth updates and more trade prints in the same window, so the feature update that usually takes 3 ms takes 28. The REST order path is competing with everyone else's panic, and you start seeing the occasional 503-and-retry that costs you a full second on its own. The correlation coefficient between signal-bar trade count and end-to-end latency in our sample was 0.41. Not enormous. Enough to ruin a constant.
And the strategies that survive a first-pass screen are, overwhelmingly, the ones that trade breakouts, reversals, dislocations, liquidation cascades. They fire in exactly the bucket where the latency is worst. A momentum signal that only trips when the minute moved 40 bps is a signal that has arranged, structurally, to always be served by the slowest version of your stack.
We once spent a day chasing a 3.6-second outlier that showed up at roughly the same time every night. It was logrotate, copy-truncating a 2 GB file on the same EBS volume the process was writing to, stalling the event loop. The market was innocent. Measure first, then blame microstructure.
What that does to a fill
Price drifts while you wait. On a random minute, the average adverse move over our 195 ms median was about 1.1 bps against the signal direction. On the top-quartile minutes at 312 ms, 5.8 bps. On the top-1% minutes at 840 ms, roughly 14 bps of adverse drift before the order even reaches the book.
On BTC perps the spread barely participates in this: one tick on an 84,000 quote is 0.012 bps, so the damage is adverse drift rather than crossing cost. Move the same strategy to a mid-cap perp and the second term wakes up — top-of-book spread in our sample went from 1.4 bps in calm minutes to 6.2 bps in the top quartile, and the depth at that touch thinned by more than half. You pay the drift and the wider spread and the worse depth, all at once, all because they share a cause.
When we replayed the same 31 days with per-bar conditional latency instead of a flat 200 ms, the backtest's average entry price moved 3.1 bps worse, concentrated almost entirely in 9% of the trades. Backtest-versus-paper entry-price divergence dropped from 4.2 bps to 0.9 bps. In-sample Sharpe on that strategy went from 1.7 to 1.1, which is the honest number and always was.
So what do you actually put in the config
Not a scalar. What we run now:
- Sample, don't set. Draw the latency for each simulated order from an empirical distribution conditioned on that bar's own activity — trade count and absolute return are enough, and both are already in a standard kline. Fit it once per venue per order type, refit monthly.
- Keep the tail. Do not fit a lognormal and call it done; the p99 lives in retry storms and GC pauses that no smooth distribution reproduces. We use the empirical quantiles with a resampled tail above p95.
- Latency applies to cancels too. A stop-replacement or a hedge cancel is subject to the same delay, and in fast markets your cancel is racing a fill. If your simulator cancels instantly, your risk model is fiction.
- Add a staleness guard and simulate it. Ours drops any signal whose bar is more than 1.5 seconds old on arrival. That's a real behaviour with real cost — it silently removes trades, and it has to remove the same ones in the backtest.
- Recalibrate against paper. Paper trading is where you get the ground truth. Log the four hop timestamps on every order and diff the distributions against what the simulator assumed, weekly.
Other asset classes, same disease. US equities: the consolidated tape adds its own aggregation delay on top of your broker's, and it widens under message-rate pressure for the same structural reason. Prediction markets: REST-only order entry on most venues, and we've measured sub-100 ms medians that go to multiple seconds when a resolution event hits and everyone arrives simultaneously. Crypto options: the quote you're pricing off may be two seconds stale in the exact minute the underlying moved.
The general form of the mistake is treating an execution parameter as an independent constant when it's a function of the same state variable driving your signal. Fees behave; funding behaves; latency does not. If you're going to carry one number in your head from all this, carry the ratio: our p99 is fifteen times our median, and our signal arrives disproportionately near the p99 end.
Go instrument the four hops. It's an afternoon of work and it will tell you more about your strategy than another week of parameter search.
← All posts
