This is the question I get asked most, in some form, by people who have just finished their first strategy: how many trades do I need before the result is real? Usually there's a number attached. "I have 1,200 trades, that's enough, right?" And the honest answer annoys everyone, because it isn't a trade count at all.
Here's the whole thing in one line, and then I'll spend the rest of the post on why it hurts.
What's the standard error of a Sharpe ratio?
For a Sharpe computed on n periodic returns, assuming they're independent and roughly normal, the sampling error is:
SE(s) ≈ √((1 + s²/2) / n)
where s is the Sharpe measured at that same frequency. Annualize both sides and something clean falls out. With k observations per year and T years, the annualized Sharpe S has:
SE(S) ≈ √((1 + S²/(2k)) / T)
Look at that 2k in the denominator. For daily returns k = 252, so even a Sharpe of 2 contributes 4/504 ≈ 0.008 to the numerator. The whole term collapses to 1. The standard error of an annualized Sharpe is approximately 1/√T, where T is your calendar years of data. It barely depends on the Sharpe itself, and it does not depend on your sampling frequency, and it absolutely does not depend on how many trades you took.
So:
| Years of data | SE(Sharpe) | 95% CI if you measured 1.5 |
|---|---|---|
| 1 | 1.00 | −0.46 to 3.46 |
| 2 | 0.71 | 0.11 to 2.89 |
| 3 | 0.58 | 0.37 to 2.63 |
| 5 | 0.45 | 0.62 to 2.38 |
| 10 | 0.32 | 0.88 to 2.12 |
A backtest showing Sharpe 1.5 over two years of history cannot statistically distinguish itself from a coin flip at the 95% level. That's the baseline situation for most crypto research, because most people's clean, survivorship-corrected, fee-accurate data starts somewhere around 2022 and half of the interesting symbols didn't exist before 2023.
But I have 40,000 trades. Doesn't that fix it?
No, and this is the misunderstanding I most want to kill.
Trade count and sample length are different quantities and only one of them is in the formula. If your strategy fires 40,000 times over six months, you have T = 0.5 and SE ≈ 1.41. Your 95% interval on a measured Sharpe of 2.0 runs from −0.8 to 4.8. Forty thousand trades and the honest conclusion is "might be good, might be negative."
The reason is that those 40,000 trades are not 40,000 independent bets on 40,000 different market states. They're 40,000 samples from about 180 days of market behaviour, and the days are correlated with each other, and the regimes are correlated within themselves. A high-frequency mean-reversion book that makes money every day for four months has really made one bet — that short-horizon reversion holds in this liquidity regime — and has observed that bet resolve maybe a handful of independent times.
The mental substitution I use: count regimes, not fills. How many genuinely different market conditions did this strategy survive? A funding-carry strategy that ran through one long basis-positive stretch has seen n = 1, regardless of how many hourly rebalances it logged.
Quick diagnostic: block-bootstrap your daily P&L with a block length of 20 days and look at the spread of resampled Sharpes. If the 5th percentile is below zero, your trade count is irrelevant. It takes about nine lines of numpy and it has talked me out of more strategies than any other single check.
Does the formula hold for real strategies?
Not exactly, and it errs in the direction you won't like. The 1/√T result assumes IID normal returns. Real strategy returns are autocorrelated and skewed, and the skew is usually negative for anything that looks like carry, short vol, or mean reversion. Mertens' correction adds those moments back:
SE(s) ≈ √((1 + s²/2 − γ₃·s + (γ₄−3)·s²/4) / n)
with γ₃ the skew and γ₄ the kurtosis. Negative skew makes the −γ₃·s term positive, which widens the interval. Excess kurtosis widens it further. For a typical crypto carry P&L with skew around −1.2 and kurtosis around 9, I've seen the corrected standard error come out 30–40% larger than the naive one. Lo's 2002 paper handles the autocorrelation side and the effect is the same sign: positive serial correlation in returns inflates the naive Sharpe and shrinks the apparent error, which is a nasty pairing.
So treat 1/√T as a floor on your uncertainty. It's the best case.
How high does the Sharpe need to be if I tested 500 variants?
Now we get to the part that matters for anyone running an automated research pipeline, which is the situation we're in every day at Stratmill: the generation agent doesn't produce one candidate, it produces hundreds.
The expected maximum of M draws from a standard normal is roughly √(2 ln M). Multiply by your standard error and you get the Sharpe that the luckiest of M genuinely worthless strategies will display.
| Strategies tested | √(2 ln M) | Best-of-noise Sharpe, 5yr (SE 0.45) | Best-of-noise, 2yr (SE 0.71) |
|---|---|---|---|
| 10 | 2.15 | 0.96 | 1.52 |
| 100 | 3.03 | 1.36 | 2.16 |
| 500 | 3.53 | 1.58 | 2.50 |
| 5,000 | 4.13 | 1.85 | 2.93 |
Read the second-to-last row. If you generated 500 candidates against two years of data and the winner shows Sharpe 2.4, you have found precisely nothing. That is below the noise floor. Bailey and López de Prado's deflated Sharpe formalizes this with the variance of your trial Sharpes standing in for the crude √(2 ln M), and it's worth implementing properly, but the crude version is enough to change behaviour today.
Two things make this worse than the table suggests. First, M is not the number of strategies you saved, it's the number you evaluated, including every parameter tweak, every "let me just try a 30-period lookback instead," every rerun after a bugfix. Nobody counts honestly. Second, your candidates aren't independent draws, so √(2 ln M) overstates the effective breadth, though correlated trials also mean a single lucky regime lifts a whole cluster of them together.
The most dangerous number in quant research is the one nobody logged: how many times you looked.
So what do I actually do?
Five things, in the order I'd do them.
- Log every evaluation. A counter that increments on each backtest run, persisted, never reset. If you can't say what M is, you can't say what your threshold is. This is the single highest-value hour of engineering in the whole list.
- Report Sharpe with its interval, always. Never print a bare number. Our backtest reports emit
1.72 ± 0.51 (2.9y, skew-adjusted)and the ± is not optional. It changes how the whole team talks about results. - Set the bar from M and T before you look. Pull the number out of the table above and write it down. Post-hoc thresholds are how everyone talks themselves into a curve fit.
- Prefer breadth over frequency when you're sample-starved. You can't manufacture more calendar time, but you can run the same signal across 40 uncorrelated symbols. That genuinely raises effective n in a way that raising trade frequency does not. Watch the correlations though — 40 crypto perps in a risk-off hour are one instrument.
- Then paper trade it. Out-of-sample time accumulates at one day per day and there's no shortcut, which is exactly why it's the only sample nobody can overfit.
None of this makes short samples usable. It makes you honest about what a short sample can support, which is a much weaker claim than most backtest reports are written to imply. A two-year Sharpe of 1.5 is a hypothesis worth paper trading. It is not a finding.
← All posts
