Backtesting, optimisation and gates

How results are produced with real costs, how parameters are chosen out of sample, and how a strategy earns promotion.

Real exchange fees and costs

Backtests apply each venue's published maker and taker fees, funding payments on perpetual futures, borrow costs on shorts, a spread and a market-impact model on every fill. Equity is marked to market bar by bar, and leveraged positions can be liquidated at the venue's maintenance margin. Nothing is reported gross of costs.

Position sizing and leverage

Strategies size positions relative to account equity rather than in fixed amounts. The requested leverage is plumbed into the engine and clamped to the venue's cap: spot and options venues stay at 1x, futures venues allow more. Spot accounts cannot short, and the strategy base class enforces it.

How optimisation works

Optimisation uses the strategy's full available history. The most recent part is set aside as a holdout. The rest, the development period, is split into a sequence of walk-forward windows: parameters are searched on each window's training span and judged on the test span that follows it, so every out-of-sample result comes from data the search never saw.

The final parameters are then chosen once on the whole development period and tested exactly once on the holdout. Sensitivity, stability and significance statistics are reported alongside, and the promotion gates read the out-of-sample and holdout evidence, never the in-sample fit.

The holdout

The holdout is the last part of the history: at least 20% of it, at least 15 days, and long enough to expect about 20 trades, up to 35% of the history. A holdout that hits the 35% cap is labelled underpowered. It is never used to choose parameters, and the agents see results only up to the start of the holdout, so their decisions cannot be tuned to it. You, as the owner, see the full history on the strategy page.

Nested walk-forward windows

Walk-forward windows are sized by the number of trades, not only by time: each test window is lengthened until it can expect about 20 trades. Windows step monthly, or quarterly for long histories, with a short gap between training and test. Inside each training span the search is nested: it is split 70/30 again so parameters are picked on inner validation data. At least 3 windows and enough out-of-sample coverage are needed for a conclusive result; otherwise the result is marked inconclusive for lack of trades.

Parameter search and the final refit

Parameters are searched with Bayesian optimisation, about 30 trials per tunable parameter per window, between 30 and 150. Instead of the single best trial, the selector favours parameters whose neighbours also score well, under the strategy's risk constraints. The final refit applies the same selector to the whole development period; the holdout then tests exactly those parameters. Each optimisation has a time budget, and long searches can be run on remote compute.

Sensitivity and plateaus

Every numeric parameter is moved by 10%, 25% and 50% in both directions, and whole-number parameters by single and double steps. The plateau is how far a parameter can move before the Sharpe ratio drops by more than its statistical uncertainty; a sharper drop is reported as a cliff. A wide plateau is reassuring evidence, a cliff is a warning. This is a report, not a gate.

Parameter stability

Stability compares how much the chosen parameters vary from window to window with how much a random choice would vary. Parameters that stay close together are labelled stable; parameters that jump around are labelled unstable. It is a warning shown with the results, never a gate on its own. A per-window replay of the final parameters is also shown, labelled as partly in-sample because those parameters were chosen on the same data.

Significance and the deflated Sharpe ratio

Out-of-sample returns are tested with a t-test that allows for autocorrelation, and with a deflated Sharpe ratio. The deflated Sharpe ratio asks whether the result beats what the best of many random tries would reach, counting the effective number of trials: the out-of-sample variants actually tried, including each extra revision after a review. Internal search trials are not counted against the strategy, because they never see out-of-sample data.

Combinatorial cross-validation and the probability of backtest overfitting are computed on the training data as diagnostics. They inform the analyst but no longer decide promotion on their own.

Return windows and breadth

Slow strategies may trade too rarely to fill trade-count windows. They are then judged on time windows of period returns instead, with tests that respect autocorrelation. A breadth test runs the unchanged rule on 5 or more related markets: consistent results across markets can turn an inconclusive result into a pass, and a failure elsewhere can turn a pass into a warning.

Low-confidence results

When the evidence is not strong enough to be conclusive but nothing failed outright, and the analyst still recommends promotion, the strategy can enter paper trading labelled low confidence, not statistically proven. The label is shown on the strategy and paper pages and in notifications. A low-confidence strategy is not ready for live trading until its own paper trading record becomes significant.

Promotion gates

Promotion floors are enforced in code from a published criteria file. Hard gates cannot be waived: enough history, profitable in most calendar years with exposure, an average trade that covers the round-trip cost, no excess margin use, out-of-sample and holdout results present and not statistically rejected, drawdown within the registered limit, and complete, consistent accounting.

Soft gates and waivers

Soft gates cover drawdown against the strategy's own limit, Calmar ratio, profit factor, win rate together with payoff, holding period, the holdout compared with walk-forward results, the deflated Sharpe ratio, concentration of profit in a single year, market-impact cost and losses in high-volatility periods. The analyst may waive a soft gate only with a written reason that names the metric; the waiver is stored on the run, the Risk Officer can reject it, and the Portfolio Manager re-checks it.

Reading a backtest honestly

Compare the equity curve with the drawdown, check how many trades the result rests on, and read the average return per trade against the round-trip cost. A high return from a handful of trades, or a Sharpe ratio that disappears out of sample, is a warning rather than a result.