Bootstrap Strategy Backtests with Randomized Signal Timing

Summary

The document compares bootstrapping realized strategy returns with resampling the underlying price series, and explains why neither approach directly provides a convincing test of a strategy’s added value. Resampling strategy returns can obscure whether performance is abnormal, while resampling prices may weaken the strategy’s ability to earn returns even when its logic is reasonable.

It proposes a signal-timing benchmark inspired by Cowles: model the distribution of bars between actual buy and sell signals, generate pairs of signal times from that distribution, and apply them at random points in the original series. Repeating this process produces benchmark equity curves and performance statistics, such as profit factor, drawdown, Ulcer measure, and Sharpe ratio, for comparison with the actual strategy. The note outlines a method rather than presenting empirical validation; its usefulness depends on the timing model and does not specify how to account for dependence or other design choices.

Key ideas

• Resampling strategy returns alone may not establish whether a strategy produces abnormal performance.
• Bootstrapping the original price series can undermine performance even for a reasonable strategy.
• A proposed alternative models the spacing between buy and sell signals and randomizes their placement in the original series.
• Repeated randomized trials provide benchmark equity curves and performance statistics for comparison with observed results.

Tags

# Proper way to backtest strategy using bootstrap method


# Proper way to backtest strategy using bootstrap method












Should I back-test in a single (original) price series and bootstrap the strategy returns to get statistics of interest? Or should I create bootstrapped price series using bootstrapped returns from the original price series and run the strategy on those? The latter one requires significantly more computational power.

## Answer by Alexander Didenko (score 1)

https://quant.stackexchange.com/a/75483

Straight bootstrap of the returns of the strategy would result in inconclusive evidence about the ability of the strategy to generate added value in terms of some "abnormal" returns. Bootstrapping original time series would undermine ability of the strategy to generate returns even if the strategy is reasonable. You could instead use solution in the style of Cowles, described above. For example, something like that:

- Model distribution of number of bars between two signals (buy and sell) of the actual strategy.

- Generate N buy/sell pairs from the distribution (1) and apply it to random points in time series.

- Finally, calculate equity curve and all statistics like profit factor, average drawdown, Ulcer and Sharpe ratio, etc.

- Repeat (2) and (3) many times, say B = 10 000+ times.

- Compare actual metrics and bootstrapped ones.

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.