Evaluating Strategies with Profitable Periods and Later Drawdowns
Summary
The document raises a backtesting problem: an algorithm shows a substantial gain over an initial run of trades, then gives back those gains and finishes below its starting balance. Results also depend on the chosen start date, and the author suspects the strategy performs differently in trending and choppy markets. Possible responses under consideration include finding a regime filter or stopping after a drawdown threshold.
The central question is whether win rate, expectancy, and Sharpe ratio over a favorable segment justify treating the strategy as promising despite its full-period loss. The note provides no performance data beyond the author’s stated example and offers no answer or tested method for resolving the question. Its useful implication is that a selected profitable window and positive summary metrics do not, by themselves, establish robustness or show that a strategy would recover if left running. Regime dependence, start-date sensitivity, and the full equity path warrant examination; the suggested drawdown stop would itself need evaluation.
Key ideas
- The strategy’s apparent success changes with the backtest start date and evaluation endpoint.
- A profitable early segment can be followed by losses that erase its gains.
- The author suspects different performance in trending and choppy conditions but has not validated a filter.
- Win rate, expectancy, and Sharpe ratio are raised as metrics, but the document does not establish that they prove robustness.
- A drawdown threshold is proposed as a possible stopping rule, not as a tested solution.
Tags
Full text
# Can a losing algo strategy be good if it was winning for 1000 trades? # Can a losing algo strategy be good if it was winning for 1000 trades? My algo, in back test, runs for 1000 trades and makes a 10% profit (over a few weeks of candles), from research it seems 1000 trades is statistically significant. Obvs not all 1000 trades were winners but overall it was in profit due to the big winners. After about 1000 trades it starts losing. Were I to evaluate success based on final account balance at 1100 trades ,the test would be a failure, it would have lost 5% of the initial balance. But, if this algo was in production and I stopped it running after 1000 trades, I would be pleased with the 10% profit. I notice that the start datetime for my backtest is important regarding profit levels reached. Clearly there are some conditions that suit the algorithm, I have not managed to program into a successful filter yet. I suspect it is trending conditions that are more favourable, and periods of ranging choppiness lead to big losses. It is a work in progress to work out how to know when to turn off the algorithm, programmatically. I guess I could look at a drawdown floor, beyond which I bail out. But the main question is, could there be metrics that would show the algorithm favourably, since 90% of the backtest it was in profit, even though in the final stretch it lost all the profits. Which metrics should I be looking at? The WinRate was always 0.4 or above, Expectancy was always positive. If I calculate Sharpe Ratio and it is positive..are these three enough to say the algorithm is worth investigation, can I exclude the final balance in my consideration since there is maybe an element of luck as to where I stop the backtest? In other words, if the WinRate, Expectancy and Sharpe Ratio are all decent over 1000 trades, then can I ignore the final balance being a loss and assume that the algo is solid and would have got back into profit had I left it run long enough.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.