Choosing Fold Lengths for Trading Strategy Cross-Validation
Summary
The note considers how to score trades that remain open at the end of a fold in a K-fold strategy backtest. The response recommends closing the position at the fold boundary and recording its resulting profit or loss, rather than dropping the trade. It warns that a fold containing only a few trades can make the validation result depend heavily on luck.
As a rule of thumb, the answer proposes sizing each fold to fit roughly ten average-duration trades and having at least four folds. This yields a suggested data requirement of about forty times the average trade duration. These figures are presented as illustrative guidance, not a formal statistical guarantee. The note does not address temporal dependence between folds, leakage from overlapping trades, or the suitability of ordinary K-fold splitting for time series.
Key ideas
- Record the profit or loss of open trades by closing them at the fold boundary.
- A fold should contain enough trades to reduce sensitivity to a small number of outcomes.
- The response suggests fold length of about ten average trade durations.
- It recommends at least four folds, while framing the counts as a rule of thumb.
Tags
Full text
# Kfold cross validation: how to handle hold-out periods # Kfold cross validation: how to handle hold-out periods I want to backtest a strategy using K-fold cross validation. Assume I have a period of 300 days in my backtest. I divide it into 30 folds of 10 days each. On day 1 of a fold, I enter a trade, and exit after 6 days. On day 7 I enter another trade. But when we hit day 10, the final day in the fold, the trade is still open. What is the best way to handle this? Should I just close the trade at the end of day 10, and record the pnl at that point? Or just ignore the trade altogether since it was unfinished (ie, still open). ## Answer by Ishan Shah (score 1) https://quant.stackexchange.com/a/43941 Ideally, you should close the trade and book the pnl and do the analysis with respect to that. But the problem here is you are making the decision based on two trades. You can tackle this problem using the following way - Determine the average duration of your trade say d days - The number of days in the fold should be at least able to accommodate 10 trades. This size of the fold should be 10 * d - There should be atleast 4 folds for you to confidently cross validate your strategy. Thus the number of data points required are 10 * d * 4. That is 40 times the average duration of the trade. The numbers given are only for representation purpose but the general rule of thumb is there should be more number of trades per fold to effectively validate your trading strategy. Otherwise there is a high possibility that by sheer luck 2 trades happen to be profitable. And as the number of trades increases, early closing of trade shouldn't be a problem.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.