When to Detrend Data in Strategy Backtests and Statistical Tests
Summary
The discussion separates signal generation in a realistic backtest from statistical inference about predictive relationships. For price-based rules such as moving-average crossovers or Bollinger Bands, the answers advise using the original historical prices, provided each decision uses only information available at that time. Detrending before generating signals can change the strategy being evaluated.
Detrending may matter when testing whether a signal predicts future returns, because price series can be highly autocorrelated and long-run market drift can make random market exposure appear profitable. One proposed check subtracts a fitted trend from prices for evaluating trading outcomes while still generating signals from actual prices. Another answer favors comparing the strategy on unadjusted data with a buy-and-hold benchmark, using returns and risk measures such as volatility or drawdown. These are methodological perspectives rather than a universal rule; the appropriate treatment depends on whether the goal is strategy simulation or statistical inference.
Key ideas
- Generate price-based trading signals from the price series that would have been observable at decision time.
- Detrending can alter the signals and therefore change the strategy under evaluation.
- Statistical tests of predictive relationships should account for price autocorrelation and market drift.
- A fitted trend can be subtracted for an evaluation of whether apparent profits exceed drift.
- Comparing a strategy with buy and hold on the same unadjusted data is another proposed benchmark.
Tags
Full text
# Detrending price series for back testing # Detrending price series for back testing If I am testing a trend-following strategy, should I detrend the data before applying the rules or should I generate signals based on the original price series but use detrended data for performance evaluation ? If I use a trend-following strategy with detrended data, it may not generate signals as it would with real price series. ## Answer by oronimbus (score 5) https://quant.stackexchange.com/a/75446 Depends on what the goal is. If you want to backtest a priced based signal (e.g. RSI, SMA Crossovers, Bollinger Bands or other technical indicators) then it wouldn’t make much sense to detrend the time series just for the sake of it. The backtest doesn’t really care about the nature of the signal as long as you don’t use information unavailable at the time of financial decision making (i.e. future data). Detrending becomes important for statistical analysis and inference. For example if you want to answer something like “how well does signal $x_t$ predict returns of series $y$ at $t+1$”. You will be led to a wrong conclusion if you don’t account for the fact that the price series is highly autocorrelated. ## Answer by nbbo2 (score 4) https://quant.stackexchange.com/a/75461 I am not sure why you want to use detrending as part of a backtest. The only book I know that advocates such an approach is D. Aronson's Evidence based technical analysis, Wiley, 2006. The valid point he makes is that if you test a stock market timing strategy that goes into and out of the market at random times, it will make money over a long period simply because the stock market rises over the long run. But that does not make it an attractive strategy. He advocates fitting a trend line to the S&P 500 prices over the backtest period (by connecting the starting and ending prices) and simulating buys and sells at an adjusted price equal to the actual price minus the trend line. For signal generation you would use the real prices. If you buy and sell at random using these adjusted prices you will make a P&L of zero, correctly showing the strategy is worthless. Some people call these prices the "drift ajusted prices" (it is a term I like more than 'detrended' which has many meanings). The approach I prefer is to backtest 2 strategies using the same software and (unadjusted) data, the strategy you are interested and a Buy and Hold strategy that is fully invested in the S&P 500 at all times. Then you compare the stats for the 2 strategies and try to see if your strategy has an advantage over BH either in terms of excess returns or lower volatility, lower drawdown, etc. I think it is easier and cleaner to compare two realistic strategies rather than looking at a strategy that trades at made up prices. But that is just my opinion.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.