Why Broad MACD Parameter Searches Can Mislead Backtests
Summary
The document examines a reported study in which MACD strategies using many parameter combinations outperformed buy and hold in a high percentile. A response cautions that this result does not by itself challenge market efficiency, especially for Bitcoin, whose pricing efficiency may have changed over time. It argues that the reported performance could arise from chance or from flaws in how the indicator was calculated.
In particular, the response questions treating an exponentially weighted moving average span as a rolling-window length. Because the resulting indicator values across parameter choices may be highly related, many tested strategies may effectively share similar signals rather than represent independent evidence. The apparent breadth of outperformance could therefore exaggerate the strength of the finding. The post recommends further statistical testing but does not provide a corrected backtest, formal significance analysis, transaction-cost treatment, or enough detail to establish whether the strategies truly outperform. The reported percentile is a claim from the original study, not a validated conclusion.
Key ideas
- Searching many MACD parameter combinations can produce apparent winners through chance.
- An exponential moving average span controls decay and is not a rolling-window length.
- Closely related parameter choices may yield correlated signals and weakly independent tests.
- A high outperformance percentile alone does not establish a robust trading edge.
- Backtest conclusions need further statistical scrutiny and careful indicator implementation.
Tags
Full text
# Can MACD with a broad range of parameter combinations beat Buy and Hold under the Efficient Market Hypothesis? # Can MACD with a broad range of parameter combinations beat Buy and Hold under the Efficient Market Hypothesis? I conducted a study with Moving Average Convergence Divergence (MACD)s in the range of ``` short_periods = range(10, 60) # Example range long_periods = range(40, 200) # Example range signal_periods = range(2, 10) # Example range ``` The result was that MACD strategies have a better than buy and hold annualized return performance in the 99.75th percentile. Can that happen under the Efficient Market Hypothesis? Data (and scroll for the code below data): https://gist.github.com/rolandkofler/67c342cdad48b485356f70a5b877b3a2 Statistics of the result: https://gist.github.com/rolandkofler/a53a580f1ddc62b533137be3ecbddb16 ## Answer by danospanos (score 2) https://quant.stackexchange.com/a/78699 The informational efficiency of bitcoin pricing increases over time, so if your results were based on a dataset from the very early stages of bitcoin trading, I would not attempt to dispute your findings, although a 99% outperformance rate seems too good to be true, even for the early days. However, when it comes to the question of whether the MACD can outperform a buy-and-hold strategy over a wide range of parameters, given the current efficiency of bitcoin pricing, the answer is a definite NO. The results you've presented are flawed and could just be the result of a chance. To calculate the MACD, you used: ``` data['price'].ewm(span=short_period, adjust=False).mean() ``` This may not work as you expect. According to the documentation found here and here, the ewm() function operates as an expanding window, not a rolling window. Thus, the span parameter does not indicate the size of the rolling window, but decay in terms of the span. This is not the correct way to calculate this technical indicator. Based on that, I intuitively conclude that most of your strategies can be very similar because they rely on nearly identical values of calculated indicators as they will be dependent on each other. Then, the observed high outperformance across a wide range of parameters is probably just an artifact. In reality, it's likely that these strategies, which boast a 99% outperformance rate, have very similar entry points. Viewed as a singular (strategy) result, it could be attributed to pure luck, suggesting that further statistical testing is needed.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.