Evaluating Market-Timing Backtests Against Risk and Chance
Summary
The discussion examines how to tell whether a market-timing strategy adds skill beyond exposure to broad market returns. A strategy that is mostly long during a rising market may appear successful while offering little independent evidence of an edge. One response suggests comparing the backtest with a distribution of zero-skill outcomes: simulate many random trading paths under the same constraints and initial conditions, and compare the strategy’s result with that distribution. Testing different starting portfolios can also reveal sensitivity to the initial setup.
Another contribution points to a framework for estimating the information ratio expected from timing strategies. It distinguishes repeated timing decisions in one market from breadth across multiple strategies or markets, and reports that volatility-weighted position sizes maximize expected timing information ratios under the paper’s assumptions. The discussion also notes that market timing should be judged against buy-and-hold and that long equity exposure can benefit from an upward market bias. These are discussion-board recommendations and a paper summary, not a complete empirical comparison; conclusions depend on the strategy constraints, sample, and model assumptions.
Key ideas
- A mostly long strategy can appear successful because it captures the market’s upward movement.
- Compare a backtest with random strategies that follow the same trading constraints.
- Varying the initial portfolio can show how much results depend on the starting point.
- Evaluate market timing against buy-and-hold to assess whether trading adds value.
- The cited framework separates timing frequency from breadth across strategies and markets.
Tags
Full text
# What are the risk factors in analysing strategies? # What are the risk factors in analysing strategies? What do you think of strategies displayed on timelyportfolio.blogspot.com? I really like the fact that there is some code to reproduce the strategies, but they seem very elementary because he does not control for risk factors. By that I mean that you can have a super complex strategy with lots of bells and whistles, but if in the end your strategy is mostly long, and that for the period you consider the market is up, you are merely looking at 1 big point and not 5000 independent samples from your strategy even though you might have this illusion. How do professional deal with this, can you point at good papers for it ? ## Answer by timelyportfolio (score 8) https://quant.stackexchange.com/a/1437 Glad people are reading. Simple with more history in terms of time and indexes is better in my book. I have spent 13 years reading over 200 research papers, incorporating complicated and advanced techniques, and studying very reputable buy side research with no improvement in results. Readers are on their own to extend to lots of markets including Nikkei with 90% drawdown over 20 years. Also equity markets have an upward bias making reliable robust shorting techniques extremely difficult. I save the currency, bond, and commodity markets with long term mean reversion and no upward bias for potential shorts. See and hear Emmanuel Derman for much more discussion from someone who knows far better than me. Absolutely love the discussion. Look forward to reading other responses. ## Answer by Frank Fingerman (score 6) https://quant.stackexchange.com/a/1428 I don't think risk factors are that important here. This is a simple market timing strategy where you're either fully exposed to the market or not exposed at all. All he needs to show is that he's adding value above buy-and-hold by all of the in-and-out trading. ## Answer by Patrick Burns (score 5) https://quant.stackexchange.com/a/1881 I think the way to see the real effect in a backtest is to produce the distribution achieved with zero skill. You can get one point from this distribution by starting with the same initial portfolio, then do random trading through the time period conditional on obeying the same set of constraints. Do that several times to get the approximate distribution. The initial portfolio can have a big effect on what you see. So at the very least using different starting portfolios can give you some information. A fuller discussion of this is at http://www.portfolioprobe.com/2010/11/05/backtesting-almost-wordless/ ## Answer by Tal Fishman (score 4) https://quant.stackexchange.com/a/1879 On the Expected Performance of Market Timing Strategies, a recent working paper by Hallerbach from Robeco Asset Management, attempts to construct a rigorous framework for evaluating market-timing strategies. > We derive expressions for the Information Ratio (IR) that can be expected from market timing strategies in non-parametric and parametric settings. Our results hold as superior approximations and lift Grinold’s [1989] “Fundamental Law of Active Management” to an operational level. In addition, we separate “time series breadth” (the timing frequency per strategy) from “cross-section breadth” (the number of separate sub-strategies) because they contribute differently to performance. We provide theoretical proofs that implementing volatility-weighted bet sizes, both in the time series context of a single underlying market and in the cross-section context of multiple underlying markets, maximizes the expected timing IR. Our theoretical results can be used as a benchmark for and reality check on the back tested performance of timing strategies. We confirm the accuracy of our results by simulating timing strategies for currencies, equities and fixed income.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.