Market Regime Changes, Model Stability, and Backtest Length

# How to identify a change in market dynamics?


# How to identify a change in market dynamics?












As a beginner, I'm learning how to make good trading strategies. One of the things to consider for reliable backtesting is the Minimum Backtest Length, whose selection is basically a tradeoff:

Too short a backtest duration $\implies$ statistically unreliable backtest

Too long a backtest duration $\implies$ underlying dynamics of the market may have changed.

I don't have a concrete idea of what "underlying dynamics of the market" means in the context of trading strategies. Secondly, from the first answer to this question: Backtesting Period, I gathered that the "market dynamics" are different for trading strategies of different frequencies. So the underlying dynamics relevant to an HFT strategy change more frequently than those relevant to a strategy with weekly turnover.

Firstly, what exactly is meant/covered by market dynamics that affect a particular trading strategy?

More importantly, could someone please explain exactly how one can identify when and why the underlying dynamics of the market changed?

## Answer by Forgottenscience (score 1, accepted)

https://quant.stackexchange.com/a/32382

Markets are adaptive, highly dynamic systems that depend on the input of a multitude of risk attitudes, investment ideas, trading frequencies, economic expectations and much more.

One way of understanding market dynamics is through assuming you have a parameterized model for explaining or trading the market, $m$, that depends on a set of parameters $\eta_i$, $i \in \{1,2,\ldots, n\}$. Such a model could be the CAPM model where the interest rate of an asset $r$ is decided via $$r = \alpha + \beta r_{market} + \epsilon, \quad\epsilon \sim N(0,\sigma)$$ where your set of parameters is $\{\alpha, \beta, \sigma\}$. If you go on to estimate this model (via OLS, for example), you have an implicit assumption that your parameter set won't change significantly in the out-of-sample period, otherwise the model is not usable for trading as the world you have looked at is very different than the world in the future, and it is only good for explaining the past. If you are using daily return data and you estimate the $\beta$ variable for the last 30 years, chances are you will have gone through a number of different market-types that all are lumped together in your static parameter estimate. Thus you often want to create models that change dynamically over time, such as using rolling estimation of $\beta$. And of course, the above model is pretty much useless at extremely high frequencies, and you will not care too much about the interest rate policy when you are working with tick data.

It also boils down to what a trading strategy is supposed to $do$. If we assume you aren't market making, then you will be exploiting some regularly occurring pattern or anomaly via some sort of indicator that gives you signals. This pattern can disappear if other traders latch onto it, or if the market dynamics change due to an external event.

If you want to detect regime shifts in the parameters, there are a number of methods. This paper gives a very large number of different methods developed for this purpose. Some of the most popular in finance are (Hidden) Markov Models and changepoint analysis. You can find good questions on this under the market-regime tag here.

I will note that I don't believe that a backtest can be too long if you are at lower frequencies, as the longer data will just give you more confidence that your model actually could be useful in a multitude of market regimes rather than just the recent past. Many people go to great lengths to generate artificial data with particular properties to vastly increase their backtest length, synthetically.