Skip to content
All library documents

Covariance Penalties for Reducing Backtest Overfitting

Article arXiv papers · Author: Adriano Koshiyama et al.
Read the original

Summary

The paper examines how strategy selection based on historical data can produce misleading performance estimates. It presents a covariance-penalty correction that lowers a risk metric to account for the number of parameters and the amount of data underlying a trading strategy. The authors place this method alongside approaches that address data snooping, inflated performance estimates, and evaluation through cross-validation.

The empirical investigation compares covariance-penalty approaches using ordinary least squares and total least squares across more than 1,300 assets. The reported findings support covariance penalties as a way to mitigate backtesting overfitting and indicate stronger performance for total least squares than for ordinary least squares in the study. The document gives no detailed asset breakdown, performance measures, or implementation guidance, so it does not establish how the correction will behave for every strategy or dataset.

Key ideas

  • Backtest parameter selection can make historical strategy performance appear more reliable than it is.
  • A covariance penalty adjusts a risk metric for the parameters and data used to develop a strategy.
  • The study compares ordinary and total least squares versions across more than 1,300 assets.
  • The reported results favor total least squares, though the summary provides limited detail about evaluation conditions.

Tags

BacktestingStatisticsRisk management
Full text
# 1905.05023


# Avoiding Backtesting Overfitting by Covariance-Penalties: an empirical investigation of the ordinary and total least squares cases









Systematic trading strategies are rule-based procedures which choose portfolios and allocate assets. In order to attain certain desired return profiles, quantitative strategists must determine a large array of trading parameters. Backtesting, the attempt to identify the appropriate parameters using historical data available, has been highly criticized due to the abundance of misleading results. Hence, there is an increasing interest in devising procedures for the assessment and comparison of strategies, that is, devising schemes for preventing what is known as backtesting overfitting. So far, many financial researchers have proposed different ways to tackle this problem that can be broadly categorised in three types: Data Snooping, Overestimated Performance, and Cross-Validation Evaluation. In this paper, we propose a new approach to dealing with financial overfitting, a Covariance-Penalty Correction, in which a risk metric is lowered given the number of parameters and data used to underpins a trading strategy. We outlined the foundation and main results behind the Covariance-Penalty correction for trading strategies. After that, we pursue an empirical investigation, comparing its performance with some other approaches in the realm of Covariance-Penalties across more than 1300 assets, using Ordinary and Total Least Squares. Our results suggest that Covariance-Penalties are a suitable procedure to avoid Backtesting Overfitting, and Total Least Squares provides superior performance when compared to Ordinary Least Squares.

Shown in full with attribution under the source's licence. Licence: abstract CC0

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.