Evaluating Cryptocurrency Machine Learning Backtests Beyond Returns

Summary

The document describes a backtest for a support vector machine that classifies the next day’s movement in Bitcoin, Ethereum, and Litecoin as buy, hold, or sell. The author says the simulation accounts for fees, spread, and market depth, and reports a positive combined return. They compare its performance with buy and hold and the CCi30 cryptocurrency index, then ask what additional metrics could provide a more informative assessment and how to prioritize them.

No particular metric or evaluation method is supplied as an answer. The document therefore offers a research question rather than evidence that the strategy is robust or profitable in practice. Its stated cost and liquidity assumptions make the backtest more realistic than one that ignores trading frictions, but the description does not specify the test period, validation design, risk exposures, or stability across market conditions. Those omissions limit what can be concluded from the reported positive return.

Key ideas

• The strategy uses a support vector machine to classify daily price direction for three cryptocurrencies.
• The backtest accounts for fees, spread, and market depth, according to the author.
• The author compares aggregate returns with buy and hold and a cryptocurrency index.
• The document asks which risk and performance metrics should supplement nominal returns, but provides no answer.

Tags

# What are the best metrics to evaluate an ML algo backtest, other than the nominal returns?


# What are the best metrics to evaluate an ML algo backtest, other than the nominal returns?












I have designed an algorithm that uses Support Vector Machines to classify the next day's price movement for several prominent cryptocurrencies on a `[1,0,-1] (buy/hold/sell)` basis. These cryptocurrencies are namely Bitcoin, Ethereum & Litecoin. I have desgined a backtest that takes into account fees, spread and depth too and therefore can assume a backtesting environment somewhat more realistic than most. When aggregating the returns from across these 3 cryptocurrencies, the overall return is positive.

But there must be other, more insightful and prudent metrics that indicate the success of a strategy other than just pure returns. The only metric that I have so far compared the strategy to is a `Buy and hold (B&H)` and comparing returns to the CCi30 index, an index for cryptocurrencies. Whilst I have seen a similar answer here, I found the answers slightly vague and not necessarily relevant to this particular asset class.

Are there any other metrics that can evaluate the performance of a backtest? And which ones should receive the most weighting?

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.