Evaluating Trading Algorithms with Task-Specific Metrics

# The quality of an trading algorithm


# The quality of an trading algorithm












Good day. I am currently writing a term paper on the creation of trading algorithms in the foreign exchange market (by an algorithm I mean the one that follows the alpha model, for example, signals from some kind of technical analysis indicator). And I wondered, what indicator can be considered reliable when assessing the quality of an algorithm? Indicators such as Sharp ratio or LR correlation are, as I understand it, relative, i.e. the bigger, the better. But how can we give absolute analytics of the efficiency of a single algorithm? This leads to the question, are there any reference values for these indicators for comparison?

## Answer by Stéphane (score 1, accepted)

https://quant.stackexchange.com/a/59098

As regards a commentary made by another user, the Sharp Ratio is not a standalone measure because it requires the selection of a benchmark rate of return. While it is common practice in financial economics to use some proxy for the risk-free rate as a benchmark, it is worth nothing that financial economists are usually focusing on data as it relates to representative agents. Your average trader will probably find other rates than the yield on US Treasury bonds more meaningful.

Now, how do you evaluate a trading algorithm depends on what the algorithm is tasked to achieve. As @noob2 pointed out, algorithms are used to automate tasks and trading involves many tasks. It is not uncommon, from what I understand, for one or many algorithms to focus exclusively on identifying trading signals. Alpha models are usually designed to flag short, long or neutral trading opportunities -- again, that's my understanding. Those signals can then be fed into another algorithm used to determine a new target portfolio which poses the problem of sizing your target positions and designing an execution strategy which presumably would split trades into blocks to avoid moving prices in too much of an adverse direction. Each step in there can be automated with more or less sophisticated methods and every solution can be more or less integrated. Your entire strategy consist in identifying desirable changes to your portfolio and chasing after this moving target, hopefully turning a profit large enough to justify operational costs and to compensate the risks involved.

Now, how do you evaluate it? Well, in the end, if you automated the whole thing, you could use a Sharp ratio to evaluate the WHOLE strategy. You'd also potentially look at the length of time you can spend loosing money, under which condition you seem to be loosing money, and how much money you can expect to loose. For the later, I would personally use my back testing data as input for an EVT kind of analysis to see what happens with the extreme lows. Basically, you need an idea of whether the approach works well enough to be profitable, what kind of bank roll you need to ensure any Law of Large Numbers effects kick in, and I'd also look into the Giacomini-Rossi fluctuation tests for my alpha models to make sure things aren't bouncing around too much -- they check for forecasting performance under model instability (i.e., when the best model changes over time).

I'm sure you can easily find meaningful metrics and methods to do all of the above very easily.