Skip to content
All library documents

Historical Kelly Optimization and the Limits of Empirical Returns

Article Quant Q&A · Author: danuker

Summary

The document considers whether portfolio weights can be chosen by maximizing average portfolio log returns directly from observed data, without fitting a named probability distribution. One response explains that this still amounts to modeling returns: the historical observations define an empirical distribution, whose usefulness depends on the amount and representativeness of the data. Estimation of returns and cross-asset dependence remains vulnerable to change over time.

The questioner reports that allocations among stocks and bonds vary sharply with the return horizon and data construction, and worries that tail events may be poorly represented. Another response treats this instability as a warning that a limited historical record may not reveal the true scale of extreme outcomes. The discussion does not establish a universally superior allocation method or prove that empirical optimization is invalid; it highlights sensitivity to sample choices, fat tails, and unstable correlations. Half-Kelly scaling is raised as an idea, but the document does not analyze its effectiveness.

Key ideas

  • Maximizing historical average log returns uses an empirical distribution even when no parametric distribution is fitted.
  • The empirical approach depends on past returns and correlations being informative about future conditions.
  • Reported asset weights change substantially with return frequency and horizon, signaling model sensitivity.
  • A short sample may fail to capture the magnitude of rare losses in fat-tailed returns.
  • The discussion offers no definitive allocation prescription and leaves stability as a central concern.

Tags

Full text
# Can I apply the Kelly criterion directly, without fitting any distributions?


# Can I apply the Kelly criterion directly, without fitting any distributions?












#### Problem

I want to apply the Kelly criterion to asset returns, so that I know how much to hold of each, ideally (and how much I should keep as a cash reserve).

As far as I understand the Kelly criterion, it's about maximizing the expected logarithmic returns - which computes as $$\frac{1}{n}\sum_{t=1}^{n} log(\frac{wealth_t}{wealth_{t-1}})$$

This correctly weighs losses, since summing in log-space is equivalent to multiplying in regular-space: even just one complete loss bankrupts my portfolio with no chance of recovery (it has a log of $-\infty$). Thanks to Matthew Gunn for explaining.

#### Research

In many "tricks" I've found online, people fit distributions to the returns, or find a neat formula.

What I dislike is there's some assumption of normality, or appeal to the law of large numbers - but I am skeptical they hold. For example, this answer and this page mention the variance - but variance can't capture fat tails.

Taleb shows that a fat-tailed distribution finds it much harder to converge to a mean, so the "large numbers" need to be much larger.

Another problem is the multivariate scenario: n-dimensional distributions are a headache to get right. Many mention covariance, but I have the same objection as for variance: it does not apply for non-normal distributions.

#### My question

To get to the point: Is it a good idea to side-step fitting any distribution and optimize directly on the data?

For example, say I compare stocks and bonds. Stocks have a high return, but also high risk, while bonds have a low return and low risk.

I'd get the two series, compute their log-returns, and find a linear combination with a maximum average log-return.

The coefficients of the linear combination will be the amount I invest in each.

Perhaps I divide the result by 2, to get half-Kelly, because past performance is no guarantee of future results.

But is skipping any distribution modelling a good idea?

I am inclined to believe it is, at least in the case when the data includes several instances of crashes. But I have a hunch I might be missing something, maybe related to ergodicity. Can you invalidate my belief?

##### Results

In case anyone is curious, I did it, and it seems very strange:

- when using monthly returns, the algorithm likes crazy leverage, like 3x stocks and 6.67x bonds.

- when using yearly rolling returns, it only goes 1.7x stocks and 2.34x bonds.

- 5-year returns make it go 2x stocks and .16x bonds.

- 10-year returns, both go up again to 2.45x and bonds go down to .263x.

I guess I don't even know what to ask. What is this? Serial correlation?

In any case, I suspect tail risk is not captured properly, so I can't recommend my method as it is. It doesn't seem to even converge.

## Answer by FP0 (score 2, accepted)

https://quant.stackexchange.com/a/71666

I think that your worries regarding normality, in general the right choice of a distribution, and the headache regarding high-dimensional distributions are valid.

However, how could you maximise the expected value of your total log-returns without modelling these individual log-returns?

By using a couple of series of log-returns and creating combinations of them, you are already modelling your log-returns. Instead of having a "usual" distribution like a normal distribution derived from some fitting procedure, you are using some kind of empirical distribution derived from past observations of the log-returns. Depending on how much data you have on hand and whether you think that observations of past returns will be representative of future returns, you could give it a try. But I do not think that there is a definitive answer to your question since there are too many questions being raised. I think that the main issue will be the stability of the distribution of the returns, and of their correlations, in the end.

## Answer by danuker (score 0)

https://quant.stackexchange.com/a/71698

My allocations varied wildly from 3x stocks/6x bonds to 2x stocks/.16x bonds, depending on how exactly I made the fit.

This is a red flag. Extreme data in the tails were very affected by what I believed to be minimal processing. But the "tails" were wagging the dog. So I dug deeper.

I read more from Taleb, and he refers to this as the "Lucretius fallacy, which as we saw can be paraphrased as: the fool believes that the tallest river and tallest mountain there is equals the tallest ones he has personally seen".

I am the fool, and I thank Taleb for letting me know!

Here is his monograph, "Statistical Consequences of Fat Tails", which proved to me an illuminating piece of work. Chapter 9 begins with the previous quote.

Needless to say, I have more studying to do before I juggle with estimating my exposures.

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.