Information Theory and Speech Models in Quantitative Trading

# How are cryptography and speech recognition technology applied to forecasting financial markets?


# How are cryptography and speech recognition technology applied to forecasting financial markets?












One of the answers to my previous question regarding the strategy of Renaissance Technologies, there was a reference to The Quants: How a New Breed of Math Whizzes Conquered Wall Street and Nearly Destroyed It.

After doing some browsing in the book I found that it states that Renaissance Technologies obviously very successfully employs cryptography and speech recognition technology for forecasting financial time series.

Do you know of any good papers (or other references) where the use of either of these technologies in connection with finance is shown?

## Answer by HaltingState (score 29, accepted)

https://quant.stackexchange.com/a/1077

By "cryptography" you mean information theory. Information theory is useful for portfolio optimization and for optimally allocating capital between trading strategies (a problem which is not well addressed by other theoretical frameworks.)

See:

--- J. L. Kelly, Jr., "A New Interpretation of Information Rate," Bell System Technical Journal, Vol. 35, July 1956, pp. 917-26

--- E. T. Jaynes, Probability Theory: The Logic of Science http://amzn.to/dtcySD

--- http://en.wikipedia.org/wiki/Gambling_and_information_theory

- http://en.wikipedia.org/wiki/Kelly_criterion

In the simple case, you would use "The Kelly Rule". More complicated information theory based strategies for allocating capital between trading strategies take into account correlations between the performance of trading strategies and the relationship between market conditions and strategy performance.

As for Natural Language Processing and speech recognition; when you examine the founders of Renaissance Technology, you will notice that many of the early employees had backgrounds in natural language processing. Naively, you might assume that RT is using NLP based strategies.

However, you will find that all of RT's NLP related hires have backgrounds (published research, Phd thesis's) in speech recognition and specifically in Hidden Markov Models and Kalman filters. The academic background and published research of RT employees gives you a good idea of the algorithms they are using.

The information that has leaked out of RT suggests that RT heavily uses "hierarchical hidden markov models" for latent variable extraction from market time series. It is also believed that RT has developed a proprietary algorithm for "layering" multiple trading strategies for trade signal generation.

RT does not have a single secret trading strategy that magically generates billions of dollars a year. Renaissance Technology's trading strategies are based upon the integration of information from multiple mathematical models.

## Answer by markbruns (score 13)

https://quant.stackexchange.com/a/1167

In a very, very general sense, what Renaissance Technologies does well [and others try to do, many do less well] is understand where the "true" signal is (i.e. where prices should be) and what is noise (i.e. over-/under-reactions by others in the market) in the total signal of market prices. Generally, trading profits are made by taking the opposing position to someone who over- / under-reacts to where the market price is going to be because the market will "come back" to the true price.

Cryptographic algorithms and speech recognition algorithms have been developed to accomplish essentially the same thing ... it's necessary to separate noise from underlying signal in those and other applications of information theory, basically this is machine learning ... in general, becoming proficient in machine learning is skill that applies in many fields and for the mathematically-inclined individual, it is well worth studying [because there are so many applications, not just trading]. A good starting point would be the lectures by Andrew Ng of Stanford University

Attempting to beat the hedge funds really isn't something that should be copied by amateurs, i.e. it's an expensive way to get an education ... the algorithms are dynamic; the people using these algorithms must be faster, better, smarter than others who are using these algorithms ... continually updated, adjusted, advanced, accelerated, sharpened by teams of very smart people ... Renaissance isn't alone; it has smart competitors; there's a constant struggle by well-capitalized new firms with smart founders to make these adjustments with smarter, brighter, more capable teams of people who have access to better, faster technologies ... for example, many these algorithms must now execute so rapidly that it's necessary to use specifically-designed hardware that use ASIC integrated circuits or similar hardware, i.e. software on a supercomputer isn't fast enough to execute the trades ... the oscillation of the noise around the signal happens faster and faster.

Generally, the world benefits by the development of these algorithms and hardware, because the advances in this technology can eventually "spill over" into other projects in the rest of the world ... you aren't likely to find out what Renaissance Technologies and its competitors are doing primarily mostly because the people doing it don't have time [while they are doing it] to write papers and explain; it's not really that they want to keep secrets (i.e. there's some advantage to having your competition look at what you were doing last month, last year because by the time they understand it, you already know how to beat your old strategy), it's mostly that they don't have time / inclination to write up the explanation just yet ... as with all pursuits, the day eventually arrives where a person just doesn't really care any more and maybe wants to answer questions or tell the story.

Anyone who smugly tells you that they can't tell you because it's a secret basically is telling you they're just somebody like Mr. HR-clerical-job-app-screening-person and they couldn't begin to understand -- so it is easier for them to tell you that it's a secret. It's not that much of a secret ... mostly, it just happens fast ... so you need to be able to pick it up on your own without somebody explaining ... if you need to learn by having it explained, you shouldn't try to use it.

## Answer by SRKX (score 12)

https://quant.stackexchange.com/a/1009

I honestly think that most people who could be able to answer to this question simply won't either because they actually work for Renaissance, or because they work in a top quant hedge fund and they'll keep it a secret.

I discussed this topic once during an interview and the guy said "we'll discuss this further if you get the job" lol.

About papers, I'm sure that there have been incredible findings in the industry which are just kept secret until they are found by academy, or simply become obsolete....

And honestly, it's quite fair...

## Answer by Ram Ahluwalia (score 9)

https://quant.stackexchange.com/a/1905

I have been learning more about speech recognition motivated by its application to financial forecasting. I have identified a couple connect points. Turns out each of these tools can and are regularly used in financial modeling as well.

- Use of Markov Models

- Use of Fourier transforms (sine/cosine decompositions)

- Use of component analysis

## Answer by Dr. Jochen L. Leidner (score 5)

https://quant.stackexchange.com/a/1146

The Brown et al. paper and its connection with trading is discussed here: http://jochenleidner.posterous.com:80/from-speech-recognition-to-trading (mirror)

## Answer by Barnaby (score 5)

https://quant.stackexchange.com/a/17433

Speech recognition signal processing is complex and possibly similar to the complexity of financial markets. They are similar as per characterictics the non stationarity, noise types and other aspects such us the existence of a spectrum etc conceptual frequency and the grammar to construct and articulate concepts is not evenly and randomly distributed; so pattern discovery can occur and therefore signal processing methods may be similar.

In language processing evolution of speech incorporates new words and old words are eliminated. In this respect financial markets are the same where signals of certain characteristics appear and have limited life spans. Once a person at Renaissance mentioned to me that since the early start of the models today only 25% of the original signals are being used while 75% of the pass signals used have lost all predicting power. This is the same as saying that 75% of the patterns encountered in the past which had statistical significance in relation to returns have since vanished. So any model if fixed as per the patterns to encounter is deemed to fail after a few years of using it. Maybe Language processing as per financial prediction is related to the discovery of new patterns which are formed or derived from previous existing patterns.

Cryptology may be in the techniques employed as to pint point the true signal from noise types and in this respect a decryption algorithm will be the equivalent of having the mechanism to automatically transform an encrypted signal into a decrypted one. Therefore is related to denoising.

## Answer by alexa (score 3)

https://quant.stackexchange.com/a/3445

As we all know, it will not be in the public unless academical research. However there is a clue. We can search for the paper that current CEO of Renaissance published. That can be worked as a stepping stone. In my opinion, voice recognition algorithm was used to eliminate noises to see the clear pattern. For specific voice to be recognized, it has to eliminate a lot of noises. Let's share the papers if anyone find written by current CEO of Renaissance. Simons has retired and the current one is the one with voice recognition, if I am not mistaken.

## Answer by bill_080 (score 2)

https://quant.stackexchange.com/a/1187

Here's a $40 million Twitter-based hedge fund from Derwent Capital Markets:

Twitter-hedge fund article

My guess is, you'll get a wide swing of opinions on this scheme.

Edit 1 (05/25/2012) =================================

Here's an update on the "Twitter Fund":

Update on Twitter hedge fund

No big surprise.