Limits of Price Prediction and Where Quantitative Finance Finds Structure

# What is currently predictable in the stock and bond markets and what is not


# What is currently predictable in the stock and bond markets and what is not












Disclaimer: I have some knowledge of statistics, machine learning and probability theory, but next to zero knowledge of finance (I had to look up Wikipedia to refresh my knowledge of the difference between a bond and a stock), so please don't shoot the newbie :)

I was reading this question on Cross Validated, and I noted that some users (included the original poster) noted that the task of predicting the stock market was impossible. To be pedantic, I think they were actually referring to the simpler (?) task of predicting the future values of a specific stock price based on the stock price past values, not predicting the overall trend of the whole stock market, but you get my point. This is something which I've seen often in pop-finance books, and which is usually explained intuitively this way: if there was a model which could be used to predict reliably the future price of one or more stock, everyone would use it, this would affect the future stock exchanges and prices, changing the data generating process (the stochastic process corresponding to the stock price time series), and thus the model wouldn't work anymore. Another argument which is sometimes given is that market crises are never predicted (the subprime mortgage crisis, dot-com bubble, etc.).

However, there are companies, university degrees, research centers, etc. which work in the field of quantitative finance, so, even if the endeavor of predicting the stock and bond markets is understandably hard, there must be some degree of success. What is currently predictable with some margin of accuracy, and what is not? Concerning the level of the answer, I can follow you if you talk about expectation, stochastic processes, martingales, Monte Carlo and Markov Chain Monte Carlo, neural networks, etc.. However I've only heard about stuff such as the Black-Scholes equation, without knowing the actual details. I think I could understand if you explained me the concept, but I cannot say for sure.

## Answer by Will Gu (score 1, accepted)

https://quant.stackexchange.com/a/31634

I'll add my two cents even though there have been a couple of answers. In terms of the actual price of stock XYZ tomorrow, it's not likely to be predicable. The price is an combination of both market move and individual stock move, and all come down to supply and demand. There are market research that gives all kinds of target prices, but are mostly just used to advertise to clients.

But just like you can forecast weather based on both historical information and surrounding atmosphere, you can generate some subjective view in terms of the stock price changes (going up or down) based on your experience and observation of the current market, just like what day traders do. In this case, other people may have gotten similar view and started actions before you do, so price would have changed already. That's called factor-in or price-in.

In some scenarios, certain people can predict the stock price because of information asymmetry, e.g. knowing ahead of time the earning announcement details. Some other people may piggy-back one way or another (could even be their alphas), while most of the uninformed traders just trade randomly and get exploited. It's essentially a zero-sum game. I've heard that most of the liquidity in the US stock market comes from institutional trading, but still, some institutions are better than others.

Down to the micro-structure level, stock-price is predictable if you look at tick-by-tick. The market orders are highly autocorrelated. This is likely because of order execution. This regime is not accessible to average people though.

Not being able to predict stock price doesn't mean you can't make money by investment. The fundamental theory of investment is risk-reward trade-off. You can form/sort your portfolio based on some criteria and hope for growth better than the market portfolio (based on SP500 for example). The relative value here is probably more predictable than individual stock prices.

The quant finance people don't necessarily care about the price levels, as a lot of the theories are based risk-neutral assumptions.

In summary, I don't think that the exact level of stock/bond price is predicable, but it shouldn't affect a lot of the quant finance work.

## Answer by vonjd (score 3)

https://quant.stackexchange.com/a/31608

The renowned CXO Advisory Group has a section "What Works Best?".

Here some general information is given and many links to their research articles which e.g. summarize lots of current academic research (although most of the linked articles are behind a paywall the links to the original papers are normally provided).

The article closes with "In summary, strategic diversification and momentum and value strategies applied at the asset class level via low-fee funds (especially with momentum and value in combination) may be among the best approaches [...]"

In any case I think this is a valuable place to start. I myself have been a subscriber to their service for many years and it has helped me to keep some perspective.

## Answer by JoshK (score 2)

https://quant.stackexchange.com/a/31633

I think you have a small misunderstanding in terms of what the folks with all of the various incarnations of quantitative degrees are doing. There are always people trying to punt on the direction of the stock market for sure - and there is always an appetite with investors to try to outperform the market.

What you are missing is that humanity is net long assets as a whole. That means that people have investments that need to be managed. What do you do with those investments? How to manage them? Lend them? Optimize them to suit certain objectives (think of pensions and insurers). So many of the quantitative people in finance work on these kinds of problems.

Many banks will have teams of quantitative people working to improve the process of lending their shares and properly collateralizing them. Just as one example.

I think a lot of times quantitative people are disappointed when they get to a finance job and realize that they are not going to be predicting where a stock is going but are instead trying to solve a problem like figuring out which broker is giving the best value for their execution services.

## Answer by LazyCat (score 1)

https://quant.stackexchange.com/a/31607

I think, the popular consensus is that, while in general the stock prices are unpredictable, there are certain market regimes it's possible. That's pretty much the premise of a well-written book by Lo and MacKinlay: http://press.princeton.edu/titles/6558.html which I recommend.

In terms of ML forecasting methods, it often translates to the situation, when the most difficult part is to figure out in what particular situations to use these forecasting methods, and not so much which particular type of a regression technique or a neural net is optimal.