Choosing References for Stochastic and Dynamic Programming

Summary

The document distinguishes stochastic programming from stochastic dynamic programming and recommends different references for each. It identifies Birge and Louveaux’s second edition as a useful introduction to stochastic programming, highlighting its coverage of iterative and approximation methods. It also cautions against relying on the first edition because of formatting problems described by the respondent.

For stochastic dynamic programming, the answer recommends Puterman’s work on Markov decision processes, noting its theoretical depth and coverage of some continuous-time results, while expressing uncertainty about its breadth of applications. It points readers toward stochastic processes as useful background and approximate dynamic programming as a way to reformulate or relax difficult stochastic problems so they can be solved more efficiently. The recommendations are based on the respondent’s reading and judgment; the document provides no systematic comparison of the books or trading-specific applications.

Key ideas

• Stochastic programming and stochastic dynamic programming are distinct topics and may call for different references.
• Birge and Louveaux’s second edition is recommended for iterative and approximation techniques in stochastic programming.
• Markov decision process texts provide a foundation for stochastic dynamic programming, including some continuous-time theory.
• Approximate dynamic programming can reformulate or relax stochastic problems to make them more tractable.
• The recommendations reflect an individual reader’s assessment rather than a comparative evaluation.

Tags

# stochastic programming book recommendations


# stochastic programming book recommendations












Hi: Can anyone recommend an introductory book on stochastic programming ? There are obviously so many books on Amazon but I can't tell easily which ones could be useful. It would be good if it had some balance between theory and application. Thanks.

## Answer by kurtosis (score 1, accepted)

https://quant.stackexchange.com/a/57456

I think you will want a few books since the best book for stochastic programming (but not dynamic, i.e. across time) is different than the best book(s) for stochastic dynamic programming.

For stochastic programming, Birge and Louveaux's Introduction to Stochastic Programming 2nd Ed. is the book I found most helpful. It covers many iterative and approximation techniques. It hurts me to say this (since Birge is a very good human), but I would not get the first edition: it has serious flaws with formatting in a few places. So make sure to get the 2nd edition.

For stochastic dynamic programming, Puterman's Markov Decision Processes is outstanding and even has enough theory to cover some continuous-time results. The jumping off point is stochastic processes, which I found very helpful and intuitive. I'm not sure, though, if it has as much on applications as the other two books I mention here.

You should also read up on approximate dynamic programing since that often lets you relax or reframe a stochastic problem enough to solve it more efficiently. We just read papers on the topic, but since then Powell has written Approximate Dynamic Programming which appears to be very good.

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.