Duality Between Relative Entropy and Log-Exponential Expectations

Summary

The document asks for a proof of a variational identity connecting the log of an exponential expectation, called free energy, with relative entropy. For a bounded random variable, the stated relation expresses this log-expectation as the supremum, over probability measures absolutely continuous with respect to a reference measure, of the variable's expectation under the chosen measure minus that measure's relative entropy.

The accompanying answer points readers toward statistical mechanics and a related explanation, but does not prove the identity. It also speculates about a connection to selecting an equivalent martingale measure, without establishing that interpretation. The useful mathematical idea is the trade-off between favoring measures that assign more weight to high values of a random variable and penalizing departures from the reference distribution through entropy. The post leaves the derivation and its financial application open, so the suggested market interpretation should not be treated as demonstrated.

Key ideas

• The log of an exponential expectation can be represented through an optimization over probability measures.
• Relative entropy measures how much a candidate probability measure differs from a reference measure.
• The variational objective rewards higher expected values while penalizing divergence from the reference measure.
• The post asks for a proof but offers only pointers and a tentative financial interpretation.

Tags

# Answer by Magic is in the chain (score 1)


# The duality of the free energy and relative entropy used to deduce deduce the stochastic game between an agent and the market












I am reading the article Pricing via utility maximization and entropy by Richard Rouge and Nicole El Karoui. They talk about the relative entropy of a probability measure $Q$ with respect to the probability measure $P$ defined by $h(Q \vert P) := E[dQ/dP \ln(dQ/dP)]$ if $Q \ll P$, $+\infty$ else. They also talk about the concept of free energy of a random variable $B$, and this is equal to $\ln E[\exp B]$. They claim that for a bounded random variable $B$, entropy and free energy are in relation by the following duality:

\begin{align} \ln E[\exp B] = \sup_{Q \ll P} [ E^{Q}[B] - h(Q \vert P) ] \end{align}

Does anyone know an article that shows the proof? or does anyone know how to deduce this equation?

The interesting feature is that with this formula we can deduce the stochastic game between an agent and the market.

## Answer by Magic is in the chain (score 1)

https://quant.stackexchange.com/a/41768

Such relationships are commonly covered in statistical mechanics, so any decent statsictal mechanics book should help. Here is an article that gives a very nice summary of the key concepts:

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2716075/#!po=4.74138

I don’t have access to Rouge and Karoui article, but I think it is just trying to find an equivalent Martingale measure that maximises the free energy (think of it as a measure of sort of stability).

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.