Deriving the Bregman Mean from Expected Divergence

Summary

The document defines a Bregman divergence from a differentiable convex function and asks how to find the point minimizing its expected divergence from a distribution. The answer writes the expected divergence as an objective in the candidate point, separates the terms that depend on that point, and differentiates. Setting the derivative to zero gives the condition that the gradient of the generator at the minimizer equals the distributional expectation of the generator's gradient. Applying the inverse gradient yields the stated Bregman mean formula.

This derivation explains the optimization step, but a critical point is not automatically a valid unique minimizer in every setting. The formula relies on the required expectations being defined and on the gradient being invertible over the relevant domain; the document's setup also specifies convexity and differentiability conditions. The result is a general statistical concept rather than a trading strategy or empirical market finding.

Key ideas

• The Bregman mean minimizes expected divergence from a distribution.
• Differentiating the objective gives a condition on the generator's gradient at the minimizing point.
• When the gradient is invertible, the minimizer is the inverse gradient applied to the expected gradient of the observations.
• The derivation assumes the expectations exist and the stated convexity and differentiability conditions hold.
• A stationary point requires suitable conditions to ensure it is a unique minimizer.

Tags

# Bregman Mean of a Distribution


# Bregman Mean of a Distribution












In a paper (link), author writes, given that $\gamma:R\rightarrow \bar{R}$ is a convex function, $dom_{\gamma}:=\{x\in R:\gamma(x)<+\infty\}$ is a non-empty open set and $\gamma$ a closed proper differentiable function in the interior of $dom_{\gamma}$, $d$ is Bregman divergence $$d_{\gamma}(x,x')=\gamma(x)-\gamma(x')-\gamma'(x')(x-x')$$ Define the Bregman mean as the unique point $b$ in the support of $\mu$ satisfying $$\int d_{\gamma}(b,x)\mu(dx)=\min_{m\in dom_{\gamma}}\int d_{\gamma}(m,x)\mu(dx)$$.

He says that it is very easy to obtain $b$ by differentiating: $b=\gamma'^{-1}[\int\gamma'(x)\mu(dx)]$.

Can anyone explain to me the definition and how he gets the formula for $b$?

## Answer by Gordon (score 3)

https://quant.stackexchange.com/a/45334

Note that \begin{align*} f(m) &= \int d_{\gamma}(m,x)\mu(dx)\\ &=\int \big[\gamma(m)-\gamma(x)-\gamma'(x)(m-x)\big]\mu(dx)\\ &=\gamma(m) - \int \big[\gamma(x)+\gamma'(x)(m-x)\big]\mu(dx). \end{align*} Then, \begin{align*} \frac{df}{dm} = \gamma'(m) - \int \gamma'(x)\mu(dx), \end{align*} and the critical point is given by \begin{align*} b = \big(\gamma'\big)^{-1}\Big( \int \gamma'(x)\mu(dx)\Big). \end{align*}

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.