Conditional Densities and the Joint Distribution of Dependent Variables
Summary
The document compares two ways of expressing a joint probability density. For independent variables, it gives the familiar factorization into marginal densities. For dependent variables, it describes the chain rule factorization into conditional densities, where each variable is conditioned on the preceding variables. A three-variable expression illustrates that this construction uses the full relevant history of earlier variables.
It then defines the sum of variables up to each index and proposes a product of densities conditioned only on those partial sums. The central question is whether this product can replace the joint density or convey equivalent information. The document does not include an answer or a proof. In general, conditioning on a partial sum is not the same as conditioning on the full vector of previous variables; equivalence would require additional conditions. The piece therefore raises a useful distinction between conditional representations but does not establish a general identity.
Key ideas
- Independent variables have a joint density that factors into their marginal densities.
- Dependent variables can be represented using a chain of conditional densities.
- The proposed product conditions each variable on a preceding partial sum rather than the full preceding vector.
- The document asks whether these representations are equivalent but does not provide a resolution.
Tags
Full text
# Finding a representation for Joint probability density function of dependent random variables
# Finding a representation for Joint probability density function of dependent random variables
Suppose that $\underline{\mathbf{X}}=(X_1, X_2, ..., X_n)$ is a vector of independent random variables. Then, we know that the joint probability density function $f_{\underline{\mathbf{X}}}(x_1, x_2, ..., x_n)$ is given by the following relation
\begin{equation} f_{\underline{\mathbf{X}}}(x_1, x_2, ..., x_n) = \prod_{j=1}^{n}f_{X_j}(x_j) \end{equation} Now, assume that the above random vector contains dependent random variables, in which case the joint probability density function is given by
\begin{equation} f_{\underline{\mathbf{X}}}(x_1, x_2, ..., x_n) = \prod_{j=1}^{n}f_{{X_j}|\underline{\mathbf{X}_{j-1}}}(x_j) \end{equation} where ${X_j}|\underline{\mathbf{X}_{j-1}}$ denotes the conditional distribution of $X_j$ given the random vector $\underline{\mathbf{X}_{j-1}} = (X_i, X_2, ..., X_{j-1})$. In a simple languhge, for $n=3$, we have that \begin{equation} f_{\underline{\mathbf{X}}}(x_1, x_2, x_3) = f_{X_3|X_1, X_2}(x_3)\times f_{X_2|X_1}(x_2) \times f_{X_1}(x_1) \end{equation} We define a new random variable $T_n = \sum_{i=1}^{n}X_i$. Let's consider the following arbitrary function: \begin{equation} G = \prod_{j=1}^{n}f_{X_j|T_{j-1}}(x_j) \end{equation} where $f_{X_j|T_{j-1}}$ denotes the conditional distribution of $X_j$ given $T_{j-1}$. For example, for $n=3$, we have that
\begin{equation} G = f_{X_3|T_2}(x_3)\times f_{X_2|T_1} \times f_{X_1}(x_1), \end{equation} My question is as follows:
If we consider the situation where $X_j's$ are dependent, for which we use the second relation for the joint density function, then what we can say about the relationship between $G$ and $f$. Is it possible to replace them with each other? Can one say that the identity holds between $G$ and $f$ in terms of the information we get from them?Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.