Why the OLS Residual-to-Outcome Density Jacobian Is One
Summary
The document explains a change of variables used when writing a likelihood for a linear regression with normally distributed errors. In the stated model, the error is the outcome minus the fitted linear expression. When the predictors and regression coefficients are held fixed, differentiating that residual with respect to the outcome gives one, so the absolute Jacobian factor in the density transformation is one.
The replies motivate this by noting that a one-unit change in the outcome produces a one-unit change in the residual, while the predictors are treated as fixed in this conditional calculation. This is a basic likelihood and econometrics explanation rather than a trading method. The question’s displayed density appears to omit the square on the residual in the normal exponent, and the discussion does not develop the full likelihood or address cases where predictors are modeled jointly with the outcome. Its takeaway is limited to the stated regression setup and change-of-variable step.
Key ideas
- The regression error is defined as the outcome minus its fitted value.
- With predictors and coefficients held fixed, the residual changes one-for-one with the outcome.
- The absolute derivative in the density transformation is therefore one.
- The explanation concerns a conditional OLS likelihood setup, not a joint model of predictors and outcome.
Tags
Full text
# maximum likelihood pdf
# maximum likelihood pdf
I am looking at the topic maximum likelihood, and I cannot understand why we set the pdf of $y_{t}$ equal to 1. It is with regards to a OLS example.
The information i got is this:
Model: $y_{t}=\beta_{0}+\beta_{1} x_{1, t}+\beta_{2} x_{2, t}+u_{t}$ where $u_{t} \sim N\left(0, \sigma^{2}\right)$
We want to estimate the parameter $\theta=\left\{\beta_{0}, \beta_{1}, \beta_{2}, \sigma^{2}\right\}$
The PDF of $u_{t}$ = $f\left(y_{t}\right)=\frac{1}{\sqrt{2 \pi \sigma^{2}}} \exp \left[-\frac{\left(u_{t}\right)^{2}}{2 \sigma^{2}}\right]$
We thus get that PDF of $y_{t}$ is: $f\left(y_{t}\right)=f\left(u_{t}\right)\left|\frac{\partial u_{t}}{\partial y_{t}}\right|=\frac{1}{\sqrt{2 \pi \sigma^{2}}} \exp \left[-\frac{\left(y_{t}-\beta_{0}-\beta_{1} x_{1, t}-\beta_{2} x_{2, t}\right)}{2 \sigma^{2}}\right]$
Where we use $\frac{\partial u_{t}}{\partial y_{t}}=\frac{\partial}{\partial y_{t}}\left[y_{t}-\beta_{0}-\beta_{1} x_{1, t}-\beta_{2} x_{2, t}\right]=1$
I dont understand why this equals 1, do we just set it equal to 1?
## Answer by Valometrics.com (score 0)
https://quant.stackexchange.com/a/50894
$y_t$ is independent from $x_{1,t}$ and $x_{2,t}$ by definition, otherwise $u_t$ should be deterministic.
It means that $\frac{dx_{1,t}}{dy_t}=0$ and $\frac{dx_{2,t}}{dy_t}=0$. All betas are constant so $\frac{du_t}{dy_t}=1$.
## Answer by demully (score 0)
https://quant.stackexchange.com/a/61302
Because u = y - b0 - b1x1 - b2x2, by definition. So any derivatives must surely follow from this...
I suspect your problem here lies in the econometric distinction of “reality” where the above holds true; versus the model, where you are estimating betas and hoping normally-distributed residuals etc.
Seen thus, changing actual-Y will also change your residual by 1 times this - given any current model. And thus your model error, given the equivalence above.
## Answer by Dave Harris (score 0)
https://quant.stackexchange.com/a/68829
I think that you are overthinking it.
If $y=\beta_0+\beta_1x_1+\beta_2x_2+u$ note that a 1 unit increase in $u$ results in a 1 unit increase in $y$. The same is true for a two unit causing a two unit change or negative seven unit change causing a negative seven unit change. That is also true in the opposite direction.
$u=y-\beta_0-\beta_1x_1-\beta_2x_2$
Neither $x_1$ nor $x_2$ are a function of $y$, so $$\frac{\partial{}u}{\partial{y}}=1-0-0-0$$
The derivative of $y$ with respect to $y$ is one.Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.