Jacobian transformation and uniform priors in Bayesian inference

This blog post describes the Jacobian transformation and how a uniform prior on a parameter can turn into a non-uniform prior for related parameters.

The Jacobian transformation is an algebraic method for determining the probability distribution of a variable $y$ when we know the probability distribution for $x$ and some transfer function which relates $x$ and $y$.

Let $x$ be a variable with probability density function $f(x)$ and
$$F(x) = \int f(x)dx,$$ and
$$F(y) = \int f(y)dy.$$ We can determine f(y) as
$$f(y) = J(x, y)f(x).$$ where $J(x, y)$ is the Jacobian. In this example, the Jacobian is given by
$$J(x, y) = \left|\frac{\partial x}{\partial y}\right|.$$ Now assume you perform a likelihood analysis and you put a uniform prior on a parameter $\alpha$. The uniform prior on $\alpha$ between $\alpha_{\rm low}$ and  $\alpha_{\rm high}$ means that
$$f(\alpha) = \begin{cases}    1 & \text{ for } \alpha_{\rm low} < \alpha < \alpha_{\rm high},\\    0 & \text{ otherwise }. \end{cases}$$ The result of the analysis will be some posterior distribution for $\alpha$. Now assume you want to use that likelihood to infer something about another parameter $\beta = \frac{1}{\alpha}$. The uniform prior which was applied to $\alpha$ is now distorted and we can calculate it using the Jacobian transformation above.

First, we calculate the Jacobian
$$\left|\frac{\partial \alpha}{\partial \beta}\right| = \frac{1}{\beta^2},$$ which together with $f(\alpha)$ results in
$$f(\beta) = \frac{1}{\beta^2}.$$

Figure 1: A flat prior on $\alpha$ transfers into a strongly scale-dependent prior on $\beta = 1/\alpha$.
While the prior on $\alpha$ gives the same weight to all scales between $1.5$ and $10$,
in $\beta$ much more weight is given to large values of $\beta$.

Figure 1 shows the prior for $\alpha$ and $\beta$ using $1.5 < \alpha < 10$. This means that the uniform prior on $\alpha$ now looks nowhere near a uniform prior on $\beta$. Given that Bayesian inference always assumes some prior, one has to take into account how priors change in parameter transformations.

Please leave comments/questions below.
best
Florian