If \(X\sim N(\mu,\sigma^2)\), then \(Z=\frac{X-\mu}{\sigma}\sim N(0,1)\).
\(Z\) has
moment generating function \(M_Z(t)=\mathbb{E}[e^{uZ}] = e^{\frac{t^2}{2}}\)
characteristic function \(\phi_Z(t)=\mathbb{E}[e^{itZ}] = e^{-\frac{t^2}{2}}\)
and cumulant generating function \(K_Z(t)=\log M_Z(t) = \frac{t^2}{2}\).
\(X\) has
moment generating function \(M_X(t)=\mathbb{E}[e^{uX}] = e^{\mu t + \frac{\sigma^2 t^2}{2}}\)
characteristic function \(\phi_X(t)=\mathbb{E}[e^{itX}] = e^{i\mu t - \frac{\sigma^2 t^2}{2}}\)
and cumulant generating function \(K_X(t) = \log M_X(t) = \mu t + \frac{\sigma^2 t^2}{2}\).
The gamma distribution is given by the PDF
\[\,\mathrm{d}\mu_{\alpha,\beta}(t) = \frac{1_{t\ge 0}}{\Gamma(\alpha)\beta^\alpha} t^{\alpha} e^{-\frac{t}{\beta}} \frac{\mathrm{d} t}{t}\]
Its moment generating function is easily derived as
\[M_{\Gamma(\alpha,\beta)}(t) = (1-\beta t)^{-\alpha}\]
The characteristic function is given by
\[\phi_{\Gamma(\alpha,\beta)}(t) = (1-i\beta t)^{-\alpha}\]
From which we know that
\(\mathbb{E}[X] = \alpha \beta\)
\(\mathbb{E}[X^2] = \beta^2 \alpha(\alpha+1)\)
\(\mathrm{Var}(X) = \beta^2 \alpha\).
The chi-squared distribution is a special case of the gamma distribution with \(\alpha = \frac{n}{2}\) and \(\beta = 2\). But why do we want it in the first place?
Definition. The chi-squared distribution with \(p\) degrees of freedom is given by the distribution of a sum of \(p\) independent standard normal random variables, i.e., if \(Z_1,\ldots,Z_p\sim N(0,1)\) are independent, then
\[\chi^2_p \sim Z_1^2 + \ldots + Z_p^2.\]
Consider the distribution of the square of a standard normal random, we have
\[\begin{align*} \,\mathrm{d}\mu_{Z^2}(t) &\propto 1_{t\ge 0} e^{-\frac{t}{2}} \frac{\mathrm{d} t}{\sqrt{t}}\mathrm{d}t\\ &\propto 1_{t\ge 0} e^{-\frac{t}{2}} t^{\frac{1}{2}} \frac{\mathrm{d} t}{t}\\ \end{align*}\]
therefore it has the same distribution as \(\Gamma(\frac{1}{2},2)\), which is the gamma distribution with shape parameter \(\frac{1}{2}\) and scale parameter \(2\).
Therefore we can use the moment generating function for gamma distribution, we quickly see that its moment generating function is given by
\[M_{\chi^2_p}(t) = (1-2t)^{-\frac{p}{2}}.\]
For a normal distribution \(X\sim N(\mu,\sigma^2)\), if \(S^2\) is the sample variance, we can compute
\[V' = \frac{(n-1)S^2}{\sigma^2} = \sum (Z_i - \bar{Z})^2\]
where \(Z_i = \frac{X_i - \mu}{\sigma}\) are the standardized variables and \(\bar{Z}\) is the sample mean of the standardized variables.
Note that \(V' = \sum Z_i^2 - n\bar{Z}^2\), we conclude \(V' + n\bar{Z}^2 = \sum Z_i^2 \sim \chi^2_n\). Basu’s theorem tells that \(\overline{X}\) and \(S^2\) are independent, as well as \(V'\) and \(n\bar{Z}^2\). This means
\[M_{V'} M_{n\bar{Z}^2} = M_{\chi^2_n} = (1-2t)^{-\frac{n}{2}}.\]
and
\[M_{V'} = (1-2t)^{-\frac{n}{2}} / M_{n\bar{Z}^2} = (1-2t)^{-\frac{n}{2}} (1-2t)^{\frac{1}{2}} = (1-2t)^{-\frac{n-1}{2}}.\]
This proves that \(V' = \frac{(n-1)S^2}{\sigma^2} \sim \chi^2_{n-1}\).
\(V'\) and \(\overline{X} \sim N(\mu,\frac{\sigma^2}{n})\) are independent, so we can easily compute the joint distribution of \(V'\) and \(\overline{X}\).
The student t-statistic is defined as
\[T_{n-1} = \frac{\overline{X} - \mu}{S / \sqrt{n}}\]