Derivation of the Student's t-Distribution [Basic Statistics I Studied #6]

English translation of the original Statistics #6 post. Source notation and the author’s original claims are preserved. Separate explanatory notes identify and correct source errors.

First, why is this distribution called the Student t-distribution?

W. S. Grosset, who worked at a brewery, first discovered and used it while working there.

Apparently, he did not want other breweries to use it in the same way, so he published it under the pen name “student t.”

First, let us look at what the t-distribution is used for.

To estimate the population mean μ, we use the sample mean Sample mean X bar . The distribution related to Sample mean X bar repeated is

and we use Z statistic .

When we do not know Population variance sigma squared , we cannot use the Z statistic,

so, as an alternative, we use , the T statistic.

The distribution followed by this T statistic is called the “t-distribution.”

The probability density function of this t-distribution is

Student t density , apparently.....

We cannot just skip past this without deriving it, can we????????

Actually, the derivation is the whole post, lol lol lol lol hehe.

Let us get into the derivation!

Rewriting T and introducing Z and U

First, let us change the form of T a little.

\frac{\bar X-\mu}{S/\sqrt n}=\frac{\bar X-\mu}{\sigma/\sqrt n}\cdot\frac{\sigma}{S}

=\frac{\bar X-\mu}{\sigma/\sqrt n}\cdot\frac{1}{S/\sigma}

=\frac{\bar X-\mu}{\sigma/\sqrt n}\cdot\frac{1}{\sqrt{\frac{nS^2}{\sigma^2}\cdot\frac1n}}

=\frac{\frac{\bar X-\mu}{\sigma/\sqrt n}}{\sqrt{\frac{nS^2/\sigma^2}{n}}}

Phew...

After rewriting it this way, the numerator is

\frac{\bar X-\mu}{\sigma/\sqrt n}=Z\sim N(0,1)

and we saw the denominator’s nS²/σ² when discussing the chi-squared distribution!

There, we proved that (n−1)S/σ² follows χ² with n−1 degrees of freedom, so we can say that nS²/σ² follows χ² with n degrees of freedom.

\frac{(n-1)S}{\sigma^2}\sim\chi^2_{n-1},\qquad\frac{nS^2}{\sigma^2}\sim\chi^2_n

Then, for that T, substitute

Z=\frac{\bar X-\mu}{\sigma/\sqrt n},\qquad U=\frac{nS^2}{\sigma^2}

Z\sim N(0,1),\qquad U\sim\chi^2_n

Preserved original img_024.png

Preserved original Korean source img_024.png

A correction to the sample-size notation

This note is separate from the author’s translated panel. Its earlier variance citation omits the square on S, and its next step incorrectly treats a change in multiplier as a change in degrees of freedom.

For N independent, identically distributed observations from a normal population with mean μ and standard deviation σ > 0, define the unbiased sample variance and degrees of freedom by

S^2=\frac1{N-1}\sum_{i=1}^{N}(X_i-\bar X)^2,\qquad\nu=N-1,\qquad N\ge2.

Then the standardized sample mean is standard normal, the scaled sample variance is chi-squared, and Z and U are independent:

Z=\frac{\bar X-\mu}{\sigma/\sqrt N}\sim N(0,1),\qquad U=\frac{(N-1)S^2}{\sigma^2}\sim\chi^2_\nu.

Therefore the sample statistic has the exact Student t distribution:

T=\frac{\bar X-\mu}{S/\sqrt N}=\frac Z{\sqrt{U/\nu}}\sim t_\nu.

Multiplying U by N/(N−1) cannot change its degrees of freedom. The scaled variable has mean N but variance

\operatorname{Var}\!\left(\frac N{N-1}U\right)=\frac{2N^2}{N-1}\ne2N,

so it does not have the chi-squared distribution with N degrees of freedom claimed in the panel. Dividing the squared deviations by N instead gives

V^2=\frac1N\sum_{i=1}^{N}(X_i-\bar X)^2,\qquad\frac{NV^2}{\sigma^2}\sim\chi^2_{N-1}.

This changes the variance estimate but still leaves N−1 degrees of freedom. With that convention, the exact t statistic is

\frac{\bar X-\mu}{V/\sqrt{N-1}}\sim t_{N-1};

using V/√N instead would give a scaled t statistic. The later density derivation is valid when its n is read as an abstract degree-of-freedom parameter ν > 0, with independent Z and U. For a normal sample of N observations, ν = N−1. H is an antiderivative of h in the differentiation step. NIST sample-mean inference.

Joint density and cumulative distribution

As we have always done, let us calculate the cumulative distribution of this guy:

T=\frac{Z}{\sqrt{U/n}}

Since Z and U are statistically independent,

their joint probability density function (joint probability density function) is

If “statistically independent” or “joint probability density function” is unfamiliar, check the mathematical definition and have a think... It is not difficult.

g(z,u)=\frac{1}{\sqrt{2\pi}}e^{-z^2/2}\cdot\frac{u^{n/2-1}e^{-u/2}}{2^{n/2}\Gamma(n/2)}

Now, to calculate T’s cumulative distribution:

F(t)=P(T\le t)

=P\!\left(\frac{Z}{\sqrt{U/n}}\le t\right)

=P\!\left(Z\le t\sqrt{\frac Un}\right)

Using the inequality this way...

-\infty\le z\le t\sqrt{\frac un},\qquad 0\le u\le\infty

(Because U follows χ²ₙ.) We can do the double integral this way.

F(t)=\int_0^\infty\int_{-\infty}^{t\sqrt{u/n}}g(z,u)\,dz\,du

=\int_0^\infty\int_{-\infty}^{t\sqrt{u/n}}\frac1{\sqrt{2\pi}}e^{-z^2/2}\frac{u^{n/2-1}e^{-u/2}}{2^{n/2}\Gamma(n/2)}\,dz\,du

=\frac1{\sqrt{2\pi}\,2^{n/2}\Gamma(n/2)}\int_0^\infty\int_{-\infty}^{t\sqrt{u/n}}e^{-z^2/2}u^{n/2-1}e^{-u/2}\,dz\,du

=\frac1{\sqrt{2\pi}\,2^{n/2}\Gamma(n/2)}\int_0^\infty{\color{red}\Bigl(}\int_{-\infty}^{t\sqrt{u/n}}e^{-z^2/2}\,dz{\color{red}\Bigr)}u^{n/2-1}e^{-u/2}\,du

Let us call e to the power −z²/2 “h(z).”

e^{-z^2/2}=h(z)

Preserved original img_025.png

Preserved original Korean source img_025.png

Differentiating and simplifying the density

F(t)=\frac1{\sqrt{2\pi}\,2^{n/2}\Gamma(n/2)}\int_0^\infty\left[H\!\left(t\sqrt{\frac un}\right)-H(-\infty)\right]u^{n/2-1}e^{-u/2}\,du

And now differentiate both sides with respect to t.

{\color{red}\frac d{dt}}F(t)=\frac1{\sqrt{2\pi}\,2^{n/2}\Gamma(n/2)}\int_0^\infty{\color{red}\frac d{dt}}\left[H\!\left(t\sqrt{\frac un}\right)-H(-\infty)\right]u^{n/2-1}e^{-u/2}\,du

f(t)=\frac1{\sqrt{2\pi}\,2^{n/2}\Gamma(n/2)}\int_0^\infty h\!\left(t\sqrt{\frac un}\right)\sqrt{\frac un}\,u^{n/2-1}e^{-u/2}\,du

=\frac1{\sqrt{2\pi}\,2^{n/2}\Gamma(n/2)}\int_0^\infty e^{-\frac12\frac un t^2}\sqrt{\frac un}\,u^{n/2-1}e^{-u/2}\,du

=\frac1{\sqrt{2n\pi}\,2^{n/2}\Gamma(n/2)}\int_0^\infty\sqrt u\,u^{n/2-1}e^{-\frac12\frac un t^2}e^{-u/2}\,du

=\frac1{\sqrt{2n\pi}\,2^{n/2}\Gamma(n/2)}\int_0^\infty u^{(n+1)/2-1}e^{-\frac u2(1+t^2/n)}\,du

If we let u(1+t²/n)=y,

u\left(1+\frac{t^2}n\right)=y,\qquad u=\frac y{1+t^2/n},\qquad du=\frac1{1+t^2/n}\,dy

=\frac1{\sqrt{2n\pi}\,2^{n/2}\Gamma(n/2)}\int_0^\infty\left[\frac y{1+t^2/n}\right]^{(n+1)/2-1}e^{-y/2}\cdot\frac1{1+t^2/n}\,dy

=\frac1{\sqrt{2n\pi}\,2^{n/2}\Gamma(n/2)}\cdot\frac1{(1+t^2/n)^{(n+1)/2}}\int_0^\infty y^{(n+1)/2-1}e^{-y/2}\,dy

=\frac1{\sqrt{2n\pi}\,2^{n/2}\Gamma(n/2)}\cdot\frac1{(1+t^2/n)^{(n+1)/2}}\int_0^\infty\frac{2^{(n+1)/2}\Gamma((n+1)/2)\,y^{(n+1)/2-1}e^{-y/2}}{2^{(n+1)/2}\Gamma((n+1)/2)}\,dy

=\frac{2^{(n+1)/2}\Gamma((n+1)/2)}{\sqrt{2n\pi}\,2^{n/2}\Gamma(n/2)}\cdot\frac1{(1+t^2/n)^{(n+1)/2}}\int_0^\infty\frac{y^{(n+1)/2-1}e^{-y/2}}{2^{(n+1)/2}\Gamma((n+1)/2)}\,dy

The integral’s density is the chi-squared distribution with n+1 degrees of freedom. Integrating over 0 to ∞ gives:

The integral is 1.

=\frac{\Gamma((n+1)/2)}{\sqrt{n\pi}\Gamma(n/2)}\cdot\frac1{(1+t^2/n)^{(n+1)/2}}

Done!

\therefore\ f(t)=\frac{\Gamma((n+1)/2)}{\sqrt{n\pi}\Gamma(n/2)}\cdot\frac1{(1+t^2/n)^{(n+1)/2}}

Preserved original img_026.png

Preserved original Korean source img_026.png

Now let us briefly go over the properties of the t-distribution and finish this post.

First, the t-distribution is symmetric about the origin. (It is very easy to see that it is an even function.)

Also, scholars usually say that using the t-distribution is appropriate when “the sample size is less than 30.”

(Well, apparently some say 100, and others say 10,

but we can just think of 30 as the number they use on average.)

Next, the mean and variance of the t-distribution:

Mean: E(T) = 0, n > 1. (This also says that a t-distribution with one degree of freedom has no mean.)

Variance: Var(T) = n/n−2, n > 2. (Again, t-distributions with one or two degrees of freedom have no variance!!!!)

Finally, let us briefly cover how to read a t-distribution table.

(Although we might not need to,,,)

The t-distribution table writes the (1−α) quantile as t alpha,n ,

which means Upper-tail probability alpha .

For example, if a table says t0.05,9 equals1.833 ... what does that mean?!

Hmm..... shall we do one example and finish?

(This problem comes from Walpole, et al., Probability and Statistics for Engineers and Scientists.)

Ex. 8.11 A chemical engineer claims that the yield of a batch process is 500 g per liter of raw material.

To demonstrate this, he selects and tests 25 batches every month.

It is agreed that his claim will be regarded as reasonable if the t value calculated from the test results lies between −t₀.₀₅ and t₀.₀₅.

If the results for the 25 batches give a sample mean of 518 and a standard deviation of 40 g, what conclusion can be drawn?

Assume that the population follows an approximately normal distribution.

Calculating the t value from the test results gives

With 24 degrees of freedom, t₀.₀₅ = 1.711.

Since the calculated t value is greater than t₀.₀₅, we can say that the actual yield is greater than 500 g.

In fact, I have seen the t-distribution used a lot in t-tests,

but we have not discussed testing at all yet, so solving a t-test problem here does not seem quite right.

We will talk about testing later and solve one then.

n = [1., 2., 5., 100000000000000]
for i in n:                   # i = n
    x = np.linspace(-5, 5, 100)
    y = sc.t(i).pdf(x)
    plt.plot(x, y, linewidth=2.0, label = 'n=%s' % i)
plt.grid(True)
plt.legend()
plt.ylabel('p(x)')
plt.xlabel('x')
plt.title('Student-t Distribution')
# plt.savefig('5.Student-t Distribution.jpeg')

Student t distributions for increasing degrees of freedom

As the number gets bigger and bigger and bigger, it converges to the Gaussian.

Shall we check that on a computer too!??!

n = 5.
x = np.linspace(-0.3, 0.3, 100)
for i in range(3):
    y1 = sc.t(n).pdf(x)
    plt.plot(x, y1, linewidth=2.0, label = 'n=%s' % n)
    n = n * 5
 
y2 = sc.norm(0, 1).pdf(x)
plt.plot(x, y2, linewidth=1.0, label = 'Gaussian')
 
plt.grid(True)
plt.legend()
plt.ylabel('p(x)')
plt.xlabel('x')
plt.ylim(0.37, 0.4)
plt.title('Student-t & Gaussian Distribution')
# plt.savefig('6.Student-t & Gaussian Distribution.jpeg')

After plotting it a few times, the middle seemed to catch up really, really fast,

so I adjusted the range a little.

After limiting it to that range,

let us make the number bigger and bigger again.

n = 50.
x = np.linspace(-0.3, 0.3, 100)
for i in range(3):
    y1 = sc.t(n).pdf(x)
    plt.plot(x, y1, linewidth=2.0, label = 'n=%s' % n)
    n = n * 5
 
y2 = sc.norm(0, 1).pdf(x)
plt.plot(x, y2, linewidth=1.0, label = 'Gaussian')
 
plt.grid(True)
plt.legend()
plt.ylabel('p(x)')
plt.xlabel('x')
plt.ylim(0.37, 0.4)
plt.title('Student-t & Gaussian Distribution(2)')
# plt.savefig('7.Student-t & Gaussian Distribution(2).jpeg')

Student t and Gaussian distributions second comparison

When I made n extremely large,

the curves looked completely overlapping to the eye,

so I chose a suitably large n at a sensible point......

Mean, variance and the number 30

These explanations are separate from the translated author text. The source’s ungrouped “n/n−2” means

\frac\nu{\nu-2}.

A standard t distribution has mean 0 for ν > 1 and finite variance only for ν > 2:

\operatorname{E}(T)=0\quad(\nu>1),\qquad\operatorname{Var}(T)=\frac\nu{\nu-2}\quad(\nu>2).

Symmetry alone does not give a finite mean at ν = 1. The normal-sample theorem applies at every N ≥ 2, including large samples. Thirty is a rough guideline, not a universal boundary for t inference or a guarantee that nonnormal data meet its assumptions. As ν grows, the t distribution approaches the standard normal. NIST t distribution.

About the name

The brewer’s name is William Sealy Gosset; his 1908 paper appeared under the name Student. “Grosset” and “student t” above reproduce the original author’s wording. The claim that he wanted to keep other breweries from using the method is an anecdote, not an established motive. Historical accounts discuss Guinness’s publication rules and differ about why anonymity was required. Zabell’s historical research; the original paper.

What the example supports

Here the sample size, degrees of freedom and observed statistic are

N=25,\qquad\nu=24,\qquad t=\frac{518-500}{40/\sqrt{25}}=2.25.

The source uses upper-tail notation:

t_{0.05,24}\approx1.711,\qquad P(T>t_{0.05,24})=0.05.

Its stated interval from −1.711 to 1.711 is central 90%, so the corresponding two-sided test has total significance 10%. The result lies outside that interval, rejecting μ = 500 under the stated rule. A prespecified one-sided 5% test against μ > 500 also rejects equality. This is statistical evidence for a larger population mean under the sampling assumptions, rather than proof of the true yield. A result inside the interval would not establish equality either. For a two-sided 5% test, the critical value is about 2.064; 2.25 also exceeds it. NIST table and tail conventions.

Comments