The Central Limit Theorem

The law of large numbers says ${\bar{X}}_n\to \mu$; the central limit theorem describes the error. Rescaled by $\sqrt{n}$ — the only scaling that gives a non-degenerate limit — the fluctuation converges in distribution to a Gaussian, whatever the underlying law, provided only that it has finite variance.

The universality is the striking part, and the proof explains it: the characteristic function of a normalized sum depends on the summand law only through its first two moments, to the order that survives the limit.

References: Durrett, Probability, ch. 3; Billingsley, Probability and Measure, §27; Feller, vol. II, ch. XV–XVII.

1. Statement

Lindeberg–Lévy CLT. Let $X_1,X_2,\dots$ be i.i.d. with mean $\mu$ and variance $0<{\sigma }^2<\infty$, and $S_n={\sum }_{i\le n}{X}_i$. Then

$$\frac{S_n-n\mu }{\sigma \sqrt{n}}⟹N(0,1),$$

i.e. $P(\frac{S_n-n\mu }{\sigma \sqrt{n}}\le a)\to \Phi (a)$ for every $a\in \mathbb{R}$.

Convergence is in distribution (convergence-modes.md) and nothing stronger is true: the normalized sums do not converge a.s. or in probability to anything (the law of the iterated logarithm shows they oscillate). Because the limiting CDF $\Phi$ is continuous, convergence holds at every point, and in fact uniformly (Pólya).

The scaling is forced. $\mathrm{Var}(S_n-n\mu )=n{\sigma }^2$, so dividing by $n^{\alpha }$ gives variance $n^{1-2\alpha }{\sigma }^2$: for $\alpha >1/2$ the limit is $0$ (that is the law of large numbers), for $\alpha <1/2$ the variance blows up, and $\alpha =1/2$ is the unique exponent leaving a non-trivial limit.

2. Proof by Characteristic Functions

Proof. Assume $\mu =0$, $\sigma =1$ (replace $X_i$ by $(X_i-\mu )/\sigma$). Let $\phi ={\phi }_{X_1}$. Since $E[X_1^2]<\infty$, expectation.md §5 gives the two-term expansion

$$\phi (t)=1-\frac{t^2}{2}+o(t^2),t\to 0.$$

By independence the characteristic function of $S_n/\sqrt{n}$ is $\phi (t/\sqrt{n}{)}^n$, so

$${\phi }_{S_n/\sqrt{n}}(t)={\left(1-\frac{t^2}{2n}+o\left(\frac{t^2}{n}\right)\right)}^n⟶e^{-t^2/2},$$

the elementary limit $(1+z/n+o(1/n){)}^n\to e^z$. Since $e^{-t^2/2}$ is the characteristic function of $N(0,1)$ and is continuous at $0$, Lévy's continuity theorem (expectation.md §5) upgrades pointwise convergence of the transforms to convergence in distribution. $\square$

Where universality comes from. The summand law enters only through $\phi$'s Taylor coefficients at $0$; normalization kills the constant and the linear term, the quadratic term becomes the Gaussian exponent, and everything of higher order is $o(t^2/n)$ and vanishes. Two distributions with the same first two moments are indistinguishable at this resolution — that is the theorem, and it is also its limitation: the CLT says nothing about the tails at fixed $n$ (Berry–Esseen does).

3. Beyond i.i.d.

Lindeberg–Feller CLT. For each $n$ let $X_{n,1},\dots ,X_{n,n}$ be independent with mean $0$ and ${\sum }_{i}E[X_{n,i}^2]\to {\sigma }^2$. If the Lindeberg condition holds — for every $\epsilon >0$,

$$\sum _{i=1}^{n}E\left[X_{n,i}^2{1}_{\{|X_{n,i}|>\epsilon \}}\right]⟶0$$

— then ${\sum }_i{X}_{n,i}\Rightarrow N(0,{\sigma }^2)$.

The Lindeberg condition is a uniform-smallness requirement: no single summand contributes a non-negligible share of the variance. It is essentially necessary (Feller's converse, under an asymptotic-negligibility hypothesis), and it is why the CLT is not a licence to assume normality for any sum: a sum dominated by one heavy term is not Gaussian. Lyapunov's condition (${\sum }_{i}E|X_{n,i}{|}^{2+\delta }\to 0$) is a convenient sufficient version.

Versions also hold with independence weakened to a martingale-difference structure (martingales.md) or to sufficiently fast mixing, which is what makes the Gaussian ubiquitous in practice rather than merely in the i.i.d. idealization.

4. Quantitative Forms

Berry–Esseen (stated). If additionally $\rho =E|X_1-\mu {|}^3<\infty$, then

$$\underset{a}{\sup }\left|P\left(\frac{S_n-n\mu }{\sigma \sqrt{n}}\le a\right)-\Phi (a)\right|\le \frac{C\rho }{{\sigma }^3\sqrt{n}},$$

with an absolute constant $C$ (best known $<0.47$).

The rate $n^{-1/2}$ is sharp. Note that this bounds the error uniformly in $a$, which means it is weak exactly where one often wants it: in the far tails, where both terms are tiny and the relative error can be enormous. Tail estimates come instead from large-deviation theory (Cramér's theorem, giving exponential rates) or directly from the concentration bounds of expectation.md §6 — one reason Chernoff/Hoeffding, not the CLT, is what a rigorous randomized-algorithm analysis uses.

5. Failure and Generalization: Stable Laws

The finite-variance hypothesis is essential. For heavy tails the correct scaling and limit change.

Definition. A law is stable if for i.i.d. copies, $X_1+\cdots +X_n$ has the same law as $a_n{X}_1+b_n$ for some constants. Stable laws are parametrized by an index $\alpha \in (0,2]$, with $\alpha =2$ the Gaussian.

Generalized CLT. If $P(|X_1|>x)\sim c{x}^{-\alpha }$ with $0<\alpha <2$ (regularly varying tails), then $n^{-1/\alpha }(S_n-b_n)$ converges to an $\alpha$-stable law, not a Gaussian.

The Cauchy law ($\alpha =1$) is the standard example: ${\bar{X}}_n$ is Cauchy for every $n$, so no scaling produces a Gaussian and there is no law of large numbers either. The Gaussian is thus the unique stable law with finite variance, and the CLT is the statement that it attracts everything in its domain — universality with a precise boundary, not a general fact about sums.

Why the Gaussian is a fixed point. Stability says the family is closed under convolution up to affine rescaling; in transform terms, $\phi (t{)}^n=>\phi (a_{n}t)e^{i{b}_{n}t}$. For $\phi =e^{-t^2/2}$ this holds with $a_n=>\sqrt{n}$, which is the same computation as §2 read backwards. The CLT is the statement that this fixed point is attracting on the finite-variance domain — a renormalization-group argument in all but name, and the reason the same Gaussian shows up in Brownian motion, diffusion, and free-field propagators.

6. Where This Is Used

• Brownian motion — the CLT is Donsker's invariance principle in miniature: rescaled random walks converge to Wiener measure.
• Modes of convergence — the whole statement is about $\Rightarrow$.
• Expectation §5 — characteristic functions and Lévy continuity, the machinery used here.
• Complex analysis — the saddle-point method is the analytic cousin of §2's expansion, and gives the local (density) form of the CLT.