Random Variables and Distributions

A random variable is not a variable and is not random: it is a measurable function on the sample space. The name is a historical accident that survives because the objects behave, notationally, like the "random quantities" of informal usage.

This page makes the definition, introduces the law (the pushforward measure that carries all the information anyone ever uses), and works through the standard apparatus — distribution functions, densities, quantiles — as consequences. The punchline is §5: the sample space $\Omega$ of kolmogorov-axioms.md drops out of every computation.

References: Billingsley, Probability and Measure, ch. 3; Durrett, Probability: Theory and Examples, §1.2; Kallenberg, Foundations of Modern Probability, ch. 1.

1. Definition

Definition. Let $(\Omega ,\mathcal{F},P)$ be a probability space. A random variable is a measurable map $X:\Omega \to \mathbb{R}$, i.e. $X^{-1}(B)\in \mathcal{F}$ for every $B\in \mathcal{B}(\mathbb{R})$. A random vector is a measurable map into ${\mathbb{R}}^n$; a random element of a measurable space $(E,\mathcal{E})$ is a measurable map into it.

By the generator criterion (measure-theory.md §7) it suffices that $\{X\le a\}\in \mathcal{F}$ for every $a\in \mathbb{R}$. The abbreviations $\{X\in B\}$ for $X^{-1}(B)$ and $P(X\in B)$ for $P(X^{-1}(B))$ are universal and used throughout.

Measurability is exactly the condition that makes the questions we want to ask be events. It is not a restriction on which functions are interesting; it is the requirement that $P$ be applicable.

Definition ($\sigma$-algebra generated by $X$). $\sigma (X)=\{X^{-1}(B):B\in \mathcal{B}(\mathbb{R})\}\subseteq \mathcal{F}$ — the smallest $\sigma$-algebra making $X$ measurable.

$\sigma (X)$ is the information carried by $X$: an event lies in it iff observing $X$ decides it. This reading is developed in conditional-expectation.md, and the following lemma is its technical backbone.

Doob–Dynkin lemma. $Y$ is $\sigma (X)$-measurable $⟺$ $Y=g(X)$ for some Borel $g$.

"Determined by $X$" and "a function of $X$" are therefore the same thing — a statement with no analogue in the informal vocabulary, and one of the clearest payoffs of the measure-theoretic definition.

2. The Law

Definition. The law (or distribution) of $X$ is the pushforward measure $P_X=X_{\ast }P$ on $(\mathbb{R},\mathcal{B}(\mathbb{R}))$:

$$P_X(B)=P(X\in B),B\in \mathcal{B}(\mathbb{R}).$$

$P_X$ is a probability measure ($X^{-1}$ preserves disjoint countable unions), so every random variable induces a probability space on $\mathbb{R}$. Conversely every probability measure on $\mathbb{R}$ is the law of some random variable — take $\Omega =\mathbb{R}$, $P=\mu$, $X=\mathrm{id}$. Laws and random variables are thus in many-to-one correspondence, and only the law is ever observable.

Two random variables with the same law are equal in distribution, written $X\stackrel{d}{=}Y$. They may live on different spaces and be quite different functions: on $\Omega =\{0,1\}$ with fair $P$, the variables $X(\omega )=\omega$ and $Y(\omega )=1-\omega$ satisfy $X\stackrel{d}{=}Y$ while $P(X=Y)=0$. Equality in distribution is much weaker than equality, and keeping the two apart is the main notational discipline of the subject.

3. Distribution Functions

Definition. The cumulative distribution function of $X$ is $F_X(a)=P(X\le a)$.

Theorem (characterization). $F:\mathbb{R}\to [0,1]$ is the CDF of some random variable iff it is (i) non-decreasing, (ii) right-continuous, and (iii) $F(-\infty )=0$, $F(+\infty )=1$. The correspondence $\mu \leftrightarrow F$ between Borel probability measures on $\mathbb{R}$ and such functions is a bijection.

Proof. ($\Rightarrow$) Monotonicity is monotonicity of $P$; right-continuity is continuity from above along $\{X\le a+1/k\}\downarrow \{X\le a\}$; the limits are continuity along $\{X\le \pm k\}$. ($\Leftarrow$) Given $F$, the half-open intervals $(a,b]$ form a $\pi$-system, $\mu ((a,b])=F(b)-F(a)$ is a premeasure on the algebra of their finite disjoint unions, and Carathéodory extends it (measure-theory.md §5). Uniqueness is the $\pi$–$\lambda$ corollary. $\square$

This is the Lebesgue–Stieltjes correspondence, and it is why a one-dimensional distribution can be handed over as a single function of a real variable. Note which direction is nontrivial: writing down $F$ is easy, but that a measure exists with that CDF is Carathéodory's theorem.

Right-continuity is a convention forced by using $\le$; the jump of $F$ at $a$ is $P(X=a)$, so $F$ is continuous at $a$ iff $a$ is not an atom.

Lebesgue's trichotomy. By the decomposition theorem (lebesgue-integral.md §7), every law on $\mathbb{R}$ splits uniquely as

$$P_X={\alpha }_{ac}{\mu }_{ac}+{\alpha }_d{\mu }_d+{\alpha }_{sc}{\mu }_{sc},$$

an absolutely continuous part (has a density), a discrete part (atoms), and a singular continuous part — continuous $F$, but concentrated on a Lebesgue-null set. The third is not pathological bookkeeping: the Cantor distribution is exactly such a law, and singular continuous spectra occur in quantum mechanics.

4. Densities

Definition. $X$ is absolutely continuous with density $p$ if $P_X\ll {\lambda }^1$, in which case

$$p=\frac{d{P}_X}{d{\lambda }^1},P(X\in B)={\int }_{B}pd{\lambda }^1$$

by Radon–Nikodym (lebesgue-integral.md §7).

So "has a density" is not a mild regularity assumption but a genuine dichotomy: densities exist exactly when the law assigns zero probability to every Lebesgue-null set. Discrete variables have no density (they charge points), the Cantor distribution has none (it charges a null set), and neither omission is repairable.

The density is unique only up to ${\lambda }^1$-a.e. equality, so pointwise statements about $p$ are meaningless unless a version has been fixed. When $F$ is $C^1$, $p=F^{\prime }$ and the usual calculus applies.

For random vectors everything is identical with ${\lambda }^n$ in place of ${\lambda }^1$; marginals are obtained by integrating out coordinates, which is Fubini (product-measures.md §3).

5. Why $\Omega$ Does Not Matter

Every probabilistic quantity attached to $X$ — its expectation, variance, moments, characteristic function, tail probabilities — is computed from $P_X$ alone. The change-of-variables formula for pushforwards,

$$E[g(X)]={\int }_{\Omega }g(X(\omega ))dP(\omega )={\int }_{\mathbb{R}}gd{P}_X,$$

proved in expectation.md §2, is the formal statement: the sample space is integrated away. Two consequences:

• One may always take $\Omega =\mathbb{R}$ (or ${\mathbb{R}}^n$, or ${\mathbb{R}}^{\mathbb{N}}$) with $X$ the identity or the coordinate maps. This canonical model is the standard device for proving existence — see extension-theorem.md.
• Conversely, a single rich enough $\Omega$ carries every distribution at once:

Theorem (quantile/Skorokhod representation). Let $U$ be uniform on $(0,1)$ and $F$ any CDF. Define the generalized inverse $F^{-}(u)=\inf \{x:F(x)\ge u\}$. Then $F^{-}(U)$ has CDF $F$.

Proof. Right-continuity and monotonicity of $F$ give $F^{-}(u)\le x⟺u\le F(x)$; hence $P(F^{-}(U)\le x)=P(U\le F(x))=F(x)$. $\square$

So $((0,1),\mathcal{B},{\lambda }^1)$ supports a random variable of every one-dimensional law — and, using a measurable bijection $(0,1)\to (0,1{)}^{\mathbb{N}}$, of every countable family at once. The apparatus of "choosing a probability space" is therefore never a real constraint, which is the sense in which $\Omega$ is scaffolding (kolmogorov-axioms.md §2).

What is not determined by the laws. The joint behaviour. $P_X$ and $P_Y$ fix every marginal question but no question about $(X,Y)$ together: the joint law $P_{(X,Y)}$ on ${\mathbb{R}}^2$ is extra data, and many joint laws have the same marginals (the set of them is the coupling problem). Independence (independence.md) is precisely the choice of the product coupling.

6. Standard Families

Named distributions are just laws on $\mathbb{R}$ that recur; nothing in the theory distinguishes them. The ones used elsewhere in this folder:

The Cauchy law earns its place by being a stable law with no expectation: it shows that the hypotheses of the limit theorems are not decoration.

7. Where This Is Used

• Independence — joint laws that factor.
• Expectation — $E[g(X)]=\int gd{P}_X$, made precise.
• Modes of convergence — convergence in distribution is convergence of the laws $P_{X_k}$, not of the functions $X_k$.
• The Kolmogorov extension theorem — the canonical model of §5 in its general form.