The Kolmogorov Extension Theorem

Everything so far has assumed the probability space handed to us. For finitely many variables this is harmless — put the joint law on ${\mathbb{R}}^n$ and take coordinates. For an infinite family it is a real existence question: "let $(X_t{)}_{t\ge 0}$ be a process with these finite-dimensional distributions" is not a definition until some $(\Omega ,\mathcal{F},P)$ carries it.

The Kolmogorov extension theorem answers it in the greatest possible generality: consistent finite-dimensional distributions always come from a process. It is the dependent generalization of the infinite product measure (product-measures.md §5), and §4 explains the price it exacts — the resulting $\sigma$-algebra cannot see path properties.

References: Kallenberg, Foundations, ch. 6; Billingsley, Probability and Measure, §36; Durrett, Probability, §A.3.

1. Stochastic Processes and Finite-Dimensional Distributions

Definition. A stochastic process indexed by a set $T$ is a family $(X_t{)}_{t\in T}$ of random variables on a common probability space. For fixed $\omega$, the map $t\mapsto X_t(\omega )$ is a sample path.

A process is thus a single random element of the path space ${\mathbb{R}}^T$, and it is a matter of emphasis whether one thinks of a family of variables or one path-valued variable.

Definition. The finite-dimensional distributions (fdds) of $(X_t)$ are the laws ${\mu }_{t_1,\dots ,t_n}$ of the vectors $(X_{t_1},\dots ,X_{t_n})$ on ${\mathbb{R}}^n$, for all finite $\{t_1,\dots ,t_n\}\subseteq T$.

These are what a model actually specifies (a random walk by its step law, Brownian motion by its Gaussian increments). They cannot be arbitrary:

Consistency (Kolmogorov's conditions).

1. Permutation. For a permutation $\pi$, ${\mu }_{t_{\pi (1)},\dots ,t_{\pi (n)}}$ is the pushforward of ${\mu }_{t_1,\dots ,t_n}$ under the corresponding coordinate permutation.
2. Marginalization. ${\mu }_{t_1,\dots ,t_{n-1}}(B)=>{\mu }_{t_1,\dots ,t_n}(B\times \mathbb{R})$.

Both say the specifications must agree wherever they overlap. They are obviously necessary; the theorem says they are sufficient.

2. Cylinder Sets and the Product $\sigma$-Algebra

Definition. On ${\mathbb{R}}^T$, a cylinder set is $\{x:(x_{t_1},\dots ,x_{t_n})\in B\}$ for finite $\{t_i\}$ and $B\in >\mathcal{B}({\mathbb{R}}^n)$. The product $\sigma$-algebra $\mathcal{B}(\mathbb{R}{)}^{\otimes T}$ is generated by them.

Cylinders form an algebra and a $\pi$-system, which is exactly what the construction and uniqueness machinery of measure-theory.md needs.

3. The Theorem

Theorem (Kolmogorov, 1933). Let $T$ be an arbitrary index set and $\{{\mu }_{t_1,\dots ,t_n}\}$ a consistent family of Borel probability measures on the spaces ${\mathbb{R}}^n$. Then there is a unique probability measure $P$ on $({\mathbb{R}}^T,\mathcal{B}(\mathbb{R}{)}^{\otimes T})$ whose coordinate maps $X_t(x)=x_t$ have exactly these finite-dimensional distributions.

Proof sketch. Define $P$ on cylinders by $P(\text{cyl})={\mu }_{t_1,\dots ,t_n}(B)$; consistency makes this well defined (a cylinder has many representations). Finite additivity is immediate. The work is countable additivity on the algebra of cylinders, and it is here that a topological hypothesis enters: given cylinders $C_k\downarrow \varnothing$ with $P(C_k)\ge \epsilon$, one uses inner regularity of the $\mu$'s (measure-theory.md §6) to shrink each $C_k$ to a compact-based cylinder, and compactness then produces a point in ${\bigcap }_k{C}_k$ — a contradiction. Carathéodory extension finishes; uniqueness is $\pi$–$\lambda$. $\square$

What the hypotheses are for. Nothing is assumed about $T$: it may be uncountable, unordered, anything. The assumption is on the state space — the compactness argument needs the $\mu$'s to be Radon, which holds for Borel measures on Polish spaces (so ${\mathbb{R}}^n$, ${\mathbb{R}}^{\infty }$, any complete separable metric space) but can fail for exotic state spaces. This is the same standard-Borel hypothesis that regular conditional distributions require (conditional-expectation.md §5), and for the same reason.

Special cases worth naming:

• $T=\mathbb{N}$ with product fdds: the infinite product measure, i.e. existence of i.i.d. sequences (independence.md §3).
• $T=\mathbb{N}$ with fdds built from an initial law and a transition kernel: existence of Markov chains — the fdds ${\mu }_{1,\dots ,n}(d{x}_1\cdots d{x}_n)=\nu (d{x}_1)K(x_1,d{x}_2)\cdots K(x_{n-1},d{x}_n)$ are consistent by construction.
• $T=[0,\infty )$ with Gaussian fdds of covariance $\min (s,t)$: a process with the finite-dimensional distributions of Brownian motion (brownian-motion.md) — but see §4.

4. What the Product $\sigma$-Algebra Cannot See

Proposition. Every $A\in \mathcal{B}(\mathbb{R}{)}^{\otimes T}$ depends on only countably many coordinates: there is a countable $S\subseteq T$ such that membership in $A$ is determined by $(x_t{)}_{t\in S}$.

Proof. The sets with this property form a $\sigma$-algebra (a countable union of countable index sets is countable) containing all cylinders. $\square$

For uncountable $T$ this is fatal to path questions. The sets

$$\{x\in {\mathbb{R}}^{[0,1]}:x\text{ is continuous}\},\{x:\underset{t\in [0,1]}{\sup }{x}_t\le c\}$$

each depend on all uncountably many coordinates and are therefore not measurable in the product $\sigma$-algebra. Kolmogorov's theorem hands us a process whose fdds are those of Brownian motion, but on a space where "the path is continuous" is not an event and has no probability.

Two standard repairs:

Definition. $X$ is a version (or modification) of $X$ if $P(X_t=X_t)=1$ for each $t$ separately. Versions have identical fdds but may have completely different path properties — one may be continuous everywhere and the other nowhere.

Kolmogorov–Chentsov continuity criterion. If for some $\alpha ,\beta ,C>0$

$$E[|X_t-X_s{|}^{\alpha }]\le C|t-s{|}^{1+\beta }\text{for all }s,t,$$

then $X$ has a version with locally Hölder-continuous paths of every exponent $\gamma <\beta /\alpha$.

Alternatively, one abandons ${\mathbb{R}}^T$ and constructs the measure directly on a path space with a decent topology — Wiener measure on $C[0,\infty )$ with its Borel $\sigma$-algebra, where continuity is automatic and the sup is measurable. Both routes are taken in brownian-motion.md.

The foundational reading. "The process exists" is ambiguous. Kolmogorov's theorem gives existence of a law on coordinates, which is all the finite-dimensional specification can determine. Anything about paths — continuity, unboundedness, hitting times — is additional structure, obtained by choosing a version or a different carrier space, and is not determined by the fdds at all. Two processes with identical fdds can differ on every path property one cares about.

5. Where This Is Used

• Brownian motion — the fdds are consistent by §3, but the interesting content is precisely §4's repair.
• Martingales — filtrations index information by time; the process must exist first.
• Exchangeability and de Finetti — an exchangeable family is a consistent fdd specification invariant under permutations.
• Product measures §5 — the independent special case, proved there.