Conditional Expectation

Elementary probability defines $P(A\mid B)=P(A\cap B)/P(B)$ and stops, because the definition is vacuous when $P(B)=0$ — and the interesting conditions ("given that the particle is at $x$", "given the path so far") are exactly the null ones. The measure-theoretic repair is to condition not on an event but on a $\sigma$-algebra, and to define the result as a Radon–Nikodym derivative.

This is the deepest definition in the folder and the one with the most foundational consequence: §6 shows that conditioning on a null event has no answer independent of the $\sigma$-algebra one conditions on, so ordinary probabilistic talk ("given that $\theta =0$") is genuinely ambiguous.

References: Williams, Probability with Martingales, ch. 9; Billingsley, Probability and Measure, §33–34; Kallenberg, Foundations, ch. 6.

1. The Elementary Case and Its Limits

For $P(B)>0$, $P(\cdot \mid B)$ is a probability measure on $(\Omega ,\mathcal{F})$ and $E[X\mid B]=E[X{1}_B]/P(B)$. Nothing here needs justification, and Bayes' rule $P(A\mid B)=P(B\mid A)P(A)/P(B)$ is immediate.

Now let $X,Y$ be jointly continuous with density $p(x,y)$ and ask for $E[X\mid Y=y]$. Every event $\{Y=y\}$ is null, so the quotient is $0/0$. The elementary fix — define $p(x\mid y)=p(x,y)/p_Y(y)$ and integrate — works, but it is a definition by fiat that depends on the chosen versions of densities and does not extend beyond the absolutely continuous case. What follows replaces it with something canonical.

2. Conditioning on a $\sigma$-Algebra

The change of viewpoint: a sub-$\sigma$-algebra $\mathcal{G}\subseteq \mathcal{F}$ represents information — the events one can decide — and $E[X\mid \mathcal{G}]$ is the best prediction of $X$ using that information. It is a random variable, not a number.

Definition/Theorem. Let $X\in L^1(\Omega ,\mathcal{F},P)$ and $\mathcal{G}\subseteq \mathcal{F}$ a sub-$\sigma$-algebra. There exists a random variable $Z$, unique up to a.s. equality, such that

1. $Z$ is $\mathcal{G}$-measurable, and $Z\in L^1$;
2. ${\int }_{G}ZdP={\int }_{G}XdP$ for every $G\in \mathcal{G}$.

We write $Z=E[X\mid \mathcal{G}]$, and $E[X\mid Y]:=E[X\mid \sigma (Y)]$. Also $P(A\mid \mathcal{G}):=E[1_A\mid \mathcal{G}]$.

Proof (existence). Assume first $X\ge 0$ and define $\nu (G)={\int }_{G}XdP$ for $G\in \mathcal{G}$. Then $\nu$ is a finite measure on $(\Omega ,\mathcal{G})$ and $\nu \ll P{|}_{\mathcal{G}}$, since $P(G)=0⟹{\int }_{G}XdP=0$. By Radon–Nikodym (lebesgue-integral.md §7) there is a $\mathcal{G}$-measurable $Z=d\nu /dP{|}_{\mathcal{G}}$ satisfying (2). For general $X$ apply this to $X^{\pm }$. Uniqueness: if $Z,Z^{\prime }$ both work then ${\int }_G(Z-Z^{\prime })dP=0$ for all $G\in \mathcal{G}$; taking $G=\{Z>Z^{\prime }\}\in \mathcal{G}$ forces $P(Z>Z^{\prime })=0$, and symmetrically. $\square$

Three readings of the same object, all used:

• Averaging. Condition (2) says $Z$ has the same integral as $X$ over every event $\mathcal{G}$ can see; $Z$ is $X$ averaged out over the unresolved variation.
• Projection. For $X\in L^2$, $E[X\mid \mathcal{G}]$ is the orthogonal projection of $X$ onto the closed subspace $L^2(\mathcal{G})\subseteq L^2(\mathcal{F})$ (hilbert-spaces.md). Condition (2) is exactly $⟨X-Z,1_G⟩=0$, i.e. orthogonality to the subspace. Hence $E[X\mid \mathcal{G}]$ minimizes $E[(X-W{)}^2]$ over $\mathcal{G}$-measurable $W$: it is the best mean-square predictor.
• Derivative. Literally a Radon–Nikodym derivative, which is where the a.s.-uniqueness (and the need to speak of versions) comes from.

"Unique up to a.s. equality" is not a technicality. $E[X\mid \mathcal{G}]$ is an equivalence class; its value at any individual $\omega$ is meaningless. Every identity below holds a.s., and §6's paradox is precisely a case where two natural version-choices disagree on a null set that one wants to ask about.

3. Properties

All hold almost surely, for $X,Y\in L^1$ and sub-$\sigma$-algebras $\mathcal{H}\subseteq \mathcal{G}\subseteq \mathcal{F}$.

The convergence theorems (monotone, Fatou, dominated) also hold in conditional form. The tower property is the one used constantly: coarser information overrides finer. It is the identity that fails for finitely additive probability (finite-additivity.md), which is a good measure of how much $\sigma$-additivity is doing here.

Two degenerate cases anchor intuition: $\mathcal{G}=\{\varnothing ,\Omega \}$ (no information) gives the constant $E[X]$; $\mathcal{G}=\mathcal{F}$ (full information) gives $X$.

4. Conditioning on a Random Variable

$E[X\mid Y]$ is $\sigma (Y)$-measurable, so by the Doob–Dynkin lemma (random-variables.md §1) there is a Borel $g$ with $E[X\mid Y]=g(Y)$ a.s.

Definition. $E[X\mid Y=y]:=g(y)$, defined for $P_Y$-a.e. $y$.

This recovers the elementary notion — and exposes its exact status. The function $g$ is determined only up to $P_Y$-null sets, so $E[X\mid Y=y]$ has no canonical value at any particular $y$ when $P(Y=y)=0$. The familiar formula falls out: with joint density $p(x,y)$,

$$g(y)={\int }_{\mathbb{R}}x\frac{p(x,y)}{p_Y(y)}dx\text{for }{p}_Y(y)>0,$$

now derived rather than stipulated, and with the version-dependence visible.

5. Regular Conditional Distributions

One would like $P(\cdot \mid \mathcal{G})(\omega )$ to be a probability measure in $A$ for each fixed $\omega$. From §2 it is only a family of a.s.-defined random variables: countable additivity holds a.s. for each sequence of disjoint sets, but the exceptional null set depends on the sequence, and there are uncountably many sequences. The two quantifiers do not commute for free.

Definition. A regular conditional distribution (r.c.d.) of $X$ given $\mathcal{G}$ is a map $\kappa :\Omega \times \mathcal{B}(\mathbb{R})\to [0,1]$ with (i) $\kappa (\omega ,\cdot )$ a probability measure for every $\omega$, (ii) $\kappa (\cdot ,B)$ a version of $P(X\in B\mid \mathcal{G})$ for every $B$.

Theorem (existence). If $X$ takes values in a standard Borel space (e.g. ${\mathbb{R}}^n$, or any Polish space with its Borel $\sigma$-algebra), a regular conditional distribution exists and is a.s. unique.

Proof idea. Build the conditional CDF on rationals, repair monotonicity and right-continuity off a single null set (countably many constraints, hence one exceptional set), and extend by the Lebesgue–Stieltjes correspondence (random-variables.md §3). $\square$

The Polish hypothesis is not removable: there are probability spaces with sub-$\sigma$-algebras admitting no r.c.d. Given one, the disintegration $P(X\in B,Y\in C)={\int }_C\kappa (y,B)d{P}_Y(y)$ holds, which is the rigorous form of "condition on $Y=y$, then average over $y$", and the basis of the conditional-density calculus and of Bayesian updating.

6. The Borel–Kolmogorov Paradox

Setup. Let $(\Theta ,\Phi )$ be a uniformly distributed point on the unit sphere. Condition on the event that the point lies on a fixed great circle. What is the conditional distribution along that circle?

• Parametrize by longitude $\varphi$ and condition on $\varphi =0$ (a meridian): the conditional density in the polar angle $\theta$ is proportional to $\frac{1}{2}\sin \theta$ — not uniform, concentrating near the equator.
• Parametrize by latitude and condition on the equator $\theta =\pi /2$: the conditional distribution is uniform in $\varphi$.

Both circles are great circles, congruent under a rotation that preserves the uniform law. Both events have probability zero. The answers differ.

Resolution. There is no error: conditioning is defined relative to a $\sigma$-algebra, not to an event. "Condition on the meridian" means condition on $\sigma (\Phi )$ and evaluate the resulting function at $\varphi =0$; "condition on the equator" means condition on $\sigma (\Theta )$ and evaluate at $\theta =\pi /2$. These are different sub-$\sigma$-algebras, so there is no reason for the answers to agree — and by §4 the value at any single null point is only defined up to a null set of the conditioning variable anyway. The limiting procedures implicit in the two descriptions ("thin strips around the circle") shrink in genuinely different ways, and the Jacobian of the parametrization is what differs.

The moral for the foundations: the axioms do not determine what "given that $E$" means when $P(E)=0$; a conditioning family must be supplied as part of the model, exactly as a measure must be supplied in Bertrand's paradox §4. The two paradoxes are the same phenomenon at different stages — underdetermination by verbal description, cured only by specifying a mathematical object.

Where the null event arises as a limit of positive-probability events in a specified way (a filtration, a limiting experiment), the ambiguity disappears. That is why martingales, which fix the conditioning family in advance, are the right framework for sequential conditioning.

7. Where This Is Used

• Martingales — $E[X_{n+1}\mid {\mathcal{F}}_n]=X_n$; the tower property is the whole theory.
• Exchangeability and de Finetti — the representing mixture is a conditional distribution given the exchangeable $\sigma$-algebra.
• Countable vs. finite additivity — non-conglomerability is failure of the tower property.
• The Lebesgue integral §7 — Radon–Nikodym, the engine of §2.