Probability

Probability from the foundations. The organizing question of this folder is not how do I compute this expectation but what is a probability, mathematically, and what do the axioms actually entail — where the definitions come from, which results are theorems about the measure rather than facts about the world, and what the alternatives to the standard package look like.

The whole subject rests on one identification, due to Kolmogorov in 1933:

$$\text{a probability is a measure of total mass }1.$$

Everything else — random variables, expectation, independence, conditioning, the limit theorems — is measure theory with a change of vocabulary, and the vocabulary is worth learning precisely because it makes the right theorems obvious.

Prerequisites

This folder assumes measure theory, the Lebesgue integral, and product measures, and never re-derives them. $\sigma$-algebras, countable additivity, measurable functions, the convergence theorems, Radon–Nikodym, and Fubini are all cited from there. Readers coming from the physics tree who want only the dictionary can start at quantum probability and follow the links backwards.

Contents

A. The Kolmogorov axioms

1. Kolmogorov's Axiomatization — the triple $(\Omega ,\mathcal{F},P)$, what the axioms fix and what they leave free, the classical and geometric models recovered, and Bertrand's paradox.
2. Random Variables and Distributions — random variables as measurable maps, the law as a pushforward, CDFs, densities, and why $\Omega$ almost never matters.
3. Independence — independence of events, variables, and $\sigma$-algebras as a product-measure condition; Borel–Cantelli.
4. Expectation — expectation as an integral, the standard inequalities, moments, characteristic functions, and concentration (Chernoff/Hoeffding).
5. Conditional Expectation — conditioning on a $\sigma$-algebra via Radon–Nikodym, regular conditional probability, and the Borel–Kolmogorov paradox.

B. Limit theorems

6. Modes of Convergence — a.s., in probability, in $L^p$, in distribution; the implication lattice, uniform integrability, weak convergence and tightness.
7. Laws of Large Numbers — the weak and strong LLN, and the ergodic theorem as their generalization.
8. The Central Limit Theorem — Lindeberg–Lévy via characteristic functions, Lindeberg–Feller, Berry–Esseen, stable laws.
9. Zero–One Laws — Kolmogorov's tail law and Hewitt–Savage: whole classes of questions decided by the structure of the measure alone.

C. Processes and existence

10. The Kolmogorov Extension Theorem — consistent finite-dimensional distributions determine a process; cylinder $\sigma$-algebras and what they cannot see.
11. Martingales — filtrations as information, stopping times, optional stopping, and the convergence theorem.
12. Brownian Motion — existence, Wiener measure, non-differentiable paths.

D. Foundations and alternatives

13. Exchangeability and de Finetti — symmetric laws are mixtures of i.i.d. laws.
14. Countable vs. Finite Additivity — what $\sigma$-additivity buys and what merely finitely additive probability costs.
15. Algorithmic Randomness — randomness as a property of an individual sequence: Kolmogorov complexity and Martin-Löf tests.
16. Non-Standard Probability — hyperfinite spaces, Loeb measure, and infinitesimal probabilities.
17. Quantum Probability — probability on a projection lattice, Gleason's theorem, POVMs, and the Born rule.
18. Why This Matters — The Physics Bridge — statistical mechanics, typicality, randomized algorithms.

Dependency graph

graph TD
  MT[../04-analysis/14-measure-theory] --> K[1 kolmogorov-axioms]
  LI[../04-analysis/15-lebesgue-integral] --> E[4 expectation]
  PM[../04-analysis/16-product-measures] --> IND[3 independence]
  K --> RV[2 random-variables]
  RV --> IND
  RV --> E
  E --> CE[5 conditional-expectation]
  LI --> CE
  E --> CM[6 convergence-modes]
  CM --> LLN[7 laws-of-large-numbers]
  IND --> LLN
  CM --> CLT[8 central-limit-theorem]
  E --> CLT
  IND --> Z[9 zero-one-laws]
  PM --> EX[10 extension-theorem]
  CE --> M[11 martingales]
  EX --> BM[12 brownian-motion]
  M --> BM
  EX --> DF[13 exchangeability-de-finetti]
  LLN --> DF
  K --> FA[14 finite-additivity]
  LLN --> AR[15 algorithmic-randomness]
  BM --> NS[16 nonstandard-probability]
  E --> QP[17 quantum-probability]
  ST[../06-functional-analysis/04-spectral-theorem] --> QP
  QP --> PB[18 physics-bridge]
  LLN --> PB

Reading order

Read A → B in order; that is the spine, and pages 1–8 are what "knowing probability" means. Part C is needed only once processes in continuous time or infinite sequences with dependence are in play. Part D is independent of C and can be read directly after B — it is where this folder differs from a standard course, and where the axioms of page 1 are tested against the alternatives.

Scope

• No measure theory. Developed in 04-analysis/, cited here.
• No statistics. Estimation, hypothesis testing, and inference are a different subject and are not covered.
• No stochastic calculus. The Itô integral is stated in brownian-motion.md and goes no further.
• No interpretations. Frequentism, propensity, subjective Bayesianism, Dutch-book and Cox-style justifications, and the Principal Principle are philosophy, not mathematics, and are deliberately absent. Where a theorem is commonly enlisted in that debate — the strong law, de Finetti's theorem — the page states precisely what is and is not proved, and stops. That debate is taken up in Philosophy of Probability, which cites this folder throughout and re-proves none of it.