Logical Probability and Inductive Logic

The logical interpretation holds that probability is an objective relation between propositions: $P(h\mid e)=r$ says that the evidence $e$ supports the hypothesis $h$ to degree $r$, in the same impersonal way that a valid argument's premises support its conclusion. Deductive entailment is the limiting case $r=1$, contradiction the case $r=0$, and the interval between them is the province of a discipline the tradition called inductive logic.

The ambition is worth stating at full strength, because it is what makes the program interesting and its failure instructive. If the view is right, then given a body of evidence there is exactly one rational degree of belief in any hypothesis, and it is fixed by logic alone. Disagreement between two people with the same evidence would be a logical error, as demonstrable as a mistake in arithmetic. Induction would have the same kind of objectivity as deduction, and the problem of induction would be, if not solved, at least relocated into a discipline with rules.

This page sets out the thesis, its relation to logicism in the philosophy of mathematics, the tradition that developed it, the five results that broke it, and what survived.

The thesis

Take the sentences of some language and consider the relation "$e$ confirms $h$ to degree $r$". The logical interpretation makes four claims about it:

1. The relation is objective — it holds or fails independently of anyone's beliefs, like entailment.
2. It is a priori — determinable by analysis of the propositions, without empirical investigation.
3. It is unique — for a given $e$ and $h$ there is one correct $r$.
4. It is normative for belief — an agent whose evidence is exactly $e$ ought to have credence $r$ in $h$.

Claims 1–3 are semantic and 4 is epistemic; a position can hold the first three and reject the fourth, but nobody in the tradition wanted to, since the point of the enterprise was to guide inference.

The relation is not the material conditional and not entailment, and it behaves in ways deductive logic does not prepare one for. Most importantly it is non-monotonic: adding premises can lower the degree of support, whereas in deduction, if $e$ entails $h$ then $e\wedge f$ entails $h$ for any $f$. That evidence can be defeated by further evidence is the characteristic feature of inductive support, and any formal treatment must accommodate it.

Relation to logicism

The parallel with logicism in the philosophy of mathematics — Frege's and Russell's thesis that arithmetic reduces to logic — is real, historically connected, and easy to overstate. It is worth pinning down, since the analogy is the natural one to reach for and it misleads if taken too far.

What is shared. Both programs aim to show that a domain apparently requiring substantive extra-logical assumptions is really logic in disguise, and both were pursued with the formal machinery of Principia Mathematica. The connection is not merely thematic: Keynes wrote the Treatise in Cambridge in Russell's orbit, Russell reviewed it approvingly, and Carnap's program is continuous with the logical empiricists' broader attempt to show that everything legitimate in knowledge is either logic or observation. Both also found that the reduction required a principle whose logical credentials were doubtful — the Axiom of Reducibility in one case, the choice of a measure over state descriptions in the other.

What is different. Logicism is a thesis about the ontology and epistemology of mathematics: numbers are logical objects, arithmetic truths are analytic. Logical probability is a thesis about the status of a relation between propositions, and it is compatible with any view about what numbers are. One can be a logicist and a subjectivist about probability, or a platonist about mathematics and a logical probabilist, without strain. The two theses are neither entailing nor incompatible; they are parallel ambitions in different domains, and the status of mathematical objects is a separate question from the status of confirmation relations.

The instructive similarity is in how they failed. Neither was refuted by a knock-down argument. Both were undermined by technical results showing that the reduction required choices the reducing discipline could not make on its own — and in both cases the surviving descendants (neo-Fregeanism, objective Bayesianism) keep the ambition while conceding that some substantive posit is ineliminable.

The tradition

• W. E. Johnson (1920s) proved an early representation result. His sufficientness postulate — that the probability of the next observation falling in a category depends only on the number of previous observations in that category and the total — plus symmetry yields, for more than two categories, exactly Carnap's later continuum. It is the first demonstration that plausible qualitative constraints pin down a family of inductive methods, and it is also the first sign that they do not pin down a single one.
• Keynes, A Treatise on Probability (1921), is the classic statement. Probability is a primitive logical relation, apprehended directly rather than defined, holding between a proposition and a body of evidence. Keynes is notably more cautious than his successors on two points that later proved to be the sticking ones: he held that many probabilities are merely ordinal or not comparable at all, so the relation need not always deliver a number; and he restricted the Principle of Indifference to cases where the alternatives are "indivisible", trying to head off the paradoxes of indifference.
• Wittgenstein, Tractatus 5.15, gave a purely syntactic version: the probability of a proposition is the proportion of the truth-grounds of the evidence that are also truth-grounds of the hypothesis — the ratio of rows in a truth table. Waismann extended it. This is logical probability in its most literal form, and it inherits every problem about the choice of atomic propositions.
• Jeffreys, Theory of Probability (1939), pursued the objective assignment of priors by requiring invariance under reparameterisation, producing the Jeffreys prior. He is the bridge to modern objective Bayesianism, and the one member of the tradition whose technical apparatus is still in routine use.
• Carnap, Logical Foundations of Probability (1950) and The Continuum of Inductive Methods (1952), is the most systematic development. In a formal language with finitely many predicates, a state description specifies the properties of every individual; a structure description specifies how many individuals fall under each combination, ignoring which. Carnap's preferred measure $m^{\ast }$ distributes probability equally over structure descriptions and then equally within each — precisely so that learning from experience is possible. The resulting confirmation function $c^{\ast }$ makes observed regularities raise the probability of their continuing, which the naive uniform measure over state descriptions does not.

Carnap later generalised to the $\lambda$-continuum. For a language with $k$ predicates, after $n$ observations of which $n_i$ fall in category $i$, the probability that the next falls in category $i$ is

$$P=\frac{n_i+\lambda /k}{n+\lambda }$$

with $\lambda \in [0,\infty ]$ measuring how strongly prior symmetry is weighted against observed frequency. At $\lambda =0$ one gets the straight rule — predict the observed frequency, learn fast, overfit. At $\lambda =\infty$ one gets $1/k$ regardless of evidence — never learn at all. Carnap's $c^{\ast }$ is $\lambda =k$.

The five results that broke it

The program was not refuted from outside. It was dismantled by results internal to it, and the details matter because each identifies a distinct failure mode.

1. Language dependence. The confirmation function depends on the predicates of the language. Carnap's measures treat the predicates as primitive and distribute probability by counting structure descriptions built from them — so a language with grue and bleen as primitives, in which "green" and "blue" are the defined terms, yields a confirmation function on which observed green emeralds confirm that future ones are grue. Goodman's new riddle of induction is thus not an objection from outside the formalism but a demonstration that the formalism has a free parameter it cannot fix. Logic does not privilege one vocabulary, so if confirmation is relative to vocabulary it is not a matter of logic alone.

2. No principled choice of $\lambda$. Carnap could not select a member of his own continuum on logical grounds, and eventually did not claim to: the choice reflects how much inductive caution one wants, which is a decision rather than a theorem. But the whole point of the program was uniqueness — claim 3 above. A continuum of admissible inductive methods, with the choice left to the agent, is subjectivism with extra structure.

3. Universal generalisations get probability zero. On $c^{\ast }$ in an infinite domain, any universal generalisation has probability zero on any finite evidence, and stays there. No number of observed black ravens raises the probability that all ravens are black above zero. Since confirming laws is the central case of scientific inference, this is fatal for the program's stated purpose. Hintikka's systems recovered non-zero probabilities for universals, at the cost of further free parameters — which is to say, at the cost of problem 2 again.

4. Putnam's diagonal argument. Putnam showed that for any effectively computable inductive method there is a computable sequence on which it fails to converge to the truth, although a different method would succeed. There is therefore no algorithmic inductive logic that is universally reliable, and the aspiration to a mechanical logic of confirmation cannot be met in full generality.

5. The indifference paradoxes. Everything that afflicts the classical interpretation — wine–water, the cube factory, Bertrand — applies here, since the measure over state descriptions is chosen by symmetry considerations of exactly the kind those paradoxes undermine.

What survived

The program failed as an attempt to make induction a branch of logic. Three of its components remain in good standing.

• Cox's theorem (1946) supplies the explanation of the calculus that the tradition wanted. If a degree of plausibility is a real number, depends on the proposition and the evidence, and satisfies weak structural conditions — consistency of different derivation routes, and a functional relationship between the plausibility of a conjunction and those of its parts — then it is isomorphic to a probability measure. This is a genuine result and is the strongest available argument that any graded notion of support must obey the axioms. Its assumptions have been scrutinised: Halpern exhibited finite counterexamples showing that the continuity and differentiability conditions do real work and cannot simply be dropped.
• Maximum entropy and invariance. Jaynes's proposal — choose the distribution maximising entropy subject to known constraints, and use transformation groups to fix the measure — is the logical program's ambition applied to priors rather than to a complete inductive logic. It addresses language dependence by demanding invariance under a specified group, which is a real advance over "no reason to discriminate", though it faces the objection that choosing the group reintroduces the arbitrariness one level up. This is the subject of objective Bayesianism.
• Evidential probability. Kyburg's and later Jon Williamson's systems keep the idea that evidence objectively constrains belief without claiming a unique confirmation function for every pair of propositions. The contemporary uniqueness thesis — that a body of evidence permits exactly one rational credence — is claim 3 of the original program, now debated on its own rather than as part of a formal reduction.

Assessment

The pattern is that the logical interpretation scores well on every criterion conditional on there being a unique confirmation function, and the results above are jointly a demonstration that there is not. That is a clean failure, and a more informative one than a muddled success.

Where this sits

The logical interpretation was the most serious attempt to answer the epistemic question of the introduction objectively — to have degrees of belief that are neither arbitrary nor merely personal. Its collapse is the main reason the field's centre of gravity moved to subjective Bayesianism, which abandons uniqueness and keeps coherence, and to objective Bayesianism, which keeps as much of uniqueness as the constraints will bear.

It also leaves a permanent lesson about method, applied throughout this section: a formal result about a confirmation function is a result relative to a language and a measure, and the choice of those is not itself a formal matter. That is the same moral as Bertrand's paradox, arriving by a different route.

The next page turns from the epistemic side to the objective one, and to the interpretation that dominated statistical practice for most of the twentieth century: frequentism.