Underdetermination, Simplicity, and Model Choice
Evidence does not always discriminate. For any body of data there are, in principle, many hypotheses compatible with it, and sometimes indefinitely many that fit it perfectly. If theory choice is to be rational, something beyond fit to the data must do work — and the leading candidate is simplicity.
This creates an obligation. Simplicity is a preference for one kind of hypothesis over another, and unless it can be shown to be truth-conducive, favouring simple theories is an aesthetic policy dressed as a method. This page sets out the varieties of underdetermination, the Bayesian treatments of simplicity, and the extent to which the appeal succeeds.
Varieties of underdetermination
Three distinct claims travel under the name, and they differ enormously in strength.
Holist underdetermination (Duhem–Quine). Hypotheses face evidence only in bundles, so a recalcitrant observation never refutes a single claim; any hypothesis can be retained by adjusting auxiliaries. This is uncontroversial as stated and, as Bayesian confirmation shows, well handled: disconfirmation distributes over the conjuncts according to their priors and likelihoods, so retention is possible but has a probabilistic cost.
Transient underdetermination. The evidence available now does not settle the matter, though further evidence would. This is the ordinary condition of science and poses no philosophical problem.
Strong (empirical-equivalence) underdetermination. There exist rival theories with identical empirical consequences, so no possible evidence could discriminate. This is the philosophically interesting claim, and its instances are fewer than often supposed. The standard examples are genuine: Lorentzian ether theory versus special relativity; Newtonian mechanics with the universe at absolute rest versus in uniform motion; Bohmian versus standard quantum mechanics. But manufacturing empirically equivalent rivals for an arbitrary theory — say, by conjoining "and reality is as the theory says, except that an undetectable demon exists" — produces hypotheses nobody takes seriously, and the reason is precisely that they are gratuitously complex.
Curve fitting
The cleanest instance. Given data points, a polynomial of degree passes exactly through all of them, and infinitely many curves fit the data perfectly. Yet a straight line through slightly scattered points is preferred to a wiggling high-degree polynomial that hits each one.
The justification is not that the simple curve fits better — it fits worse — but that it will predict better. A curve fitted to noise reproduces the noise, and noise does not recur. This is overfitting, and its existence is a mathematical fact rather than a methodological preference: as model flexibility increases, error on the training data falls monotonically while error on new data eventually rises.
This is the single strongest argument that simplicity is truth-conducive rather than merely convenient, and it is worth being clear about what it establishes. It shows that, given that data contain noise and that the underlying signal is not itself arbitrarily complex, restricting flexibility improves prediction. The second condition is an inductive assumption — it is a claim about the world — so the argument does not escape the problem of induction. What it does establish is that the preference for simplicity is not arbitrary: it has a precise rationale, quantifiable in the bias–variance decomposition, and it can be checked by cross-validation.
The model-selection criteria formalise the trade-off. AIC penalises the log-likelihood by the number of parameters, targeting predictive accuracy; BIC penalises more heavily and, under assumptions, converges on the true model if it is among those considered. That the two criteria have different goals — prediction versus truth — and therefore recommend different models is itself philosophically instructive: "the best model" is not well defined independently of what one wants it for.
Bayesian treatments of simplicity
Bayesianism can locate a preference for simplicity in either the prior or the likelihood, and only the second is explanatory.
In the prior. Assign simpler hypotheses higher prior probability. This works formally and explains nothing: it encodes the preference rather than justifying it, and invites the question why the world should be expected to be simple. The Solomonoff prior, weighting hypotheses by for Kolmogorov complexity , is the principled version and has attractive convergence properties, but it is uncomputable and depends on the choice of universal machine — the machine-dependence being a formal echo of Goodman's language-dependence.
In the likelihood. The better answer, and the one that constitutes a genuine explanation. A complex theory with many adjustable parameters can accommodate many possible data sets, so it must spread its probability thinly across them; a simple theory concentrates its probability on the few data sets it predicts. When those data are observed, the simple theory's likelihood is much higher, and it wins the Bayes factor comparison decisively.
This is the Ockham factor, and it derives the preference for simplicity from the probability axioms plus the requirement that likelihoods be normalised. No separate simplicity prior is needed: flexible theories are penalised automatically because their predictive commitments are diffuse. It also explains why the penalty is proportional to unused flexibility — a complex theory whose extra parameters are all doing genuine work is not penalised in the same way.
The qualification is that this requires priors over the parameters, so the Bayes factor is not prior-free and can be sensitive to how parameter space is carved.
What "simple" means
Both treatments presuppose a measure of simplicity, and here the difficulty is Goodman's, arriving for the third time in this section.
Simplicity is language-relative. Counting parameters depends on the parameterisation; counting symbols depends on the vocabulary; Kolmogorov complexity depends on the universal machine. A language with grue as a primitive makes grue-hypotheses simple.
Three responses, none decisive. One may take the natural-language or scientific vocabulary as given, which is honest but concedes that simplicity judgements presuppose an already-inductively-shaped conceptual scheme. One may appeal to natural properties in Lewis's sense — the joint-carving ones — which grounds simplicity objectively at the cost of a further primitive, exactly as in the best-system account of laws. Or one may note that machine-dependence in Kolmogorov complexity is bounded by a constant, so that for sufficiently complex hypotheses the ordering is machine-independent — true, but the constant can swamp the comparisons that actually arise.
There is also more than one virtue here. Parsimony (few entities), elegance (few or simple equations), and unification (one mechanism for many phenomena) are distinct and can conflict; a theory positing fewer entities may require uglier laws. Treating them as a single desideratum obscures real trade-offs.
Does underdetermination undermine realism?
The sceptical argument: if empirically equivalent rivals exist, and evidence cannot discriminate, belief in any particular theory is unwarranted, so scientific realism fails.
The Bayesian reply is that empirical equivalence leaves the likelihoods equal but not the posteriors, since the priors differ. If the demon-augmented theory has a lower prior — because it is more complex, or gratuitously adds structure — its posterior stays lower however much evidence arrives.
Whether this vindicates realism or relocates the problem is exactly the question. It makes theory choice depend on priors, and if those merely encode a taste for simplicity, the realist has shown only that a preference plus evidence yields a preference. If the simplicity preference is grounded — by the overfitting argument, or by the Ockham factor — the reply has real force. The honest position is that the Bayesian framework converts a question about evidence into a question about priors, and that the priors can be partly but not wholly defended.
There is also a deflationary observation worth recording: in the actual history of science, cases of genuine strong underdetermination are rare, and the ones that occur — Lorentz versus Einstein, Bohm versus Copenhagen — are treated as live interpretive disputes rather than as demonstrations that belief is impossible. The philosophical problem is more acute in principle than in practice.
Where this sits
Simplicity is where confirmation theory meets the problem of induction most directly, and the results are mixed but not empty. The overfitting argument shows the preference is not arbitrary; the Ockham factor derives it from the probability calculus rather than positing it; and the residual language-relativity is Goodman's problem, which no framework in this section escapes.
The pattern established in the logical interpretation recurs: a formal apparatus does substantial work once a vocabulary is fixed, and cannot fix the vocabulary. What is new here is that the work done is considerable — enough to explain why science prefers simple theories and why that preference tends to pay.
The last page of this folder turns from evidence about the world to evidence from other people: testimony and disagreement.