Testimony and Disagreement

Most of what anyone believes rests on testimony. Almost no one has verified the existence of Antarctica, the mechanism of vaccination, or the value of the fine-structure constant; we believe them because others report them. Probabilistic epistemology treats testimony as evidence like any other — a report is an event whose likelihood depends on whether the reported proportion is true — and this yields a surprisingly powerful analysis, along with some sharp negative results about combining opinions.

The basic model

Let $H$ be a hypothesis and $R$ the event that a witness reports $H$ . Bayes' theorem gives

$\frac{P ( H ∣ R )}{P ( \neg H ∣ R )} = \frac{P ( R ∣ H )}{P ( R ∣ \neg H )} \cdot \frac{P ( H )}{P ( \neg H )}$

The testimony's force is the likelihood ratio $P (R ∣ H) / P (R ∣ \neg H)$ — how much more likely the witness is to report $H$ if it is true than if it is false. A perfectly reliable witness has an infinite ratio; a witness who reports at random has a ratio of $1$ and provides no evidence at all.

Two consequences follow immediately, and both are important for the harder cases.

Reliability is not enough; discrimination is what matters. A witness who says "yes" to everything is highly likely to report a truth when there is one, but equally likely to report a falsehood, so the ratio is $1$ . What matters is the difference between behaviour under $H$ and under $\neg H$ .

Priors still dominate for extraordinary claims. A witness with a likelihood ratio of $1000$ raises the odds by a factor of $1000$ . If the prior odds are $1 0^{- 9}$ , the posterior odds remain about $1 0^{- 6}$ . This is the probabilistic core of Hume's argument about miracles, and it is developed at length in miracles and testimony. Note that this is not a general bar on believing testimony for improbable events — it says only that the required likelihood ratio scales with the prior improbability, which is a quantitative claim rather than a prohibition.

Multiple witnesses and the independence assumption

The multiplication of independent testimony is the model's most striking result. If $n$ witnesses each report $H$ independently conditional on $H$ and on $\neg H$ , the likelihood ratios multiply:

$Λ_{total} = i = 1 \prod n Λ_{i}$

Ten witnesses each only modestly reliable, with $Λ = 3$ , jointly give $3^{10} \approx 59, 000$ . Concurring testimony from independent sources can therefore establish very improbable claims, which is the standard reply to Hume's blanket scepticism about miracle reports.

Everything hinges on the independence assumption, and it fails in most real cases in a way that is easy to miss. Witnesses who talked to each other, read the same source, share a culture, or have a common motive are not conditionally independent, and treating them as such compounds error rather than cancelling it. Ten people repeating one rumour provide roughly the evidential weight of one person. A correlated error term places a ceiling on the total likelihood ratio no matter how many witnesses are added — which is why the number of reports is a poor guide to evidential strength, and why the genuinely hard work in assessing testimony is establishing independence rather than counting sources.

This is the same structure as the common-cause analysis: correlated reports are screened off by their shared source, and once the source is conditioned on, the extra reports add nothing.

Peer disagreement

Now consider testimony from someone as well-informed and capable as oneself. You and a colleague examine the same data and reach different conclusions — you assign $0.7$ , they assign $0.3$ . What should you do?

The disagreement itself is evidence: it indicates that at least one of you has erred, and by hypothesis you have no antecedent reason to think it is them.

Conciliationism (the equal-weight view) says you should move substantially toward their credence, in the symmetric case splitting the difference. The rationale: from a neutral standpoint you are equally likely to be the one who erred, so treating your own view as privileged is unjustified partiality. The bootstrapping objection to the alternative is forceful — if you retain your view on the grounds that you are right, you are using the disputed judgement to certify itself.

Steadfastness says you may retain your credence. Your evidence includes not just the data but your having reasoned through it, and that first-person access is evidence they lack. Kelly's version: if your original reasoning was in fact correct, conciliating makes you less accurate, and the equal-weight view licenses splitting the difference with someone who is simply wrong.

The total-evidence view (Kelly's later position) holds that the first-order evidence and the psychological fact of disagreement both bear, with their relative weight varying by case. This is plainly right and correspondingly unspecific.

Two structural observations are worth more than adjudicating the dispute.

Conciliationism is self-undermining in a specific way. It is itself a philosophical thesis about which epistemologists disagree, so its adherents should conciliate about it and lower their credence in it. This is not a refutation, but it is an awkwardness that the steadfast view does not face.
Disagreement is higher-order evidence — evidence about the reliability of one's own reasoning rather than about the world. The framework's difficulty in handling it reflects the logical omniscience idealisation: an agent who is certain of all logical truths cannot be uncertain about whether they reasoned correctly, so standard Bayesianism has no natural place for the phenomenon.

Pooling and its impossibility results

A related formal question: given several agents' credences, is there a principled way to combine them into one?

Linear pooling takes a weighted average, $P (A) = \sum_{i} w_{i} P_{i} (A)$ . It is simple and always yields a probability function. It also fails to preserve judgements of independence: if every agent regards $A$ and $B$ as independent, the pooled function generally does not, which is a serious defect when the independence structure is what the model is about. Linear pooling also does not commute with conditionalization — pooling then updating differs from updating then pooling.

Geometric pooling takes a normalised weighted geometric mean. It commutes with conditionalization and preserves unanimous independence judgements, but it does not preserve unanimous judgements of the probability of a disjunction, and it gives a veto to any agent assigning zero.

The situation generalises into an impossibility result of the Arrow type: no pooling method satisfies all of a small set of individually attractive conditions — unanimity preservation, independence preservation, commuting with conditionalization, and non-dictatorship. Aggregating credences is subject to the same structural obstacles as aggregating preferences, and the reason is the same: pairwise-defined operations cannot respect global structure.

The practical upshot is that the aggregate opinion of a group is not well defined independently of a choice of aggregation rule, and different rules give different answers. This bears directly on how to read expert consensus, and on the temptation to treat a distribution of expert opinion as though it were a single credence.

Where this sits

Testimony is where probabilistic epistemology touches ordinary life most directly, and it delivers real results: the multiplication of independent evidence, the ceiling imposed by correlation, and the scaling of required reliability with prior improbability. These are quantitative and often counterintuitive, and they discipline arguments — about miracles, about consensus, about the weight of numbers — that are otherwise conducted on intuition.

Its limits are the framework's usual ones. Peer disagreement involves higher-order evidence that a logically omniscient model cannot represent, and pooling faces impossibility results showing that "the group's credence" is not a well-defined object.

This completes the treatment of induction and confirmation. The folder's arc is that probability systematises evidential reasoning powerfully — the ravens, severity, the Ockham factor, the multiplication of testimony — while leaving the inductive commitment itself undischarged, resident in the priors and in the choice of vocabulary.

The next folder turns from evidence to structure: what, if anything, probabilistic dependence can tell us about causation.