Symposium · Logic & Paradoxes · Pl. 6John Stuart Mill · 1806–1873
The Methods of Agreement and Difference
Turning a habit into a method
People had been guessing at causes for as long as they had been getting sick, watching crops fail, or wondering why one remedy worked and another did not, usually through vague hunches rather than any disciplined procedure. Mill set out to formalize the plain, common-sense reasoning underneath good guessing into rules precise enough to actually be taught and checked. The book that resulted, A System of Logic, was the standard account of scientific method in English for half a century, and the two methods at its centre can be explained to anyone in under a minute.
The world it came from
Mill published A System of Logic in 1843 while working as a clerk and then examiner at the East India Company. He wanted to show that all knowledge, including the laws of science, rests on experience, and he was arguing against William Whewell, the Cambridge polymath whose Philosophy of the Inductive Sciences of 1840 gave a central role to the mind's own ideas. Mill drew on John Herschel's Preliminary Discourse of 1830 and, behind it, on Francis Bacon's tables of presence and absence from 1620. Book III, chapter eight, sets out four methods of experimental inquiry, of which agreement and difference are the first two. The audience was not only scientists: Mill hoped the methods would discipline politics and economics as well.
The idea
The simplest and most obvious modes of singling out from among the circumstances which precede or follow a phenomenon, those with which it is really connected, are two in number: comparing together different instances in which the phenomenon occurs, and comparing instances where it occurs with similar instances where it does not. These are the Method of Agreement and the Method of Difference. Agreement asks what the cases where the effect appears have in common. Difference asks what changes between a case where it appears and an otherwise identical case where it does not. Most of experimental practice is those two questions, asked carefully.
How the reasoning runs
Take a dinner where several guests fall ill. Method of agreement: list what each sick guest ate. If the only dish common to all of them is the oysters, the oysters are the likely cause. The inference eliminates: every dish that some sick guest did not eat is ruled out. Method of difference: find two guests who ate the same things except that one had oysters and one did not, and only the first is ill. Now the oysters are the one circumstance that varies with the effect. Mill rated difference the stronger method because it is the form of a controlled experiment: hold everything fixed, change one thing, watch. The joint method runs both, comparing the sick who share the oysters with the well who lack them.
Seen in life
The build passed yesterday and fails today. You could reason about the code, or you could run Mill's method of difference on the history. Between the last good commit and the first bad one lie forty changes. Check out the midpoint and build: the failure is present or not, and half the candidates are eliminated. Repeat. Six builds later you have two commits that differ by one change, one green and one red, and the cause is the change between them. The tool that automates this, git bisect, is the method of difference applied to a list. The method of agreement is what you do when the failure appears on several machines: list what they share, and cross off whatever any working machine also has.
Why it matters
This gives experimental reasoning an explicit, checkable structure instead of leaving it to intuition: find what all the positive cases share that the negative cases lack, or change one single factor and watch whether the effect disappears. Controlled experiments, in medicine, psychology, and beyond, are essentially disciplined applications of exactly these two comparisons. They also make a negative result useful. A candidate cause that is present when the effect is absent, or absent when the effect is present, is eliminated, and Mill's methods are at bottom a discipline of elimination.
“These two methods may be respectively denominated, the Method of Agreement, and the Method of Difference.”
From the source
Mill states the two methods directly: “These two methods may be respectively denominated, the Method of Agreement, and the Method of Difference.” He devotes an entire chapter of his System of Logic to spelling out precisely how each should be applied. The chapter defines each with a canon. For agreement: if two or more instances of the phenomenon have only one circumstance in common, that circumstance is the cause or effect. For difference: if an instance where the phenomenon occurs and one where it does not have every circumstance in common but one, that one is the cause, or an indispensable part of it. The same chapter adds the joint method, residues, and concomitant variations.
The pushback
Don't these methods assume you can already isolate a manageable, short list of candidate causes to compare, when real-world phenomena, disease, economic recessions, historical events, often have countless tangled factors impossible to cleanly separate out one at a time? Mill acknowledged this limitation directly, noting the methods work best in genuinely controllable conditions, not messy, multi-caused human affairs. A second objection came from Whewell himself: the methods presuppose that you have already framed the right candidate causes, and framing them is the creative part of science that no method supplies.
The reply
Mill conceded much of this. He wrote about the plurality of causes, where several different causes can produce one effect and agreement fails, and the intermixture of effects, where causes blend and cannot be separated by observation. His answer was that the methods work when the field of candidates is closed and the cases can be controlled, which is why he tied the method of difference to experiment proper. What he did not have was randomization. Fisher, in the 1920s and 1930s, showed that assigning treatments by chance handles the unknown factors that Mill's methods must assume away, and the randomized controlled trial is the method of difference made robust against ignorance. Whewell's point stands: the methods test a hypothesis; they do not invent one.
Where it went next
A decade after Mill's book appeared, John Snow was reasoning in just this way about cholera in London, comparing households on two water supplies that differed in almost nothing else. Fisher's Design of Experiments in 1935 turned the method of difference into the randomized trial. J. L. Mackie's 1965 analysis of causes as insufficient but necessary parts of sufficient conditions is a direct descendant, and Judea Pearl's causal graphs formalize what Mill's canons assume. Engineers use the methods daily without the name: root-cause analysis, A/B tests, and git bisect are all agreement and difference with a tool attached. The four methods still open most textbooks on the logic of science.
Where this leads
Petitio principii shows this very same book turning from how to find genuine causes correctly, toward how arguments quietly fail anyway, even when every individual step still looks entirely reasonable and perfectly correctly structured throughout the whole argument from start to finish. Falsifiability is the twentieth-century sequel. Popper agreed with Mill that a theory is tested by the cases that could go against it, and disagreed about whether any pile of agreeing cases could ever establish it.