Linda problem
A bank teller, against a bank teller who is also active in a feminist organisation.
What this measures
The subject reads four lines about Linda — a philosophy graduate, outspoken, once a regular at demonstrations — and picks which of two statements about her is more likely. One says she works as a bank teller. The other says she works as a bank teller and is active in a feminist organisation.
The second statement is the first with a further claim attached, so it cannot be the likelier of the two: every world in which Linda is a feminist bank teller is a world in which she is a bank teller.
People pick the conjunction anyway, 85% of the time. The sketch is built to make them: “bank teller” is a poor likeness of the woman described, while “feminist” is the word the sketch invites the reader to supply and never says. As a portrait the conjunction wins; as a probability it cannot. Which of the two an LLM answers is the measurement.
Human result benchmark
85% of human subjects rank “bank teller and active in a feminist organisation” above “bank teller”, in the two-statement form where both are on the page together. This is representativeness standing in for probability: the longer statement is the better likeness of the woman described, and likeness is what the judgement runs on.
The comparison as presented
Linda is 31 years old, single, very bright and quite outspoken. She studied philosophy at university. As a student she was deeply concerned with issues of discrimination and social justice, and she often took part in demonstrations.
| Statement | Claims |
|---|---|
| Linda works as a bank teller. | teller |
| Linda works as a bank teller and is active in a feminist organisation. conjunction | teller + feminist |
The marked statement claims everything the other one claims and one thing more, so it cannot be the likelier of the two.
Experimental setup
Each LLM is asked the question directly and answers with one of the two statements. One run is a block of ten trials, each in its own context, so the measurement is a proportion rather than a single draw.
Which statement appears first alternates from trial to trial, so each one leads exactly half the time and the reported rate is clear of positional effect. Both orders are below, exactly as they are sent.
The prompts
the conjunction as B
Linda is 31 years old, single, very bright and quite outspoken. She studied philosophy at university. As a student she was deeply concerned with issues of discrimination and social justice, and she often took part in demonstrations. In your opinion, which statement is more likely to be true? A. Linda works as a bank teller. B. Linda works as a bank teller and is active in a feminist organisation. Reply with the letter of the statement you think is more likely, and one sentence explaining why.
the conjunction as A
Linda is 31 years old, single, very bright and quite outspoken. She studied philosophy at university. As a student she was deeply concerned with issues of discrimination and social justice, and she often took part in demonstrations. In your opinion, which statement is more likely to be true? A. Linda works as a bank teller and is active in a feminist organisation. B. Linda works as a bank teller. Reply with the letter of the statement you think is more likely, and one sentence explaining why.
How often did the LLMs pick the conjunction?
Run it yourself
No run yet.
The run
Bibliography
- (1983). Extensional versus Intuitive Reasoning: The Conjunction Fallacy in Probability Judgment. Psychological Review 90(4), 293–315. The Linda problem. In the two-statement form, 85% of respondents ranked the conjunction higher.
- (1972). Subjective Probability: A Judgment of Representativeness. Cognitive Psychology 3(3), 430–454. The heuristic the conjunction fallacy is evidence for — likeness standing in for probability.