The Columnist's Error

The newspaper had started running opinion columns about the case, which Wrinje found both useful and infuriating. Useful because they sometimes contained new information. Infuriating because they contained reasoning errors that, now that he knew what to look for, made him want to tear the pages apart.

The latest column was by a man named Brenzik, who wrote with the confidence of someone who had never been wrong about anything, or at least never admitted to it.

Brenzik's column was about the DNA evidence. He wrote:

"The hair found at the scene has been analyzed. DNA testing has a random match probability of approximately 1 in a million. This means that if the DNA matches a suspect, there is only a 1-in-a-million chance that the suspect is innocent. The science is clear: a DNA match is, for all practical purposes, proof of guilt."

Wrinje read this twice. Something felt wrong, but he could not articulate what. He brought the newspaper to Glagalbagal's apartment.

Glagalbagal: Ah, the Prosecutor's Fallacy.

Wrinje: It has a name?

Glagalbagal: It does. And it is one of the most common reasoning errors in criminal cases. It has probably contributed to wrongful convictions.

Wrinje: What is the error?

Glagalbagal: The columnist says: "If the DNA matches, there is a 1-in-a-million chance the suspect is innocent." But that is not what the 1-in-a-million number means. The 1-in-a-million is the probability of a match given that the person is innocent — it is the random match probability. What the columnist claims it is, is the probability that the person is innocent given a match. Those are not the same thing.

Wrinje: That sounds like the same mistake I was making with the eyewitness testimony. I said "the witness is 70% reliable" and concluded "there is a 70% chance Jansu is guilty."

Glagalbagal: Exactly the same structure. You are confusing the probability of the evidence given the theory with the probability of the theory given the evidence. They can be wildly different.

Wrinje: Can you show me why?

Glagalbagal: Let us use numbers. Suppose our city has a million people. One of them is the killer. The police have no other information — they test everyone's DNA against the hair found at the scene.

Wrinje: All million people?

Glagalbagal: Hypothetically. The random match probability is 1 in a million. So for each innocent person, there is a 1-in-a-million chance their DNA falsely matches. There are 999,999 innocent people. How many false matches do you expect?

Wrinje: 999,999 divided by a million... about 1.

Glagalbagal: About 1 false match, yes. And the actual killer also matches. So you have about 2 matches total — 1 real and 1 false.

Wrinje: So if you pick a match at random, there is a 50% chance they are the real killer. Not a 99.9999% chance.

Glagalbagal: Exactly. With no other information, a DNA match in a city of a million people — given a random match probability of 1 in a million — gives you roughly a 50% chance the matched person is guilty. That is very far from "proof of guilt."

Wrinje: But the columnist said 1-in-a-million chance of innocence.

Glagalbagal: The columnist confused two numbers. The probability of the evidence given innocence — that is the 1-in-a-million random match rate. The probability of innocence given the evidence — that is what he should have calculated but did not. He swapped them. This swap is the Prosecutor's Fallacy.

Wrinje: Why is it called that?

Glagalbagal: Because prosecutors in real trials have made this exact argument to juries. "The DNA test has a false positive rate of 1 in a million, so there is only a 1-in-a-million chance the defendant is innocent." Juries hear this and think the case is proven beyond doubt. But it is not — not without considering the prior probability and the size of the population.

Wrinje: So the DNA evidence is not as strong as it sounds?

Glagalbagal: It depends on the context. If the police already have a suspect for independent reasons — motive, opportunity, witnesses — and then the DNA matches, that is very strong. Because the prior probability is already much higher than 1-in-a-million. But if the match comes from searching a large database with no prior suspicion, the evidence is weaker than most people think.

Wrinje: This is the same lesson from the beginning. The strength of the evidence depends on what you already know.

Glagalbagal: Always. Evidence does not exist in a vacuum. It interacts with everything else you know. That is why Bayes' theorem works the way it does — the prior matters.

Wrinje: I want to write a letter to the newspaper correcting Brenzik.

Glagalbagal: You could. Though columnists who write with that much confidence rarely enjoy being corrected by twelve-year-olds.

Wrinje: Maybe that is exactly why someone should do it.