Research reveals the problem with ‘overwhelming evidence’
A new study has suggested that total agreement between witnesses or overwhelming evidence in a court case should raise suspicions of bias and lower confidence in a result.
In a study to be published in The Proceedings of The Royal Society A, researchers found that unanimity should raise questions in situations where there is a lot of prior uncertainty.
The paper shows, for instance, that a police line-up where victims all agree on the identity of an attacker is actually less likely to be correct than one where there is some disagreement between witnesses.
Similarly, a set of DNA samples that are all in perfect agreement could be evidence of an underlying systemic error.
Speaking with Lawyers Weekly, lead researcher Professor Derek Abbott, a physicist and electronic engineer at The University of Adelaide, said the study, while principally mathematical, was highly relevant to the legal profession.
Professor Abbott said the principle uncovered in the paper is not a new one and has been in the “collective subconscious of society” for some time.
One example of this, cited in the paper, is Jewish law in the classical era. As stated in the Talmud, defendants facing the death sentence were acquitted if all 23 judges (‘the Sanhedrin’) found them guilty.
“There is some evidence, like the Jewish [law] from centuries ago, that show that people have some inkling of this sort of thing because they would let somebody off being executed if everybody agreed that they should be executed,” said Professor Abbott.
“It seems counter-intuitive, it seems that defies logic, but it's also saying that perhaps if everybody is in total agreement there's been a collusion or a bias so you have to be careful of that.”
Professor Abbott said his research team were, however, surprised by their finding and branched into diverse areas, including law, as a “sanity check”.
“When something is counter-intuitive you try to apply it to something that is more familiar to people and see if it still makes sense,” he said.
But there are some obvious examples where this logic clearly holds, he continued. For instance, when a party receives 100 per cent of a vote in an election there is usually reason to suspect rigging.
“You can see that straight away,” he said. “If there is too much unanimity, there is something wrong.”
“So the next question you have to ask is, 'but hang on a minute, surely there are instances where unanimity is a good thing?’” Professor Abbott continued. “That's right. You've got to distinguish the two cases.”
“If I have, instead of a police line-up, a line-up of bananas and one apple and a million people will come along one after another and say, ‘yep, that's the apple’, there will be no disagreement about that.
"It will be absolutely unanimous. That form of unanimity is correct. That's incontrovertible.”
The difference between this and a real police line-up is that there is prior uncertainty in the later scenario. According to Professor Abbott, the misidentification rate can be as high as 48 per cent in police line-ups.
“So that's almost 50 per cent – that's almost like tossing a coin,” he said. “So if you are expecting almost 50 per cent of the time that your witnesses should get it wrong, if 20 witnesses are all agreeing then that can't be right, because you are already expecting half of them to be wrong.”
The study, which is grounded in Bayesian analysis, could also hold for modern juries or even High Court decisions, according to Professor Abbott.
“If a case is unanimous, one has to then question why it is unanimous,” he said. “Is it because the case was a clear one or was it because there was some systemic bias there?”