Bad Evidence
by Liliana Segura and Jordan Smith, The Intercept
May 5 2019
As an alternative theme for the 2019 AAFS (American Academy of Forensic Sciences) meeting, “complacency kills” might not have been half bad, at least if anyone had wished to inject some urgency into things. The Baltimore event began amid an ongoing crisis within forensic science that remained woefully unresolved. When we first wrote about the AAFS for The Intercept following its 2016 conference in Las Vegas, we encountered an embattled field facing rising public scrutiny over some of its most cherished and longstanding disciplines. Wrongful convictions rooted in junk science, crime labs embroiled in scandal, and a devastating revelation about the FBI’s hair microscopy division in 2015 had turned the image of forensics popularized by shows like “CSI: Crime Scene Investigation” on its head. The implications of the FBI scandal were particularly alarming. Hair analysts testifying on the stand had made erroneous statements in at least 33 death penalty cases, according to the agency. “Nine of these defendants have already been executed and five died of other causes while on death row.”
But whereas there was some reason in Vegas to feel optimistic about the prospects for reform — the theme that year was “Transformation: Embracing Change” — things have seemed to go backward since then. A report critiquing the scientific validity of certain forensic techniques, released by the President’s Council of Advisors on Science and Technology in the fall of 2016, met with aggressive pushback by the FBI and Barack Obama’s Department of Justice. Then there was the election of Donald Trump and the elevation of Jeff Sessions, who put a halt on a number of federal initiatives that were just starting to get underway.
Much of the recent upheaval in the forensics world can be traced back to a landmark study released by the National Academy of Sciences in 2009. Titled “Strengthening Forensic Science in the United States: A Path Forward,” the report questioned the scientific basis for virtually every forensic discipline used to convict people and send them to prison. With the exception of DNA analysis, it found, “no forensic method has been rigorously shown to have the capacity to consistently, and with a high degree of certainty, demonstrate a connection between evidence and a specific individual or source.”
The NAS report was particularly damning for the so-called pattern-matching disciplines, in which an analyst examines a piece of evidence — say a bloody fingerprint found at a crime scene — and tries to match it to a sample belonging to a suspect. At AAFS, where forensic areas are divided into 11 different sections, many members of such fields responded with a mix of denial and defiance. While some practitioners took up the call issued by the NAS report — the fingerprint community, for example, has worked to develop objective comparison methods and determine error rates — others insisted the old ways of doing things were just fine.
Oh, what are those “old ways”? John Oliver is happy to describe them.
Now keep in mind The Intercept report is about “Medical Examiners”, presumably Physicians or people with Degrees, not the vast majority of “Coroners”, many of whom are Morticians (or in other words Cosmetologists for Dead People).
There’s been a fair amount of change since the report’s release, and the National Commission on Forensic Science, of which King was a member, made a number of recommendations that were adopted — including a ban on practitioners using the phrase “reasonable degree of scientific certainty” when testifying about their confidence in the matches they’ve made. The terminology has no meaning outside the courtroom and yet suggests a strong scientific foundation that cannot be said about most forensic practices. Yet a number of promised reforms have not materialized.
At the 2016 conference in Las Vegas, then-Deputy Attorney General and NCFS co-chair Sally Yates announced that the DOJ would be conducting a “stress test” on various disciplines performed in the FBI lab — not because there were any particular concerns, she was quick to say, but as a means of ensuring “the public’s ongoing confidence in the work we do.” The decision seemed a prudent one, given the alarming results of a joint review of thousands of FBI hair analysis cases, which revealed that FBI analysts had overstated their conclusions 95 percent of the time. “This doesn’t necessarily mean that there were problems with the underlying science,” Yates explained to the plenary audience. “It means that the probative value of the scientific evidence wasn’t always properly communicated to juries.”
Seven months later, Lynch announced that she was adopting the NCFS recommendations on testimonial language, but that wider stress test simply never happened. “We were doing great until the Trump administration came about and … the things that were happening at the federal level, the policy stuff and so on, that came to a screeching halt,” said Alicia Carriquiry, a professor of statistics at Iowa State University and director of the Center for Statistics and Applications in Forensic Evidence.
Carriquiry says that conversations happening among stakeholders and the DOJ just fizzled out after Trump was elected and Sessions was brought on board. And the NCFS wasn’t the only advisory panel to get the hook. While Trump continued the PCAST on paper, he hasn’t appointed any members to it. In December 2018, the DOJ disbanded its Science Advisory Board, which provided input to the department on what types of research — including in forensics — it should fund.
Even gains that had been made in curbing unsupportable testimony have since weakened, she said. “When the DOJ was under the Obama administration there were conversations. In fact, I participated in many of those where we would sit with the DOJ people and think about the type of language that should come out of crime labs and what type of reviews we should be doing of the disciplines,” she said. “The second Sessions came on board those initiatives were killed dead.”
It wasn’t just that things stopped. “Worse, you know. They’re trying to walk back many of the things we were making progress on,” she said. “For example, we had come to some sort of an agreement about the fact that we were going to do a broad review of the disciplines.” But when the Trump administration took over the attitude was, “‘What review?’ So that was completely done. Then we had come to an agreement on the type of language that should be used in reporting and testimony. That was squashed and the language went back to exactly what it used to be.”
So, what is real science? It must have a refutable hypothesis meaning If I assert the Sun rises in the East it damn well better. It must be predictive meaning it should happen tomorrow just the same. It must be replicable meaning that you can go outside your house at 5:30 am and look around and watch the Sun rise just the same as I can (unless it’s raining).
Oh, and it should be Peer Reviewed and published in a reputable Academic Journal. That’s kind of overrated according to Aristotle and Ptolemy who both got rave reviews and were catastrophically wrong about nearly everything though the Math was beautiful.
Recent Comments