By Laura Spinney, a science journalist and writer based in Paris. Her book “Proto: How One Ancient Language Went Global” will be published on May 13. Originally published at Undark.
When does context become bias?
It’s a question that engenders anxiety in many scientific disciplines, and with good reason. Information about a patient’s family history may nudge a doctor towards the right diagnosis, or tip them towards a wrong one. Awareness of a suspect’s criminal past may help an investigator read a crime scene correctly, or unwittingly inculpate an innocent person. When the ground truth is inaccessible, it’s difficult to know where the line lies. The age-old dilemma has erupted anew, in forensic science, and it’s dividing the field.
The catalyst was the publication, in 2022, of a document that went unnoticed beyond the field. Sixteen forensic scientists from six countries, in what they called the Sydney Declaration, claimed that their discipline was in an “intractable state of crisis.” The core purpose of the forensic scientist, as defined at the field’s birth in the early 20th century, was the study of the trace — any vestige of human activity — in the context of a crime scene. Their job was to examine the scene, formulate hypotheses of what had happened there, and direct the collection of evidence that would allow those hypotheses to be tested. Instead, the authors claimed, the scientist had become a technician, performing specialized analyses in a lab, on evidence divorced of all context. Forensic science had lost sight of its raison d’être.
The paper dropped at what was already a time of intense soul-searching for forensic science. In 2009, the National Research Council, or NRC, published an excoriating report on the field’s practices. Though the report highlighted a number of problems, the one that got the most airtime was that many routinely used forensic techniques, including fingerprint and firearms examinations, were neither accurate nor reliable.
Its aura of infallibility shattered, the field was stung into action. By 2019, Peter Neufeld, co-founder of the U.S.-based nonprofit Innocence Project, was praising its efforts to reinforce the scientific foundations of the techniques that had been criticized. Neufeld said in a press release that the progress forensic science had taken to becoming a “neutral truth teller” could not be understated; innocent people had been freed and lives saved. Legal scholars and scientists expressed their approval too.
But other problems highlighted by the NRC had not been addressed. There was still no central body providing oversight, and forensic services were still mostly under the control of law enforcement agencies or prosecutors’ offices. This meant that crime scenes were usually examined by police officers or technicians with expertise in specific types of trace evidence, many of whom had not been trained to assess the scene holistically. The report dealt with the U.S., but these things were true elsewhere too. Simon Cole, a professor of criminology, law, and society at the University of California, Irvine, told me that, by the late 20th century, “forensic science had become an adjunct of law enforcement without allegiance to science in the sense of being unbiased and following the evidence where it leads.”
It was this state of affairs that the Sydney Declaration was reacting to, and its timing was not coincidental. Several of its authors told me that the technical improvements, while welcome, have created a strange disjunction in which extremely precise answers are being provided to ill-defined problems. The evidence that is collected tends to be that which is visible or likely to lead to the identification of a suspect — DNA, fingerprints, CCTV footage — rather than that which will support one hypothesis over another and facilitate the correct reconstruction of events.
In 2023, three of the declaration’s co-authors illustrated their arguments with a real-life case. While attempting to escape, a hospitalized prisoner had shot and killed the police officer guarding him. Pleading not guilty to a homicide charge, the prisoner claimed that he had acted in self-defense. The initial examination of the scene by the police produced no evidence to challenge that plea, but a second investigation conducted by a forensic scientist demonstrated that the prisoner’s actions had been premeditated.
Many in the field agree with the declaration’s diagnosis, but not necessarily with its solution — namely, to return to an older world in which the forensic scientist oversees the processing of the scene. The problem, the skeptics say, is that crime scenes are no longer what they were. Many crimes now have a digital dimension, if only because many people have an online existence. This fact, combined with the ever-increasing sensitivity of forensic tools, means that a scene generates a huge volume of information — more than a single person can assess holistically. Then there is the risk of bias, about which we know much more than we did 20 years ago.
Irrelevant contextual information can skew the decisions of even the most competent and self-aware scientist. This happened in the case of Brandon Mayfield, the Oregon lawyer who was wrongly arrested for the 2004 Madrid bombing on the basis of a false fingerprint identification. Understanding of bias has fueled moves to separate forensic scientists from police and from each other, so that sub-disciplines increasingly work in informational siloes — the very trend the Sydney Declaration decries.
But Itiel Dror, a London-based neuroscientist who studies bias, thinks the declaration sets up a false dichotomy. Between the two bad options of having police do the scientific work or exposing scientists to bias, Dror says there is a good third option: Scientists examine the scene, and different scientists or technicians conduct the laboratory analyses. Those in the lab are given only task-relevant information which is unveiled to them sequentially, so that the potential for bias is minimized at each stage of their decision-making.
But then who decides what information is relevant? How much does each technician need to know about the case to conduct the right tests and make the evidence “speak” as accurately and informatively as possible? Isn’t bias in the eye of the beholder, shaped by their lived experience? Or to reiterate the initial question, where does context end and bias begin? To the authors of the Sydney Declaration, bias is ubiquitous and shape-shifting, and the scientific method is the best shield against it. And they have at least one ally in the director of Scotland’s forensics service, Fiona Douglas.
Scotland is unusual in that both its police force and its forensics service answer to the Scottish Police Authority but operate independently. In an interview, Douglas cited a 2017 case that was solved because forensic scientists directed the collection of evidence at the scene and pooled their expertise in the lab. Previously, only the surface of guns had been routinely swabbed, but working with DNA and fingerprint experts, a ballistics specialist took a gun apart and found traces of blood inside. This led to the identification of a man connected with an illegal firearms network and, ultimately, the dismantling of the network.
Three years on from the Sydney Declaration, the field still lacks a unified vision. Lacking that vision, the authors write, forensic science is turning out recruits who don’t know what they don’t know — technicians who never learned to examine a scene. If a discipline barely a century old wants to avoid falling victim to its own success, they warn, it must reinvent itself. Otherwise, it will forfeit the accolade of neutral truth teller, justice will suffer, and all of us will suffer with it.
