There are fundamental scientific inferential challenges to using neuroscientific data in the courts and to making policy, said Joshua Buckholtz. One does not have to look very far to find so-called experts willing to make extraordinary claims about the coming neurolegal revolution, he said. This is why proactive and thoughtful efforts are needed to define the general and specific criteria that neuroscientific evidence, primarily neuroimaging, needs to fulfill before using it to make valid inferences about the behaviors, actions, or intentions of an individual. Judge Patti Saris agreed, expressing her hope that the workshop would be a first step toward developing scientific standards for evaluating neuroimaging technologies as a means of helping judges and policy makers understand the reliability of current neuroscientific methods.
ADMISSIBILITY OF EXPERT TESTIMONY: BATTLE OF THE EXPERTS
Francis Shen, associate professor of law and McKnight Presidential Fellow at the University of Minnesota, noted that thousands of cases have included some type of neuroscientific evidence going back to the 1940s (Denno, 2015; Farahany, 2015). Importantly, he said, this evidence is nearly always introduced through testimony from a human expert who, given the adversarial system under which the law operates, is cross-
examined and/or countered by an opposing expert. According to Judge Saris, one of the most difficult decisions a judge must make is in regard to the admissibility of scientific expert testimony. For example, should this type of testimony be admitted to determine whether a defendant had the mens rea (criminal intent) to be capable of a crime?
Daubert hearings, discussed in Chapter 2, typically involve a battle of experts presented by both the plaintiff or prosecution and defense, and culminate in the judge issuing elaborate fact findings that are reviewable on appeal, said Judge Saris. David Faigman, the John F. Digardi Distinguished Professor of Law at the University of California Hastings College of Law and professor of psychiatry at the University of California, San Francisco, School of Medicine, added that the adversarial process leads to what might be called “expert mining,” where both sides find experts who agree with their litigation position. He argued that Daubert has actually made it more difficult to get expert testimony admitted, citing the Daubert case itself, as well as in her cases of the “Daubert trilogy”—Joiner and Kumho Tire—in which the expert testimony was ultimately excluded.1
Judge Saris added that the multiple requirements specified by the Daubert decision have also made the judge’s job more difficult. Before Daubert, judges could admit expert testimony if it was generally accepted by the scientific community. Now in federal courts and many state courts, the judge must be a “gate keeper” to determine if the evidence is reliable and relevant. The court must consider the five Daubert reliability factors2:
- Testability of scientific theory or technique
- “Whether the technique has been subject to peer review and publication
- Known or potential rate of error
- The existence and maintenance of standards controlling the technique’s operation
- General acceptance within the relevant scientific community”
Judge Saris highlighted the challenges of assessing these criteria; for example, can research conducted in animal studies published in a peer-reviewed journal translate to humans? What if the methodology and results in one peer-reviewed article is counter to what other researchers have
___________________
1General Electric Company v. Joiner, 522 U.S. 136 (1997); Kumho Tire Company v. Carmichael, 526 U.S. 137 (1999).
2Daubert v. Merrell Dow Pharmaceuticals, 509 U.S. 579 (1994).
shown in other journals? According to Shen, however, most evidence is never presented to a jury because the vast majority of civil cases are settled and criminal cases nearly always end with a plea bargain, in which the Federal Rules of Evidence do not apply. As part of a plea bargain, attorneys may present neuroscientific evidence regardless of its significance with the hope of getting a better deal for their client. The Federal Rules of Evidence also do not apply to what types of technology the police or government can use in their investigations, added Shen.
When presented in the courtroom, however, judges do have to apply Federal Rule of Evidence 702, as well as Rule 403, which Shen called a kind of “failsafe” rule. Rule 403 says the judge can “exclude relevant evidence if its probative value is substantially outweighed by danger of one or more of the following: unfair prejudice, confusing the issues, misleading the jury, undue delay, wasting time, or needlessly presenting cumulative evidence” (The National Court Rules Committee, 2018). As an example of how this has played out in the courtroom, Shen described a federal case in Tennessee, where a doctor was charged with defrauding the government by improperly billing for Medicare and Medicaid services.3 To prove their case, the government had to show that the doctor did this knowingly. The defense presented expert testimony from another doctor who claimed that fMRI scans indicated that the accused doctor was telling the truth about not cheating or defrauding the government. The judge excluded the evidence, based on Rule 702, ruling the technology was not ready for courtroom use, and also ruled that the “danger of unfair prejudice substantially outweighed the probative value of the evidence” (Rule 403).
When neuroscientific evidence is allowed to be introduced, it is often in the sentencing phase of a trial, where the Federal Rules of Evidence do not apply. Shen said little is known about how that evidence affects juror decision making. Shen mentioned one death penalty case in which quantitative EEG evidence was offered by the defense to show that the defendant had “a broken brain.” Three jurors interviewed by a reporter after the man was sentenced to life in prison indicated that the brain scans swayed their decisions (Miller, 2010). Shen and colleagues recently summarized empirical evidence regarding the effect of EEG memory recognition evidence on assessments of defendant credibility, concluding that there is “a tangled mess of contradictory findings” (Shen et al., 2017). He suggested that de-
___________________
3United States v. Semrau (2010), U.S. District Court for the Western District of Tennessee, No. 07-10074.
cisions based on Rule 403 will continue to be very contextualized and individualized, yet the data suggest that presentation of neuroscientific evidence can change outcomes. Shen compared the admissibility challenge of informative but incomplete brain evidence to instant replay in sports: It is used when available (meaning when the plaintiff has sufficient resources), when the stakes are high, and when the referee’s decision is not trusted (Shen, 2016).
TRANSLATING SCIENTIFIC CONCEPTS INTO LEGAL CONSTRUCTIONS (AND VICE VERSA)
Faigman discussed three issues that must be considered in translating scientific research into constructs appropriate for legal decision making: (1) moving from group data in science to individual decision making in law (known as G2i); (2) operationally defining legal/normative concepts into scientifically measurable terms; and (3) presenting scientific research in court using probabilities rather than qualitative or categorical substitutes.
According to Faigman, the inherent challenges of G2i arise from the different perspectives of science versus law. As he wrote in his 1999 book Legal Alchemy: The Use and Misuse of Science in the Law, “While science attempts to discover the universals hiding among the particulars, trial courts attempt to discover the particulars hiding among the universals” (Faigman, 1999, p. 69). Science, said Faigman, first gathers evidence at the “G” level, asking whether what is being presented in court represents a general phenomenon, and second whether scientific research supports it. In the courtroom, however, a different question must be answered, that is, whether this case represents an instance of that phenomenon. For example, if battered woman syndrome is offered as a defense, one must ask first whether this syndrome exists and examine the scientific evidence supporting that claim, if, and only if, research supports the existence of the general phenomenon, one must next ask whether this case is an instance of that phenomenon (Faigman, 2008). However, scientific evidence is inherently probabilistic, said Faigman, while the law requires categorical decisions, in other words, whether the plaintiff’s action is or is not attributable to the phenomenon offered as evidence. Jack Gallant noted that science’s focus on “G” may be changing as cognitive neuroscience research increasingly focuses on individual studies of individual subjects, leading to the acquisition of more information on individual differences.
Translating scientific evidence to support legal or normative concepts such as intelligence, competency, or volitional control also requires operational definitions that enable measurement, said Faigman. Competency is a particularly difficult concept to operationalize, he said, because it can mean many different things depending on whether one is asking if a person is competent to be executed, to receive a life sentence without parole, to waive Miranda rights, to consent to institutionalization in a psychiatric hospital, or to make a decision about abortion, for example. Faigman described a series of cases in which the Supreme Court addressed competency to be executed based on intellectual disability.4 These cases have been argued based on an evolving operational definition of intellectual disability under the Eighth Amendment, which “prohibits the imposition of cruel and unusual punishment.” In Atkins v. Virginia, the Court left the decision to the states to define intellectual disability. In Hall v. Florida, Florida defined intellectual disability as an IQ score of 70 or below. In Moore v. Texas, the Court ruled that intellectual disability must be informed by the Diagnostic and Statistical Manual of Mental Disorders, 5th ed. (DSM-5) (American Psychiatric Association, 2013). Faigman suggested that this may not be the last word on determining competency for execution because the DSM-5 requires the presence of three factors: significantly subaverage intellectual functioning (IQ), deficits in adaptive functioning, and onset of deficits during the developmental period. “In a death penalty case, intellectual functioning is important because of its correlation with the ability to understand the gravity of the crime and the purpose of the penalty, as well as the ability to resist a momentary impulse or the influence of others.”5 However, Faigman argued that adaptive functioning is more important in determining eligibility for social services, and onset during early development may have no bearing on the question of competency to be executed. Moreover, Faigman argued that the Supreme Court has yet to explain how these three factors associated with intellectual disability in the DSM-5 fit the two principles that help define the Eighth Amendment standard for whether a person is competent to be executed: retribution (i.e., that the plaintiff is worthy of blame) and deterrence (i.e., that the plaintiff is cognitively capable of being deterred or that others might be deterred by imposition of the death penalty).
___________________
4Roper v. Simmons, 543 U.S. 551 (2005); Atkins v. Virginia, 536 U.S. 304 (2002); Hall v. Florida, 572 U.S. (2014); Moore v. Texas, 581 U.S. (2017).
5Hall v. Florida, 572 U.S. (2014) (Alito, J., dissenting).
Buckholtz suggested that a lingua franca—a common language—between neuroscientists and legal scholars could allow them to work together to generate operational definitions by clarifying legal standards, articulating what the law requires, and then deciding if legally meaningful inferences are scientifically valid and if there is incremental value to neuroscientific evidence over the much less expensive behavioral data. Faigman, however, argued that it may not be possible to use common terms to describe what are very different concepts. For example, evidentiary reliability does not necessarily correspond with scientific validity, volitional control may not be accurately reflected in studies that use delayed-reward discounting, and insanity does not equate with mental illness. Moreover, presenting statistical and probabilistic information in court will remain a challenge given the need for courts to make categorical decisions. Shen added that with regard to the admissibility of patterns of brain activity or other measures discussed above as biomarkers, it may be more useful to think of these measures as “biosuggestors” because they are not conclusive. Moreover, handling biomarker evidence in a legal framework will be very challenging, he said.
Faigman added that decision thresholds are often very different for science than for law. For example, the cost of making a mistake in diagnosing mental illness for the purposes of drug treatment is far lower than for deciding whether that person would face civil commitment to an institution.
EVALUATING NEUROSCIENTIFIC EVIDENCE
A problem with meeting requirements of the Daubert rule, said Faigman, is that there is currently no way for the court to know what the mainstream academic opinion is. Moreover, Judge Nancy Gertner, who served as U.S. District Judge of the U.S. District Court for the District of Massachusetts until her retirement in 2011, suggested that the courts may not be the best gatekeepers regarding which neuroscientific evidence should be admitted. Scientifically unsound forensic evidence is frequently admitted, she said, noting that the National Research Council in 2009 published a report that concluded that the courts have been utterly ineffective in assessing the research basis for forensic science (NRC, 2009). More recently, the President’s Council of Advisors on Science and Technology (PCAST) issued a report to the president on ensuring scientific validity of forensic evidence admitted in criminal courts, concluding that clearer
standards are needed to establish the validity and reliability of forensic methods (PCAST, 2016).
Neuroscientific evidence raises different kinds of questions in comparison to other forensic evidence, said Judge Gertner, in part because forensic evidence was established outside of any scientific context, and many types of forensic evidence have little scientific information to support them. Indeed, she said that criminal law is based, in large part, on normative value judgments. Legal concepts related to addiction, alcoholism, the impact of toxic stress, self-defense, and mens rea have not been adequately mapped onto validated neuroscientific concepts, she said, leading to policies such as Florida’s stand-your-ground doctrine that not only have nothing to do with science but may even run counter to what neuroscience would suggest. The law treats addiction and alcoholism as voluntary choices, although neuroscience has made clear that there is a continuum in terms of the consequences of that first drink or first use of a drug. After several years of substance abuse, the concept of choice may become an illusion, she said. Moreover, while there are rules for some legal concepts such as provocation, these rules are gendered and troubling, based on normative decisions and value judgments, said Judge Gertner.
Neuroscience as it relates to the law also must be viewed in the context of the broader legal system, where lack of resources, ineffective assistance of counsel, and the absence of discovery may affect the outcome of a trial, she added. Decisions about admitting evidence become especially problematic at sentencing, where there are profound resource problems and the Federal Rules of Evidence do not apply, said Judge Gertner, leading to evidence being accepted that is “good enough,” but not necessarily good. She said there are virtually no limits on what a judge can consider at sentencing, including untested generalizations about public safety, deterrence, risk assessment and prediction, and other factors that have nothing to do with the individual.
As described earlier, neuroscience may help elucidate legal constructs such as mitigation and aggravation, although such evidence may be less relevant to the question of culpability than it is to sentencing or rehabilitation and thus may require different rules regarding admissibility, said Judge Gertner.
Neuroforensics Training for Judges, Attorneys, and Law Enforcement
Tor Wager commented that one of the challenges for judges, attorneys, and law enforcement is addressing the knowledge gap—knowing whom and what to trust and under what circumstances. Judges need to understand the reasons why brain models may be wrong, for example, relying on evidence collected using different populations and different scanners, he said. They also need to understand probability and the limitations of evidence as conclusive proof, said Faigman. For example, he noted that for decades, firearms experts have been allowed to testify (usually without challenge) that a cartridge case can be linked to a particular gun to the exclusion of all other guns in the world, although this is a clear overstatement of what the evidence actually supports.
Judge Barbara Parker Hervey of the Texas Court of Criminal Appeals suggested that training of judges, attorneys, and law enforcement can bridge the gap between science and law, but it requires new legislation and adequate funding. Judge Hervey oversees a grant from the Texas legislature to educate judges, defense attorneys, and prosecutors on issues such as actual innocence and the challenges relating to eyewitness identification, false confessions, mental illness, and the use of informants. In 2008, they established the Texas Criminal Integrity Unit to look at wrongful convictions with the aim of making changes at the front end of the system rather than simply catching mistakes at the back end. Judge Hervey emphasized the need to collaborate with scientists, lawyers, and other stakeholders to ensure that all are trained at the same level. Shen added that some of the problems might be mitigated by training not only judges but also law students.
However, Judge Gertner countered that judicial education and training are not remotely enough to ensure appropriate decision making with respect to admission of evidence. She cited multiple examples of cognitive bias in courts, which have led to asymmetric decision making. For example, given time pressures facing judges, they typically only write opinions in cases that have been dismissed, which results in the law evolving on the basis of losing cases. Courts of appeals only see cases in which forensic evidence was admitted and the defendant was convicted, thus skewing the law in favor of admitted evidence. Precedent also strongly influences acceptance of evidence, she said, leading to old science being favored over new science. An indicator of this, she noted, is the fact that the admissibil-
ity of DNA evidence was held up for years while trace evidence was routinely admitted without objection. Courts also tend to rely on information that readily comes to mind and have difficulty with self-correction, she said.
Faigman also suggested that many lawyers and judges lack an understanding of basic research methods and statistics and thus may not be educable on the intricacies and limitations of neuroscience. Judges have to recognize when they are at the edge of their understanding, said Judge Gertner. She said the PCAST report attempted to address this issue, suggesting that with regard to expert scientific testimony, there may be a range within which a judge may appropriately base a decision, but beyond which additional technical advice is needed (PCAST, 2016).
Providing Judges with Peer-Reviewed Advice
Given that judges need to evaluate scientific evidence in real time and make decisions, Faigman suggested providing them with technical advisors with whom they can discuss evidentiary matters. He and others have started companies that send out expert reports for peer review from mainstream academic scientists. He said the American Association for the Advancement of Science (AAAS) also has established a program to identify court-appointed experts or technical advisors. Buckholtz advocated for a peer-review system to evaluate scientific information under the special master template, which would allow scientific information to be evaluated by an independent review conducted by scientists with relevant knowledge. He acknowledged, however, that there is no standardized system for conducting peer review of evidence such as the system that exists for peer review of grants and journals. Faigman added that many judges believe in the adversarial process and have been resistant to the idea of technical advisors or court-appointed experts.
