6
Findings and Recommendations
Eyewitnesses make mistakes. Our understanding of how to improve the accuracy of eyewitness identifications is imperfect and evolving. In the previous chapters, we described law enforcement procedures to elicit accurate eyewitness identifications; the courts’ handling of eyewitness identification evidence; the science of visual perception and memory as it applies to eyewitness identifications; and the contributions of scientific research to our understanding of the variables that affect the accuracy of identifications. On the basis of its review, the committee offers its findings and recommendations for
- identifying and facilitating best practices in eyewitness procedures for the law enforcement community;
- strengthening the value of eyewitness identification evidence in court; and
- improving the scientific foundation underpinning eyewitness identification.
OVERARCHING FINDINGS
The committee is confident that the law enforcement community, while operating under considerable pressure and resource constraints, is working to improve the accuracy of eyewitness identifications. These efforts, however, have not been uniform and often fall short as a result of insufficient training, the absence of standard operating procedures, and the continuing
presence of actions and statements at the crime scene and elsewhere that may intentionally or unintentionally influence eyewitness’ identifications.
Basic scientific research on human visual perception and memory has provided an increasingly sophisticated understanding of how these systems work and how they place principled limits on the accuracy of eyewitness identification (see Chapter 4).1 Basic research alone is insufficient for understanding conditions in the field and thus has been augmented by studies applied to such specific practical problem of eyewitness identification (see Chapter 5). Such applied research has identified key variables that affect the accuracy and reliability of eyewitness identifications and has been instrumental in informing law enforcement, the bar, and the judiciary of the frailties of eyewitness identification testimony.
A range of best practices has been validated by scientific methods and research and represents a starting place for efforts to improve eyewitness identification procedures. A number of law enforcement agencies have, in fact, adopted research-based best practices. This report makes actionable recommendations on, for example, the importance of adopting “blinded” eyewitness identification procedures. It further recommends that standardized and easily understood instructions be provided to eyewitnesses and calls for the careful documentation of eyewitness’ confidence statements. Such improvements may be broadly implemented by law enforcement now. It is important to recognize, however, that, in certain cases, the state of scientific research on eyewitness identification is unsettled. For example, the relative superiority of competing identification procedures (i.e., simultaneous versus sequential lineups) is unresolved.
The field would benefit from collaborative research among scientists and law enforcement personnel in the identification and validation of new best practices that can improve eyewitness identification procedures. Such a foundation can be solidified through the use of more effective research designs (for example, those that consider more than one variable at a time, and in different study populations to ensure reproducibility and generalizability), more informative statistical measures and analyses (i.e., methods from statistical machine learning and signal detection theory to evaluate the performance of binary classification tasks), more probing analyses of research findings (such as analyses of consequences of data uncertainties), and more sophisticated systematic reviews and meta-analyses (that take
_______________
1Basic research on vision and memory seeks a comprehensive understanding of how these systems are organized and how they operate generally. The understanding derived from basic research includes principles that enable one to predict how a system (such as vision or memory) might behave under specific conditions (such as those associated with witnessing a crime), and to identify the conditions under which it will operate most effectively and those under which it will fail. Applied research, by contrast, empirically evaluates specific hypotheses about how a system will behave under a particular set of real-world conditions.
account of current guidelines, including transparency and reproducibility of methods).
In view of the complexity of the effects of both system and estimator variables and their interactions on eyewitness identification accuracy, better experimental designs that incorporate selected combinations of these variables (e.g., presence or absence of a weapon, lighting conditions, etc.) will elucidate those variables with meaningful influence on eyewitness performance, which can, in turn, inform law enforcement practice of eyewitness identification procedures. To date, the eyewitness literature has evaluated procedures mostly in terms of a single diagnosticity ratio or an ROC (Receiver Operating Characteristic) curve; even if uncertainty is incorporated into the analysis, many other powerful tools for evaluating a “binary classifier” are available and worthy of consideration.2 Finally, syntheses of eyewitness research has been limited to meta-analyses that have not been conducted in the context of systematic reviews. Systematic reviews of stronger research studies need to conform to current standards and be translated into terms that are useful for decision-makers.
The committee offers the following recommendations to strengthen the effectiveness of policies and procedures used to obtain accurate eyewitness identifications.
RECOMMENDATIONS TO ESTABLISH BEST PRACTICES FOR THE LAW ENFORCEMENT COMMUNITY
The committee’s review of law enforcement practices and procedures, coupled with its consideration of the scientific literature, has identified a number of areas where eyewitness identification procedures could be strengthened. The practices and procedures considered here involve acquisition of data that reflect a witness’ identification and the contextual factors that bear on that identification. A recurrent theme underlying the committee’s recommendations is development of, and adherence to, guidelines that are consistent with scientific standards for data collection and reporting.
Recommendation #1: Train All Law Enforcement Officers in Eyewitness Identification
The resolution and accuracy of visual perceptual experience, as well as the fidelity of our memories to events perceived, may be compromised by many factors at all stages of processing (see Chapter 4). Perceptual experiences are limited by uncertainties and biased by expectations. Unknown
_______________
2T. Hastie, R. Tibshirani, and J. H. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction (New York: Springer, 2009).
to the individual, memories are forgotten, reconstructed, updated, and distorted. An eyewitness’s memory can be contaminated by a wide variety of influences, including interaction with the police.
The committee recommends that all law enforcement agencies provide their officers and agents with training on vision and memory and the variables that affect them, on practices for minimizing contamination, and on effective eyewitness identification protocols. In addition to instruction at the police academy, officers should receive periodic refresher training, and officers assigned to investigative units should receive in-depth instruction. Dispatchers should be trained not to “leak” information from one caller to the next and to ask for information in a non-leading way. Police officers should be trained to ask open-ended questions, avoid suggestiveness, and efficiently manage scenes with multiple witnesses (e.g., minimize interactions among witnesses).
Recommendation #2: Implement Double-Blind Lineup and Photo Array Procedures
Decades of scientific evidence demonstrate that expectations can bias perception and judgment and that expectations can be inadvertently communicated.3 Even when lineup administrators scrupulously avoid comments that could identify which person is the suspect, unintended body gestures, facial expressions, or other nonverbal cues have the potential to inform the witness of his or her location in the lineup or photo array.
Double-blinding is central to the scientific method because it minimizes the risk that experimenters might inadvertently bias the outcome of their research, finding only what they expected to find. For example, in medical clinical trials, double-blind designs are crucial to account for experimenter biases, interpersonal influences, and placebo effects.
To minimize inadvertent bias, double-blinding procedures are sometimes used in which the test administrator does not know the composition of the photo array or lineup. If administrators are not involved with construction of the lineup and are unaware of the placement of the potential suspect in the sequence, then they cannot influence the witness.
Some in the law enforcement community have responded to calls for double-blind lineup administration with concern, citing the potential for increased financial costs and human resource demands. The committee believes there are ways to reduce these costs and recommends that police departments consider procedures and new technologies that increase efficiency of data acquisition under double-blind procedures or those procedures that closely approximate double-blind procedures. If an administrator who does
_______________
3See Box 2-1.
not know the identity of the suspect cannot be assigned to the task, then a non-blind administrator (one knowing the status of the individuals in the lineup) might use a computer-automated presentation of lineup photos. If computer-based presentation technology is unavailable, then the administrator could place photos in numbered folders that are then shuffled, as is current practice in some jurisdictions.
The committee recommends blind (double-blind or blinded) administration of both photo arrays and live lineups and the adoption of clear, written policies and training on photo array and live lineup administration. Police should use blind procedures to avoid the unintentional or intentional exchange of information that might bias an eyewitness. The “blinded” procedure minimizes the possibility of either intentional or inadvertent suggestiveness and thus enhances the fairness of the criminal justice system. Suggestiveness during an identification procedure can result in suppression of both out-of-court and in-court identifications and thereby seriously impair the prosecutions’s ability to prove its case beyond a reasonable doubt. The use of double-blind procedures will eliminate a line of cross-examination of officers in court.
Recommendation #3: Develop and Use Standardized Witness Instructions
The committee recommends the development of a standard set of easily understood instructions to use when engaging a witness in an identification procedure.
Witnesses should be instructed that the perpetrator may or may not be in the photo array or lineup and that the criminal investigation will continue regardless of whether the witness selects a suspect. Administrators should use witness instructions consistently in all photo arrays or lineups, and can use pre-recorded instructions or read instructions aloud, in the manner of the mandatory reading of Miranda Rights. Accommodations should be made when questioning non-English speakers or those with restricted linguistic ability. Additionally, the committee recommends the development and use of a standard set of instructions for use with a witness in a showup.
Recommendation #4: Document Witness Confidence Judgments
Evidence indicates that self-reported confidence at the time of trial is not a reliable predictor of eyewitness accuracy.4 The relationship between the witness’ stated confidence and accuracy of identifications may be greater at the moment of initial identification than at the time of trial. However, the strength of the confidence-accuracy relationship varies, as it depends on complex interactions among such factors as environmental conditions, persons involved, individual emotional states, and more.5 Expressions of confidence in the courtroom often deviate substantially from a witness’ initial confidence judgment, and confidence levels reported long after the initial identification can be inflated by factors other than the memory of the suspect. Thus, the committee recommends that law enforcement document the witness’ level of confidence verbatim at the time when she or he first identifies a suspect, as confidence levels expressed at later times are subject to recall bias, enhancements stemming from opinions voiced by law enforcement, counsel and the press, and to a host of other factors that render confidence statements less reliable. During the period between the commission of a crime and the formal identification procedure, officers should avoid communications that might affect a witness’ confidence level. In addition, to avoid increasing a witness’ confidence, the administrator of an identification procedure should not provide feedback to a witness. Following a formal identification, the administrator should obtain level of confidence by witness’ self-report (this report should be given in the witness’ own words) and document this confidence statement verbatim. Accommodations should be made for non-English speakers or those with restricted linguistic ability.
Recommendation #5: Videotape the Witness Identification Process
The committee recommends that the video recording of eyewitness identification procedures become standard practice.
_______________
4See, e.g., C. M. Allwood, J. Knutsson, and P. A. Granhag, “Eyewitnesses Under Influence: How Feedback Affects the Realism in Confidence Judgements,” Psychology, Crime, and Law 12(1): 25–38 (2006); B. H. Bornstein and D. J. Zickafoose, “‘I Know I Know It, I Know I Saw It’: The Stability of the Confidence-Accuracy Relationship Across Domain,” Journal of Experimental Psychology-Applied 5(1): 76–88 (1999); P. A. Granhag, L. A. Stromwall, and C. M. Allwood, “Effects of Reiteration, Hindsight Bias, and Memory on Realism in Eyewitness Confidence,” Applied Cognitive Psychology 14(5): 397–420 (2000); and H. L. Roediger III, J. T. Wixted, and K. A. DeSoto, “The Curious Complexity between Confidence and Accuracy in Reports from Memory” in Memory and Law, ed. L. Nadel and W. P. Sinnott-Armstrong (Oxford: Oxford University Press, 2012).
5See, e.g., J. M. Talarico and D. C. Rubin, “Confidence, Not Consistency, Characterizes Flashbulb Memories,” Psychological Science 14(5): 455–461 (September 2003).
Although videotaping does have drawbacks (e.g., costs, witness advocates opposing videotaping of witnesses’ faces, and witnesses not wanting to be videotaped), it is necessary to obtain and preserve a permanent record of the conditions associated with the initial identification. When necessary, efforts should be made to obtain non-intrusive recordings of the initial identification process and to accommodate non-English speakers or those with restricted linguistic ability. Measures should also be taken to protect the identity of eyewitnesses who may be at risk of harm because they make an identification.
RECOMMENDATIONS TO STRENGTHEN THE VALUE OF EYEWITNESS IDENTIFICATION EVIDENCE IN COURT
The best guidance for legal regulation of eyewitness identification evidence comes not from constitutional rulings, but from the careful use and understanding of scientific evidence to guide fact-finders and decision-makers. The Manson v. Brathwaite test under the Due Process Clause of the U.S. Constitution for assessing eyewitness identification evidence was established in 1977, before much applied research on eyewitness identification had been conducted. That test evaluates the “reliability” of eyewitness identifications using factors derived from prior rulings and not from empirically validated sources. As critics have pointed out, the Manson v. Brathwaite test includes factors that are not diagnostic of reliability. Moreover, the test treats factors such as the confidence of a witness as independent markers of reliability when, in fact, it is now well established that confidence judgments may vary over time and can be powerfully swayed by many factors. While some states have made minor changes to the due process framework, (e.g., by altering the list of acceptable “reliability” factors; see Chapter 3), wholesale reconsideration of this framework is only a recent development (e.g., the recent decisions by state supreme courts in New Jersey and Oregon; see Chapter 3).
Recommendation #6: Conduct Pretrial Judicial Inquiry
Eyewitness testimony is a type of evidence where (as with forms of forensic trace evidence) contamination may occur pre-trial. Judges rarely make pre-trial inquiries about evidence in criminal cases without one of the parties first raising an objection. In cases involving eyewitness evidence, however, parties may not be sufficiently knowledgeable about the relevant scientific research to raise concerns.
Judges have an affirmative obligation to insure the reliability of evidence presented at trial. To meet this obligation, the committee recommends that, as appropriate, a judge make basic inquiries when eyewitness
identification evidence is offered. While the contours of such an inquiry would need to be established on a case-by-case basis, at a minimum, the judge could inquire about prior lineups, what information had been given to the eyewitness before the lineup, what instructions had been given to the eyewitness in connection with administering the lineup, and whether the lineup had been administered “blindly.” The judge could also entertain requests from the parties for additional discovery and could ask the parties to brief any issues raised by these inquiries. A judge also could review reports of the eyewitness’ confidence and any recordings of the identification procedures. When assessing the reliability of an identification, a judge could also inquire as to what eyewitness identification procedures the agency had in place and the degree to which they were followed. Both pre-trial judicial inquiries and any subsequent judicial review would create an incentive for agencies to adopt written eyewitness identification procedures and to document the identifications themselves.
If these initial inquiries raise issues with the identification process, a judge could conduct a pre-trial hearing to review the reliability and admissibility of eyewitness identification evidence and to assess how it should be treated at trial if found admissible. If indicia of unreliable eyewitness identifications are present, the judge should apply applicable law in deciding whether to exclude the identifications or whether some lesser sanction is appropriate. As discussed in the sections that follow, a judge may limit portions of the testimony of the eyewitness. A judge can also ensure that the jury is provided with a scientific framework within which to evaluate the evidence.
Recommendation #7: Make Juries Aware of Prior Identifications
The accepted practice of in-court eyewitness identifications can influence juries in ways that cross-examination, expert testimony, or jury instructions are unable to counter effectively. Moreover, as research suggests (see Chapters 4 and 5), the passage of time since the initial identification may mean that a courtroom identification is a less accurate reflection of an eyewitness’ memory. In-court confidence statements may also be less reliable than confidence judgments made at the time of an initial out-of-court identification; as memory fails and/or confidence grows disproportionately. The confidence of an eyewitness may increase by the time of the trial as a result of learning more information about the case, participating in trial preparation, and experiencing the pressures of being placed on the stand.
An identification of the kind dealt with in this report typically should not occur for the first time in the courtroom. If no identification procedure was conducted during the investigation, a judge should consider ordering that an identification procedure be conducted before trial. In any case,
whenever the eyewitness identifies a suspect in the courtroom, it is important for jurors to hear detailed information about any earlier identification, including the procedures used and the confidence expressed by the witness at that time. The descriptions of prior identifications and confidence at the time of those earlier out-of-court identifications provide more useful information to the fact-finders and decision-makers. Accordingly, the committee recommends that judges take all necessary steps to make juries aware of prior identifications, the manner and time frame in which they were conducted, and the confidence level expressed by the eyewitness at the time.
Recommendation #8: Use Scientific Framework Expert Testimony
The committee finds that a scientific framework describing what factors may influence a witness’ visual experience of an event and the resolution and fidelity of that experience, as well as factors that underlie and influence subsequent encoding, storage, and recall of memories of an event, can inform the fact-finder in a criminal case. As discussed throughout this report, many scientifically established aspects of eyewitness memory are counter-intuitive and may defy expectations. Jurors will likely need assistance in understanding the factors that may affect the accuracy of an identification. In many cases this information can be most effectively conveyed by expert testimony.
Contrary to the suggestion of some courts, the committee recommends that judges have the discretion to allow expert testimony on relevant precepts of eyewitness memory and identifications. Expert witnesses can explain scientific research in detail, capture the nuances of the research, and focus their testimony on the most relevant research. Expert witnesses can convey current information based on the state of the research at the time of a trial. Expert witnesses can also be cross-examined, and limitations of the research can be expressed to the jury.
Certainly, qualified experts will not be easy to locate in a given jurisdiction; and indigent defendants may not be able to afford experts absent court funds. Moreover, once the defense secures an expert, the prosecution may retain a rebuttal expert, adding complexity to the litigation. Further investigation may explore the effectiveness of expert witness presentation of relevant scientific findings compared with jury instructions. Until there is a clearer understanding of the strengths and weaknesses of this technique, the committee views expert testimony as an appropriate and effective means of providing the jury with information to assess the strength of the eyewitness identification.
Expert witnesses should not be permitted to testify without limits. An expert explaining the relevant scientific framework can describe the state of the research and focus on the factors that are particularly relevant in a
given case. However, an expert must not be allowed to testify beyond the limits of his or her expertise. Although current scientific knowledge would allow an expert to inform the jury of factors bearing on their evaluation of an eyewitness’ identification, the committee has seen no evidence that the scientific research has reached the point that would properly permit an expert to opine, directly or through an equivalent hypothetical question, on the accuracy of an identification by an eyewitness in a specific case.
In many jurisdictions, expert witnesses who can testify regarding eyewitness identification evidence may be unavailable. In state courts, funding for expert witnesses may be far more limited than funding in federal courts. The committee recommends that local jurisdictions make efforts to ensure that defendants receive funding to obtain access to qualified experts.
Recommendation #9: Use Jury Instructions as an Alternative Means to Convey Information
The committee recommends the use of clear and concise jury instructions as an alternative means of conveying information regarding the factors that the jury should consider.
Jury instructions should explain, in clear language, the relevant principles. Like the New Jersey instructions,6 the instructions should allow judges to focus on factors relevant to the specific case, since not all cases implicate the same factors. Jury instructions do not need to be as detailed as the New Jersey model instructions and do not need to omit all reference to underlying research. With the exception of the New Jersey instructions, jury instructions have tended to address only certain subjects, or to repeat the problematic Manson v. Brathwaite language, which was not intended as instructions for jurors.
Appropriate legal organizations, together with law enforcement, prosecutors, defense counsel, and judges, should convene a body to establish model jury instructions regarding eyewitness identifications.
_______________
6New Jersey Criminal Model Jury Instructions, Identification (July 19, 2012), available at: http://www.judiciary.state.nj.us/pressrel/2012/jury_instruction.pdf. New Jersey Court Rule 3:11, Record of an Out-of-Court Identification Procedure (July 19, 2012), available at: http://www.judiciary.state.nj.us/pressrel/2012/new_rule.pdf, New Jersey Court Rule 3:13-3. Discovery and Inspection (July 19, 2012), available at: http://www.judiciary.state.nj.us/pressrel/2012/rev_rule.pdf.
RECOMMENDATIONS TO IMPROVE THE SCIENTIFIC FOUNDATION UNDERPINNING EYEWITNESS IDENTIFICATION RESEARCH
Basic scientific research on visual perception and memory provides important insight into the factors that can limit the fidelity of eyewitness identification (see Chapter 4). Research targeting the specific problem of eyewitness identification (see Chapter 5) complements basic scientific research. However, this strong scientific foundation remains insufficient for understanding the strengths and limitations of eyewitness identification procedures in the field. Many of the applied studies on key factors that directly affect eyewitness performance in the laboratory are not readily applicable to actual practice and policy. Applied research falls short because of a lack of reliable or standardized data from the field, a failure to include a range of practitioners in the establishment of research agendas, the use of disparate research methodologies, failure to use transparent and reproducible research procedures, and inadequate reporting of research data. The task of guiding eyewitness identification research toward the goal of evidence-based policy and practice will require collaboration in the setting of research agendas and agreement on methods for acquiring, handling, and sharing data.
Recommendation #10: Establish a National Research Initiative on Eyewitness Identification
To further our understanding of eyewitness identification, the committee recommends the establishment of a National Research Initiative on Eyewitness Identification (hereinafter, the Initiative). The Initiative should involve the academic research community, law enforcement community, the federal government, and philanthropic organizations. The Initiative should (1) establish a research agenda to guide research for the next decade; (2) formulate practice- and policy-relevant research questions; (3) identify opportunities for additional data collection; (4) systematically review research to examine emerging findings on the impact of system and estimator variables; (5) translate research findings into policies and procedures that are both practical and appropriate for law enforcement; and (6) set priorities and timelines for issues to be addressed, the conduct of research, the development of best practices, and formal assessments.
The committee notes that there appear to be few existing partnerships between the scientific community and law enforcement organizations and therefore recommends that the National Science Foundation (NSF) and the National Institute of Standards and Technology (NIST) take a leadership role working with other federal agencies, such as the National Institute of
Justice (NIJ), the Bureau of Justice Statistics (BJS), and the Federal Bureau of Investigation (FBI), to support such collaborations.
The impact on society of innocents being incarcerated while perpetrators remain free, in conjunction with limited federal resources, highlights the need for both public and private support for this Initiative.
To enhance the scientific foundation of eyewitness identification research and practice, the Initiative should commit to the following:
a. Include a practice- and data-informed research agenda that incorporates input from law enforcement and the courts and establishes methodological and reporting standards for research to assess the fundamental performance of various aspects of eyewitness identification procedures as well as synthesize research findings across studies.
b. Develop protocols and policies for the collection, preservation, and exchange of field data that can be used jointly by the scientific and law enforcement communities. Data collection procedures used in the field should be developed to ensure the relevance of the collected data, to facilitate analysis of the data, and to minimize potential bias and loss of data through incomplete recording strategies.
Law enforcement agencies should take the lead in collecting, maintaining, and sharing relevant data from the field. Much of the data that would be useful for the evaluation of eyewitness identification procedures have been collected in the form of administrative records and may be readily adapted for use in research. Comprehensive data should be collected on lineup composition and witness selections (i.e., fillers, non-identifications, and position of suspect in lineup).
c. Develop and adopt guidelines for the conduct and reporting of applied scientific research on eyewitness identification that conform to the highest scientific standards. All eyewitness research, including field-based studies, laboratory-based studies, and research synthesis, should use rigorous research methods and provide detailed reporting of both methods and results, including (1) pre-registration of all study protocols; (2) investigation of research questions and hypotheses informed by the needs of practice and policy; (3) adoption of strict operationalization of key measures and objective data collection; (4) development of experimental designs informed by analytical concerns; (5) use of proper statistical procedures that account for the often nontraditional nature of data in this field (e.g., estimates of effects with appropriate state-
ments of uncertainty, multiple responses from different scenarios from the same individuals, effects of order and time of presentation when important, treatment of extreme observations or outliers); (6) reporting of participant recruitment and selection and assignment to conditions; (7) complete reporting of findings including effect sizes and associated confidence intervals for both significant and non-significant effects; and (8) derivation of conclusions that are grounded firmly in the findings of the study, are framed in the context of the strengths and limitations of study methodology, and clearly state their implications for practice and policy decisions.
Strict adherence to guidelines for eyewitness identification research will result in more credible research findings that can guide policy and practice. Research that conforms to guidelines will withstand rigorous scrutiny by peers, will be verifiable through replication, and will permit inclusion in systematic reviews, leading to greater confidence in the validity and generalizability of findings.
d. Adopt rigorous standards for systematic reviews and meta-analytic studies. Meta-analyses of primary studies should be conducted only in the context of systematic reviews that locate and critically appraise all research findings, including those from unpublished studies. Analyses should consistently appraise and account for possible biases in the included research. Studies that do not adequately conduct or report research methods, such as randomization, should be identified in the findings. Sensitivity analyses considering impacts of lower quality or inadequately reported studies on pooled effect estimates should be conducted and reported. When attempting to draw conclusions from studies with missing data, reviewers should first attempt to contact the authors of the research for additional information. When missing data cannot be retrieved from researchers, imputation methods should, if used, be specific, transparent, and reproducible. Statistical methods for meta-analysis should conform to current best practice, using models appropriate to the level of heterogeneity of results across studies, computing both point estimates and confidence intervals around effect sizes, and translating the results of meta-analyses into terms that are both understandable and useful to practice and policy decision makers.
e. Provide basic instruction for police, prosecutors, defense counsel, and judges on aspects of the scientific method relevant to eyewitness identifications procedures (e.g., the rationale for blinded administration), including principles of research design and the uncertainties associated with data analysis. Training should cover the
importance of data collection and interpretation, including the role of standardized eyewitness identification procedures and documentation of witness statements of confidence. Competencies acquired through such training (quantitative reasoning, understanding principles of research design, and recognition of data uncertainties) are likely to apply to issues beyond eyewitness identification. For example, the knowledge and skills from training can be applied to other issues that personnel face, either in forensic science technologies or in process administration, evaluation, and quality improvement. Similarly, scientists will benefit from a greater knowledge of legal issues, standards, and procedures related to the problem of eyewitness identification. Training of both communities (law and science) will enhance communication and lead to productive collaborations.
The collaborative research initiative between researchers and law enforcement communities will be challenging as it will necessitate (1) standardized police procedures;7 (2) systematic valid evidence collection and data entry and analysis; and (3) education and training for both researchers and law enforcement professionals on the differences between these two communities in their use of terms and considerations of standards of evidence and uncertainties in data. These three elements of a collaborative initiative are critical to advancing the science related to eyewitness identifications, as each bears directly on the integrity of the foundation upon which the efficacy and validity of current and future practices will be judged. Without such a foundation, practical advances in our scientific understanding are unlikely to occur.
The committee further recommends that the Initiative support research to better understand the following: (1) the variables that affect the accuracy, precision, and reliability of eyewitness identifications, and how those variables interact and vary in practice; (2) the (possibly joint) impact of estimator and system variables on both identification accuracy and response bias; (3) best practices for probing witness memory with the least potential for bias or contamination; (4) best strategies to assess witnesses’ confidence levels when making an identification; (5) appropriate types of instructions for police, witnesses, and juries to best inform and facilitate the collection and interpretation of eyewitness identifications; (6) photo array composi-
_______________
7The term standardized procedures refers to the notion that professionals reliably follow the same set of steps or procedures. Such standardization ensures that data across cases can be considered comparable and, to a greater extent, more reliable. Although reliability is not equivalent to validity, it is essential before researchers can assess questions of validity. Without standardized procedures, valid comparisons between departments and regions of the country cannot be achieved.
tion and procedures; (7) identification procedures in the field (showups); (8) innovative technologies that might increase the reliability of eyewitness testimony (e.g., algorithm-based computer face recognition software, computer administered photo arrays, and mobile technologies with photo identification programs); and (9) the most effective means of informing jurors how to consider the factors that affect the strengths and weaknesses of eyewitness identification evidence.
Recommendation #11: Conduct Additional Research on System and Estimator Variables
Among the many variables that can affect eyewitness identification, the procedures for constructing a lineup have received the greatest attention in recent years. As discussed in Chapter 5, the question as to whether a simultaneous or sequential lineup is preferred is a specific case of the more general question of what conditions might improve the performance of an eyewitness. The answer to that question depends upon the criteria used to evaluate performance, and much of the debate has thus focused on the analysis tools for evaluation. These tools have improved significantly over the years, beginning with the use of a diagnosticity ratio, which uses the likelihood that the person identified is actually guilty as an evaluation criterion. More recently, the diagnosticity ratio approach has been augmented by analysis of Receiver Operating Characteristics (ROC analysis), which uses a measure of discriminability (i.e., a measure of how well the witness can discriminate between different possible matches to his or her memory of the face of the culprit) as an evaluation criterion. In principle, ROC analysis is a positive step, if only because it incorporates more information (i.e., the earlier diagnosticity ratio is one component of the ROC analysis). But a more complex question concerns how policy-makers and practitioners should weigh the two evaluation criteria that have been considered thus far—likelihood of guilt and discriminability—when making a decision about which lineup procedures to adopt. The answer is particularly nuanced because the two criteria do not always lead to the same conclusion; one lineup procedure may yield poorer discriminability while at the same time increasing the likelihood that the identified person is actually guilty.
The committee concludes that there should be no debate about the value of greater discriminability—to promote a lineup procedure that yields less discriminability would be akin to advocating that the lineup be performed in dim instead of bright light. For this reason, the committee recommends broad use of statistical tools that can render a discriminability measure to evaluate eyewitness performance. But a lineup procedure that improves discriminability can yield greater or lesser likelihood of correct identification, depending on how the procedure is applied (see Chapter 5).
For lineup procedures that yield greater discriminability, greater likelihood of correct identification would appear preferable and can be achieved by methods that elicit a more conservative response bias, such as a sequential (relative to simultaneous) lineup procedure.8 The committee thus recommends a rigorous exploration of methods that can lead to more conservative responding (such as witness instructions) but do not compromise discriminability.
In view of these considerations of performance criteria and recommendations about analysis tools, can we draw definitive conclusions about which lineup procedure (sequential or simultaneous) is preferable? At this point, the answer is no. Using discriminability as a criterion, there is, as yet, not enough evidence for the advantage of one procedure over another. The committee thus recommends that caution and care be used when considering changes to any existing lineup procedure, until such time as there is clear evidence for the advantages of doing so. From a larger perspective, the identification of factors (such as specific lineup procedures or states of other system variables) that can objectively improve eyewitness identification performance must be among the top priorities for this field. This leads us to three additional recommendations.
a. The committee recommends a broad exploration of the merits of different statistical tools for use in the evaluation of eyewitness performance. ROC analysis represents an improvement over a single diagnosticity ratio, yet there are well-documented quantitative shortcomings to the ROC approach. But are there alternatives? As noted in Chapter 5, the task facing an eyewitness is a binary classification task and there exist many powerful statistical tools for evaluation of binary classification performance that are widely used, for example, in the field of machine learning. While none of these tools has been vetted for application to the problem of eyewitness identification, they offer a potentially rich resource for future investigation in this field.
b. The alternative (sequential) lineup procedure was introduced as part of an effort to improve eyewitness performance. While, as noted above, it remains unclear whether the procedure has improved eyewitness performance, that goal is still primary. In an effort to achieve that goal, many studies over the past three decades have explored the possibility that other factors may also affect performance, but until recently these investigations have not
_______________
8The committee stresses, however, that adoption of a more conservative response bias necessitates a compromise by which fewer lineup “picks” are made overall and thus fewer guilty suspects are identified (see Chapter 5).
evaluated performance using a discriminability measure. The committee therefore recommends a broad exploration of the effects of different system variables (e.g., additional variants on lineup procedures, witness lineup instructions) and estimator variables (e.g. presence or absence of weapon, elapsed time between incident and identification task, levels of stress) and—importantly—interactions between these variables using either the ROC approach or other tools for evaluation of binary classifiers that can be shown to have advantages over existing analytical methods.
c. Building upon the committee’s call for a practice- and data-informed research agenda that incorporates input from law enforcement and the courts and establishes methodological and reporting standards for research, the committee recommends that the scientific community engaged in studies of eyewitness identification performance work closely with law enforcement to identify other system and estimator variables that might influence performance and practical issues that might preclude certain strategies for influencing performance. In addition, the committee recommends that policy decisions regarding changes in procedure should be made on the basis of evidence of superiority and should be made in consultation with police departments to determine which procedure yields the best combination of performance and practicality.
CONCLUSION
Eyewitness identification can be a powerful tool. As this report indicates, however, the malleable nature of human visual perception, memory, and confidence; the imperfect ability to recognize individuals; and policies governing law enforcement procedures can result in mistaken identifications with significant consequences. New law enforcement training protocols, standardized procedures for administering lineups, improvements in the handling of eyewitness identification in court, and better data collection and research on eyewitness identification can improve the accuracy of eyewitness identifications.
