Review of Proposals for Research on Statistical
Methodologies for Assessing Variables in

Eyewitness
Performance

Committee on the Review of Proposals for
Research on Statistical Methodologies for Assessing
Variables in Eyewitness Performance

Committee on Science, Technology, and Law
Policy and Global Affairs

Committee on Applied and Theoretical Statistics
Division on Engineering and Physical Sciences

A Report of

THE NATIONAL ACADEMIES PRESS
Washington, DC
www.nap.edu

THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001

This activity was funded by a grant between the National Academies of Sciences, Engineering, and Medicine and the Laura and John Arnold Foundation. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of the organization that provided support for the project.

Digital Object Identifier: 10.17226/23633

Copyright 2016 by the National Academy of Sciences. All rights reserved.

Printed in the United States of America

Suggested citation: National Academies of Sciences, Engineering, and Medicine. 2016. Review of Proposals for Research on Statistical Methodologies for Assessing Variables in Eyewitness Performance. Washington, DC: The National Academies Press. doi: 10.17226/23633.

The National Academy of Sciences was established in 1863 by an Act of Congress, signed by President Lincoln, as a private, nongovernmental institution to advise the nation on issues related to science and technology. Members are elected by their peers for outstanding contributions to research. Dr. Marcia McNutt is president.

The National Academy of Engineering was established in 1964 under the charter of the National Academy of Sciences to bring the practices of engineering to advising the nation. Members are elected by their peers for extraordinary contributions to engineering. Dr. C. D. Mote, Jr., is president.

The National Academy of Medicine (formerly the Institute of Medicine) was established in 1970 under the charter of the National Academy of Sciences to advise the nation on medical and health issues. Members are elected by their peers for distinguished contributions to medicine and health. Dr. Victor J. Dzau is president.

The three Academies work together as the National Academies of Sciences, Engineering, and Medicine to provide independent, objective analysis and advice to the nation and conduct other activities to solve complex problems and inform public policy decisions. The Academies also encourage education and research, recognize outstanding contributions to knowledge, and increase public understanding in matters of science, engineering, and medicine.

Learn more about the National Academies of Sciences, Engineering, and Medicine at www.national-academies.org.

Reports document the evidence-based consensus of an authoring committee of experts. Reports typically include findings, conclusions, and recommendations based on information gathered by the committee and committee deliberations. Reports are peer reviewed and are approved by the National Academies of Sciences, Engineering, and Medicine.

Proceedings chronicle the presentations and discussions at a workshop, symposium, or other convening event. The statements and opinions contained in proceedings are those of the participants and have not been endorsed by other participants, the planning committee, or the National Academies of Sciences, Engineering, and Medicine.

For information about other products and activities of the National Academies, please visit nationalacademies.org/whatwedo.

COMMITTEE ON THE REVIEW OF PROPOSALS FOR RESEARCH ON STATISTICAL METHODOLOGIES FOR ASSESSING VARIABLES IN EYEWITNESS PERFORMANCE

Chair

DAVID BANKS, Professor of Statistics, Institute of Statistics and Decision Sciences, Duke University

Members

THOMAS D. ALBRIGHT (NAS), Professor and Director, Vision Center Laboratory and Conrad T. Prebys Chair in Vision Research, Salk Institute for Biological Studies

RICHARD A. BERK, Professor of Criminology and Statistics, Department of Criminology and Department of Statistics, University of Pennsylvania

SCOTT MCNAMARA, Oneida County District Attorney, Oneida County District Attorney's Office

EUGENE WONG (NAE), Professor Emeritus, University of California, Berkeley

SANDY L. ZABELL, Professor of Mathematics and Statistics, Northwestern University

Staff

STEVEN KENDALL, Study Director and Program Officer, Committee on Science, Technology, and Law

ANNE-MARIE MAZZA, Senior Director, Committee on Science, Technology, and Law

SCOTT T. WEIDMAN, Director, Board on Mathematical Sciences and their Applications

KAROLINA KONARZEWSKA, Program Coordinator, Committee on Science, Technology, and Law

COMMITTEE ON SCIENCE, TECHNOLOGY, AND LAW

Co-Chairs

DAVID BALTIMORE (NAS/NAM), President Emeritus and Robert Andrews Millikan Professor of Biology, California Institute of Technology

DAVID S. TATEL, Judge, U.S. Court of Appeals for the District of Columbia Circuit

Members

THOMAS D. ALBRIGHT (NAS), Professor and Director, Vision Center Laboratory and Conrad T. Prebys Chair in Vision Research, Salk Institute for Biological Studies

ANN ARVIN (NAM), Lucile Salter Packard Professor of Pediatrics and Professor of Microbiology and Immunology, Stanford University School of Medicine, and Vice Provost and Dean of Research, Stanford University

CLAUDE R. CANIZARES (NAS), Bruno Rossi Professor of Physics, Massachusetts Institute of Technology

ARTURO CASADEVALL (NAM), Professor and Chair, W. Harry Feinstone Department of Molecular Microbiology and Immunology, Johns Hopkins University Bloomberg School of Public Health

JOE S. CECIL, Project Director, Program on Scientific and Technical Evidence, Division of Research, Federal Judicial Center

R. ALTA CHARO (NAM), Warren P. Knowles Professor of Law and Bioethics, University of Wisconsin at Madison

HARRY T. EDWARDS, Judge, U.S. Court of Appeals for the District of Columbia Circuit

DREW ENDY, Associate Professor, Bioengineering, Stanford University and President, The BioBricks Foundation

MARCUS FELDMAN (NAS), Burnet C. and Mildred Wohlford Professor of Biological Sciences, Stanford University

JEREMY FOGEL, Director, The Federal Judicial Center

HENRY T. GREELY, Deane F. and Kate Edelman Johnson Professor of Law and Professor, by courtesy, of Genetics, Stanford University

MICHAEL GREENBERGER, Law School Professor and Director, Center for Health and Homeland Security, University of Maryland

MICHAEL IMPERIALE, Arthur F. Thurnau Professor of Microbiology and Immunology, University of Michigan

GREG KISOR, Chief Technologist, Intellectual Ventures

ROBERT S. LANGER (NAS/NAE/NAM), David H. Koch Institute Professor, Massachusetts Institute of Technology

GOODWIN LIU, Associate Justice, California Supreme Court

JENNIFER MNOOKIN, Dean and David G. Price and Dallas P. Price Professor of Law, University of California, Los Angeles School of Law

R. GREGORY MORGAN, Senior Vice President and Secretary of the Corporation, Massachusetts Institute of Technology

DAVID RELMAN (NAM), Thomas C. and Joan M. Merigan Professor, Departments of Medicine and of Microbiology and Immunology, Stanford University and Chief, Infectious Disease Section, VA Palo Alto Health Care System

MARTINE A. ROTHBLATT, Chairman and Chief Executive Officer, United Therapeutics

DAVID VLADECK, Professor and Co-Director, Institute for Public Representation, Georgetown Law School

Staff

ANNE-MARIE MAZZA, Senior Director

STEVEN KENDALL, Program Officer

KAROLINA KONARZEWSKA, Program Coordinator

D. ALLEN AMMERMAN, Financial Officer

COMMITTEE ON APPLIED AND THEORETICAL STATISTICS

Chair

CONSTANTINE GATSONIS, Henry Ledyard Goddard University Professor and Chair, Department of Biostatistics, and Director, Center for Statistical Sciences, Brown University

Members

DEEPAK AGARWAL, Director of Engineering, LinkedIn

MICHAEL DANIELS, Professor and Chair, Department of Statistics & Data Sciences, University of Texas at Austin

KATHERINE BENNETT ENSOR, Professor and Chair, Department of Statistics, Rice University

MONTSERRAT FUENTES, Dean, College of Humanities and Sciences, Virginia Commonwealth University

ALFRED O. HERO III, John H. Holland Distinguished University Professor of Electrical Engineering and Computer Science and the R. Jamison and Betty Williams Professor of Engineering, University of Michigan

AMY HERRING, Professor and Associate Chair, Department of Biostatistics, University of North Carolina

DAVID M. HIGDON, Professor, Social Decision Analytics Laboratory, Biocomplexity Institute of Virginia Tech

ROBERT E. KASS, Maurice Falk Professor, Departments of Statistics and Machine Learning, Carnegie Mellon University

JOHN LAFFERTY, Louis Block Professor, Departments of Statistics, Computer Science, and the College, University of Chicago

JOSÉ M.F. MOURA (NAE), Philip and Marsha Dowd University Professor and Associate Department Head for Research and Strategy, Department of Electrical and Computer Engineering, Carnegie Mellon University

SHARON-LISE T. NORMAND, Professor of Health Care Policy (Biostatistics), Harvard Medical School and Professor, Department of Biostatistics, Harvard School of Public Health

ADRIAN RAFTERY (NAS), Professor, Departments of Statistics and Sociology, University of Washington

LANCE WALLER, Rollins Professor and Chair, Department of Biostatistics and Bioinformatics, Emory University

EUGENE WONG (NAE), Professor Emeritus, Department of Electrical Engineering and Computer Science, University of California, Berkeley

Staff

MICHELLE K. SCHWALBE, Director

LINDA CASOLA, Senior Project Assistant

BETH DOLAN, Financial Associate

Acknowledgments

ACKNOWLEDGMENT OF PRESENTERS

The committee gratefully acknowledges the contributions of the following individuals who provided important and useful information at the first meeting of the committee:

ACKNOWLEDGMENT OF EXTERNAL REVIEW

This report has been reviewed in draft form in accordance with procedures approved by the National Academies of Sciences, Engineering, and Medicine’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the process.

We wish to thank Joe S. Cecil, Federal Judicial Center, for his review of this report and for overseeing the review process. Appointed by the National Academies, he was responsible for making certain that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution.

This page intentionally left blank.

Contents

1 INTRODUCTION

2 THE DEVELOPMENT OF THE REQUEST FOR PROPOSALS/EVALUATION OF FORMAL PROPOSALS

APPENDIXES

A BIOGRAPHICAL INFORMATION OF COMMITTEE AND STAFF

B COMMITTEE MEETING AGENDAS

C LAURA AND JOHN ARNOLD FOUNDATION REQUEST FOR PROPOSALS

This page intentionally left blank.