Personnel Selection in the
PATTERN EVIDENCE DOMAIN
of Forensic Science
Proceedings of a Workshop
Julie Anne Schuck, Rapporteur
Board on Human-Systems Integration
Division of Behavioral and Social Sciences and Education
THE NATIONAL ACADEMIES PRESS
Washington, DC
www.nap.edu
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001
This activity was supported by Contract No. 60NANB150D340 from the U.S. Department of Commerce. Any opinions, findings, conclusions, or recommendations expressed in this publication do not necessarily reflect the views of any organization or agency that provided support for the project.
International Standard Book Number-13: 978-0-309-45140-6
International Standard Book Number-10: 0-309-45140-X
Digital Object Identifier: 10.17226/23681
Additional copies of this report are available from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.
Copyright 2017 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
Suggested citation: National Academies of Sciences, Engineering, and Medicine. (2017). Personnel Selection in the Pattern Evidence Domain of Forensic Science: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23681.
The National Academy of Sciences was established in 1863 by an Act of Congress, signed by President Lincoln, as a private, nongovernmental institution to advise the nation on issues related to science and technology. Members are elected by their peers for outstanding contributions to research. Dr. Marcia McNutt is president.
The National Academy of Engineering was established in 1964 under the charter of the National Academy of Sciences to bring the practices of engineering to advising the nation. Members are elected by their peers for extraordinary contributions to engineering. Dr. C.D. Mote, Jr., is president.
The National Academy of Medicine (formerly the Institute of Medicine) was established in 1970 under the charter of the National Academy of Sciences to advise the nation on medical and health issues. Members are elected by their peers for distinguished contributions to medicine and health. Dr. Victor J. Dzau is president.
The three Academies work together as the National Academies of Sciences, Engineering, and Medicine to provide independent, objective analysis and advice to the nation and conduct other activities to solve complex problems and inform public policy decisions. The National Academies also encourage education and research, recognize outstanding contributions to knowledge, and increase public understanding in matters of science, engineering, and medicine.
Learn more about the National Academies of Sciences, Engineering, and Medicine at www.national-academies.org.
Reports document the evidence-based consensus of an authoring committee of experts. Reports typically include findings, conclusions, and recommendations based on information gathered by the committee and committee deliberations. Reports are peer reviewed and are approved by the National Academies of Sciences, Engineering, and Medicine.
Proceedings chronicle the presentations and discussions at a workshop, symposium, or other convening event. The statements and opinions contained in proceedings are those of the participants and have not been endorsed by other participants, the planning committee, or the National Academies of Sciences, Engineering, and Medicine.
For information about other products and activities of the National Academies, please visit nationalacademies.org/whatwedo.
COMMITTEE ON WORKFORCE PLANNING MODELS FOR FORENSIC SCIENCE: A WORKSHOP
Frederick L. Oswald (Chair), Department of Psychology, Rice University
Winfred Arthur, Jr., Department of Psychology, Texas A&M University
D. Zachary Hambrick, Department of Psychology, Michigan State University
Andrew S. Imada, A.S. Imada & Associates, Laguna Niguel, CA
Randall S. Murch, Biosecurity and Forensics, Research Program Development, National Capital Region, Virginia Polytechnic Institute and State University
Ann Marie Ryan, Department of Psychology, Michigan State University
Jay A. Siegel, Independent Consultant, Michigan State University (emeritus)
Nancy T. Tippins, CEB
Julie Anne Schuck, Program Officer
Dixie Gordon, Information Officer
BOARD ON HUMAN-SYSTEMS INTEGRATION
Nancy J. Cooke (Chair), Cognitive Science and Engineering and College of Technology and Innovation, Arizona State University
Ellen J. Bass, College of Computing and Informatics and College of Nursing & Health Professions, Drexel University
Sara J. Czaja, Department of Psychiatry and Behavioral Sciences, Center on Aging, University of Miami Miller School of Medicine
Francis T. Durso, Department of Psychology, Georgia Institute of Technology
Andrew S. Imada, A.S. Imada & Associates, Laguna Niguel, CA
Edmond W. Israelski, Abbvie Combination Product Development, Chicago, IL
Elizabeth Loftus, Psychology and Social Behavior, Criminology, Law and Society, Cognitive Sciences, University of California, Irvine
Frederick L. Oswald, Department of Psychology, Rice University
Karl S. Pister, Chancellor (Emeritus) University of California, Santa Cruz, and Dean & Roy W. Carlson Professor of Engineering (Emeritus), Department of Civil and Environmental Engineering, University of California, Berkeley
David Rempel, Department of Medicine, University of California, San Francisco
Emilie M. Roth, Roth Cognitive Engineering, Stanford, CA
Barbara Silverstein, Safety and Health Assessment and Research for Prevention Program, Washington State Department of Labor and Industries
David H. Wegman, Department of Work Environment, University of Massachusetts Lowell (emeritus)
Poornima Madhavan, Director (until September 2016)
Toby Warden, Interim Director (as of September 2016)
Preface
The field of forensic science is often in the news these days and under particular scrutiny. National efforts are underway to develop standards and guidelines for forensic techniques and to push research that tests the accuracy and reproducibility of forensic examinations. Notably, most recently following the workshop, the President’s Council of Advisors on Science and Technology published an important report, Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods.
In the midst of all these improvements, the allocation and maintenance of organizational, technological, and human resources will continue to play a critical role in the overall performance of a forensic lab. Today, and in the foreseeable future, the quality and reliability of forensic analyses depend in large part on the expertise and capabilities of forensic examiners. In the pattern evidence domain, the field of forensic science is looking to move away from reliance on apprentice-like models of hiring and training toward innovative research-based applications that will effectively identify and test for critical skills and competencies necessary to perform the pattern recognition tasks.
On July 14-15, 2016, the Board on Human-Systems Integration (BOHSI) of the National Academies of Sciences, Engineering, and Medicine sponsored a workshop on personnel selection in forensic science that brought together scholars in industrial and organizational (I-O) psychology, practicing forensic scientists and forensic lab directors, and legal experts. The
purpose of the workshop was to develop a better understanding of the current status of selection and training of forensic scientists who specialize in pattern evidence, tools used in I-O psychology to understand elements of a task, and ways that aptitude and performance can be reliably, feasibly, and fairly measured. I thank the National Institute of Standards and Technology (NIST) for the generous support to enable this workshop and publication, particularly Melissa Taylor for having the foresight to bring the I-O psychology, forensic science, and legal disciplines together. In the workshop, we learned a great deal about the task of analyzing pattern evidence and about the state-of-the-art approaches to designing, validating, and evaluating instruments that are used for personnel selection. This summary provides an objective report of what occurred at the workshop, drawing on views presented by individual participants and focusing on the possibilities to develop strategic next steps for a newly formed multidisciplinary community to coordinate collective energies and to continue discussion and improvement efforts.
First and foremost, let me extend my thanks to other workshop steering committee members, who gave generously of their knowledge and time to frame the workshop agenda, identify the presenters, and lend their own expertise to the workshop discussions: Winfred Arthur, Jr., Texas A&M University; D. Zachary Hambrick, Michigan State University; Andrew S. Imada, A.S. Imada & Associates; Randall S. Murch, Virginia Polytechnic Institute and State University; Ann Marie Ryan, Michigan State University; Jay A. Siegel, independent consultant; and Nancy T. Tippins, CEB. The success of the workshop was critically dependent on a talented and thoughtful group of experts, who took time out of their valuable schedules to present their relevant experiences and research and to participate in what was an interesting and invigorating multidisciplinary discussion: Mark W. Becker, Michigan State University; Wendy S. Becker, Shippensburg University; John M. Collins, Jr., Forensic Foundations Group; Melissa R. Gische, Federal Bureau of Investigation; Rockne P. Harmon, independent consultant; Scott Highhouse, Bowling Green State University; Bethany Jurs, Transylvania University; Jessica LeCroy, Defense Forensic Science Center; S. Mort McPhail, Society for Industrial and Organizational Psychology; Mara Merlino, Kentucky State University; Liberty Munson, Microsoft Learning Experiences; Daniel Murrie, University of Virginia; Dan Putka, HumRRO; Maria Weir Ruggiero, Los Angeles County Sheriff’s Department Crime Laboratory; Marvin E. Schechter, defense attorney; and Lisa Scott, University of Florida. In addition, I would particularly like to recognize Susan Ballou, National Institute of Standards and Technology; Wesley Grose, Los Angeles County Sheriff’s Department Crime Laboratory; and Heidi Eldridge, RTI International, for
their insightful contributions to the discussion, as well as the many others whose interest led them to take time out of their schedules and attend the workshop, either in person or by viewing the Webcast.
Also, let me thank Poornima Madhavan, board director of BOHSI until September 2016, for her ongoing consultation in coordinating and preparing for the workshop. Julie Schuck from the National Academies’ Committee on Law and Justice deserves special thanks, as she provided invaluable guidance from start to finish: the development of this workshop, assistance in inviting and preparing the participants, coordinating the workshop itself, and serving as the rapporteur for these proceedings, where she thoughtfully captures the many perspectives and major messages presented at the workshop. Dixie Gordon from the Division of Engineering and Physical Sciences and Annalee Gonzales from the Board on Children, Youth, and Families, enlisted to help with administrative aspects of the workshop, kept the workshop on track and running smoothly. My thanks also go to the staff from Sparkstreet for producing the Webcast, as well as to other staff within the Division of Behavioral and Social Sciences and Education (DBASSE), especially Viola Horek and Doug Sprunger, who managed communications around this workshop; Eugenia Grohman, who consulted with staff on the editing of this summary; Kirsten Sampson Snyder, who managed the report review process; and Yvonne Wise, who managed the production process.
This workshop summary has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the Report Review Committee of the National Academies. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the charge. The review comments and draft manuscript remain confidential to protect the integrity of the process.
I thank the following individuals for their review of this report: Heidi Eldridge, Center for Forensic Sciences, RTI International, Research Triangle Park, NC; Henry Swofford, Latent Print Branch, U.S. Army Criminal Investigation Laboratory, Forest Park, GA; and Wesley P. Grose, Scientific Services Bureau, Los Angeles County Sheriff’s Department.
Although the reviewers listed above provided many constructive comments and suggestions, they were not asked to endorse the content of the report, nor did they see the final draft of the report before its release. The review of this report was overseen by John Monahan, University of Virginia. Appointed by the National Academies, he was responsible for making certain that an independent examination of this report was car-
ried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the author and the institution.
Frederick L. Oswald, Chair
Committee on Workforce Planning Models for Forensic Science: A Workshop
Contents
2 THE TASK OF PATTERN RECOGNITION
Nature of the Job in Forensic Laboratories
Hiring and Training of Pattern Evidence Examiners
Research on Pattern Recognition and Developing Expertise
3 INDUSTRIAL AND ORGANIZATIONAL PSYCHOLOGY
Assessment for the Purposes of Selection