Reference Manual on Third Edition |
Committee on the Development of the Third Edition of the
Reference Manual on Scientific Evidence
Committee on Science, Technology, and Law
Policy and Global Affairs
FEDERAL JUDICIAL CENTER
NATIONAL RESEARCH COUNCIL
OF THE NATIONAL ACADEMIES
THE NATIONAL ACADEMIES PRESS
Washington, D.C.
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, N.W. Washington, DC 20001
The Federal Judicial Center contributed to this publication in furtherance of the Center’s statutory mission to develop and conduct educational programs for judicial branch employees. The views expressed are those of the authors and not necessarily those of the Federal Judicial Center.
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.
The development of the third edition of the Reference Manual on Scientific Evidence was supported by Contract No. B5727.R02 between the National Academy of Sciences and the Carnegie Corporation of New York and a grant from the Starr Foundation. The views expressed in this publication are those of the authors and do not necessarily reflect those of the National Academies or the organizations that provided support for the project.
International Standard Book Number-13: 978-0-309-21421-6
International Standard Book Number-10: 0-309-21421-1
Library of Congress Cataloging-in-Publication Data
Reference manual on scientific evidence. — 3rd ed.
p. cm.
Includes bibliographical references and index.
ISBN-13: 978-0-309-21421-6 (pbk.)
ISBN-10: 0-309-21421-1 (pbk.)
1. Evidence, Expert—United States. I. Federal Judicial Center.
KF8961.R44 2011
347.73´67—dc23
2011031458
Additional copies of this report are available from the National Academies Press, 500 Fifth Street, N.W., Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area); Internet, http://www.nap.edu.
Copyright 2011 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
THE FEDERAL JUDICIAL CENTER
The Federal Judicial Center is the research and education agency of the federal judicial system. It was established by Congress in 1967 (28 U.S.C. §§ 620–629), on the recommendation of the Judicial Conference of the United States, with the mission to “further the development and adoption of improved judicial administration in the courts of the United States.” By statute, the Chief Justice of the United States chairs the Federal Judicial Center’s Board, which also includes the director of the Administrative Office of the U.S. Courts and seven judges elected by the Judicial Conference.
The Center undertakes empirical and exploratory research on federal judicial processes, court management, and sentencing and its consequences, often at the request of the Judicial Conference and its committees, the courts themselves, or other groups in the federal system. In addition to orientation and continuing education programs for judges and court staff on law and case management, the Center produces publications, videos, and online resources. The Center provides leadership and management education for judges and court employees, and other training as needed. Center research informs many of its educational efforts. The Center also produces resources and materials on the history of the federal courts, and it develops resources to assist in fostering effective judicial administration in other countries.
Since its founding, the Center has had nine directors. Judge Barbara J. Rothstein became director of the Federal Judicial Center in 2003
THE NATIONAL ACADEMIES
Advisers to the Nation on Science, Engineering, and Medicine
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council.
Committee on the Development of the Third Edition of the Reference Manual on Scientific Evidence
Co-Chairs:
JEROME P. KASSIRER (IOM), Distinguished Professor, Tufts University School of Medicine
GLADYS KESSLER, Judge, U.S. District Court for the District of Columbia
Members:
MING W. CHIN, Associate Justice, The Supreme Court of California
PAULINE NEWMAN, Judge, U.S. Court of Appeals for the Federal Circuit
KATHLEEN MCDONALD O’MALLEY, Judge, U.S. Court of Appeals for the Federal Circuit
JED S. RAKOFF, Judge, U.S. District Court, Southern District of New York
CHANNING R. ROBERTSON, Ruth G. and William K. Bowes Professor, School of Engineering, and Professor, Department of Chemical Engineering, Stanford University
JOSEPH V. RODRICKS, Principal, Environ
ALLEN WILCOX, Senior Investigator, Institute of Environmental Health Sciences
SANDY L. ZABELL, Professor of Statistics and Mathematics, Weinberg College of Arts and Sciences, Northwestern University
Consultant to the Committee:
JOE S. CECIL, Project Director, Program on Scientific and Technical Evidence, Division of Research, Federal Judicial Center
Staff:
ANNE-MARIE MAZZA, Director
STEVEN KENDALL, Associate Program Officer
GURUPRASAD MADHAVAN, Program Officer (until November 2010)
Board of the Federal Judicial Center
The Chief Justice of the United States, Chair
Judge Susan H. Black, U.S. Court of Appeals for the Eleventh Circuit
Magistrate Judge John Michael Facciola, U.S. District Court for the District of Columbia
Judge James B. Haines, U.S. Bankruptcy Court for the District of Maine
Chief Judge James F. Holderman, U.S. District Court for the Northern District of Illinois
Judge Edward C. Prado, U.S. Court of Appeals for the Fifth Circuit
Chief Judge Loretta A. Preska, U.S. District Court for the Southern District of New York
Chief Judge Kathryn H. Vratil, U.S. District Court for the District of Kansas
James C. Duff, Director of the Administrative Office of the U.S. Courts
Committee on Science, Technology, and Law National Research Council
DAVID KORN (Co-Chair), Professor of Pathology, Harvard Medical School, and formerly, Inaugural Vice Provost for Research, Harvard University
RICHARD A. MESERVE (Co-Chair), President, Carnegie Institution for Science, and Senior of Counsel, Covington & Burling LLP
FREDERICK R. ANDERSON, JR., Partner, McKenna, Long & Aldridge LLP
ARTHUR I. BIENENSTOCK, Special Assistant to the President for Federal Research Policy, and Director, Wallenberg Research Link, Stanford University
BARBARA E. BIERER, Professor of Medicine, Harvard Medical School, and Senior Vice President, Research, Brigham and Women’s Hospital
ELIZABETH H. BLACKBURN, Morris Herzstein Professor of Biology and Physiology, University of California, San Francisco
JOHN BURRIS, President, Burroughs Wellcome Fund
ARTURO CASADEVALL, Leo and Julia Forchheimer Professor of Microbiology and Immunology; Chair, Department of Biology and Immunology; and Professor of Medicine, Albert Einstein College of Medicine
JOE S. CECIL, Project Director, Program on Scientific and Technical Evidence, Division of Research, Federal Judicial Center
ROCHELLE COOPER DREYFUSS, Pauline Newman Professor of Law and Director, Engelberg Center on Innovation Law and Policy, New York University School of Law
DREW ENDY, Assistant Professor, Bioengineering, Stanford University, and President, The BioBricks Foundation
PAUL G. FALKOWSKI, Board of Governors Professor in Geological and Marine Science, Department of Earth and Planetary Science, Rutgers, The State University of New Jersey
MARCUS FELDMAN, Burnet C. and Mildred Wohlford Professor of Biological Sciences, Stanford University
ALICE P. GAST, President, Lehigh University
JASON GRUMET, President, Bipartisan Policy Center
BENJAMIN W. HEINEMAN, JR., Senior Fellow, Harvard Law School and Harvard Kennedy School of Government
D. BROCK HORNBY, U.S. District Judge for the District of Maine
ALAN B. MORRISON, Lerner Family Associate Dean for Public Interest and Public Service, George Washington University Law School
PRABHU PINGALI, Deputy Director of Agricultural Development, Global Development Program, Bill and Melinda Gates Foundation
HARRIET RABB, Vice President and General Counsel, Rockefeller University
BARBARA JACOBS ROTHSTEIN, Director, The Federal Judicial Center
DAVID S.TATEL, Judge, U.S. Court of Appeals for the District of Columbia Circuit
SOPHIE VANDEBROEK, Chief Technology Officer and President, Xerox Innovation Group, Xerox Corporation
Staff
ANNE-MARIE MAZZA, Director
STEVEN KENDALL, Associate Program Officer
Foreword
In 1993, in the case Daubert v. Merrell Dow Pharmaceuticals, Inc., the Supreme Court instructed trial judges to serve as “gatekeepers” in determining whether the opinion of a proffered expert is based on scientific reasoning and methodology. Since Daubert, scientific and technical information has become increasingly important in all types of decisionmaking, including litigation. As a result, the science and legal communities have searched for expanding opportunities for collaboration.
Our two institutions have been at the forefront of trying to improve the use of science by judges and attorneys. In Daubert, the Supreme Court cited an amicus curiae brief submitted by the National Academy of Sciences and the American Association for the Advancement of Science to support the view of science as “a process for proposing and refining theoretical explanations about the world that are subject to further testing and refinement.” Similarly, in Kumho Tire Co. v. Carmichael (1999) the Court cited an amicus brief filed by the National Academy of Engineering for its assistance in explaining the process of engineering.
Soon after the Daubert decision the Federal Judicial Center published the first edition of the Reference Manual on Scientific Evidence, which has become the leading reference source for federal judges for difficult issues involving scientific testimony. The Center also undertook a series of research studies and judicial education programs intended to strengthen the use of science in courts.
More recently the National Research Council through its Committee on Science, Technology, and Law has worked closely with the Federal Judicial Center to organize discussions, workshops, and studies that would bring the two communities together to explore the nature of science and engineering, and the processes by which science and technical information informs legal issues. It is in that spirit that our organizations joined together to develop the third edition of the Reference Manual on Scientific Evidence. This third edition, which was supported by grants from the Carnegie Corporation of New York and the Starr Foundation, builds on the foundation of the first two editions, published by the Center. This edition was overseen by a National Research Council committee composed of judges and scientists and engineers who share a common vision that together scientists and engineers and members of the judiciary can play an important role in informing judges about the nature and work of the scientific enterprise.
Our organizations benefit from the contributions of volunteers who give their time and energy to our efforts. During the course of this project, two of the chapter authors passed away: Margaret Berger and David Freedman. Both Margaret and David served on NRC committees and were frequent contributors to Center judicial education seminars. Both were involved in the development of the Reference Manual from the beginning, both have aided each of our institutions through their services on committees, and both have made substantial contributions to our understanding of law and science through their individual scholarship.
They will be missed but their work will live on in the thoughtful scholarship they have left behind.
We extend our sincere appreciation to Dr. Jerome Kassirer and Judge Gladys Kessler and all the members of the committee who gave so generously to make this edition possible.
THE HONORABLE BARBARA J. ROTHSTEIN | RALPH J. CICERONE |
Director | President |
Federal Judicial Center | National Academy of Sciences |
Acknowledgments
This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the National Academies’ Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, accuracy, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the process.
We wish to thank the following individuals for their review of selected chapters of this report: Bert Black, Mansfield, Tanick & Cohen; Richard Bjur, University of Nevada; Michael Brick, Westat; Edward Cheng, Vanderbilt University; Joel Cohen, Rockefeller University; Morton Corn, Morton Corn and Associates; Carl Cranor, University of California, Riverside; Randall Davis, Massachusetts Institute of Technology; John Doull, University of Kansas; Barry Fisher, Los Angeles County Sheriff’s Department; Edward Foster, University of Minnesota; David Goldston, Natural Resources Defense Council; James Greiner, Harvard University; Susan Haack, University of Miami; David Hillis, University of Texas; Karen Kafadar, Indiana University; Graham Kalton, Westat; Randy Katz, University of California, Berkeley; Alan Leshner, American Association for the Advancement of Science; Laura Liptai, Biomedical Forensics; Patrick Malone, Patrick Malone & Associates; Geoffrey Mearns, Cleveland State University; John Monahan, The University of Virginia; William Nordhaus, Yale University; Fernando Olguin, U.S. District Court for the Central District of California; Jonathan Samet, University of Southern California; Nora Cate Schaeffer, University of Wisconsin; Shira Scheindlin, U.S. District Court for the Southern District of New York; and Reggie Walton, U.S. District Court for the District of Columbia.
Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the report, nor did they see the final draft of the report before its release. The review of this report was overseen by D. Brock Hornby, U.S. District Judge for the District of Maine. Appointed by the National Academies, he was responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution.
Preface
Supreme Court decisions during the last decade of the twentieth century mandated that federal courts examine the scientific basis of expert testimony to ensure that it meets the same rigorous standard employed by scientific researchers and practitioners outside the courtroom. Needless to say, this requirement places a demand on judges not only to comprehend the complexities of modern science but to adjudicate between parties’ differing interpretations of scientific evidence. Science, meanwhile, advances. Methods change, new fields are born, new tests are introduced, the lexicon expands, and fresh approaches to the interpretation of causal relations evolve. Familiar terms such as enzymes and molecules are replaced by microarray expression and nanotubes; single-author research studies have now become multi-institutional, multi-author, international collaborative efforts.
No field illustrates the evolution of science better than forensics. The evidence provided by DNA technology was so far superior to other widely accepted methods and called into question so many earlier convictions that the scientific community had to reexamine many of its time-worn forensic science practices. Although flaws of some types of forensic science evidence, such as bite and footprint analysis, lineup identification, and bullet matching were recognized, even the most revered form of forensic science—fingerprint identification—was found to be fallible. Notably, even the “gold standard” of forensic evidence, namely DNA analysis, can lead to an erroneous conviction if the sample is contaminated, if specimens are improperly identified, or if appropriate laboratory protocols and practices are not followed.
Yet despite its advances, science has remained fundamentally the same. In its ideal expression, it examines the nature of nature in a rigorous, disciplined manner in, whenever possible, controlled environments. It still is based on principles of hypothesis generation, scrupulous study design, meticulous data collection, and objective interpretation of experimental results. As in other human endeavors, however, this ideal is not always met. Feverish competition between researchers and their parent institutions, fervent publicity seeking, and the potential for dazzling financial rewards can impair scientific objectivity. In recent years we have experienced serious problems that range from the introduction of subtle bias in the design and interpretation of experiments to overt fraudulent studies. In this welter of modern science, ambitious scientists, self-designated experts, billion-dollar corporate entities, and aggressive claimants, judges must weigh evidence, judge, and decide.
As with previous editions of the Reference Manual, this edition is organized according to many of the important scientific and technological disciplines likely to be encountered by federal (or state) judges. We wish to highlight here two critical issues germane to the interpretation of all scientific evidence, namely issues of causation and conflict of interest. Causation is the task of attributing cause and effect, a normal everyday cognitive function that ordinarily takes little or
no effort. Fundamentally, the task is an inferential process of weighing evidence and using judgment to conclude whether or not an effect is the result of some stimulus. Judgment is required even when using sophisticated statistical methods. Such methods can provide powerful evidence of associations between variables, but they cannot prove that a causal relationship exists. Theories of causation (evolution, for example) lose their designation as theories only if the scientific community has rejected alternative theories and accepted the causal relationship as fact. Elements that are often considered in helping to establish a causal relationship include predisposing factors, proximity of a stimulus to its putative outcome, the strength of the stimulus, and the strength of the events in a causal chain. Unfortunately, judges may be in a less favorable position than scientists to make causal assessments. Scientists may delay their decision while they or others gather more data. Judges, on the other hand, must rule on causation based on existing information. Concepts of causation familiar to scientists (no matter what stripe) may not resonate with judges who are asked to rule on general causation (i.e., is a particular stimulus known to produce a particular reaction) or specific causation (i.e., did a particular stimulus cause a particular consequence in a specific instance). In the final analysis, a judge does not have the option of suspending judgment until more information is available, but must decide after considering the best available science. Finally, given the enormous amount of evidence to be interpreted, expert scientists from different (or even the same) disciplines may not agree on which data are the most relevant, which are the most reliable, and what conclusions about causation are appropriate to be derived.
Like causation, conflict of interest is an issue that cuts across most, if not all, scientific disciplines and could have been included in each chapter of the Reference Manual. Conflict of interest manifests as bias, and given the high stakes and adversarial nature of many courtroom proceedings, bias can have a major influence on evidence, testimony, and decisionmaking. Conflicts of interest take many forms and can be based on religious, social, political, or other personal convictions. The biases that these convictions can induce may range from serious to extreme, but these intrinsic influences and the biases they can induce are difficult to identify. Even individuals with such prejudices may not appreciate that they have them, nor may they realize that their interpretations of scientific issues may be biased by them. Because of these limitations, we consider here only financial conflicts of interest; such conflicts are discoverable. Nonetheless, even though financial conflicts can be identified, having such a conflict, even one involving huge sums of money, does not necessarily mean that a given individual will be biased. Having a financial relationship with a commercial entity produces a conflict of interest, but it does not inevitably evoke bias. In science, financial conflict of interest is often accompanied by disclosure of the relationship, leaving to the public the decision whether the interpretation might be tainted. Needless to say, such an assessment may be difficult. The problem is compounded in scientific publications by obscure ways in which the conflicts are reported and by a lack of disclosure of dollar amounts.
Judges and juries, however, must consider financial conflicts of interest when assessing scientific testimony. The threshold for pursuing the possibility of bias must be low. In some instances, judges have been frustrated in identifying expert witnesses who are free of conflict of interest because entire fields of science seem to be co-opted by payments from industry. Judges must also be aware that the research methods of studies funded specifically for purposes of litigation could favor one of the parties. Though awareness of such financial conflicts in itself is not necessarily predictive of bias, such information should be sought and evaluated as part of the deliberations.
The Reference Manual on Scientific Evidence, here in its third edition, is formulated to provide the tools for judges to manage cases involving complex scientific and technical evidence. It describes basic principles of major scientific fields from which legal evidence is typically derived and provides examples of cases in which such evidence was used. Authors of the chapters were asked to provide an overview of principles and methods of the science and provide relevant citations. We expect that few judges will read the entire manual; most will use the volume in response to a need when a particular case arises involving a technical or scientific issue. To help in this endeavor, the Reference Manual contains completely updated chapters as well as new ones on neuroscience, exposure science, mental health, and forensic science. This edition of the manual has also gone through the thorough review process of the National Academy of Sciences.
As in previous editions, we continue to caution judges regarding the proper use of the reference guides. They are not intended to instruct judges concerning what evidence should be admissible or to establish minimum standards for acceptable scientific testimony. Rather, the guides can assist judges in identifying the issues most commonly in dispute in these selected areas and in reaching an informed and reasoned assessment concerning the basis of expert evidence. They are designed to facilitate the process of identifying and narrowing issues concerning scientific evidence by outlining for judges the pivotal issues in the areas of science that are often subject to dispute. Citations in the reference guides identify cases in which specific issues were raised; they are examples of other instances in which judges were faced with similar problems. By identifying scientific areas commonly in dispute, the guides should improve the quality of the dialogue between the judges and the parties concerning the basis of expert evidence.
In our committee discussions, we benefited from the judgment and wisdom of the many distinguished members of our committee, who gave time without compensation. They included Justice Ming Chin of the Supreme Court of California; Judge Pauline Newman of the U.S. Court of Appeals for the Federal Circuit in Washington, D.C.; Judge Kathleen MacDonald O’Malley of the U.S. Court of Appeals for the Federal Circuit; Judge Jed Rakoff of the U.S. District Court for the Southern District of New York; Channing Robertson, Ruth G. and William K. Bowes Professor, School of Enginering, and Professor, Department of Chemical Engineering, Stanford University; Joseph Rodricks,
Principal, Environ, Arlington, Virginia; Allen Wilcox, Senior Investigator, Institute of Environmental Health Sciences, Research Triangle Park, North Carolina; and Sandy Zabell, Professor of Statistics and Mathematics, Weinberg College of Arts and Sciences, Northwestern University.
Special commendation, however, goes to Anne-Marie Mazza, Director of the Committee on Science, Technology, and Law, and Joe Cecil of the Federal Judicial Center. These individuals not only shepherded each chapter and its revisions through the process, but provided critical advice on content and editing. They, not we, are the real editors.
Finally, we would like to express our gratitude for the superb assistance of Steven Kendall and for the diligent work of Guru Madhavan, Sara Maddox, Lillian Maloy, and Julie Phillips.
JEROME P. KASSIRER AND GLADYS KESSLER
Committee Co-Chairs
Summary Table of Contents
A detailed Table of Contents appears at the front of each chapter.
Stephen Breyer
The Admissibility of Expert Testimony
Margaret A. Berger
David Goodstein
Reference Guide on Forensic Identification Expertise
Paul C. Giannelli, Edward J. Imwinkelried, & Joseph L. Peterson
Reference Guide on DNA Identification Evidence
David H. Kaye & George Sensabaugh
David H. Kaye & David A. Freedman
Reference Guide on Multiple Regression
Daniel L. Rubinfeld
Reference Guide on Survey Research
Shari Seidman Diamond
Reference Guide on Estimation of Economic Damages
Mark A. Allen, Robert E. Hall, & Victoria A. Lazear
Reference Guide on Exposure Science
Joseph V. Rodricks
Reference Guide on Epidemiology
Michael D. Green, D. Michal Freedman, & Leon Gordis
Bernard D. Goldstein & Mary Sue Henifin
Reference Guide on Medical Testimony
John B. Wong, Lawrence O. Gostin, & Oscar A. Cabrera
Reference Guide on Neuroscience
Henry T. Greely & Anthony D. Wagner
Reference Guide on Mental Health Evidence
Paul S. Appelbaum
Reference Guide on Engineering
Channing R. Robertson, John E. Moalli, & David L. Black