THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.
This study was supported by Contract No. EP-C-09-003, TO#6 between the National Academy of Sciences and the Environmental Protection Agency. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the organizations or agencies that provided support for this project.
International Standard Book Number-13: 978-0-309-13034-9
International Standard Book Number-10: 0-309-13034-4
Additional copies of this report are available from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.
For more information about the Institute of Medicine, visit the IOM home page at: www.iom.edu.
Copyright 2013 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
The serpent has been a symbol of long life, healing, and knowledge among almost all cultures and religions since the beginning of recorded history. The serpent adopted as a logotype by the Institute of Medicine is a relief carving from ancient Greece, now held by the Staatliche Museen in Berlin.
Suggested citation: IOM (Institute of Medicine). 2013. Environmental decisions in the face of uncertainty. Washington, DC: The National Academies Press.
THE NATIONAL ACADEMIES
Advisers to the Nation on Science, Engineering, and Medicine
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council.
COMMITTEE ON DECISION MAKING UNDER UNCERTAINTY
FRANK A. SLOAN (Chair), J. Alexander McMahon Professor of Health and Management and Professor of Economics, Duke University, Durham, NC
JAMES S. HOYTE, Assistant to the President and Associate Vice President, Adjunct Lecturer on Public Policy, Harvard University, Boston, MA (Retired)
ROGER E. KASPERSON, Research Professor and Distinguished Scientist, Clark University, Worcester, MA
EMMETT B. KEELER, Senior Mathematician and Professor of Health Economics, Pardee RAND Graduate School, Santa Monica, CA
SARAH B. KOTCHIAN, Associate Director for Planning, University of New Mexico, Albuquerque (Retired)
JOSEPH V. RODRICKS, Principal, ENVIRON International Corporation, Arlington, VA
SUSAN L. SANTOS, Assistant Professor, University of Medicine and Dentistry of New Jersey, Piscataway
STEPHEN H. SCHNEIDER, 1 Melvin and Joan Lane Professor for Interdisciplinary Environmental Studies, Department of Biology, and Senior Fellow, Woods Institute for the Environment, Stanford University, CA
STEPHANIE TAI, Assistant Professor of Law, University of Wisconsin Law School, Madison
DETLOF VON WINTERFELDT, Professor of Industrial and Systems Engineering and Professor of Public Policy and Management, University of Southern California, Los Angeles
ROBERT B. WALLACE, Irene Ensminger Steecher Professor of Epidemiology and Internal Medicine, University of Iowa, Iowa City
IOM Staff
MICHELLE C. CATLIN, Study Director (from September 2011)
KATHLEEN STRATTON, Study Director (through August 2011)
KRISTINA SHULKIN, Senior Project Assistant (until July 2008)
HOPE HARE, Administrative Assistant
ROSE MARIE MARTINEZ, Director, Board on Population Health and Public Health Practice
___________________
1 Deceased, July 2010.
Reviewers
This report has been reviewed in draft form by persons chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the National Research Council’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report:
Ann Bostrom, University of Washington
E. D. Elliott, Yale University School of Law
William H. Farland, Colorado State University
Adam M. Finkel, University of Medicine and Dentistry of New Jersey
Dennis G. Fryback, University of Wisconsin–Madison
Marianne Horinko, The Horinko Group
Ronald A. Howard, Stanford University
David O. Meltzer, University of Chicago
Kara Morgan, Food and Drug Administration
Richard D. Morgenstern, Resources for the Future
Mary D. Nichols, California Air Resources Board
Gregory M. Paoli, Risk Sciences International
Melissa J. Perry, George Washington University
David Spiegelhalter, Centre for Mathematical Sciences, Cambridge
Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations nor did they see the final draft of the report before its release. The review of this report was overseen by Chris G. Whipple, ENVIRON, and Harold C. Sox, Geisel School of Medicine at Dartmouth. Appointed by the National Research Council and the Institute of Medicine, they were responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution.
Contents
Uncertainty and Environmental Decision Making
The Context of This Report and the Charge to the Committee
Committee’s Approach to the Charge
2 RISK ASSESSMENT AND UNCERTAINTY
Uncertainty and Risk Assessment
The History of Uncertainty Analysis
Newer Approaches to Dealing with Uncertainties
3 UNCERTAINTY IN TECHNOLOGICAL AND ECONOMIC FACTORS IN EPA’S DECISION MAKING
4 UNCERTAINTY AND DECISION MAKING: LESSONS FROM OTHER PUBLIC HEALTH CONTEXTS
Uncertainty and Public Health Decisions
Bovine Spongiform Encephalopathy
Contamination of the Food Supply with Melamine
U.S. Preventive Services Task Force Recommendations
5 INCORPORATING UNCERTAINTY INTO DECISION MAKING
Incorporating Uncertainty into a Decision-Making Framework
6 COMMUNICATION OF UNCERTAINTY
Communication of Uncertainty in Risk Estimates
Considerations When Deciding on a Communications Approach
Boxes, Figures, and Tables
BOXES
FIGURES
TABLES
Multiple sources of uncertainty exist in any risk assessment including those conducted by the U.S. Environmental Protection Agency (EPA), the lead agency responsible for protecting Americans against significant risks to human health and the environment. The EPA asked the Institute of Medicine (IOM) to convene a committee to provide guidance for its decision makers and partners on approaches to manage risk in different contexts when uncertainty is present. To tackle this issue, the IOM assembled a committee of experts in the fields of risk assessment, public health, health economics, decision analysis, public policy, risk communication, and environmental and public health law. The committee met five times, including three open sessions during which committee members discussed relevant issues with outside experts and discussed the charge with the EPA.
In discussing its charge, the committee found it helpful to clarify the questions in its statement of task. When considering that question of “how … uncertainty influence[s] risk management under different public health policy scenarios,” the committee deliberated on how uncertainty can and should influence decisions and help decision makers, rather than focusing on how it currently influences such decisions. In addition, when considering tools and techniques from other areas of public health policy, the committee considered whether there are tools and techniques available from other decision-making settings of potential use to EPA decision making, what their benefits and drawbacks are, and whether and how those tools could be applied by EPA.
Uncertainty is a very broad topic with many potential implications for decision making; this presented a thorny challenge to the committee throughout its deliberations. That challenge was amplified by the broad range of
perspectives and diverse backgrounds committee members brought to the deliberations. The result was adoption of a broader approach to considering uncertainty than is typically taken for environmental decisions. In contrast, historically, much of the work related to uncertainty by EPA and others has focused on the uncertainty in the estimates of human health risks.
Despite a lengthy delay in completing this report, and after responding to excellent peer-review comments, in the end, I am proud of the work we have done and hope that the EPA and other decision makers will find the fundamental report message useful. In summary, that message is that EPA has made substantial technical progress in how it conducts uncertainty analyses in support of its human health risk assessments. However, because uncertainties pervade not only relationships between hazards and health outcomes, more emphasis is needed on the uncertainty in factors affecting EPA’s decisions in addition to estimates of uncertainties in how policies affect human health (e.g., uncertainty in economics and technological assessment that are used for regulatory purposes). Advances in accounting for these latter uncertainties are critical to more robust assessments and ultimately should lead to better decisions.
The committee would like to thank all of the individuals who contributed to the work of the committee, including those who presented to the committee (Appendix C), and the peer reviewers who gave the committee a careful assessment and a list of suggested changes that, when implemented, substantially improved the report. The committee also acknowledges the help of consultants Lynn Goldman and David Paltiel, who provided effective guidance at critical points in the Committee’s work. I would also like to acknowledge committee members Michael Taylor and Robert Perciasepe, who resigned from the committee upon being offered appointments at the FDA and EPA, respectively, Dorothy Patton, who also resigned from the committee, and Steven Schneider, who died in July 2010. All four members made early contributions to the committee’s deliberations but were not involved in the drafting and approval of the final report.
Finally, I would like to thank my colleagues on the committee for their efforts and perseverance throughout what turned out to be a lengthy process. They have argued their positions but also accommodated their colleagues and sought consensus. I would also like to acknowledge the contributions of a number of staff members from IOM, in particular Kathleen Stratton and Michelle Catlin, whose efforts were essential in information gathering, in report writing, in responding to reviewers’ comments, and in providing the committee with assistance and support. Many thanks to many other IOM staff, particularly Rose Marie Martinez, who made important contributions along the way to the final production of this report.
Frank A. Sloan, Chair
Committee on Decision Making Under Uncertainty
ACIP |
Advisory Committee on Immunization Priorities |
AHRQ |
Agency for Healthcare Research and Quality |
ANRF |
American Nonsmokers’ Rights Foundation |
ATSDR |
Agency for Toxic Substances and Disease Registry |
BAT |
best available technology economically achievable |
B/C |
benefit-to-cost |
BCA |
benefit–cost analysis |
BCT |
best conventional pollutant control technology |
BMD |
benchmark dose |
BMR |
benchmark response |
BOD5 |
biochemical oxygen demand |
BPT |
best practicable control technology currently available |
BSE |
bovine spongiform encephalopathy |
CAA |
Clean Air Act |
Cal EPA |
California Environmental Protection Agency |
CCSP |
Climate Change Science Program |
CDC |
U.S. Centers for Disease Control and Prevention |
CDER |
Center for Drug Evaluation and Review |
CDF |
cumulative distribution function |
CDRH |
Center for Devices and Radiological Health |
CEA |
cost-effectiveness analysis |
CERCLA |
Comprehensive Environmental Response, Compensation, and Liability Act |
CFSAN |
Center for Food Safety and Nutrition |
CGE |
computable general equilibrium |
CIN |
cervical intraepithelial neoplasia |
CJD |
Creutzfeldt-Jakob disease |
C–R |
concentration–response |
CWA |
Clean Water Act |
EIA |
economic impact analysis |
EPA |
U.S. Environmental Protection Agency |
EVIU |
expected value of including uncertainty |
EVPI |
expected value of perfect information |
EVSI |
expected value of sample information |
FAO |
Food and Agriculture Organization of the United Nations |
FDA |
U.S. Food and Drug Administration |
FSIS |
Food Safety and Inspection Service |
GACT |
generally available control technology |
GAO |
U.S. Government Accountability Office |
GRADE |
Grading of Recommendations Assessment, Development and Evaluation |
HAP |
hazardous air pollutant |
HCRA |
Harvard Center for Risk Analysis |
HHS |
U.S. Department of Health and Human Services |
HPV |
human papillomavirus |
IARC |
International Agency for Research on Cancer |
IOM |
Institute of Medicine |
IPCC |
Intergovernmental Panel on Climate Change |
LOAEL |
lowest-observed-adverse-effect level |
MACT |
maximum achievable control technology |
MCL |
maximum containment level |
MCLG |
maximum containment level goal |
MEI |
maximally exposed individual |
NAAQS |
National Ambient Air Quality Standards |
NAS |
National Academy of Sciences |
NESHAPS |
National Emission Standards for Hazardous Air Pollutants |
NOAEL |
no-observed-adverse-effect level |
NPV |
net present value |
NRC |
National Research Council |
NTP |
National Toxicology Program |
OMB |
Office of Management and Budget |
OSHA |
Occupational Safety and Health Administration |
OSTP |
Office of Science and Technology Policy |
probability density function |
|
POD |
point of departure |
QALY |
quality-adjusted life-year |
RCRA |
Resource Conservation and Recovery Act |
RfC |
reference concentration |
RfD |
reference dose |
RTE |
ready to eat |
SAB |
Science Advisory Board of the EPA |
SDWA |
Safe Drinking Water Act |
SHS |
secondhand smoke |
SSRI |
selective serotonin reuptake inhibitor |
TCE |
trichloroethylene |
TDI |
tolerable daily intake |
TSE |
transmissible spongiform encephalopathy |
UF |
uncertainty factor |
USDA |
U.S. Department of Agriculture |
USPSTF |
U.S. Preventive Services Task Force |
VOI |
value of information |
VSL |
value of statistical lives |
WHO |
World Health Organization |
WTP |
willingness to pay |