Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page R1
ENVIRONMENTAL
DECISIONS
IN THE FACE OF
UNCERTAINTY
Committee on Decision Making Under Uncertainty
Board on Population Health and Public Health Practice
OCR for page R2
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001
NOTICE: The project that is the subject of this report was approved by the Govern-
ing Board of the National Research Council, whose members are drawn from the
councils of the National Academy of Sciences, the National Academy of Engineer-
ing, and the Institute of Medicine. The members of the committee responsible for
the report were chosen for their special competences and with regard for appropri-
ate balance.
This study was supported by Contract No. EP-C-09-003, TO#6 between the Na-
tional Academy of Sciences and the Environmental Protection Agency. Any opin-
ions, findings, conclusions, or recommendations expressed in this publication are
those of the author(s) and do not necessarily reflect the view of the organizations
or agencies that provided support for this project.
International Standard Book Number-13: 978-0-309-13034-9
International Standard Book Number-10: 0-309-13034-4
Additional copies of this report are available from the National Academies Press,
500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202)
334-3313; http://www.nap.edu.
For more information about the Institute of Medicine, visit the IOM home page
at: www.iom.edu.
Copyright 2013 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
The serpent has been a symbol of long life, healing, and knowledge among almost
all cultures and religions since the beginning of recorded history. The serpent ad-
opted as a logotype by the Institute of Medicine is a relief carving from ancient
Greece, now held by the Staatliche Museen in Berlin.
Suggested citation: IOM (Institute of Medicine). 2013. Environmental decisions in
the face of uncertainty. Washington, DC: The National Academies Press.
OCR for page R3
“Knowing is not enough; we must apply.
Willing is not enough; we must do.”
—Goethe
Advising the Nation. Improving Health.
OCR for page R4
The National Academy of Sciences is a private, nonprofit, self-perpetuating society
of distinguished scholars engaged in scientific and engineering research, dedicated to
the furtherance of science and technology and to their use for the general welfare.
Upon the authority of the charter granted to it by the Congress in 1863, the Acad-
emy has a mandate that requires it to advise the federal government on scientific
and technical matters. Dr. Ralph J. Cicerone is president of the National Academy
of Sciences.
The National Academy of Engineering was established in 1964, under the charter
of the National Academy of Sciences, as a parallel organization of outstanding engi
neers. It is autonomous in its administration and in the selection of its members,
sharing with the National Academy of Sciences the responsibility for advising the
federal government. The National Academy of Engineering also sponsors engineer-
ing programs aimed at meeting national needs, encourages education and research,
and recognizes the superior achievements of engineers. Dr. Charles M. Vest is presi-
dent of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of
Sciences to secure the services of eminent members of appropriate professions in
the examination of policy matters pertaining to the health of the public. The Insti-
tute acts under the responsibility given to the National Academy of Sciences by its
congressional charter to be an adviser to the federal government and, upon its own
initiative, to identify issues of medical care, research, and education. Dr. Harvey V.
Fineberg is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of Sciences
in 1916 to associate the broad community of science and technology with the
Academy’s purposes of furthering knowledge and advising the federal government.
Functioning in accordance with general policies determined by the Academy, the
Council has become the principal operating agency of both the National Academy
of Sciences and the National Academy of Engineering in providing services to
the government, the public, and the scientific and engineering communities. The
C
ouncil is administered jointly by both Academies and the Institute of Medicine.
Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively,
of the National Research Council.
www.national-academies.org
OCR for page R5
COMMITTEE ON DECISION MAKING UNDER UNCERTAINTY
FRANK A. SLOAN (Chair), J. Alexander McMahon Professor of Health
and Management and Professor of Economics, Duke University,
Durham, NC
JAMES S. HOYTE, Assist to the President and Associate Vice
ant
President, Adjunct Lecturer on Public Policy, Harvard University,
Boston, MA (Retired)
ROGER E. KASPERSON, Research Professor and Distinguished Scientist,
Clark University, Worcester, MA
EMMETT B. KEELER, Senior Mathematician and Professor of Health
Economics, Pardee RAND Graduate School, Santa Monica, CA
SARAH B. KOTCHIAN, Associate Director for Planning, University of
New Mexico, Albuquerque (Retired)
JOSEPH V. RODRICKS, Principal, ENVIRON International
Corporation, Arlington, VA
SUSAN L. SANTOS, Assistant Professor, University of Medicine and
Dentistry of New Jersey, Piscataway
STEPHEN H. SCHNEIDER,1 Melvin and Joan Lane Professor for
Interdisciplinary Environmental Studies, Department of Biology,
and Senior Fellow, Woods Institute for the Environment, Stanford
University, CA
STEPHANIE TAI, Assistant Professor of Law, University of Wisconsin
Law School, Madison
DETLOF VON WINTERFELDT, Professor of Industrial and Systems
Engineering and Professor of Public Policy and Management,
University of Southern California, Los Angeles
ROBERT B. WALLACE, Irene Ensminger Steecher Professor of
Epidemiology and Internal Medicine, University of Iowa, Iowa City
IOM Staff
MICHELLE C. CATLIN, Study Director (from September 2011)
KATHLEEN STRATTON, Study Director (through August 2011)
KRISTINA SHULKIN, Senior Project Assistant (until July 2008)
HOPE HARE, Administrative Assistant
ROSE MARIE MARTINEZ, Director, Board on Population Health and
Public Health Practice
1 Deceased, July 2010.
v
OCR for page R6
OCR for page R7
Reviewers
T
his report has been reviewed in draft form by persons chosen for
their diverse perspectives and technical expertise, in accordance with
procedures approved by the National Research Council’s Report
Review Committee. The purpose of this independent review is to provide
candid and critical comments that will assist the institution in making its
published report as sound as possible and to ensure that the report meets
institutional standards for objectivity, evidence, and responsiveness to the
study charge. The review comments and draft manuscript remain confiden-
tial to protect the integrity of the deliberative process. We wish to thank the
following individuals for their review of this report:
Ann Bostrom, University of Washington
E. D. Elliott, Yale University School of Law
William H. Farland, Colorado State University
Adam M. Finkel, University of Medicine and Dentistry of New Jersey
Dennis G. Fryback, University of Wisconsin–Madison
Marianne Horinko, The Horinko Group
Ronald A. Howard, Stanford University
David O. Meltzer, University of Chicago
Kara Morgan, Food and Drug Administration
Richard D. Morgenstern, Resources for the Future
Mary D. Nichols, California Air Resources Board
Gregory M. Paoli, Risk Sciences International
Melissa J. Perry, George Washington University
David Spiegelhalter, Centre for Mathematical Sciences, Cambridge
vii
OCR for page R8
viii REVIEWERS
Although the reviewers listed above have provided many constructive
comments and suggestions, they were not asked to endorse the conclusions
or recommendations nor did they see the final draft of the report before
its release. The review of this report was overseen by Chris G. Whipple,
ENVIRON, and Harold C. Sox, Geisel School of Medicine at Dartmouth.
Appointed by the National Research Council and the Institute of Medicine,
they were responsible for making certain that an independent examination
of this report was carried out in accordance with institutional procedures
and that all review comments were carefully considered. Responsibility for
the final content of this report rests entirely with the authoring committee
and the institution.
OCR for page R9
Contents
PREFACE xvii
ABBREVIATIONS AND ACRONYMS xix
SUMMARY 1
1 INTRODUCTION 19
Uncertainty and Environmental Decision Making, 20
The Context of This Report and the Charge to the Committee, 28
Committee Process, 30
Committee’s Approach to the Charge, 31
Types of Uncertainty, 38
Overview of the Report, 43
References, 43
2 ASSESSMENT AND UNCERTAINTY 47
RISK
Risk Assessment, 47
Uncertainty and Risk Assessment, 52
The History of Uncertainty Analysis, 56
Newer Approaches to Dealing with Uncertainties, 60
Examples of EPA’s Risk Assessments, 61
Key Findings, 68
References, 69
ix
OCR for page R10
x CONTENTS
3
UNCERTAINTY IN TECHNOLOGICAL AND ECONOMIC
FACTORS IN EPA’S DECISION MAKING 73
Technology Availability, 73
Economics, 79
Other Factors, 99
Key Findings, 100
References, 102
4
UNCERTAINTY AND DECISION MAKING:
LESSONS FROM OTHER PUBLIC HEALTH CONTEXTS 107
Uncertainty and Public Health Decisions, 108
Secondhand Smoke, 113
Listeria monocytogenes, 117
Bovine Spongiform Encephalopathy, 121
Contamination of the Food Supply with Melamine, 124
Avandia®, 127
Vaccination Decisions, 131
Lessons from History, 132
U.S. Preventive Services Task Force Recommendations, 137
Key Findings, 137
References, 139
5
INCORPORATING UNCERTAINTY INTO
DECISION MAKING 147
Incorporating Uncertainty into a Decision-Making
Framework, 149
Other Considerations, 171
Stakeholder Engagement, 173
Key Findings, 176
References, 178
6 COMMUNICATION OF UNCERTAINTY 181
Communication of Uncertainty in Risk Estimates, 182
Presentation of Uncertainty, 184
Considerations When Deciding on a Communications
Approach, 198
Key Findings, 209
References, 211
7 SYNTHESIS AND RECOMMENDATIONS 217
Findings and Recommendations, 219
References, 226
OCR for page R11
CONTENTS xi
APPENDIXES
A Approaches to Accounting for Uncertainty 229
B Committee Member Biographical Sketches 247
C Meeting Agendas 255
OCR for page R12
OCR for page R13
Boxes, Figures, and Tables
BOXES
S-1 Committee’s Statement of Task, 3
S-2 Implications of Uncertainty Analysis for Decision Making, 8
1-1 The Mission of the U.S. Environmental Protection Agency, 21
1-2 Committee’s Statement of Task, 31
1-3 Uncertainty Versus Variability and Heterogeneity: The Committee’s
Use of the Terms, 39
1-4 An Example of a Decision in the Face of Deep Uncertainty, 42
2-1 Definitions, 48
2-2 Development of Estimates of Human Health Risks for Non-Cancer
Endpoints, 49
2-3 Development of Estimates of Human Health Risks for Cancer
Endpoints, 50
2-4 Trichloroethylene Risk Assessment: An Example of the Uncertainties
Present in a Cancer Risk Assessment and How They Could Affect
Regulatory Decisions, 54
3-1 EPA Control Technology Categories, 75
3-2 Definitions of Select Terms Used in Economic Analyses as Defined in
Guidelines for Preparing Economic Analysis, 80
xiii
OCR for page R14
xiv BOXES, FIGURES, AND TABLES
5-1 Definitions of Preliminary Graphical Representations of
Uncertainty, 153
5-2 Implementing Value-of-Information in a Business Context, 167
5-3 Examples of Value-of-Information Measures, 169
6-1 Strengths and Weaknesses of Numeric, Verbal, and Visual
Communication of Risk, 186
6-2 When Greater Attention to Reporting Uncertainties May Be
Needed, 201
7-1 Implications of Uncertainty Analysis for Decision Making, 219
FIGURES
1-1 The Presidential/Congressional Commission on Risk Assessment and
Risk Management’s framework for risk management decisions, 26
1-2 Factors considered in EPA’s decisions, 36
5-1a Framework for decision making, 148
5-1b Considerations for each assessment during phase 2, 149
5-2 Schematic illustrating the values that can be calculated in a value-of-
information analysis, 168
6-1 Nine displays for communicating uncertain estimates for the value
of a single variable used in experiments, 192
6-2 Examples of the most common graphical displays of uncertainty:
(a) a probability density function, (b) a cumulative distribution
function, and (c) a box-and-whisker plot, 194
6-3 Graphic used by Krupnick et al. (2006) to display sources of
uncertainty and to describe the impact of each source of uncertainty
on estimates of expected net benefits in 2025, 195
TABLES
1-1 Selected Statutory Requirements Related to Consideration of
Factors Other Than Estimates of Human Health Risks, 34
4-1 Assessment of Risks, Benefits, Other Decision-Making Factors,
and Uncertainty at Selected Public Health Agencies and
Organizations, 109
4-2 Estimates of Infected Cases of BSE in the 20 Years Following
Introduction of 500 Infected Animals into the United States, 122
OCR for page R15
BOXES, FIGURES, AND TABLES xv
5-1 Influence of the Type and Source of Uncertainty on Incorporating
Uncertainty into a Decision, 154
6-1 Estimated Reduction in Nonfatal Acute Myocardial Infarctions
Associated with Illustrative Attainment Strategies for the Revised
and More Stringent Alternative PM NAAQS in 2020, 188
6-2 Supplemental Qualitative Table Used by the Intergovernmental
Panel on Climate Change to Describe Its Confidence in Conclusions
and Results, 190
OCR for page R16
OCR for page R17
Preface
M
ultiple sources of uncertainty exist in any risk assessment in-
cluding those conducted by the U.S. Environmental Protection
Agency (EPA), the lead agency responsible for protecting Ameri-
cans against significant risks to human health and the environment. The
EPA asked the Institute of Medicine (IOM) to convene a committee to
provide guidance for its decision makers and partners on approaches to
manage risk in different contexts when uncertainty is present. To tackle this
issue, the IOM assembled a committee of experts in the fields of risk as-
sessment, public health, health economics, decision analysis, public policy,
risk communication, and environmental and public health law. The com-
mittee met five times, including three open sessions during which committee
members discussed relevant issues with outside experts and discussed the
charge with the EPA.
In discussing its charge, the committee found it helpful to clarify the
questions in its statement of task. When considering that question of “how
. . . uncertainty influence[s] risk management under different public health
policy scenarios,” the committee deliberated on how uncertainty can and
should influence decisions and help decision makers, rather than focusing
on how it currently influences such decisions. In addition, when considering
tools and techniques from other areas of public health policy, the commit-
tee considered whether there are tools and techniques available from other
decision-making settings of potential use to EPA decision making, what
their benefits and drawbacks are, and whether and how those tools could
be applied by EPA.
Uncertainty is a very broad topic with many potential implications for
decision making; this presented a thorny challenge to the committee through-
out its deliberations. That challenge was amplified by the broad range of
xvii
OCR for page R18
xviii PREFACE
perspectives and diverse backgrounds committee members brought to the
deliberations. The result was adoption of a broader approach to considering
uncertainty than is typically taken for environmental decisions. In contrast,
historically, much of the work related to uncertainty by EPA and others has
focused on the uncertainty in the estimates of human health risks.
Despite a lengthy delay in completing this report, and after responding
to excellent peer-review comments, in the end, I am proud of the work we
have done and hope that the EPA and other decision makers will find the
fundamental report message useful. In summary, that message is that EPA
has made substantial technical progress in how it conducts uncertainty
analyses in support of its human health risk assessments. However, because
uncertainties pervade not only relationships between hazards and health
outcomes, more emphasis is needed on the uncertainty in factors affecting
EPA’s decisions in addition to estimates of uncertainties in how policies
affect human health (e.g., uncertainty in economics and technological as-
sessment that are used for regulatory purposes). Advances in accounting
for these latter uncertainties are critical to more robust assessments and
ultimately should lead to better decisions.
The committee would like to thank all of the individuals who contrib-
uted to the work of the committee, including those who presented to the
committee (Appendix C), and the peer reviewers who gave the committee a
careful assessment and a list of suggested changes that, when implemented,
substantially improved the report. The committee also acknowledges the
help of consultants Lynn Goldman and David Paltiel, who provided effec-
tive guidance at critical points in the Committee’s work. I would also like
to acknowledge committee members Michael Taylor and Robert Perciasepe,
who resigned from the committee upon being offered appointments at the
FDA and EPA, respectively, Dorothy Patton, who also resigned from the
committee, and Steven Schneider, who died in July 2010. All four members
made early contributions to the committee’s deliberations but were not
involved in the drafting and approval of the final report.
Finally, I would like to thank my colleagues on the committee for
their efforts and perseverance throughout what turned out to be a lengthy
process. They have argued their positions but also accommodated their
colleagues and sought consensus. I would also like to acknowledge the con-
tributions of a number of staff members from IOM, in particular Kathleen
Stratton and Michelle Catlin, whose efforts were essential in information
gathering, in report writing, in responding to reviewers’ comments, and
in providing the committee with assistance and support. Many thanks to
many other IOM staff, particularly Rose Marie Martinez, who made im-
portant contributions along the way to the final production of this report.
Frank A. Sloan, Chair
Committee on Decision Making Under Uncertainty
OCR for page R19
Abbreviations and Acronyms
ACIP Advisory Committee on Immunization Priorities
AHRQ Agency for Healthcare Research and Quality
ANRF American Nonsmokers’ Rights Foundation
ATSDR Agency for Toxic Substances and Disease Registry
BAT best available technology economically achievable
B/C benefit-to-cost
BCA benefit–cost analysis
BCT best conventional pollutant control technology
BMD benchmark dose
BMR benchmark response
BOD5 biochemical oxygen demand
BPT best practicable control technology currently available
BSE bovine spongiform encephalopathy
CAA Clean Air Act
Cal EPA California Environmental Protection Agency
CCSP Climate Change Science Program
CDC U.S. Centers for Disease Control and Prevention
CDER Center for Drug Evaluation and Review
CDF cumulative distribution function
CDRH Center for Devices and Radiological Health
CEA cost-effectiveness analysis
CERCLA Comprehensive Environmental Response, Compensation,
and Liability Act
xix
OCR for page R20
xx ABBREVIATIONS AND ACRONYMS
CFSAN Center for Food Safety and Nutrition
CGE computable general equilibrium
CIN cervical intraepithelial neoplasia
CJD Creutzfeldt-Jakob disease
C–R concentration–response
CWA Clean Water Act
EIA economic impact analysis
EPA U.S. Environmental Protection Agency
EVIU expected value of including uncertainty
EVPI expected value of perfect information
EVSI expected value of sample information
FAO Food and Agriculture Organization of the United Nations
FDA U.S. Food and Drug Administration
FSIS Food Safety and Inspection Service
GACT generally available control technology
GAO U.S. Government Accountability Office
GRADE Grading of Recommendations Assessment, Development
and Evaluation
HAP hazardous air pollutant
HCRA Harvard Center for Risk Analysis
HHS U.S. Department of Health and Human Services
HPV human papillomavirus
IARC International Agency for Research on Cancer
IOM Institute of Medicine
IPCC Intergovernmental Panel on Climate Change
LOAEL lowest-observed-adverse-effect level
MACT maximum achievable control technology
MCL maximum containment level
MCLG maximum containment level goal
MEI maximally exposed individual
NAAQS National Ambient Air Quality Standards
NAS National Academy of Sciences
NESHAPS National Emission Standards for Hazardous Air Pollutants
NOAEL no-observed-adverse-effect level
NPV net present value
OCR for page R21
ABBREVIATIONS AND ACRONYMS xxi
NRC National Research Council
NTP National Toxicology Program
OMB Office of Management and Budget
OSHA Occupational Safety and Health Administration
OSTP Office of Science and Technology Policy
PDF probability density function
POD point of departure
QALY quality-adjusted life-year
RCRA Resource Conservation and Recovery Act
RfC reference concentration
RfD reference dose
RTE ready to eat
SAB Science Advisory Board of the EPA
SDWA Safe Drinking Water Act
SHS secondhand smoke
SSRI selective serotonin reuptake inhibitor
TCE trichloroethylene
TDI tolerable daily intake
TSE transmissible spongiform encephalopathy
UF uncertainty factor
USDA U.S. Department of Agriculture
USPSTF U.S. Preventive Services Task Force
VOI value of information
VSL value of statistical lives
WHO World Health Organization
WTP willingness to pay
OCR for page R22
