ENVIRONMENTAL
DECISIONS
IN THE FACE OF
UNCERTAINTY

Committee on Decision Making Under Uncertainty

Board on Population Health and Public Health Practice

INSTITUTE OF MEDICINE
OF THE NATIONAL ACADEMIES

THE NATIONAL ACADEMIES PRESS

Washington, D.C.

www.nap.edu



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page R1
ENVIRONMENTAL DECISIONS IN THE FACE OF UNCERTAINTY Committee on Decision Making Under Uncertainty Board on Population Health and Public Health Practice

OCR for page R1
THE NATIONAL ACADEMIES PRESS  500 Fifth Street, NW  Washington, DC 20001 NOTICE: The project that is the subject of this report was approved by the Govern- ing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineer- ing, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropri- ate balance. This study was supported by Contract No. EP-C-09-003, TO#6 between the Na- tional Academy of Sciences and the Environmental Protection Agency. Any opin- ions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the organizations or agencies that provided support for this project. International Standard Book Number-13:  978-0-309-13034-9 International Standard Book Number-10:  0-309-13034-4 Additional copies of this report are available from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu. For more information about the Institute of Medicine, visit the IOM home page at: www.iom.edu. Copyright 2013 by the National Academy of Sciences. All rights reserved. Printed in the United States of America The serpent has been a symbol of long life, healing, and knowledge among almost all cultures and religions since the beginning of recorded history. The serpent ad- opted as a logotype by the Institute of Medicine is a relief carving from ancient Greece, now held by the Staatliche Museen in Berlin. Suggested citation: IOM (Institute of Medicine). 2013. Environmental decisions in the face of uncertainty. Washington, DC: The National Academies Press.

OCR for page R1
“Knowing is not enough; we must apply. Willing is not enough; we must do.” —Goethe Advising the Nation. Improving Health.

OCR for page R1
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Acad- emy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engi­ neers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineer- ing programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is presi- dent of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Insti- tute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences ­ in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The C ­ ouncil is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council. www.national-academies.org

OCR for page R1
COMMITTEE ON DECISION MAKING UNDER UNCERTAINTY FRANK A. SLOAN (Chair), J. Alexander McMahon Professor of Health and Management and Professor of Economics, Duke University, Durham, NC JAMES S. HOYTE, Assist­­ to the President and Associate Vice ant President, Adjunct Lecturer on Public Policy, Harvard University, Boston, MA (Retired) ROGER E. KASPERSON, Research Professor and Distinguished Scientist, Clark University, Worcester, MA EMMETT B. KEELER, Senior Mathematician and Professor of Health Economics, Pardee RAND Graduate School, Santa Monica, CA SARAH B. KOTCHIAN, Associate Director for Planning, University of New Mexico, Albuquerque (Retired) JOSEPH V. RODRICKS, Principal, ENVIRON International Corporation, Arlington, VA SUSAN L. SANTOS, Assistant Professor, University of Medicine and Dentistry of New Jersey, Piscataway STEPHEN H. SCHNEIDER,1 Melvin and Joan Lane Professor for Interdisciplinary Environmental Studies, Department of Biology, and Senior Fellow, Woods Institute for the Environment, Stanford University, CA STEPHANIE TAI, Assistant Professor of Law, University of Wisconsin Law School, Madison DETLOF VON WINTERFELDT, Professor of Industrial and Systems Engineering and Professor of Public Policy and Management, University of Southern California, Los Angeles ROBERT B. WALLACE, Irene Ensminger Steecher Professor of Epidemiology and Internal Medicine, University of Iowa, Iowa City IOM Staff MICHELLE C. CATLIN, Study Director (from September 2011) KATHLEEN STRATTON, Study Director (through August 2011) KRISTINA SHULKIN, Senior Project Assistant (until July 2008) HOPE HARE, Administrative Assistant ROSE MARIE MARTINEZ, Director, Board on Population Health and Public Health Practice 1  Deceased, July 2010. v

OCR for page R1

OCR for page R1
Reviewers T his report has been reviewed in draft form by persons chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the National Research Council’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confiden- tial to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report: Ann Bostrom, University of Washington E. D. Elliott, Yale University School of Law William H. Farland, Colorado State University Adam M. Finkel, University of Medicine and Dentistry of New Jersey Dennis G. Fryback, University of Wisconsin–Madison Marianne Horinko, The Horinko Group Ronald A. Howard, Stanford University David O. Meltzer, University of Chicago Kara Morgan, Food and Drug Administration Richard D. Morgenstern, Resources for the Future Mary D. Nichols, California Air Resources Board Gregory M. Paoli, Risk Sciences International Melissa J. Perry, George Washington University David Spiegelhalter, Centre for Mathematical Sciences, Cambridge vii

OCR for page R1
viii REVIEWERS Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations nor did they see the final draft of the report before its release. The review of this report was overseen by Chris G. Whipple, ENVIRON, and Harold C. Sox, Geisel School of Medicine at Dartmouth. Appointed by the National Research Council and the Institute of Medicine, they were responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution.

OCR for page R1
Contents PREFACE xvii ABBREVIATIONS AND ACRONYMS xix SUMMARY 1 1 INTRODUCTION 19 Uncertainty and Environmental Decision Making, 20 The Context of This Report and the Charge to the Committee, 28 Committee Process, 30 Committee’s Approach to the Charge, 31 Types of Uncertainty, 38 Overview of the Report, 43 References, 43 2  ASSESSMENT AND UNCERTAINTY 47 RISK Risk Assessment, 47 Uncertainty and Risk Assessment, 52 The History of Uncertainty Analysis, 56 Newer Approaches to Dealing with Uncertainties, 60 Examples of EPA’s Risk Assessments, 61 Key Findings, 68 References, 69 ix

OCR for page R1
x CONTENTS 3  UNCERTAINTY IN TECHNOLOGICAL AND ECONOMIC FACTORS IN EPA’S DECISION MAKING 73 Technology Availability, 73 Economics, 79 Other Factors, 99 Key Findings, 100 References, 102 4  UNCERTAINTY AND DECISION MAKING: LESSONS FROM OTHER PUBLIC HEALTH CONTEXTS 107 Uncertainty and Public Health Decisions, 108 Secondhand Smoke, 113 Listeria monocytogenes, 117 Bovine Spongiform Encephalopathy, 121 Contamination of the Food Supply with Melamine, 124 Avandia®, 127 Vaccination Decisions, 131 Lessons from History, 132 U.S. Preventive Services Task Force Recommendations, 137 Key Findings, 137 References, 139 5  INCORPORATING UNCERTAINTY INTO DECISION MAKING 147 Incorporating Uncertainty into a Decision-Making Framework, 149 Other Considerations, 171 Stakeholder Engagement, 173 Key Findings, 176 References, 178 6 COMMUNICATION OF UNCERTAINTY 181 Communication of Uncertainty in Risk Estimates, 182 Presentation of Uncertainty, 184 Considerations When Deciding on a Communications Approach, 198 Key Findings, 209 References, 211 7 SYNTHESIS AND RECOMMENDATIONS 217 Findings and Recommendations, 219 References, 226

OCR for page R1
CONTENTS xi APPENDIXES A Approaches to Accounting for Uncertainty 229 B Committee Member Biographical Sketches 247 C Meeting Agendas 255

OCR for page R1

OCR for page R1
Boxes, Figures, and Tables BOXES S-1 Committee’s Statement of Task, 3 S-2 Implications of Uncertainty Analysis for Decision Making, 8 1-1 The Mission of the U.S. Environmental Protection Agency, 21 1-2 Committee’s Statement of Task, 31 1-3 Uncertainty Versus Variability and Heterogeneity: The Committee’s Use of the Terms, 39 1-4 An Example of a Decision in the Face of Deep Uncertainty, 42 2-1 Definitions, 48 2-2 Development of Estimates of Human Health Risks for Non-Cancer Endpoints, 49 2-3 Development of Estimates of Human Health Risks for Cancer Endpoints, 50 2-4 Trichloroethylene Risk Assessment: An Example of the Uncertainties Present in a Cancer Risk Assessment and How They Could Affect Regulatory Decisions, 54 3-1 EPA Control Technology Categories, 75 3-2 Definitions of Select Terms Used in Economic Analyses as Defined in Guidelines for Preparing Economic Analysis, 80 xiii

OCR for page R1
xiv BOXES, FIGURES, AND TABLES 5-1 Definitions of Preliminary Graphical Representations of Uncertainty, 153 5-2 Implementing Value-of-Information in a Business Context, 167 5-3 Examples of Value-of-Information Measures, 169 6-1 Strengths and Weaknesses of Numeric, Verbal, and Visual Communication of Risk, 186 6-2 When Greater Attention to Reporting Uncertainties May Be Needed, 201 7-1 Implications of Uncertainty Analysis for Decision Making, 219 FIGURES 1-1 The Presidential/Congressional Commission on Risk Assessment and Risk Management’s framework for risk management decisions, 26 1-2 Factors considered in EPA’s decisions, 36 5-1a Framework for decision making, 148 5-1b Considerations for each assessment during phase 2, 149 5-2 Schematic illustrating the values that can be calculated in a value-of- information analysis, 168 6-1 Nine displays for communicating uncertain estimates for the value of a single variable used in experiments, 192 6-2 Examples of the most common graphical displays of uncertainty: (a) a probability density function, (b) a cumulative distribution function, and (c) a box-and-whisker plot, 194 6-3 Graphic used by Krupnick et al. (2006) to display sources of uncertainty and to describe the impact of each source of uncertainty on estimates of expected net benefits in 2025, 195 TABLES 1-1 Selected Statutory Requirements Related to Consideration of Factors Other Than Estimates of Human Health Risks, 34 4-1 Assessment of Risks, Benefits, Other Decision-Making Factors, and Uncertainty at Selected Public Health Agencies and Organizations, 109 4-2 Estimates of Infected Cases of BSE in the 20 Years Following Introduction of 500 Infected Animals into the United States, 122

OCR for page R1
BOXES, FIGURES, AND TABLES xv 5-1 Influence of the Type and Source of Uncertainty on Incorporating Uncertainty into a Decision, 154 6-1 Estimated Reduction in Nonfatal Acute Myocardial Infarctions Associated with Illustrative Attainment Strategies for the Revised and More Stringent Alternative PM NAAQS in 2020, 188 6-2 Supplemental Qualitative Table Used by the Intergovernmental Panel on Climate Change to Describe Its Confidence in Conclusions and Results, 190

OCR for page R1

OCR for page R1
Preface M ultiple sources of uncertainty exist in any risk assessment in- cluding those conducted by the U.S. Environmental Protection Agency (EPA), the lead agency responsible for protecting Ameri- cans against significant risks to human health and the environment. The EPA asked the Institute of Medicine (IOM) to convene a committee to provide guidance for its decision makers and partners on approaches to manage risk in different contexts when uncertainty is present. To tackle this issue, the IOM assembled a committee of experts in the fields of risk as- sessment, public health, health economics, decision analysis, public policy, risk communication, and environmental and public health law. The com- mittee met five times, including three open sessions during which committee members discussed relevant issues with outside experts and discussed the charge with the EPA. In discussing its charge, the committee found it helpful to clarify the questions in its statement of task. When considering that question of “how . . . uncertainty influence[s] risk management under different public health policy scenarios,” the committee deliberated on how uncertainty can and should influence decisions and help decision makers, rather than focusing on how it currently influences such decisions. In addition, when considering tools and techniques from other areas of public health policy, the commit- tee considered whether there are tools and techniques available from other decision-making settings of potential use to EPA decision making, what their benefits and drawbacks are, and whether and how those tools could be applied by EPA. Uncertainty is a very broad topic with many potential implications for decision making; this presented a thorny challenge to the committee through- out its deliberations. That challenge was amplified by the broad range of xvii

OCR for page R1
xviii PREFACE perspectives and diverse backgrounds committee members brought to the deliberations. The result was adoption of a broader approach to considering uncertainty than is typically taken for environmental decisions. In contrast, historically, much of the work related to uncertainty by EPA and others has focused on the uncertainty in the estimates of human health risks. Despite a lengthy delay in completing this report, and after responding to excellent peer-review comments, in the end, I am proud of the work we have done and hope that the EPA and other decision makers will find the fundamental report message useful. In summary, that message is that EPA has made substantial technical progress in how it conducts uncertainty analyses in support of its human health risk assessments. However, because uncertainties pervade not only relationships between hazards and health outcomes, more emphasis is needed on the uncertainty in factors affecting EPA’s decisions in addition to estimates of uncertainties in how policies affect human health (e.g., uncertainty in economics and technological as- sessment that are used for regulatory purposes). Advances in accounting for these latter uncertainties are critical to more robust assessments and ultimately should lead to better decisions. The committee would like to thank all of the individuals who contrib- uted to the work of the committee, including those who presented to the committee (Appendix C), and the peer reviewers who gave the committee a careful assessment and a list of suggested changes that, when implemented, substantially improved the report. The committee also acknowledges the help of consultants Lynn Goldman and David Paltiel, who provided effec- tive guidance at critical points in the Committee’s work. I would also like to acknowledge committee members Michael Taylor and Robert Perciasepe, who resigned from the committee upon being offered appointments at the FDA and EPA, respectively, Dorothy Patton, who also resigned from the committee, and Steven Schneider, who died in July 2010. All four members made early contributions to the committee’s deliberations but were not involved in the drafting and approval of the final report. Finally, I would like to thank my colleagues on the committee for their efforts and perseverance throughout what turned out to be a lengthy process. They have argued their positions but also accommodated their colleagues and sought consensus. I would also like to acknowledge the con- tributions of a number of staff members from IOM, in particular Kathleen Stratton and Michelle Catlin, whose efforts were essential in information gathering, in report writing, in responding to reviewers’ comments, and in providing the committee with assistance and support. Many thanks to many other IOM staff, particularly Rose Marie Martinez, who made im- portant contributions along the way to the final production of this report. Frank A. Sloan, Chair Committee on Decision Making Under Uncertainty

OCR for page R1
Abbreviations and Acronyms ACIP Advisory Committee on Immunization Priorities AHRQ Agency for Healthcare Research and Quality ANRF American Nonsmokers’ Rights Foundation ATSDR Agency for Toxic Substances and Disease Registry BAT best available technology economically achievable B/C benefit-to-cost BCA benefit–cost analysis BCT best conventional pollutant control technology BMD benchmark dose BMR benchmark response BOD5 biochemical oxygen demand BPT best practicable control technology currently available BSE bovine spongiform encephalopathy CAA Clean Air Act Cal EPA California Environmental Protection Agency CCSP Climate Change Science Program CDC U.S. Centers for Disease Control and Prevention CDER Center for Drug Evaluation and Review CDF cumulative distribution function CDRH Center for Devices and Radiological Health CEA cost-effectiveness analysis CERCLA Comprehensive Environmental Response, Compensation, and Liability Act xix

OCR for page R1
xx ABBREVIATIONS AND ACRONYMS CFSAN Center for Food Safety and Nutrition CGE computable general equilibrium CIN cervical intraepithelial neoplasia CJD Creutzfeldt-Jakob disease C–R concentration–response CWA Clean Water Act EIA economic impact analysis EPA U.S. Environmental Protection Agency EVIU expected value of including uncertainty EVPI expected value of perfect information EVSI expected value of sample information FAO Food and Agriculture Organization of the United Nations FDA U.S. Food and Drug Administration FSIS Food Safety and Inspection Service GACT generally available control technology GAO U.S. Government Accountability Office GRADE Grading of Recommendations Assessment, Development and Evaluation HAP hazardous air pollutant HCRA Harvard Center for Risk Analysis HHS U.S. Department of Health and Human Services HPV human papillomavirus IARC International Agency for Research on Cancer IOM Institute of Medicine IPCC Intergovernmental Panel on Climate Change LOAEL lowest-observed-adverse-effect level MACT maximum achievable control technology MCL maximum containment level MCLG maximum containment level goal MEI maximally exposed individual NAAQS National Ambient Air Quality Standards NAS National Academy of Sciences NESHAPS National Emission Standards for Hazardous Air Pollutants NOAEL no-observed-adverse-effect level NPV net present value

OCR for page R1
ABBREVIATIONS AND ACRONYMS xxi NRC National Research Council NTP National Toxicology Program OMB Office of Management and Budget OSHA Occupational Safety and Health Administration OSTP Office of Science and Technology Policy PDF probability density function POD point of departure QALY quality-adjusted life-year RCRA Resource Conservation and Recovery Act RfC reference concentration RfD reference dose RTE ready to eat SAB Science Advisory Board of the EPA SDWA Safe Drinking Water Act SHS secondhand smoke SSRI selective serotonin reuptake inhibitor TCE trichloroethylene TDI tolerable daily intake TSE transmissible spongiform encephalopathy UF uncertainty factor USDA U.S. Department of Agriculture USPSTF U.S. Preventive Services Task Force VOI value of information VSL value of statistical lives WHO World Health Organization WTP willingness to pay

OCR for page R1