Models in Environmental Regulatory Decision Making

Committee on Models in the Regulatory Decision Process

Board on Environmental Studies and Toxicology

Division on Earth and Life Studies

NATIONAL RESEARCH COUNCIL OF THE NATIONAL ACADEMIES

THE NATIONAL ACADEMIES PRESS

Washington, D.C.
www.nap.edu



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page R1
Models in Environmental Regulatory Decision Making Models in Environmental Regulatory Decision Making Committee on Models in the Regulatory Decision Process Board on Environmental Studies and Toxicology Division on Earth and Life Studies NATIONAL RESEARCH COUNCIL OF THE NATIONAL ACADEMIES THE NATIONAL ACADEMIES PRESS Washington, D.C. www.nap.edu

OCR for page R1
Models in Environmental Regulatory Decision Making THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001 NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. This project was supported by Contract No. 68-C-03-081 between the National Academy of Sciences and the U.S. Environmental Protection Agency. Additional support was provided by the U.S. Department of Transportation. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the organizations or agencies that provided support for this project. International Standard Book Number-13: 978-0-309-11000-6 International Standard Book Number-10: 0-309-11000-9 Additional copies of this report are available from The National Academies Press 500 Fifth Street, NW Box 285 Washington, DC 20055 800-624-6242 202-334-3313 (in the Washington metropolitan area) http://www.nap.edu Copyright 2007 by the National Academy of Sciences. All rights reserved. Printed in the United States of America.

OCR for page R1
Models in Environmental Regulatory Decision Making THE NATIONAL ACADEMIES Advisers to the Nation on Science, Engineering, and Medicine The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council. www.national-academies.org

OCR for page R1
Models in Environmental Regulatory Decision Making This page intentionally left blank.

OCR for page R1
Models in Environmental Regulatory Decision Making COMMITTEE ON MODELS IN THE REGULATORY DECISION PROCESS Members CHRIS G. WHIPPLE (Chair), ENVIRON, Inc., Emeryville, CA M. BRUCE BECK, University of Georgia, Warnell School of Forest Resources, Athens CLAYTON J. CLARK II, University of Florida, Gainesville ROBERT T. CLEMEN, Duke University, Fuqua School of Business, Durham, NC JUDITH A. GRAHAM, American Chemistry Council, Arlington, VA LOUIS J. GROSS, University of Tennessee, Institute for Environmental Modeling, Knoxville WINSTON HARRINGTON, Resources for the Future, Washington, DC PHILIP HOWARD, Syracuse Research Corporation, Environmental Science Center, Syracuse, NY KIMBERLY L. JONES, Howard University, Washington, DC THOMAS E. MCKONE, University of California, Berkeley, and the Lawrence Berkeley National Laboratory, Berkeley, CA NAOMI ORESKES, University of California, San Diego SPYROS N. PANDIS, University of Patras, Patra, Greece LOUISE M. RYAN, Harvard School of Public Health, Department of Biostatistics, Boston, MA MICHAEL L. STEIN, University of Chicago, Statistics Department, Chicago, IL WENDY E. WAGNER, University of Texas School of Law, Austin Staff K. JOHN HOLMES, Project Director MATTHEW RUSSELL, Associate Staff Officer RUTH E. CROSSGROVE, Senior Editor MIRSADA KARALIC-LONCAREVIC, Manager, Technical Information Center RADIAH A. ROSE, Senior Editorial Assistant Sponsor U.S. ENVIRONMENTAL PROTECTION AGENCY U.S. DEPARTMENT OF TRANSPORTATION

OCR for page R1
Models in Environmental Regulatory Decision Making BOARD ON ENVIRONMENTAL STUDIES AND TOXICOLOGY1 Members JONATHAN M. SAMET (Chair), Johns Hopkins University, Baltimore, MD RAMÓN ALVAREZ, Environmental Defense, Austin, TX JOHN M. BALBUS, Environmental Defense, Washington, DC DALLAS BURTRAW, Resources for the Future, Washington, DC JAMES S. BUS, Dow Chemical Company, Midland, MI COSTEL D. DENSON, University of Delaware, Newark E. DONALD ELLIOTT, Willkie, Farr & Gallagher LLP, Washington, DC MARY R. ENGLISH, University of Tennessee, Knoxville J. PAUL GILMAN, Oak Ridge Center for Advanced Studies, Oak Ridge, TN SHERRI W. GOODMAN, Center for Naval Analyses, Alexandria, VA JUDITH A. GRAHAM, American Chemistry Council, Arlington, VA WILLIAM P. HORN, Birch, Horton, Bittner and Cherot, Washington, DC JAMES H. JOHNSON, JR., Howard University, Washington, DC WILLIAM M. LEWIS, JR., University of Colorado, Boulder JUDITH L. MEYER, University of Georgia, Athens DENNIS D. MURPHY, University of Nevada, Reno PATRICK Y. O’BRIEN, ChevronTexaco Energy Technology Company, Richmond, CA DOROTHY E. PATTON (retired), Chicago, IL DANNY D. REIBLE, University of Texas, Austin JOSEPH V. RODRICKS, ENVIRON International Corporation, Arlington, VA ARMISTEAD G. RUSSELL, Georgia Institute of Technology, Atlanta ROBERT F. SAWYER, University of California, Berkeley LISA SPEER, Natural Resources Defense Council, New York, NY KIMBERLY M. THOMPSON, Massachusetts Institute of Technology, Cambridge MONICA G. TURNER, University of Wisconsin, Madison MARK J. UTELL, University of Rochester Medical Center, Rochester, NY CHRIS G. WHIPPLE, ENVIRON International Corporation, Emeryville, CA LAUREN ZEISE, California Environmental Protection Agency, Oakland Senior Staff JAMES J. REISA, Director DAVID J. POLICANSKY, Scholar RAYMOND A. WASSEL, Senior Program Officer for Environmental Sciences and Engineering KULBIR BAKSHI, Senior Program Officer for Toxicology EILEEN N. ABT, Senior Program Officer for Risk Analysis KARL E. GUSTAVSON, Senior Program Officer K. JOHN HOLMES, Senior Program Officer ELLEN K. MANTUS, Senior Program Officer SUSAN N.J. MARTEL, Senior Program Officer STEVEN K. GIBB, Program Officer for Strategic Communications RUTH E. CROSSGROVE, Senior Editor 1 This study was planned, overseen, and supported by the Board on Environmental Studies and Toxicology.

OCR for page R1
Models in Environmental Regulatory Decision Making OTHER REPORTS OF THE BOARD ON ENVIRONMENTAL STUDIES AND TOXICOLOGY Toxicity Testing in the Twenty-first Century: A Vision and a Strategy (2007) Sediment Dredging at Superfund Megasites: Assessing the Effectiveness (2007) Environmental Impacts of Wind-Energy Projects (2007) Scientific Review of the Proposed Risk Assessment Bulletin from the Office of Management and Budget (2007) Assessing the Human Health Risks of Trichloroethylene: Key Scientific Issues (2006) New Source Review for Stationary Sources of Air Pollution (2006) Human Biomonitoring for Environmental Chemicals (2006) Health Risks from Dioxin and Related Compounds: Evaluation of the EPA Reassessment (2006) Fluoride in Drinking Water: A Scientific Review of EPA’s Standards (2006) State and Federal Standards for Mobile-Source Emissions (2006) Superfund and Mining Megasites—Lessons from the Coeur d’Alene River Basin (2005) Health Implications of Perchlorate Ingestion (2005) Air Quality Management in the United States (2004) Endangered and Threatened Species of the Platte River (2004) Atlantic Salmon in Maine (2004) Endangered and Threatened Fishes in the Klamath River Basin (2004) Cumulative Environmental Effects of Alaska North Slope Oil and Gas Development (2003) Estimating the Public Health Benefits of Proposed Air Pollution Regulations (2002) Biosolids Applied to Land: Advancing Standards and Practices (2002) The Airliner Cabin Environment and Health of Passengers and Crew (2002) Arsenic in Drinking Water: 2001 Update (2001) Evaluating Vehicle Emissions Inspection and Maintenance Programs (2001) Compensating for Wetland Losses Under the Clean Water Act (2001) A Risk-Management Strategy for PCB-Contaminated Sediments (2001) Acute Exposure Guideline Levels for Selected Airborne Chemicals (five volumes, 2000-2007) Toxicological Effects of Methylmercury (2000) Strengthening Science at the U.S. Environmental Protection Agency (2000) Scientific Frontiers in Developmental Toxicology and Risk Assessment (2000) Ecological Indicators for the Nation (2000) Waste Incineration and Public Health (2000) Hormonally Active Agents in the Environment (1999) Research Priorities for Airborne Particulate Matter (four volumes, 1998-2004) The National Research Council’s Committee on Toxicology: The First 50 Years (1997) Carcinogens and Anticarcinogens in the Human Diet (1996) Upstream: Salmon and Society in the Pacific Northwest (1996) Science and the Endangered Species Act (1995) Wetlands: Characteristics and Boundaries (1995) Biologic Markers (five volumes, 1989-1995) Review of EPA’s Environmental Monitoring and Assessment Program (three volumes, 1994-1995) Science and Judgment in Risk Assessment (1994) Pesticides in the Diets of Infants and Children (1993)

OCR for page R1
Models in Environmental Regulatory Decision Making Dolphins and the Tuna Industry (1992) Science and the National Parks (1992) Human Exposure Assessment for Airborne Pollutants (1991) Rethinking the Ozone Problem in Urban and Regional Air Pollution (1991) Decline of the Sea Turtles (1990) Copies of these reports may be ordered from the National Academies Press (800) 624-6242 or (202) 334-3313 www.nap.edu

OCR for page R1
Models in Environmental Regulatory Decision Making PREFACE The use of computational models is an essential element of the environmental regulatory process. The complex relationship between environmental emissions, the quality of the environment, and human and ecological impacts are linked by modeling in the regulatory process. The U.S. Environmental Protection Agency (EPA) may make a scientific determination of basic environmental goals, such as how clean our air and water need to be to protect human health and the environment. But determining how those goals can be met while simultaneously allowing for basic economic services, such as transportation, energy, and agriculture, requires that we examine the links, for example, between the auto emission standards and the attainment of ambient air quality standards or between the point sources of water pollution and the quality of water. The spatial and temporal scales on which environmental controls and environmental quality are linked generally do not allow for an observational approach to understand the links between economic activity and environmental quality. These linkages are made by modeling. The task undertaken by this committee for the National Academies was to assess evolving scientific and technical issues related to the development, selection, and use of computational and statistical models in the regulatory process at EPA. In this report, the committee provides advice concerning management, evaluation, and use of models at the agency. Through public workshops and other means, the committee has considered cross-discipline issues related to model development and use, performance evaluation, peer review, uncertainty, and quality assurance–quality control. The committee assessed scientific and technical criteria that should be considered in deciding whether a model and its results could serve as a reasonable basis for environmental regulatory activities. It also examined case studies of model development, evaluation, and application as a basis for arriving at guiding principles. This report has been reviewed in draft form by persons chosen for their diverse perspectives and technical expertise in accordance with procedures approved by National Research Council (NRC) Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards of objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following for their review of this report: George V. Alexeeff, California EPA; Eula Bingham, University of Cincinnati; John Bredehoeft, the Hydrodynamics Group; E. Donald Elliott, Willkie, Farr & Gallagher, LLP; Paul Gilman, Oak Ridge Center for Advanced Studies; James Hammitt, Harvard Center for Risk Analysis; Michael Koerber, Lake Michigan Air Directors Consortium; Charles Lucas, American International Group, Inc. (retired); Virginia McConnell, Resources for the Future, Inc.; Jana Milford, University of Colorado and Environmental Defense; Lee Mulkey, University of Georgia; Kenneth Reckhow, Duke University; and Scott Zeger, Johns Hopkins University. Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations, nor

OCR for page R1
Models in Environmental Regulatory Decision Making did they see the final draft of the report before its release. The review of this report was overseen by John Bailar, University of Chicago (retired), and David Allen, University of Texas. Appointed by the NRC, they were responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the committee and the institution. The committee received oral and written presentations from the following individuals: Gary Foley, U.S. Environmental Protection Agency Tom Voltaggio, U.S. Environmental Protection Agency Albert McGartland, U.S. Environmental Protection Agency S.T. Rao, U.S. Environmental Protection Agency Joseph Merenda, U.S. Environmental Protection Agency Jim Weaver, U.S. Environmental Protection Agency David Burden, U.S. Environmental Protection Agency Tim Wool, U.S. Environmental Protection Agency Leslie Shoemaker, Tetra Tech, Inc. Jim George, Maryland Department of the Environment Cecilia Ho, Federal Highway Administration Harry Kitch, U.S. Environmental Protection Agency Timothy Miller, U.S. Geological Survey Jennifer Sass, Natural Resources Defense Council Scott Slaughter, Center for Regulatory Effectiveness Adam Finkel, University of Medicine and Dentistry of New Jersey and Princeton University Gene Tierney, U.S. Environmental Protection Agency H. Christopher Frey, North Carolina State University Margo Schwab, Office of Management and Budget John Graham, Rand Graduate School Rob Howard, Bechtel-SAIC, LLC Sheila Jasanoff, Harvard University Daniel Krewski, McLaughlin Centre for Population Health Risk Assessment Jan M. Zielinski, McLaughlin Centre for Population Health Risk Assessment Tim Ramsay, McLaughlin Centre for Population Health Risk Assessment Richard T. Burnett, McLaughlin Centre for Population Health Risk Assessment George Leavesley, U.S. Geological Survey Sam Napolitano, U.S. Environmental Protection Agency Elliot Lieberman, U.S. Environmental Protection Agency M. Granger Morgan, Carnegie Mellon University Pasky Pascual, U.S. Environmental Protection Agency Barbara Petersen, Exponent, Inc. and Durango Software, LLC James D. Schaub, U.S. Department of Agriculture Woodrow Setzer, U.S. Environmental Protection Agency Harvey Clewell, Centers for Health Research Rory Conolly, U.S. Environmental Protection Agency Richard Morgenstern, Resources for the Future Robert Perciasepe, Audubon Society Kenneth Reckhow, Duke University Paul Gilman, Oak Ridge Center for Advanced Studies

OCR for page R1
Models in Environmental Regulatory Decision Making The committee and I thank all of these individuals for their contributions. A complete list of dates, titles, and presenter names can be found in Appendix B. The committee and I are also grateful for the assistance of the NRC staff in the preparation of this report. K. John Holmes played a key role in preparing this report as project director. We also thank Raymond Wassel, senior program director of environmental sciences and engineering in the Board on Environmental Studies and Toxicology (BEST), and the other staff members contributing to this report: James Reisa, director of BEST; Steven Gibb, program officer for strategic communications; Ruth Crossgrove, senior editor; Matthew Russell, associate staff officer; Mirsada Karalic-Loncarevic, manager of the Technical Information Center; and Radiah Rose, senior editorial assistant. As chair, I thank all the members of the committee for their expertise and dedicated effort throughout the study. Chris Whipple, Chair Committee on Models in the Regulatory Decision Process

OCR for page R1
Models in Environmental Regulatory Decision Making This page intentionally left blank.

OCR for page R1
Models in Environmental Regulatory Decision Making CONTENTS     SUMMARY   1 1   STUDY BACKGROUND   18      Early Environmental Models,   19      Trends in Environmental Regulatory Model Use,   20      Model Limitations and Assumptions,   26      Origin of Study and Charge to Committee,   27      Committee Approach to the Charge,   31      Report Contents,   39 2   MODEL USE IN THE ENVIRONMENTAL REGULATORY DECISION PROCESS   40      Regulating Without Computational Models,   41      Regulatory Model Classifications,   43      Congressional and Executive Branch Influences,   62      Oversight Processes Governing Regulatory Models at EPA,   69      The Challenges of Modeling in a Regulatory Environment,   79 3   MODEL DEVELOPMENT   83      Introduction,   83      Alternative Model Development Pathways,   84      Overview of Model Development,   85      Interdependence of Models and Data from Measurements,   90      Model Development Phases,   90      Recommendations,   102 4   MODEL EVALUATION   104      Introduction,   104      Essential Objectives for Model Evaluation,   108      Elements of Model Evaluation,   112      Model Evaluation at the Problem Identification Stage,   113      Evaluation at the Conceptual Model Stage,   115      Evaluation at the Computational Model Stage,   117      Evaluation at the Model Application Stage,   134      Management of the Evaluation Process,   144      Recommendations,   160

OCR for page R1
Models in Environmental Regulatory Decision Making 5   MODEL SELECTION AND USE   170      Issues in Model Selection and Application,   171      Proprietary Models,   184      Recommendations,   188 6   FUTURE MODELING ISSUES   190      Expansion of Measurement Systems,   191      Improvements in Model Methods and Technologies,   192      Changes in Perspectives on Model Use in Regulatory Decision Making,   194      In Closing,   196     EPILOGUE   199     REFERENCES   201     GLOSSARY   226     APPENDIXES     A   Biographical Information on the Committee on Models in the Regulatory Decision Process   239 B   Public Workshop Presentations to the Committee on Models in the Regulatory Decision Process   244 C   Categories of Environmental Regulatory Models   249 FIGURES 1-1   Appendix J curve,   22 1-2   Typical EKMA diagram,   22 1-3   An orrery or physical model of the solar system,   33 1-4   The use of a mouse model for estimating human health risks,   34 1-5   Examples of dose-response models for estimating lifetime risk for male bladder cancers due to arsenic in drinking water for various exposed populations,   35 1-6   Components and functional arrangements of the IEUBK model that predict blood lead levels in children,   36 2-1   Basic modeling elements relating human activities and natural systems to environmental impacts,   46 2-2   Basic elements of risk assessment from the National Research Council’s Red Book,   52 2-3   Elements of advanced mechanistic approaches to health risk assessment,   55 2-4   Sources of information for setting various NAAQS,   67 2-5   Flow chart of general regulatory requirements for models used at the regulatory design and promulgation stage,   70

OCR for page R1
Models in Environmental Regulatory Decision Making 3-1   Iterative steps in model development proposed by Jakeman et al. (2006),   88 3-2   Conceptual model for assessing eutrophication in the European seas linking nutrient enrichments and its direct and indirect effects in the ecosystem,   97 4-1   Stages of a model’s life cycle,   107 4-2   Range of probabilities that people assign to different words absent any specific context,   133 4-3   (a) Hypothetical distribution representing uncertainty in number of lives saved by a policy; (b) distribution representing uncertainty in value of a statistical life,   139 4-4   Unconditional posterior distribution for net benefit of policy,   139 4-5   Conditional distributions of net benefit assuming different amounts for the value of a statistical life,   140 5-1   (a) Individual dose-response models, and (b) overall dose-response model fitted using the Bayesian model averaging approach,   175 TABLES 2-1   Examples of EPA’s Web Sites Containing Model Descriptions for Individual Programs,   45 2-2   Examples of Major EPA Documents That Incorporate a Substantial Amount of Computational Modeling Activities,   47 2-3   Examples of Substantive Legislative Directions for EPA Models,   63 3-1   MOBILE Model Revisions,   86 4-1   QA/QC Checks for Model Code,   120 BOXES S-1   Task Statement,   16 1-1   Ozone Modeling and the Irregular Swings Between Policy and Science,   21 2-1   Incorporating Human Behavior into Environmental Models,   50 2-2   Risk Assessment for Arsenic in Drinking Water,   53 2-3   The Development of the Requirement for Regulatory Impact Analysis for Major Federal Rules,   67 2-4   Elements of External Peer Review for Environmental Regulatory Models,   72 2-5   The Different Types of Science Advisory Panels at EPA,   74 3-1   Basic Steps in Modeling Development Process,   89 3-2   Interdependence of Models and Data from Measurements,   91 4-1   Attributes That Foster Accuracy, Precision, Parsimony, and Transparency in Models,   110 4-2   Individual Elements of Model Evaluation,   114 4-3   To Calibrate or Not To Calibrate,   124 4-5   Life-Cycle Evaluation of Models for Assessing Persistence and Long-Range Transport Potential,   151 4-6   Retrospective Analysis of Model Predictions,   158

OCR for page R1
Models in Environmental Regulatory Decision Making 5-1   Example of a Generic Model for Application to Specific Settings,   171 5-2   Appendix W: EPA’s Guidelines on Air Quality Models,   173 5-3   Arsenic in Drinking Water: Model Selection,   174 5-4   Use of Multiple Models of Varying Complexity for Estimating Mercury in Fish,   176 5-5   Model Training and Support,   177 5-6   Confidence Building in Models Through Transparency,   178 5-7   Proprietary Components of Environmental Models,   186

OCR for page R1
Models in Environmental Regulatory Decision Making Models in Environmental Regulatory Decision Making

OCR for page R1
Models in Environmental Regulatory Decision Making This page intentionally left blank.