National Academies Press: OpenBook

Science and Decisions: Advancing Risk Assessment (2009)

Chapter: Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice

« Previous: Appendix C: Timeline of Selected Environmental Protection Agency Risk-Assessment Activities
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 299
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 300
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 301
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 302
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 303
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 304
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 305
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 306
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 307
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 308
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 309
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 310
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 311
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 312
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 313
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 314
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 315
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 316
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 317
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 318
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 319
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 320
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 321
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 322
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 323
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 324
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 325
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 326
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 327
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 328
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 329
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 330
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 331
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 332
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 333
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 334
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 335
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 336
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 337
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 338
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 339
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 340
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 341
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 342
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 343
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 344
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 345
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 346
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 347
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 348
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 349
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 350
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 351
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 352
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 353
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 354
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 355
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 356
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 357
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 358
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 359
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 360
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 361
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 362
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 363
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 364
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 365
Suggested Citation:"Appendix D: Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice." National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press. doi: 10.17226/12209.
×
Page 366

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Appendix D Environmental Protection Agency Response to Recommendations from Selected NRC Reports: Policy, Activity, and Practice Table D-1 was developed as an information resource to illustrate the kinds of policies and activities that the Environmental Protection Agency (EPA) has undertaken in response to previous National Research Council recommendations (NRC 1983, 1994, 1996) for the list of bulleted topics presented below. This is not a comprehensive review. Rather, it presents representative recommendations from these key National Research Council reports, begin- ning with the so-called Red Book; related EPA policies as reflected in guidance documents and other materials; and related implementation activities, along with an assessment of some of these guidance documents and implementation activities as summarized in a 2006 report from the Government Accountability Office (GAO). Many of the individual National Research Council reports and EPA documents address the risk-assessment issues below repeatedly and with some variations in a single report. As a result, passages quoted or summarized in the table are highly selected “snapshots” and are not the only examples for the indicated topic in a given report. In addition, the “response” to recommendations in the table is considered somewhat loosely, as it simply considers whether EPA addressed the issue at some point in time. For a full picture on any topic of interest, the committee advises readers to begin with pages cited in the table and to look beyond those citations for related information. Note also that several National Research Council recommendations and EPA policy statements cover multiple topics (such as both “risk characterization” and “uncertainty” or both “models” and “defaults”). Several issues are therefore discussed under several topic headings.,   mpty E cells indicate only that the committee could not easily identify and isolate a representative quotation, not that related policies or implementation activities do not exist.   As explained in Chapter 2, the report cited as “NRC 1994” (Science and Judgment in Risk Assessment) gave special attention to issues arising under the Clean Air Act Amendments of 1990, and many of the recommenda- tions in that report focused on air issues. A recommendation directed mainly to the air program is designated by “(Directed to Air Program).” Similarly, a recommendation directed mainly to the IRIS program is designated by “(Directed to IRIS Program).” 299

300 SCIENCE AND DECISIONS: ADVANCING RISK ASSESSMENT • Aggregate and Cumulative Risk • Default Assumptions and Options • Distinguishing and Linking Risk Assessment and Risk Management • Distinguishing Science and Science Policy • Exposure Assessment (and Methods Validation) • Health-Risk and Toxicity Assessment for Cancer and Other End Points • Inference Guidelines • Interagency and Outside Collaboration • Iterative Approach to Risk Assessment • Models and Model Validation • Peer Review and Expert Panels • Priority-Setting and Data-Needs Management • Problem Formulation and Ecologic Risk Assessment • Public Review and Comment; Public Participation • Risk Characterization • Risk Communication in Relation to Risk Management • Uncertainty Analysis and Characterization • Variability and Differential Susceptibility

TABLE D-1  Environmental Protection Agency Response to National Research Council Recommendations of 1983-2006: Policy, Activity, and Practice Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc Aggregate and NRC 1994 at 240: “EPA should consider EPA 1997a Science Policy Council EPA 2003a: Human Health Research Cumulative Risk using appropriate statistical (e.g., Monte Memorandum: “This guidance directs each Strategy at E-2: “ORD’s research program on Carlo) procedures to aggregate cancer risks office to take into account cumulative risk aggregate and cumulative risk will address from exposure to multiple compounds.” issues in scoping and planning major risk the fact that humans are exposed to mixtures assessments and to consider a broader of pollutants from multiple sources. Research scope that integrates multiple sources, will provide the scientific support for effects, pathways, stressors and populations decisions concerning exposure to a pollutant for cumulative risk analyses in all cases for by multiple routes of exposure or to multiple which relevant data are available.” pollutants having a similar mode of action. ORD will also develop approaches to study EPA1997b Cumulative Risk Assessment how people and communities are affected Guidance: “Agency managers need to following exposure to multiple pollutants place special emphasis on cumulative that may interact with other environmental risk (that is, the potential risks presented stressors.” by multiple stressors in aggregate). The specific elements of risk evaluated need Also: The research strategy identified the to be determined as an explicit part of following research objectives related to the Planning and Scoping (PS) stage of cumulative risk: “(1) Determine the best and each risk assessment. . . . The Agency most cost effective ways to measure human will support research to improve our exposures in all relevant media, including understanding of cumulative risks and pathway-specific measures of multimedia to develop methods to account for the human exposures to environmental multiple elements of risks that affect contaminants across a variety of relevant humans, animals, plants and their microenvironments and exposure durations environment. In addition, the Science and conditions; (2) Develop exposure models Policy Council will support workshops and methods suitable for EPA and the for risk assessors and managers to public to assess aggregate and cumulative discuss implementation opportunities and risk, including mathematical and statistical problems, and solutions.” relationships among sources of environmental contaminants, their environmental fate, and EPA 2000a Supplementary Guidance for pathway specific concentrations; models Conducting Health Risk Assessment of linking dose and exposure from biomarker Chemical Mixtures at xiv: This guidance data; and approaches to assess population- updates the 1986 agencywide guidance on based cumulative risk, including those chemical mixtures and “describes more involving exposure to stressors other than 301 continued

TABLE D-1  Continued 302 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc detailed procedures for chemical mixture pollutants; and (3) Provide the scientific basis assessment using data on the mixture to predict the interactive effects of pollutants of concern, data on a toxicologically in mixtures and the most appropriate similar mixture, and data on the mixture approaches for combining effects and risks component chemicals. [It] is organized from pollutant mixtures.” according to the type of data available to the risk assessor, ranging from data rich GAO 2006 at 50: “The extent to which to data poor situations. . . . An evaluation program offices assess the effects of of the data may lead the user to decide cumulative and aggregate exposures is that only a qualitative analysis should be related to the regulatory responsibilities of performed. This generally occurs in cases each office and by the availability of data. where data quality is poor, inadequate For example, the hazardous air pollutant quantitative data are available, data on office routinely analyzes a mix of chemicals a similar mixture cannot be classified as from various emitting sources, such as “sufficiently similar” to the mixture of petroleum refineries, to regulate hazardous concern, exposures cannot be characterized air pollutants. Similarly, as mentioned above, with confidence, or method-specific the Office of Pesticide Programs is required to assumptions about the toxicologic action consider exposure to pesticides from various of the mixture or of its components cannot pathways, such as food, drinking water, and be met. When this occurs, the risk assessor residential uses, and various routes, such as can still perform a qualitative assessment eating, breathing, and contact with skin.” that characterizes the potential human health impacts from exposure to that Note: The Toxic Substances Control Act mixture.” does not require the Office of Pollution Prevention and Toxics to assess the risks EPA 2003b Framework for Cumulative of a new chemical that may occur through Risk Assessment at xvii: “a simple, flexible its interaction with other chemicals. The structure for conducting and evaluating office also assesses the risks of existing cumulative risk assessment within the chemicals but cannot conduct cumulative risk EPA. . . . The framework describes assessment for classes of chemical that share three main phases to a cumulative risk a common mode of action because no data assessment: (1) planning, scoping, and exist. problem formulation, (2) analysis, and (3) risk characterization…Research and GAO 2006 at 49: “The branch of the Office development needs are also discussed, of Air Quality Planning and Standards that including understanding the timing of regulates hazardous air pollutants employs exposure and its relationship to effects; the Multiple Pathways of Exposure model to

understanding the composition and toxicity assess and predict the movement and behavior of mixtures; applying the risk factor of chemicals in the environment. [It] includes approach; using biomarkers; considering procedures to estimate human exposures and hazards presented by nonchemical stressors; health risks that result from the transfer of methods for combining different types of pollutants from the air to soil and surface risk; and development of default values water bodies and the subsequent uptake of for cumulative risk assessments, among the pollutant by plants, animals, and humans. others.” The model specifically addresses exposures from breathing; consuming food, water, and EPA 2001 and 2002: General Principles for soil; and contact with skin.” Performing Aggregate Exposure and Risk Assessments: This document “focus[es] GAO 2006 at 49: EPA developed the Total on describing principles to guide the way Risk Integrated Methodology (TRIM) and in which aggregate exposure and risk created the TRIM Fate, Transport, and assessment may be performed when more Ecological Exposure model that describes extensive distributional data and more the movement of air pollutants emitted from sophisticated exposure assessment, methods any type of stationary source as well as their and tools are available. . . . [The guidance] transformation over time in water, air and looks beyond the Interim Guidance to soil. encompass the use of distributional data for all pathways of exposure when data are available. A distributional data analysis (as opposed to a point estimate approach) is preferred because this tool allows an aggregate exposure assessor to more fully evaluate exposure and resulting risk across the entire population, not just the exposure of a single, high-end individual.” (EPA 2001, p. 4) The 2002 guidance (EPA 2002a, p. ii) “provides guidance to OPP scientists for evaluating and estimating the potential human risks associated with such multichemical and multipathway exposures to pesticides.” 64 Fed. Reg. 38705[1999]: The Integrated Urban Air Toxics Strategy includes guidance on assessing cumulative risks on both the national and the urban- 303 continued

TABLE D-1  Continued 304 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc neighborhood scales. It provides “an overview of EPA’s national effort to reduce air toxics, including stationary and mobile source standards, cumulative risk initiatives, assessment approaches, and education and outreach.” The “national air toxics program includes activities under multiple Clean Air Act (Act) authorities to reduce air toxics emissions from all sources, including major industrial sources, smaller stationary sources, and mobile sources such as cars and trucks. By integrating activities under different parts of the Act, EPA can better address cumulative public health risks and adverse environmental impacts posed by exposures to multiple air toxics in areas where the emissions and risks are most significant.” EPA 2004a: Air Toxics Risk Assessment Reference Library at 14-1: The guidance states that “multipathway risk assessment may be appropriate generally when air toxics that persist and which also may bioaccumulate and/or biomagnify are present in releases. These generally will focus on the persistent bioaccumulative hazardous air pollutant (PB-HAP) compounds (Exhibit 14-1), but specific risk assessments may need to consider additional chemicals that persist and which also may bioaccumulate and/or biomagnify. For these compounds, the risk assessment generally will need to consider exposure pathways other than inhalation—in particular, pathways that involve deposition of air toxics onto soil and plants and

into water, subsequent uptake by biota, and potential human exposures via consumption of contaminated soils, surface waters, and foods. Substances that persist and bioaccumulate readily transfer between the air, water, and land. Some may travel great distances, and linger for long periods of time in the environment.” The guidance provides information on planning, scoping, problem formulation, data analysis, and risk characterization. EPA 2002a Guidance on cumulative risk assessment of pesticide chemicals that have a common mechanism of toxicity: Provides “guidance to OPP scientists for evaluating and estimating the potential human risks associated with such multichemical and multipathway exposures to pesticides. . .” (p. ii). “Cumulative risk assessments may play a significant role in the evaluation of risks posed by pesticides, and will enable OPP to make regulatory decisions that more fully protect public health and sensitive subpopulations, including infants and children. . . . The purpose of this guidance is to set forth the basic assumptions, principles, and analytical framework that are recommended for use by OPP risk assessors in conducting cumulative risk assessments. It is also intended to inform decision makers and the public of the principles and procedures generally followed in the conduct of cumulative risk assessments on pesticide chemicals” (p. 7). 305 continued

TABLE D-1  Continued 306 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc Default NRC 1994 at 8: “EPA should continue EPA 2005a Carcinogen Risk Assessment GAO 2006 at 41: “To a large degree, the use Assumptions to regard the use of default options as a Guidelines, Appendix A (71 FR 17809-12): of defaults is intertwined with EPA’s ability and Options reasonable way to deal with uncertainty The guideline “covers [five] major default to get the data it needs. As was discussed (see also Risk about underlying mechanisms in selecting options commonly employed when data previously, EPA has targeted research, both Characterization, methods and models for use in risk are missing or sufficiently uncertain in a within EPA and through its grant programs, Models, assessment.” cancer risk assessment. . . . These options to understand variability and uncertainty in Uncertainty are predominantly inferences that can help the data derived from studies of laboratory Analysis) use the data observed under empirical animals, and this research may further reduce conditions in order to estimate events and EPA’s need to rely on default options.” outcomes under environmental conditions.” NRC 1994 at 8: “EPA should explicitly EPA 2004b at 51: “EPA’s current GAO 2006 at 40: “The majority of IRIS identify each use of a default option in risk practice is to examine all relevant and assessments completed since 1997 describe assessments.” available data first when performing a the defaults used in the analysis and any risk assessment. When the chemical- and/ departures from those defaults. NRC 1994 at 8: “EPA should clearly state or site-specific data are unavailable (i.e., the scientific and policy basis for each when there are data gaps) or insufficient to “Despite the increased focus on more default option.” estimate parameters or resolve paradigms, transparency in the use of defaults, EPA EPA uses a default assumption in order to acknowledges it could more consistently continue with the risk assessment. Under describe how the default was developed and this practice EPA invokes defaults only explain why it is a reasonable assumption. after the data are determined to be not In its staff paper, EPA acknowledges it needs usable at that point in the assessment—this to ensure that the defaults are supported is a different approach from choosing by the best available data and should look defaults first and then using data to depart for opportunities to increase certainty and from them. The default assumptions confidence in the defaults and extrapolations are not chemical- or site-specific, but used.” are relevant to the data gap in the risk assessment. They are based on peer EPA 2004a: The Office of Air Quality reviewed studies and extrapolation to Planning and Standard’s Air Toxics Risk address specific data gaps. These defaults Assessment Reference Library (EPA 2004a) are based on published studies, empirical discusses defaults that should be used when observations, extrapolation from related preparing risk assessments. This is discussed, observations, and/or scientific theory.” for example, when conducting screening analyses: “For complete or potential exposure EPA 1996 Proposed Carcinogen Risk pathways identified in the exposure pathway Assessment Guidelines at 61 FR 18000: evaluation, the screening analysis may involve

Risk characterization includes “risk comparing media concentrations at points estimates and their attendant uncertainties, of exposure to ‘screening’ values (based on including key uses of default assumptions protective default exposure assumptions) when data are missing or uncertain.” and estimating exposure doses based on study area-specific exposure conditions. The Also at 17966-17970ff: Explaining the assessor then compares estimated doses with scientific and policy bases of five “major” health-based guidelines to identify substances default options. requiring further evaluation.” Also at 17964ff: “Pursuant to [the National Research Council recommendation related to criteria for departure from defaults] the following discussion presents . . . general policy guidance on using and departing from defaults in specific risk assessments.” NRC 1994 at 8: “The agency should EPA 2005a Carcinogen Risk Assessment consider attempting to give greater Guidelines at 71 FR 17770ff: “Rather than formality to its criteria for a departure viewing default options as starting points from default options, in order to give from which departures may be justified by greater guidance to the public and to lessen new scientific information, these cancer the possibility of ad hoc, undocumented guidelines view a critical analysis of all of departures from default options that the available information that is relevant would undercut the scientific credibility to assessing the carcinogenic risk as the of the agency’s risk assessments. At the starting point from which a default option same time, the agency should be aware of may be invoked if needed to address the undesirability of having its guidelines uncertainty or the absence of critical evolve into inflexible rules.” information [emphasis in original].” Also Appendix A at 17809ff: Discusses default options and alternative approaches. EPA 2000b Risk Characterization Handbook at 21: Directs risk assessors to “describe the uncertainties inherent in the risk assessment and the default positions used to address these uncertainties or gaps in the assessment.” 307 continued

TABLE D-1  Continued 308 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc Also at 41: “Risk assessors should carefully consider all available data before deciding to rely on default assumptions. If defaults are used, the risk assessment should reference the Agency guidance that explains the default assumptions or values.” NRC 1994 at 186: “EPA sometimes EPA 2005a Carcinogen Risk Assessment Note: Although both the 1996 and 2005 attempts to ‘harmonize’ risk-assessment Guidelines at 70 FR 17808: “Important guidelines refer to the scaling-factor issue (at procedures between itself and other features [of the risk characterization] 61 FR 17968 and 71 FR 17796, respectively), agencies, or among its own programs, include the constraints of available data it is not clear whether EPA has addressed by agreeing on a single common model and the state of knowledge, significant interagency harmonization to the extent assumption, even though the assumption scientific issues, an significant science and recommended. chosen might have little more scientific science policy choices that were made plausibility than alternatives (e.g., replacing when alternative interpretation of data FDA’s body-weight assumption and EPA’s exist [citations omitted]. Choices made surface-area assumption with body weight about using data or default options in the to the 0.75 power). . . . Rather than assessment are explicitly discussed in the ‘harmonizing’ risk assessments by picking course of the analysis, and if a choice is one assumption over others when several a significant issue, it is highlighted in the assumptions are plausible and none of summary.” the assumptions is clearly preferable, EPA should maintain its own default assumption for regulatory decisions but indicate that any of the methods might be accurate and present the results as an uncertainty in the risk estimate or present multiple estimates and state the uncertainty in each. However, ‘harmonization’ does serve an important purpose in the context of uncertainty analysis—it will help, rather than hinder, risk assessment if agencies cooperate to choose and validate a common set of uncertainty distributions.”

NRC 1994 at 241: “EPA’s guidelines EPA 2005a Carcinogen Risk Assessment should clearly state a default option Guidelines at 70 FR 17811: “The linear of nonthreshold low-dose linearity for approach is used when a view of the mode genetic effects on which adequate data of action indicates a linear response, for (e.g., data on chromosomal aberrations example, when a conclusion is made that or dominant or X-linked mutations) an agent directly causes alterations in might exist. This default option allows a DNA, a kind of interaction that not only reasonable quantitative estimate of, for theoretically requires one reaction but example, first-generation genetic risk due also is likely to be additive to ongoing, to environmental chemical exposure.” spontaneous gene mutation.” Distinguishing NRC 1983 at 7: “Regulatory agencies EPA 1984 at 3: “Scientists assess a risk Administrators William Ruckelshaus and Lee Linking Risk should take steps to establish and maintain to find out what the problems are. The Thomas mandated and funded a series of Assessment a clear conceptual distinction between process of deciding what to do about the training programs for (1) the entire SES corps and Risk assessment of risks and the consideration problems is ‘risk management.’ . . . The and other senior management and (2) agency Management of risk management alternatives; that is, distinction between the two activities staff in all program and regional offices. The (see also the scientific findings and policy judgments has become an attractive means for training materials were based in materials Problem embodied in risk assessments should be understanding and improving upon the developed initially by Bernard Goldstein and Formulation) explicitly distinguished from the political, two fundamental processes involved in Jack Moore. There was a heavy financial economic, and technical considerations environmental decision-making.” investment in the program, which ran for that influence the design and choice of about 5 y (approximately 1987-1992), with regulatory strategies.” EPA 1984 at 30: “First, we want to obtain remnants and updates continuing sporadically a better and more consistent information even today. NRC 1983 at 49: “Two kinds of policy base for making decisions about the can potentially affect risk assessment: that control of risk. Second, we want to use the which is inherent in the assessment process various analytic methods associated with itself and that which governs the selection risk management whenever appropriate in of regulatory options. The latter, risk developing environmental policy; we also management policy, should not be allowed want to place more emphasis on figuring to control the former, risk-assessment out what we have achieved in terms of policy.” risk reduction through past efforts and on locating and efficiently managing the serious risks remaining. Third, we must communicate to the public what we are doing, why we are doing it in risk management terms, and how the risk management approach will improve the way that EPA carries out its mission.” EPA 1986 Guidelines for Carcinogen Risk 309 Assessment at 51 FR 33993: “Regulatory continued

TABLE D-1  Continued 310 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc decision-making involves two components: risk assessment and risk management. . . . The risk assessments will be carried out independently from considerations of the consequences of regulatory action.” NRC 1994 at 267: “EPA should increase EPA 1995a Agency-wide Policy 1985: EPA established the Risk Assessment institutional and intellectual linkages Memorandum: “Risk characterization, the Forum, an interoffice standing committee between risk assessment and risk last step in risk assessment, is the starting of senior scientists. From 1985 to 2006, the management so as to create better harmony point for risk management considerations forum published more than 50 peer-reviewed between the science-policy components and the foundation for regulatory decision- reports on science-policy issues for the of risk assessment and the broader policy making, but it is only one of several information of risk assessors, risk managers, objectives of risk management. This must important components in such decisions. and the public. be done in a way that fully protects the As the last step in risk assessment, the accuracy, objectivity, and integrity of its risk characterization identifies and EPA established the Risk Management risk assessments—but the committee does highlights the noteworthy risk conclusions Council as a multioffice decision-making body not see these two aims as incompatible.” and related uncertainties. Each of the to deal with the multioffice forum products. environmental laws administered by EPA In 1993, the RMC became the Science Policy calls for consideration of other factors at Council; over the years, it has taken on a various stages in the regulatory process. broader range of activities. When the RMC As authorized by different statutes, was set up, only senior science managers decisionmakers evaluate technical feasibility and political appointees with a science (e.g., treatability, detection limits), background were members; the SPC is more economic, social, political, and legal factors broadly based, but it is an extension of the as part of the analysis of whether or not to original RMC. regulate and, if so, to what extent. Thus, regulatory decisions are usually based on GAO 2006 at 29: “The Office of Air and a combination of the technical analysis Radiation recognized the need for planning used to develop the risk assessment and and developed planning guidance as part of information from other fields.” its Air Toxics Risk Assessment Reference Library, issued in 2004. EPA acknowledged in EPA 2002b Lessons Learned from its 2004 staff paper that it needs to continue Planning and Scoping Environmental Risk to stress the importance of concerted and Assessments at vi: Was developed “to conscious planning with risk assessors and provide early feedback to agency scientists risk managers before a risk assessment is and managers regarding our experiences started. According to EPA, risk assessors need with planning and scoping as the first step to outline early in the development of

in conducting environmental assessments a risk management what will and will not be since the 1997 ‘Guidance on Cumulative addressed and how they will develop the risk Risk Assessment—Part 1’. . . . This management.” handbook is meant to reinforce the concept that formal planning and dialogue prior to the conduct of an environmental assessment can improve the final assessment product in terms of relevancy to an environmental decision and addressing the concerns of decision makers, scientists, economists and stakeholders (where applicable). This handbook is also meant to be a catalyst to encourage agency managers to adopt formal planning and scoping as part of EPA’s culture, especially when conducting significant and/or unique environmental assessments.” EPA 1998, Ecological Risk Assessment Guidelines at 13: “The characteristics of an ecological risk assessment are directly determined by agreements reached by risk managers and risk assessors during planning dialogues. These agreements are the products of planning. They include (1) clearly established and articulated management goals, (2) characterization of decisions to be made within the context of the management goals, and (3) agreement on the scope, complexity, and focus of the risk assessment, including the expected output and the technical and financial support available to complete it.” EPA 2000b Risk Characterization Handbook at 28: “Planning and scoping provides the opportunity for the risk manager(s), the risk assessor(s), and other members of the ‘team’ to define what is 311 expected to be covered in the risk continued

TABLE D-1  Continued 312 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc assessment and to explain the purposes for which the risk assessment information will be used. During the planning and scoping phase of the risk assessment process risk assessors and risk managers should engage in a dialog to identify: a) Motivating need for the risk assessment (regulatory requirements? public concern? scientific findings? other factors?); b) Management goals, issues, and policies needing to be addressed; c) Context of the risk; d) Scope and coverage of the effort; e) Current knowledge; f) What and where are the available data; g) An agreement about how to conduct the assessment . . .; h) Plans for how the results will be communicated to senior managers and to the public; and i) Information needs/data for other members of the ‘team’ to conduct their analyses (e.g., economic, social, or legal analyses).” Distinguishing NRC 1983 at 153: “Before an agency EPA 1996 Proposed Carcinogen Assessment Science and decides whether a substance should or Guidelines at 61 FR 17968: “The default Science Policy should not be regulated as a health hazard, to include benign tumors observed in a detailed and comprehensive written risk animal studies in the assessment of animal assessment should be prepared and made tumor incidence is they have the capacity publicly accessible. This written assessment to progress to the malignancies with which should clearly distinguish between the they are associated. This treats the benign scientific basis and the policy basis for the and malignant tumors as representative agency’s conclusions.” of responses to the test agent, which is scientifically appropriate. This is a science NRC 1994 at 27: “Science-policy choices policy decision that is somewhat more are distinct from the policy choices conservative of public health than not associated with ultimate decision-making including benign tumors in the assessment.” . . . The science-policy choices that regulatory agencies make in carrying out Also at 17977: These guidelines adopt the risk assessment have considerable influence science-policy position that tumor findings

on the results. . . .” in animals indicate that an agent may produce such effects in humans. Also: “Interagency and public understanding would be served by the EPA 1998 Ecological Risk Guidelines at preparation and release of a report on 110: The risk-assessment report should the science-policy issues and decisions discuss “science policy judgments or default that affect EPA’s risk-assessment and risk- assumptions used to bridge information management practices.” gaps and the basis for these assumptions.” EPA 2005a Carcinogen Risk Assessment Guidelines, at 70 FR 17774: “The agency considered both the advantages and disadvantages to extending recommended, age dependent adjustment factors for carcinogenic potency to carcinogenic agents for which the mode of action remains unknown EPA decided to recommend these factors only for carcinogens acting through a mutagenic mode of action based on a combination of analysis of available data and long-standing science policy positions which govern the Agency’s overall approach to carcinogen risk assessment.” Also at 17808: “Important features [of the risk characterization] include the constraints of available data and the state of knowledge, significant scientific issues, and significant science and science policy choices that were made when alternative interpretations of data exist.” Exposure NRC 1994 at 217: “The committee EPA 1992ad Guidelines for Exposure Assessment endorses the EPA’s use of bounding Assessment, sec. 5.3.4: “A common (and Methods estimates, but only in screening assessments approach to estimating exposure and Validation) to determine whether further levels of dose is to do a preliminary evaluation, analysis are necessary. For further levels or screening step, during which bounding of analysis, the committee supports EPA’s estimates are used, and then to proceed to development of distributions of exposure refine the estimates for those pathways 313 continued

TABLE D-1  Continued 314 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc values based on available measurements, that cannot be eliminated as of trivial modeling results, or both. These importance. distributions can also be used to estimate “The method used for bounding estimates the exposure of the maximally exposed is to postulate a set of values for the person. For example, the most likely parameters in the exposure or dose value of the exposure to the most exposed equation that will result in an exposure person is generally the 100[(N - 1)/N] or dose higher than any exposure or dose percentile of the cumulative probability expected to occur in the actual population. distribution characterizing interindividual The estimate of exposure or dose calculated variability in exposure, where N is the by this method is clearly outside of (and number of persons used to construct the higher than) the distribution of actual exposure distribution. This is a particularly exposures or doses. If the value of this convenient estimator to use because it is bounding estimate is not significant, the independent of the shape of the exposure pathway can be eliminated from further distribution. The committee recommends refinement. that EPA explicitly and consistently use an “There are two important points about estimator such as 100[(N - 1)/N], because bounding estimates. First, the only thing it, and not a vague estimate ‘somewhere the bounding estimate can establish is a above the 90th percentile,’ is responsive to level to eliminate pathways from further the language in CAAA-90 calling for the consideration. It cannot be used to make a calculation of risk to ‘the individual most determination that a pathway is significant exposed to emissions.’” (that can only be done after more (Directed to Air Program) information is obtained and a refinement of the estimate is made), and it certainly cannot be used for an estimate of actual exposure (since by definition it is clearly outside the actual distribution). Second, when an exposure scenario is presented in an assessment, it is likely that the amount of refinement of the data, information, and estimates will vary by pathway, some having been eliminated by bounding estimates, some eliminated after further refinement, and others fully developed and quantified. This is an efficient way to evaluate scenarios. In such cases, bounding estimates must not be considered to be

equally as sophisticated as an estimate of a fully developed pathway, and should not be described as such.” Note: The 1992 Guidelines do not address the 1994 recommendation for a 100[(N - 1)/N] estimator. NRC 1994 at 218: “EPA should use the EPA 2005a Guidelines for Carcinogen Risk mean of current life expectancy as the Assessment at 71 FR 17801: “Unless there assumption for the duration of individual is evidence to the contrary in a particular residence time in a high-exposure area, case, the cumulative dose received over or a distribution of residence times which a lifetime, expressed as average daily accounts for the likelihood that changing exposure prorated over a lifetime, is residences might not result in significantly recommended as an appropriate measure of lower exposure. Similarly, EPA should use exposure to a carcinogen.” a conservative estimate for the number of hours a day an individual is exposed, EPA 1992ad Sec. 4.3.1: “The Exposure or develop a distribution of the number Factors Handbook (EPA 1989a [updated of hours per day an individual spends later]) contains a summary of published in different exposure situations. Such data on activity patterns along with information can be gathered through citations. Note that the summary data neighborhood surveys, etc. in these high- and the mean values cited are for the data exposure areas. Note that the distribution sets included in the Handbook, and may would correctly be used only for individual or may not be appropriate for any given risk calculations, as total population risk is assessment.” unaffected by the number of persons whose Note: Choice of parameters within EFH exposures sum to a given total value (if is left to the discretion of the assessor risk is linearly related to exposure rate).” depending on the assessment goals, and so on. “EPA has not provided sufficient EPA 1992ad Sec. 5.3.5.1: “Some of the documentation in its exposure-assessment alternative methods for determining a high- guidelines to ensure that its point- end estimate of [exposure and] dose are: estimation techniques used to determine the (1) If sufficient data on the distribution ‘high-end exposure estimate’ (HEEE) when of doses are available, take the value data are sparse reliably yield an estimate directly for the percentile(s) of interest at the desired location within the overall within the high end. If possible, the actual distribution of exposure (which, according percentile(s) should be stated, or the to these guidelines, lies above the 90th number of persons determined in the high 315 percentile but not beyond the confines of end above the estimate, in order to give continued

TABLE D-1  Continued 316 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc the entire distribution).” (Directed to Air the risk manager an idea of where within Program) the high end-range the estimate falls. (2) If data on the distribution of doses are not available, but data on the parameters used to calculate the dose are available, a simulation (such as an exposure model or Monte Carlo simulation) can sometimes be made of the distribution. In this case, the assessor may take the estimate from the simulated distribution. (3) If some information on the distribution of the variables making up the exposure or dose equation (e.g., concentration, exposure duration, intake or uptake rates) is available, the assessor may estimate a value which falls into the high end by meeting the defining criteria of ‘high end’: an estimate that will be within the distribution, but high enough so that less than 1 out of 10 in the distribution will be as high. The assessor often constructs such an estimate by using maximum or near-maximum values for one or more of the most sensitive variables, leaving others at their mean values The exact method used to calculate the estimate of high- end exposure or dose is not critical; it is very important that the exposure assessor explain why the estimate, in his or her opinion, falls into the appropriate range, not above or below it. (4) If almost no data are available, it will be difficult, if not impossible, to estimate exposures or doses in the high end. One method that has been used, especially in screening-level assessments, is to start with a bounding estimate and back off the limits used until

the combination of parameter values is, in the judgment of the assessor, clearly in the distribution of exposure or dose. Obviously, this method results in a large uncertainty. The availability of pertinent data will determine how easily and defensibly the high-end estimate can be developed by simply adjusting or backing off from the ultra conservative assumptions used in the bounding estimates. This estimate must still meet the defining criteria of ‘high end,’ and the assessor should be ready to explain why the estimate is thought to meet the defining criteria.” NRC 1994 at 218: “EPA should provide a Note: See box immediately above for clear method and rationale for determining related guidance in the 1992 Exposure when point estimators for the HEEE can Assessment Guidelines. or should be used instead of a full Monte Carlo (or similar) approach to choosing the desired percentile explicitly. The rationale should more clearly indicate how such estimators are to be generated, should offer more documentation that such point- estimation methods do yield reasonably consistent representations of the desired percentile, and should justify the choice of such a percentile if it differs from that which corresponds to the expected value of exposure to the ‘person most exposed to emissions.’” (Directed to air program) NRC 1994 at 240: “Health-risk Note: See the Framework for Cumulative assessments should generally consider all Risk Assessment (EPA 2003b) and the possible routes by which people at risk pesticides cumulative risk guidance (EPA might be exposed, and this should be done 2002a) for related discussion. universally for compounds regulated by EPA under the Clean Air Act Amendments 317 of 1990. The agency’s risk-assessment continued

TABLE D-1  Continued 318 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc guidance for Superfund-related regulatory compliance (EPA 1989b) can serve as a guide in this regard, but EPA should take advantage of new developments and approaches to the analysis of multimedia fate and transport data. This will facilitate systematic consideration of multiroute exposures in designing and measuring compliance with Clean Air Act requirements.” (Directed to air program) NRC 1994 at 140: “EPA should explicitly Note: Nothing in the 1992 exposure consider the inclusion of noninhalation assessment guidelines prevents an assessor pathways, except where there is prevailing from considering these pathways. In Sec. evidence that noninhalation routes—such as 7.3, a reviewer of the assessment is asked, deposition, bioaccumulation, and soil and “Has the pathways analysis been broad water uptake—are negligible.” enough to avoid overlooking a significant (Directed to air program) pathway? For example, in evaluating exposure to soil contaminated with PCBs, the exposure assessment should not be limited only to evaluating the dermal contact pathway. Other pathways, such as inhalation of dust and vapors or the ingestion of contaminated gamefish from an adjacent stream receiving surface runoff containing contaminated soil, should also be evaluated as they could contribute higher levels of exposure from the same source.” Health Risk NRC 1994 at 141: “In the absence EPA 2005a Guidelines for Carcinogen and Toxicity of human evidence for or against Risk Assessment at 70FR 17772 Sec. Assessment for carcinogenicity, EPA should continue to 1.3.3: “Data from epidemiological studies Cancer and depend on laboratory-animal data for are generally preferred for characterizing Other End estimating the carcinogenicity of chemicals. human cancer hazard and risk.” Points However, laboratory-animal tumor data

should not be used as the exclusive Also: “The cancer guidelines emphasize evidence to classify chemicals as to their the importance of weighing all of the human carcinogenicity if the mechanisms evidence in reaching conclusions about operative in laboratory animals are unlikely the carcinogenic potential of agents. . to be operative in humans; EPA should . . Evidence considered includes tumor develop criteria for determining when this findings, or lack thereof, in humans and is the case for validating this assumption laboratory animal; an agent’s chemical and and for gathering additional data when the physical properties; its structure-activity finding is made that the species tested are relationships (SARS) as compared with irrelevant to humans.” other carcinogenic agents; and studies addressing potential carcinogenic processes NRC 1994 at 142: “Pharmacokinetic and and mode(s) of action, either in vivo or in pharmacodynamic data and models should vitro.” be validated, and quantitative extrapolation from animal bioassays to human should Also at 17771: “The use of mode of action continue to be evaluated and used in risk in the assessment of potential carcinogens assessment.” is a main focus of these cancer guidelines.” NRC 1994 at 9: “EPA should continue Also at 17788-91ff: “The interaction to explore and, when scientifically between the biology of the organism appropriate, incorporate pharmacokinetic and the chemical properties of the agent models of the link between exposure determine whether there is an adverse and biologically effective dose (i.e., dose effect Thus, mode of action analysis is reaching the target tissue).” based on physical, chemical and biological information that helps to explain key events in an agent’s influence on tumor development. The entire range of information developed in the assessment is reviewed to arrive at a reasoned judgment.” NRC 1994 at 141: “EPA should continue EPA 1996 Proposed Carcinogen Risk Note: It is not clear whether EPA has worked to use the results of studies in mice and Assessment Guidelines at 61 FR 17976: with NTP or other entities on the question of rats to evaluate the possibility of chemical “The default assumption is that positive testing species other than mice and rats. carcinogenicity in humans.” effects in animal cancer studies indicate that the agent under study can have NRC 1994 at 141: “EPA and NTP are carcinogenic potential in humans.” encouraged to explore the use of alternative species to test the hypothesis that results obtained in mice and rats are relevant to 319 human carcinogenesis, the use of younger continued

TABLE D-1  Continued 320 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc animals when unique sensitivity might exist for specific chemicals, and the age- dependent effects of exposure.” NRC 1994 at 142: “EPA should provide EPA 2005a Guidelines for Carcinogen Risk comprehensive narrative statements Assessment at 70FR17775: “The cancer regarding the hazards posed by carcinogens, guidelines emphasize the importance of a to include qualitative descriptions of both: clear and useful characterization narrative 1) the strength of evidence about the risks that summarizes the analyses of hazard, of a substance; and 2) the relevance to dose response, and exposure assessment. humans of the animal models and results These characterizations summarize the and of the conditions of exposure (route, assessments to explain the extent and dose, timing, duration, etc.) under which weight of evidence, major points of carcinogenicity was observed to the interpretation and rationale for their conditions under which people are likely selection, strengths and weaknesses of to be exposed environmentally. EPA should the evidence and the analysis, and discuss develop a simple classification scheme that alternative conclusions and uncertainties incorporates both these elements. A similar that deserve serious consideration [citing scheme to that set forth in Table 7-1 EPA’s Risk Characterization Handbook]. (NRC 1994) is recommended. The agency See section 5.4 of the guidelines for more should seek international agreement on a complete details.” classification system.” EPA 1996 Proposed Guidelines for NRC 1994 at 10: “EPA should develop a Carcinogen Risk Assessment at 61 FR two-part scheme for classifying evidence 17985: “Hazard classification uses three on carcinogenicity that would incorporate categories of descriptors for human both a simple classification and a narrative carcinogenic potential. . . . The descriptors evaluation. At a minimum, both parts are presented only in the context of a should include the strength (quality) of the weight of evidence narrative. . . . Using evidence, the relevance of the animal model them within a narrative preserves and and results to humans, and the relevance presents the complexity that is an essential of the experimental exposures (route, dose, part of the hazard classification.” timing, and duration) to those likely to be encountered by humans.” EPA 2005a Guidelines for Carcinogen Risk Assessment at 70 FR 17772 Sec. 1.3.3: “In order to provide some measure of clarity and consistency in an otherwise

free-form, narrative characterization, standard descriptors are used as part of the hazard narrative to express the conclusion regarding the weight of evidence for carcinogenic hazard potential. There are five recommended standard hazard descriptors: ‘Carcinogenic to Humans,’ ‘Likely to Be Carcinogenic to Humans,’ ‘Suggestive Evidence of Carcinogenic Potential,’ ‘Inadequate Information to Assess Carcinogenic Potential,’ and ‘Not Likely to Be Carcinogenic to Humans.’ Each standard descriptor may be applicable to a wide variety of data sets and weights of evidence and is presented only in the context of a weight of evidence narrative. Furthermore, as described in Section 2.5 of these cancer guidelines, more than one conclusion may be reached for an agent.” NRC 1994 at 143: “EPA should continue Also at 70 FR 17811-12: The linear default to use potency estimates—i.e., unit cancer is thought generally to provide an upper- risk—to estimate an upper bound on bound calculation of potential risk at the probability of developing cancer due low doses, for example, a 1/1,000,000 to to lifetime exposure to one unit of a 1/100,000 risk. carcinogen. However, uncertainty about the potency estimate should be described.” And at 17802: Assessments should discuss the significant uncertainties encountered in the analysis, distinguishing, if possible, among model uncertainty, parameter uncertainty, and human variation. NRC 1994 at 13: “In the analysis of EPA 1996 Proposed Guidelines for animal bioassay data on the occurrence of Carcinogen Risk Assessment at 126: “In multiple tumor types, the cancer potencies analyzing animal bioassay data on the should be estimated for each relevant occurrence of multiple tumor types, these tumor type that is related to exposure, and guidelines outline a number of biological the individual potencies should be summed and other factors to consider. The objective for those tumors.” is to use these factors to select response 321 data (including nontumor data as continued

TABLE D-1  Continued 322 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc appropriate) that best represent the biology observed. As stated in section 3 of the guidelines, appropriate options include use of a single data set, combining data from different experiments, showing a range of results from more than one data set, showing results from analysis of more than one tumor response based on differing modes of action, representing total response in a single experiment by combining animals with tumors, or a combination of these options. The approach judged to best represent the data is presented with the rationale for the judgment, including the biological and statistical considerations involved. The EPA has considered the approach of summing tumor incidences and decided not to adopt it. While multiple tumors may be independent, in the sense of not arising from metastases of a single malignancy, it is not clear that they can be assumed to represent different effects of the agent on cancer processes. In this connection, it is not clear that summing incidences provides a better representation of the underlying mode(s) of action of the agent than combining animals with tumors or using another of the several options noted above. Summing incidences would result in a higher risk estimate, a step that appears unnecessary without more reason.” EPA 2005a Guidelines for Carcinogen Risk Assessment at 71 FR 17801: “When multiple estimates can be developed, all datasets should be considered and a judgment made about how best to represent

the human cancer risk. Some options for presenting results include: adding risk estimates derived from different tumor sites” (NRC, 1994). NRC 1994 at 142: “EPA should develop EPA 2000c Benchmark Dose Technical biologically based quantitative methods Guidance Document at 1: “The purpose for assessing the incidence and likelihood of this document is to provide guidance of noncancer effects in human populations for the Agency and the outside community resulting from chemical exposure. These on the application of the benchmark dose methods should incorporate information approach to determining the point of on mechanisms of action and differences departure (POD) for linear or nonlinear in susceptibility among populations and extrapolation of health effects data. individuals that could affect risk.” This guidance discusses computation of benchmark doses and benchmark concentrations (BMDs and BMCs) and their lower confidence limits, data requirements, dose-response analysis, and reporting requirements. The document provides guidance based on today’s knowledge and understanding, and on experience gained in using this approach. The Agency is actively applying this methodology and evaluating the outcomes for the purpose of gaining experience in using it with a variety of endpoints.” NRC 1994 at 265: [Regarding IRIS], GAO 2006 at 38: “Since 1994, EPA has “EPA should enhance and expand the changed the IRIS assessment process in references in the data files on each several ways. For example, each IRIS file chemical and include information on risk- now contains a discussion of the key studies, assessment weaknesses for each chemical as well as a description of the decisions and and the research needed to remedy such default assumptions used in the assessment. weaknesses. In addition, EPA should EPA has also expanded the review that IRIS expand its efforts to ensure that IRIS assessments undergo. For example, internal maintains a high level of data quality. peer reviewers, including EPA senior health The chemical-specific files in IRIS should scientists representing program offices and include references and brief summaries of regions, review the IRIS summary and EPA health-assessment documents accompanying detailed technical information. 323 continued

TABLE D-1  Continued 324 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc and other major risk assessments of the After this review, ORD releases the document chemicals carried out by the agency, for external peer review. EPA makes draft reviews of these risk assessments by the assessments available to the public at this EPA Science Advisory Board, and the time and, following peer review, the IRIS agency’s responses to the SAB reviews. assessment discusses the key issues reviewers Important risk assessments carried out raised and EPA’s response. In addition, EPA by other government agencies or private has added a tracking system that allows IRIS parties should also be referenced and users to readily determine where an individual summarized.” assessment is in its development.” IRIS Track is a compilation of status reports on EPA’s IRIS assessments currently in progress and can be accessed at http://cfpub.epa.gov/ iristrac/index.cfm. GAO 2006 at 38: “In September 2003, EPA completed a congressionally requested review to assess the need to update information in IRIS, based on concerns that EPA and state regulators rely on potentially outdated scientific information. Input from EPA program and regional offices, the public, and other stakeholders indicated that EPA should, among other things, increase the number of new or updated assessments completed each year to 50. To date, EPA has fallen considerably short of this goal. According to a program official, EPA completed 8 IRIS assessments in 2005, plans to complete 16 in 2006, and has approximately 75 assessments under way.” “In 2004, the IRIS program also initiated a review of available scientific literature for the 460 chemicals in the database that are not under active reassessment to determine whether a reassessment based on new literature could significantly change existing

toxicity information. For 63 percent of the chemicals reviewed, no major new health effects studies were found. Such literature reviews will be conducted annually and the findings noted in the IRIS database. Some program offices maintain databases to enhance their risk assessments.” “EPA officials said a number of factors, such as the complexity of the assessment process, resource limitations, and extensive peer review, had limited EPA’s ability to complete more assessments in 2005. EPA has increased the number of staff working on IRIS assessments from 6 to 23 and may ultimately increase the number to 29. The review also indicated that EPA needs to assign staff to develop health assessments for IRIS, and provide funding for extramural research and contracts to develop IRIS files and subject them to external peer review.” Also at 39: “The Office of Air Quality Planning and Standards (OAQPS) maintains a database of dose-response values developed by various sources, including IRIS, ATSDR, and the California Environmental Protection Agency, as an aide for its risk assessors. OAQPS staff update this database as better data become available. As part of its National Air Toxics Assessment—an ongoing comprehensive evaluation of hazardous air pollutants in the United States—EPA assessed 32 air pollutants plus particulate matter in diesel exhaust in 1996. The national assessment is designed to identify air pollutants with the greatest potential to harm human health, and the results will help set priorities for collecting additional data. 325 continued

TABLE D-1  Continued 326 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc As part of its assessment, EPA compiled a national emissions inventory of hazardous air pollutants from outdoor sources, estimated population exposures to the pollutants, and characterized the potential cancer and noncancer health risks from breathing the pollutants.” Inference NRC 1983 at 162: “Uniform inference EPA 1984 at 19: “In light of the NAS Note: The recommendation for Guidelines guidelines should be developed for the use recommendations for developing risk uniform “inference guidelines” (p. 7 ff; of federal regulatory agencies in the risk assessment guidelines and procedures, we recommendations 5-9, pp. 162-169) for assessment process.” reviewed many of the technical issues that all federal agencies never really took hold, constitute components of risk assessment. but EPA has been issuing and updating its These issues are numerous, diverse, and version of such guidelines as “risk assessment cover a broad spectrum of problems. guidelines” for the last 20 years. To deal with problems like these, the Agency plans to complete new (or revise GAO 2006 at 52: “At least two-thirds of existing) guidelines on the following topics: risk assessors responding to our survey carcinogenicity, mutagenicity, reproductive who reported using guidelines or reference effects, systemic effects, chemical mixtures, documents indicated that these documents and exposure assessment. were moderately to very helpful in preparing risk assessments. In addition, between one- NRC 1994 at 5: “In 1986, EPA issued third and two-thirds of respondents who risk-assessment guidelines that were reported using policy documents said these generally consistent with the Red Book documents were moderately to very helpful in recommendations. The guidelines deal preparing risk assessments. More specifically, with assessing risks of carcinogenicity, many risk assessors said agencywide mutagenicity, developmental toxicity, and guidelines and reference documents provide effects of chemical mixtures. They include a framework to assess risks to human default options, which are essentially health that help make risk assessments more policy judgments of how to accommodate consistent. For example, some risk assessors uncertainties. They include various noted the usefulness of agency reviewed assumptions that are needed for assessing or approved procedures to support their exposure and risk, such as scaling factors assessments. In addition, some risk assessors to be used for converting test responses in said the guidelines and reference documents rodents to estimated responses helped clarify issues, and several assessors in humans.” said they were a good source for data

GAO 2006 at 53: “The Office of needed to conduct assessments. Risk Pesticide Programs periodically issues assessors responding to our survey cited the ‘hot sheets’ that describe how to apply Guidelines for Carcinogen Risk Assessment general guidance to pesticide product risk as the document most frequently used when assessments. In addition, the Office of Air preparing human health risk assessments. and Radiation created the Air Toxics Risk More specifically, several risk assessors Assessment Reference Library that provides noted that the carcinogen guidelines provide information on how to analyze the risks a useful framework for preparing risk from hazardous air pollutants.” assessments. Many risk assessors commented that agencywide guidelines and reference documents are helpful or provide useful examples. For example, a few risk assessors stated that the Exposure Factors Handbook helps provide consistency among EPA offices that conduct exposure assessments because it defines standard values for exposure, and the rationale behind those values. Another assessor said that the Review of the Reference Dose and Reference Concentration Processes provides comprehensive guidance on setting reference values and contains a case study that serves as a model for concise and well- written hazard identification. Although risk assessors responding to our survey reported that guidance documents are generally helpful, many expressed concerns about them. For example, some risk assessors consider the documents too general or too difficult to decipher. In addition, 82 percent of the risk assessors whose offices have office specific guidance said that the guidance is very or moderately helpful with regard to preparing risk assessments. According to many risk assessors, office-specific guidance provides information in a format relevant to each office’s specific needs. Over 65 percent of risk assessors reported that EPA and program offices were moderately to very effective at disseminating guidance.” 327 continued

TABLE D-1  Continued 328 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc NRC 1983 at 163: “The inference 1986-2005: EPA issued inference guidelines should be comprehensive, guidelines for carcinogenic, reproductive, detailed, and flexible. They should make developmental, mutagenic, neurotoxic, explicit the distinctions between the science and ecologic effects; for exposure and policy aspects of risk assessment. assessment; and for chemical mixtures. Specifically, they should have the following Four guidelines first issued in 1986—on characteristics: cancer, developmental toxicity, exposure, and chemical mixtures—have been updated   They should describe all components •  and reissued. See tables of contents in the of hazard identification, dose-response guidelines listed as references in this table assessment, and risk characterization and for scope and contents of each guideline. should require assessors to show that they have considered all the necessary components in each step.   They should provide detailed guidance •  on how each component should be considered, but permit flexibility to depart from the general case if an assessor demonstrates that an exception is warranted on scientific grounds.   They should provide specific guidance •  on components of data evaluation that require the imposition of risk assessment policy decisions and should clearly distinguish those decisions from scientific decisions.   They should provide specific guidance •  on how an assessor is to present the results of the assessment and the attendant uncertainties.” NRC 1983 at 166: “The process for Note: Although the congressionally chartered developing, adopting, applying, and board recommended in the report was not revising the recommended inference established, EPA undertook some of the guidelines for risk assessment should reflect activities recommended for the board. their dual scientific and policy nature.

  An expert board should be established •  EPA 1984 Risk Assessment and Management: to develop recommended guidelines for Framework for Decision Making: “We consideration and adoption by regulatory have established a Risk Assessment Forum agencies. The board’s recommended to provide an institutional locus for the guidelines should define the scientific resolution of significant risk assessment issues capabilities and limitations in assessing as they arise, and to insure that Agency health risks, delineate subjects of consensus on such issues is incorporated into uncertainty, and define the consequences the appropriate risk assessment guidelines. of alternative policies for addressing the The Forum will also provide Agency scientists uncertainties. with a regular time and place to discuss   The expert board’s report and •  problems of risk assessments in production. recommendations should be submitted Peer advice and comment of this type will to the agencies responsible for regulating help improve the quality of risk assessments, the hazards addressed by the guidelines with associated savings in time and for their evaluation and adoption. resources.” The agencies, perhaps with central coordination, should, when possible, Risk Assessment Forum–sponsored risk- choose a preferred option from among assessment guidelines and all forum reports the options that are consistent with are peer-reviewed by independent panels current scientific understanding. The in open meetings announced in the Federal procedures for adoption should afford an Register. See, for example, 70 FR 17766, opportunity for members of the public to describing the peer-review process for the comment. cancer risk-assessment guidelines issued   The process followed by the •  in 2005: “In 1996, the Agency published government for adoption of inference proposed revisions to EPA’s 1986 cancer guidelines should ensure that the guidelines for public comment. Since the resulting guidelines are uniform 1996 proposal, the document has undergone among all responsible agencies and are extensive public comment and scientific peer consistently adhered to in assessing the review, including three reviews by EPA’s risks of individual hazards. Science Advisory Board [supplemented by the   The resulting uniform guidelines •  EPA Children’s Health Protection Advisory should govern the performance of risk Committee]. Review procedures for each assessments by all the agencies that risk assessment guideline are summarized in adopt them until they are re-examined references listed for this table.” and revised; they should not prevent members of the public from disputing GAO 2006 at 36: “In addition to enhancing their soundness or applicability in its scientific leadership, EPA has also particular cases. In short, the guidelines increased its reliance on research advisory should have the status of established groups since 1994. The Science Policy 329 continued

TABLE D-1  Continued 330 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc  agency procedures, rather than binding Council and the Risk Assessment Forum regulations. play key roles in advancing the practice of   The guidelines should be reviewed •  risk assessment at EPA. The council reviews periodically with the advice and the adequacy of existing policies, establishes recommendations of the expert board. science policy as needed, and coordinates EPA The process for revising the guidelines, efforts related to methods, modeling, risk like the process for adoption, should assessment, and environmental technology. afford an opportunity for comment by all The Science Policy Council staff facilitate ad interested individuals and organizations.” hoc work groups, encourage communication and consensus building within the agency, and participate in technical work-group activities and deliberations. “The Risk Assessment Forum is a standing committee of senior EPA scientists established to promote agencywide consensus on difficult and controversial risk assessment issues and to ensure that this consensus is incorporated into guidance. According to an agency official, the forum is designed as a venue where staff can meet and discuss common risk assessment issues across program offices. One of the forum’s main contributions to risk assessment at EPA has been the issuance of a series of risk assessment guidelines. The forum is currently working on new guidelines, such as one related to adverse effects on the immune system. When more specificity is needed on an existing guideline, the forum issues companion pieces, known as ‘purple books’ because of the color of their cover, that provide additional or updated information.” NRC 1983 at 169: “The Committee 1986-2005: EPA issued guidelines for recommends that guidelines initially be carcinogenic, reproductive, developmental, developed, adopted, and applied for mutagenic, neurotoxic, and ecologic effects assessment of cancer risks. Consideration and for exposure and chemical mixtures.

of other types of health effects should See reference list in this table. follow. It may not yet be feasible to draw up as complete a set of inferences guidelines for some other health effects. For these, defining the extent of scientific knowledge and uncertainties and suggesting methods for dealing with uncertainties would constitute a useful first step.” NRC 1983 at 170: “Agencies should 1986: Exposure Guidelines were issued. develop guidelines for exposure assessment. Revised guidelines were developed in 1992. Because of diverse problems in estimating different means of exposure (e.g., through food, drinking water and consumer products), separate guidelines may be needed for each.” NRC 1994 at 266: “EPA should recognize EPA 1986d Guidelines for Carcinogen that the conduct of risk assessment does Risk Assessment at 51 FR 33993: These not require any specific methodologic guidelines describe the general framework approach and that it is best seen not as a to be followed in developing an analysis of number or even a document, but as a way carcinogenic risk and salient principles to to organize knowledge regarding potentially be used in evaluating the quality of data hazardous activities or substances and to and in formulating judgments concerning facilitate the systematic analysis of the the nature and magnitude of the cancer risks that those activities or substances hazard. . . . It is the intent of these might pose under specified conditions. The Guidelines to permit sufficient flexibility limitations of risk assessment thus broadly to accommodate new knowledge and new conceived will be clearly seen as resulting assessment methods as they emerge. from limitations in our current state of scientific understanding. Therefore, risk- assessment guidelines should be just that— guidelines, not requirements. EPA should give specific long-term attention to ways to improve this process, including changes in guidelines.” 331 continued

TABLE D-1  Continued 332 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc Interagency and NRC 1983 at 160: “When two or more GAO 2006 at 35: “In 2004, EPA and ATSDR Outside agencies share interest in and jurisdiction entered into a formal agreement to coordinate Collaboration over a health hazard that is a candidate their efforts to develop toxicological for regulation by the in the near term, a assessments for ATSDR’s work at specific joint risk assessment should be prepared highly contaminated locations and for EPA’s under the auspices of the National Integrated Risk Information System (IRIS) Toxicology Program or another appropriate database. EPA, NIEHS, and ATSDR also organization. Joint risk assessments jointly develop and annually review a list should be prepared primarily by scientific of approximately 275 hazardous substances personnel provided by the agencies and commonly found at the nation’s highly assisted as necessary by other government contaminated sites and for which ATSDR will scientists.” prepare toxicological assessments.” GAO 2006 at 36: “Each toxicological assessment contains almost everything that is known about the chemical, including its potential to harm human health or the environment. A key difference between these toxicological assessments and the ones in EPA’s IRIS database is that ATSDR includes chronic cancer and noncancer effects, as well as acute effects, while IRIS generally includes only chronic cancer and noncancer effects.” GAO 2006 at 35: “Since 1994, EPA has strengthened and formalized collaboration with a range of other federal researchers to better leverage its limited research dollars and foster the development of data to improve human health risk assessments. Specifically, EPA has developed relationships with agencies such as the National Institute for Environmental Health Sciences (NIEHS) and the Agency for Toxic Substances and Disease Registry (ATSDR). For example, in 1998, EPA established a cooperative agreement with NIEHS to develop a body of research on the relationship between exposures and children’s

health. This collaboration jointly funded Children’s Environmental Health Research Centers at seven U.S. universities and one medical center to research children’s asthma and other respiratory diseases, as well as ways to reduce farm children’s exposure to pesticides. “In addition, EPA works closely with ATSDR to help fill research gaps and develop chemical-specific toxicological assessments used in risk assessments. “At each annual review, agency staff may add chemicals to the list and identify priority research to fill gaps in knowledge. Of these 275 chemicals, approximately 150 have been identified by EPA as high priority needs.” NRC 1994 at 138: “EPA should conduct GAO 2006 at 57: “Despite the improvements more collaborative efforts with outside to collaboration at EPA, some risk assessors parties to improve the overall risk- pointed out two barriers that limit assessment process, and each step within collaboration. Specifically, assessors noted that process.” that conflicting priorities or goals among EPA offices and poor communication between some offices hinder the effectiveness of collaboration. For example, although some chemicals are studied by more than one office within EPA, the approaches and timelines differ among offices because the laws and responsibilities for each program office can differ significantly. As a result, what may be a priority chemical in one program office may not be a priority in another, thereby hindering timely collaboration. Furthermore, a couple of risk assessors found collaboration challenging because they could not find the right person in another office to communicate with on a specific issue.” 333 continued

TABLE D-1  Continued 334 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc Also at 57: “Several risk assessors suggested ways to improve and increase communication among program offices, ORD, and non- EPA organizations. For example, some risk assessors suggested more interagency work groups or meetings as a way to address research needs and foster information exchange on the development of methods. A few risk assessors suggested that a central library of risk assessment information would facilitate collaboration and avoid duplicating work already done by others. Specifically, one risk assessor said EPA could provide centralized databases of work conducted by different agencies and organizations, such as chemical-specific toxicity data, specific exposure or other values, and points of contact at each office.” GAO 2006 at 37: “The Office of Pollution Prevention and Toxics has two programs to work with industry to develop data on contaminants that can be used to better understand risks. The first is the High Production Volume (HPV) Challenge Program. This program was officially launched in late 1998 to ensure that a baseline set of data would be made available to the public on approximately 2,800 chemicals that are manufactured or imported in amounts greater than 1 million pounds per year. Diverse stakeholders, including the American Chemistry Council, Environmental Defense, and the American Petroleum Institute participate in the program. The HPV Challenge Program provides an opportunity for all stakeholders, including the public, to

comment on the tests and data summaries from the chemical sponsors—companies and consortia that volunteered to make publicly available screening-level data that allow EPA, industry, and other stakeholders to more effectively gauge the potential hazards of HPV chemicals. All comments are publicly available on the World Wide Web. As of January 2006, EPA had commitments from industry sponsors to provide data for 2,247 of the chemicals. The second program, the Voluntary Children’s Chemical Evaluation Program, is designed to provide data that will allow the public to better understand the potential health risks to children associated with certain chemical exposures. EPA asked companies that manufacture or import 23 chemicals that have been found in human tissues in various biological monitoring programs to voluntarily sponsor the evaluation of specific chemicals in a pilot program. Thirty-five companies and 10 consortia volunteered to sponsor 20 chemicals. This program was developed only after considering comments and concerns from stakeholders. Of the 23 chemicals chosen for this pilot, data gathering has been completed for 9 and is under way for another 11. The remaining 3 chemicals in the pilot program have no sponsors.” Iterative NRC 1994 at 14: “EPA should develop the EPA 1998 Ecological Risk Guidelines at GAO 2006 at 30: “Some program offices Approach to ability to conduct iterative risk assessments 92: “If risks are not sufficiently defined have also adopted an iterative—or Risk Assessment that would allow improvements to be to support a management decision, risk tiered—approach to risk assessment. . . . made in the estimates until (1) the risk is managers may elect to proceed with If this analysis indicates that the risk may below the applicable decision-making level, another iteration of one or more phases of be relatively high, assessors pursue more (2) further improvements in the scientific the risk assessment process. Reevaluating intensive analysis to determine if the risk knowledge would not significantly change the conceptual model (and associated risk is realistic or an artifact of the lower tier’s the risk estimate, or (3) EPA, the emission hypotheses) or conducting additional conservative assumptions. Despite this move 335 continued

TABLE D-1  Continued 336 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc source, or the public determines that the studies may improve the risk estimate.” toward greater use of an iterative approach, stakes are not high enough to warrant EPA acknowledges it could be clearer about further analysis. Iterative risk assessments EPA 2005a Guidelines for Carcinogen when it is taking such an approach. For would also identify needs for further Risk Assessment at 70 FR 17808: “Risk example, EPA could be more transparent research and thus provide incentives for assessment is an iterative process that about when and why it makes a risk regulated parties to undertake research grows in depth and scope in stages management decision based on a screening without the need for costly, case-by-case from screening for priority making level assessment rather than a more detailed evaluations of each individual chemical. to preliminary estimation to fuller assessment.” Iteration can improve the scientific basis of examination in support of complex risk-assessment decisions while responding regulatory decision making. Default options GAO 2006 at 30: “When a screening to risk-management concerns about such may be used at any stage, but they are assessment identifies a potential for a matters as the level of protection and predominant at screening stages. . . . There nontrivial risk, EPA decides if pursuing resource constraints.” are close to 30 provisions within the major that risk is appropriate based on its current statutes that require decisions based on priorities and available resources. If EPA NRC 1994 at 14: “EPA should develop risk, hazard or exposure assessment. . . . decides to pursue the risk, a more detailed, and use an iterative approach to risk Given this range in the scope and depth of refined risk assessment is performed. The assessment. This will lead to an improved analysis, not all risk characterizations can degree of refinement is based on the type understanding of the relationship between or should be equal in coverage or depth.” of decision, the available resources, and the risk assessment and risk management and needs of the risk manager. After refinement an appropriate blending of the two.” EPA 2005a Guidelines for Carcinogen of the estimate, EPA reviews it to see if it Risk Assessment at 70 FR 17808: “Risk will be sufficient to answer the questions NRC 1994 at 264: “Rather than a tiered assessment is an iterative process that posed. Refinements proceed iteratively risk-assessment process, EPA should grows in depth and scope in stages until the assessment provides an adequate develop the ability to conduct iterative from screening for priority making answer for the decision maker within the risk assessments, allowing improvements to preliminary estimation to fuller resources available Both the revised cancer in the process until the risk, assessed examination in support of complex guidelines and EPA’s 1995 Policy for Risk conservatively, is below the applicable regulatory decision making. Default options Characterization support an iterative decision-making level (e.g., 1 × 10-6, may be used at any stage, but they are approach to risk assessment.” etc.); until further improvements would predominant at screening stages. . . . There not significantly change the risk estimate; are close to 30 provisions within the major or until EPA, the source, or the public statutes that require decisions based on determines that the stakes are not high risk, hazard or exposure assessment. . . . enough to warrant further analysis.” Given this range in the scope and depth of analysis, not all risk characterizations can or should be equal in coverage or depth.”

64 Fed. Reg. 38705 [1999]: “In analyzing residual risk, we’ll conduct risk assessments consistent with the Agency’s human health and ecosystem risk assessment technical guidance and policies. We’ll use a tiered approach, usually first conducting a screening level assessment for a source category, and move to a refined assessment only where the risks identified in the screening assessment appear unacceptable. Depending on the characteristics of the HAPs, these assessments will address single or multiple pathways of exposure as well as human and ecological endpoints.” Models NRC 1994 at 137: “EPA should establish GAO 2006 at 41: EPA’s Agency Task Force A National Research Council committee has and Model the predictive accuracy and uncertainty on Environmental Regulatory Modeling been convened to “assess evolving scientific Validation of the methods and models and the published a report that concluded that and technical issues related to the selection quality of data used in risk assessment a need existed for, among other things, and use of computational and statistical with the high priority given to those training and technical support and agency models in decision making processes at the which support the default options. guidance on external peer review of Environmental Protection Agency (EPA). The EPA and other organizations should environmental regulatory modeling. committee will provide advice concerning also conduct research on alternative the development of guidelines and a vision methods and models that might represent EPA 1994a at 4, Model Validation of the selection and use of models at the deviations from the default options to for Predictive Exposure Assessments: agency. . . . The objective of the committee the extent that they can provide superior “Presents in outline the methods and will be to provide a report that will serve as performance and thus more accurate risk procedural steps of model validation a fundamental guide for the selection and use assessments in a clear and convincing and defines the role of validation in the of models in the regulatory process at the manner.” overall process of developing a model. . . EPA.” The committee’s report was released in . [The document] discusses the significant June 2007. role of expert opinion and qualitative judgment in determining the validation GAO 2006 at 41: “In 1997, ORD and status of a model. Finally, [it] sets out the program offices conducted an agencywide forms of evidence that will be necessary conference, called the Models 2000 in implementing a protocol for judging Workshop, to facilitate adherence to whether a model can be said to have been existing guidance on modeling, to define validated.” and implement improvements in how the agency developed and used models, and to GAO 2006 at 42: “EPA has also recommend an implementation plan for 337 continued

TABLE D-1  Continued 338 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc incorporated efforts to improve models in improving modeling within the agency. its research strategies and implementation plans. For example, in its plan for GAO 2006 at 43: “EPA is beginning research on hazardous air pollutants, EPA to embrace such new risk assessment established a long-term goal to reduce methodologies as probabilistic risk assessment uncertainties in risk assessments through and mode of action analysis. Probabilistic methods, data, and models of acute and risk assessment characterizes the variability chronic exposures and exposures through or uncertainty in risk estimates as the multiple pathways at both the national and range or distribution of the number of regional levels.” times each possible outcome will occur. In probabilistic risk assessment, one or more variables in the risk equation, such as the exposure rate, is defined as a distribution rather than as a single number. A primary advantage of probabilistic risk assessment is that it provides a quantitative description of the degree of variability or uncertainty. . . . EPA currently uses a number of models that include probabilistic analyses and is developing a new modeling framework, known as the Multimedia Integrated Modeling System, that will further enhance the agency’s ability to probabilistically model uncertainty.” GAO 2006 at 41: “EPA followed up these activities in 2000 by creating the Committee on Regulatory Environmental Modeling (CREM) to promote consistency and consensus within the agency on modeling issues (including modeling guidance, development, and application) and to enhance internal and external communications on modeling activities. CREM supports and enhances the existing modeling activities in the program offices and provides EPA with tools to support environmental decision

making. CREM also provides the public and EPA staff with a central point of inquiry about EPA’s use of models. In 2000, CREM launched agencywide activities designed to enhance the development, use, and selection of regulatory environmental models at EPA. One such activity—a workshop to facilitate discussion of good modeling practices— resulted in the development of modeling guidance. “In 2003, CREM developed guidance and created a database—called the Models Knowledge Base—of the models most frequently used in EPA.” GAO 2006 at 42: “One of ORD’s laboratories established an exposure modeling research branch and develops population exposure models, such as the Stochastic Human Exposure and Dose Simulation model for inhalation and exposures of general and sensitive subpopulations through multiple pathways. EPA has also begun to use geographic information systems (GIS) to present risk information spatially. For example, a GIS system is being developed that maps all of the drinking water intakes in the United States and their associated watersheds, so that the agency can better assess risks to drinking water supplies stemming from activities in the related watershed. For risk assessments of hazardous air pollutants, GIS can display and analyze data during planning, scoping, and problem formulation, during the exposure assessment, and during the characterization of risks. GIS can also help communicate information to risk managers and other stakeholders.” 339 continued

TABLE D-1  Continued 340 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc NRC 1994 at 142: “EPA should continue EPA 1996 Proposed Guidelines for to use the linearized multistage model Carcinogen Risk Assessment at 125: “The as a default option but should develop EPA proposes not to use a computer criteria for determining when information model such as the linearized multistage is sufficient to use an alternative model as a default for extrapolation below extrapolation model.” the observed range. The reason is that the basis for default extrapolation is a theoretical projection of the likely shape of the curve considering mode of action. For this purpose, a computer model looks more sophisticated than a straight line extrapolation, but is not. The extrapolation will be by straight line as explained in the explanation of major defaults. This was also recommended by workshop reviewers of a previous draft of these guidelines (EPA 1994b). In addition, a margin of exposure analysis is proposed to be used in cases in which the curve is thought to be nonlinear, based on mode of action. In both cases, the observed range of data will be modeled by curve fitting in the absence of supporting data for a biologically based or case- specific model.” Peer Review NRC 1983 at 156: “An agency’s risk EPA 1992b, 1994c Peer Review Policy GAO 2006 at 26: “In addition to enhancing and Expert assessment should be reviewed by an Memorandum: “Major scientifically and its scientific leadership, EPA has also Panels independent scientific advisory panel technically based work products related increased its reliance on research advisory before any major regulatory action or to agency decisions normally should be groups since 1994.” decision not to regulate. Peer review may peer reviewed. Agency managers within be performed by science panels already headquarters, Regions, laboratories, In 2003, Paul Gilman, assistant administrator established or authorized under current and field components determine and are for research and development at EPA and EPA law or, in their absence, by panels created accountable for the decision whether to science adviser, stated that “of the more than for this purpose.” employ peer review in particular instances 800 products listed in our database as either and, if so, its character, scope, and timing” having undergone peer review in 2002 or Note: By law, EPA is required to peer- (EPA 1994c, p. 2). needing peer review in the next few years, review some categories of risk assessment.

See, for example, CAA Sec. 109, FIFRA, EPA 2000d Peer Review Handbook 2nd approximately 450 were slated for external Sec.25(d), SDWA Sec. 1412(b), and edition at viii: “The goal of the Peer peer review; 67 for internal review; 225 for others. Review Policy and this Handbook is to refereed journal review; and for the balance, enhance the quality and credibility of the review mechanism has not yet been Agency decisions by ensuring that the determined (which is typical when products scientific and technical work products are a few years from completion)” (Gilman underlying these decisions receive 2003, p. 6). Looking “more closely at the appropriate levels of peer review by work products: 859 work products were independent scientific and technical reviewed by OSP [Office of Science Policy] experts.” OSP in 2002. Of that total, 113 had peer reviews completed in the past year; 273 EPA 2002c Guidelines for Ensuring and products were designated as needing peer Maximizing the Quality, Objectivity, Utility, review sometime in the future (usually within and Integrity of Information Disseminated the next 1-3 years, depending on where the by the Environmental Protection Agency: product is in its development); 362 were Discusses procedures for “ensuring and scientific articles, or compilations of several maximizing the quality of information articles, to be submitted to refereed scientific [EPA] disseminate[s]” and “administrative journals; and 111 were products that were mechanisms for EPA pre-dissemination deemed, usually because of their repetitive review of information products.” or routine nature, not to be candidates for peer review. Dividing 111 ‘peer review not EPA 2003c A Summary of General needed’ products by the 859 sum, we see Assessment Factors for Evaluating the that nearly 90 percent of our scientific and Quality of Scientific and Technical technical work products receive internal or Information at iv: Was “intended to external peer review” (p. 7). “By consistent raise the awareness of the information- and rigorous monitoring of the use of peer generating public about EPA’s ongoing review across the Agency, led by ORD’s interest in ensuring and enhancing the annual evaluation of offices’ peer review quality of information available for plans, the value of scientific peer review in Agency use. Further, it complements the ensuring the quality of EPA’s scientific and Guidelines for Ensuring and Maximizing technical products is now widely understood the Quality, Objectivity, Utility, and and accepted across the Agency” (Gilman Integrity of Information Disseminated 2003, p. 9). by the Environmental Protection Agency (EPA Information Quality Guidelines). EPA 2000e EPA Quality Manual for This summary of Agency practice is also Environmental Programs at 2-5: EPA an additional resource for Agency staff as has taken a number of activities to help they evaluate the quality and relevance of improve and ensure the quality of data and information, regardless of source.” information, beginning in 2000, with the 341 continued

TABLE D-1  Continued 342 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc EPA Quality Manual for Environmental Programs (EPA 2000e). The manual discusses EPA’s role in managing and coordinating the data-quality system, including developing a quality-management plan, and “planning, directing, and conducting assessments of the effectiveness of the quality system being applied to environmental data operations and reporting results to senior management.” NRC 1994 at 8: “EPA should continue to Risk Assessment Forum–sponsored risk- use the Science Advisory Board and other assessment guidelines and forum reports are expert bodies. In particular, the agency peer-reviewed by independent panels in open should continue to make the greatest meetings announced in the Federal Register. possible use of peer review, workshops, See, for example 70 FR 17766, describing and other devices to ensure broad peer the peer-review process for the cancer and scientific participation to guarantee risk-assessment guidelines issued in 2005: that its risk-assessment decisions will have “In 1996, the Agency published proposed access to the best science available through revisions to EPA’s 1986 cancer guidelines for a process that allows full public discussion public comment. Since the 1996 proposal, and peer participation by the scientific the document has undergone extensive public community.” comment and scientific peer review, including three reviews by EPA’s Science Advisory Board” supplemented by the EPA Children’s Health Protection Advisory Committee. GAO 2006 at 27: “The Board of Scientific Counselors (BOSC) provides objective and independent advice, information, and recommendations about ORD’s research program to ORD’s assistant administrator. BOSC is composed of scientists and engineers from academia, industry, and environmental organizations who are recognized as experts in their fields. In 1998, BOSC completed a peer review of ORD’s laboratories and centers. BOSC completed a second review

of the laboratories and centers in 2002 and 2003 that identified key accomplishments of the laboratories and centers, as well as areas for future improvement. In addition, after EPA’s Office of the Science Advisor issued its 2004 staff paper, it asked BOSC to host a workshop for EPA staff and other interested stakeholders, such as industry, environmental groups, and researchers, to provide feedback to refine EPA’s current practices and to suggest alternative approaches for specific aspects of risk assessment.” NRC 1983 at 171: “The Committee EPA 1984 Risk Assessment and Note: Although Congress did not establish the recommends to Congress that a Board on Management: Framework for Decision recommended board, EPA undertook agency- Risk Assessment Methods be established to Making at 22: “We have established specific activities, such as the Risk Assessment perform the following functions: a Risk Assessment Forum to provide Forum and risk-assessment guidelines, an institutional locus for resolution of which are analogous to recommended board   To assess critically the evolving •  significant risk assessment issues as they functions. scientific basis of risk assessment and to arise, and to insure that Agency consensus make explicit the underlying assumptions on such issues is incorporated into the and policy ramifications of the different appropriate risk assessment guidelines. The inference options in each component of Forum will also provide Agency scientists the risk assessment process. with a regular time and place to discuss   To draft and periodically to revise •  problems of risk assessments in production. recommended inference guidelines for Peer advice and comment of this type risk assessment for adoption and use by will help improve the quality of risk federal regulatory agencies. assessments, with associated savings in time   To study agency experience with risk •  and resources.” assessment and evaluate the usefulness of the guidelines.   To identify research needs in the •  risk assessment field and in relevant underlying disciplines.” Priority-Setting NRC 1994 at 10: “EPA should compile an GAO 2006 at 39: “Some program offices and Data-Needs inventory of the chemical, toxicological, maintain databases to enhance their risk Management clinical, and epidemiological literature on assessments. For example, the Office of Air 343 continued

TABLE D-1  Continued 344 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc each of the 189 chemicals identified in the Quality Planning and Standards (OAQPS) 1990 Amendments [to the Clean Air Act].” maintains a database of dose-response values [Directed to air program] developed by various sources, including IRIS, ATSDR, and the California Environmental Protection Agency, as an aide for its risk assessors. OAQPS staff update this database as better data become available. As part of its National Air Toxics Assessment—an ongoing comprehensive evaluation of hazardous air pollutants in the United States—EPA assessed 32 air pollutants plus particulate matter in diesel exhaust in 1996. The national assessment is designed to identify air pollutants with the greatest potential to harm human health, and the results will help set priorities for collecting additional data. As part of its assessment, EPA compiled a national emissions inventory of hazardous air pollutants from outdoor sources, estimated population exposures to the pollutants, and characterized the potential cancer and noncancer health risks from breathing the pollutants.” NRC 1994 at 10: “EPA should screen GAO 2006 at 36: “In addition, EPA the 189 chemicals for priorities for the has begun to establish collaborative assessment of health risks, identify the data relationships with scientific and industry- gaps, and develop incentives to expedite related researchers. For example, EPA has generation of the needed data by other cooperative agreements with the International public agencies (such as the National Life Sciences Institute’s Risk Science Institute Toxicology Program, the Agency for Toxic (ILSI-RSI), an organization that researches Substances and Disease Registry, and critical scientific issues in risk assessment, state agencies) and by other organizations such as the development of risk assessment (industry, academia, etc.).” (Directed to air methodologies. These cooperative agreements program) were specifically designed to engage the scientific community and bring together scientists from different affiliations (including

academia, other parts of government, and the private sector including industry) to address risk assessment issues. Under one agreement, ILSI-RSI is to research risk assessment approaches for cumulative and aggregate exposures. In addition, EPA has used research provided by CIIT Centers for Health Research, a chemical research laboratory funded by EPA, industry, and other federal agencies, to provide information for its formaldehyde IRIS assessment. Furthermore, EPA and industry jointly fund the Health Effects Institute (HEI)—an organization that researches the health effects of various air pollutants, including airborne particulate matter and ozone. HEI has provided data for risk assessments and convened panels of experts to review and issue reports related to risk assessment, recently on diesel exhaust.” NRC 1994 at 158: “EPA should expand GAO 2006 at 39: “ORD also maintains its efforts to gather emission and exposure personal monitoring data on the chemicals data to personal monitoring and site- in the air, foods and beverages, water, and specific monitoring.” (Directed to air dust in an individual’s personal indoor and program) outdoor environments. For example, in its National Human Exposure Assessment Survey (NHEXAS) program, which was completed in 1998, ORD collected human exposure data from hundreds of subjects from several areas of the country. NHEXAS provided data on background levels of total exposure to environmental contaminants that can be used as a baseline in exposure and risk assessments to estimate whether specific populations are exposed to increased levels of environmental contaminants.” 345 continued

TABLE D-1  Continued 346 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc Problem NRC 1996 at 3: “Risk characterization EPA 1997a Memorandum: Cumulative Risk Formulation and is the outcome of an analytic-deliberative Assessment Guidance—Phase I Planning Ecologic Risk process. Its success depends critically on and Scoping: “Recommendations from Assessment systematic analysis that is appropriate the National Research Council’s (NRC) to the problem, responds to the needs of ‘Understanding Risk: Informing Decisions the interested and affected parties, and in a Democratic Society’ and a report from treats uncertainties of importance to the the Commission on Risk Assessment and decision problem in a comprehensible way. Risk Management suggest that a variety Success also depends on deliberations that of experts, including economists and formulate the decision problem, guide social scientists, and stakeholders must be analysis to improve decision participants’ involved throughout the environmental understanding, seek the meaning of analytic risk assessment and risk management findings and uncertainties, and improve process. This guidance also recommends the ability of interested and affected involving experts and stakeholders in the parties to participate effectively in the risk planning and scoping of risk assessments. decision process. The process must have The Agency is engaged in several activities an appropriately diverse participation or that involve working with stakeholders. representation of the spectrum of interested Experience from these activities will and affected parties, of decision makers, provide the solid basis for engaging and of specialists in risk analysis, at each interested and affected parties in risk step.” assessment and risk management issues.” EPA 1998 Guidelines for Ecological Risk Assessment at 13: “The characteristics of an ecological risk assessment are directly determined by agreements reached by risk managers and risk assessors during planning dialogues. These agreements are the products of planning. They include (1) clearly established and articulated management goals, (2) characterization of decisions to be made within the context of the management goals, and (3) agreement on the scope, complexity, and focus of the risk assessment, including the expected output and the technical and financial support available to complete it.”

EPA 1998 Guidelines for Ecological Risk Assessment at 3: EPA (1998) also states that during its SAB review “most reviewers felt there was general compatibility between the Proposed Guidelines and the NRC report, although some emphasized the need for continued interactions among risk assessors, risk managers, and interested parties (or stakeholders) throughout the ecological risk assessment process and asked that the Guidelines provide additional details concerning such interactions. To give greater emphasis to these interactions, the ecological risk assessment diagram was modified to include ‘interested parties’ in the planning box at the beginning of the process and ‘communicating with interested parties’ in the risk management box following the risk assessment. Some additional discussion concerning interactions among risk assessors, risk managers, and interested parties was added, particularly to section 2 (planning).” NRC 1996 at 6: “The analytic-deliberative EPA 1998 at 13: States that “during process leading to a risk characterization planning, risk managers and risk assessors should include early and explicit attention are responsible for coming to agreement to problem formulation; representation on the goals, scope, and timing of a risk of the spectrum of interested and affected assessment and the resources that are parties at this early stage is imperative. available and necessary to achieve the The analytic-deliberative process should goals. Together they use information on the be mutual and recursive. Analysis and area’s ecosystems, regulatory requirements, deliberation are complementary and and publicly perceived environmental must be integrated throughout the values to interpret the goals for use in the process leading to risk characterization: ecological risk assessment.” deliberation frames analysis, analysis informs deliberation, and the process EPA 2003b at 63: Includes discussion of benefits from feedback between the two.” the risk characterization recommendations 347 continued

TABLE D-1  Continued 348 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc from the National Research Council 1996 report, including a box summarizing some of the points made in the report. Also states that “risk characterization is most efficiently conducted with early and continued attention to the risk characterization step in the risk assessment process (NRC 1996; EPA 2000b).” Public Review NRC 1994 at 267: “EPA should provide Note: Consistent with requirements of the GAO 2006 at 29: “Program offices involve and Comment; a process for public review and comment federal Administrative Procedure Act, as stakeholders in various ways. For example, Public with a requirement that it respond, so well as environmental laws administered the branch of the Office of Air Quality Participation that outside parties can be assured that by EPA, public notice and an opportunity Planning and Standards (OAQPS) responsible the methods used in risk assessments are for comment are provided in relation to all for setting certain air quality standards for scientifically justifiable.” EPA actions subject to those laws. six principal pollutants solicits input from stakeholders in the planning phase of its NRC 1996 at 30: “Successful risk EPA 1998 Ecological Risk Assessment periodic updates to the standards it sets. In characterization depends on input from Guidelines at 63FR 11-12: “In some addition, the public may officially comment three kinds of actors: public officials risk assessments, interested parties on draft air quality standards once they . . . analytic experts . . . and the interested also take an active role in planning, are publicly released. The Office of Water and affected parties to the decision. The particularly goal development. . . . pursues stakeholder and public involvement interested and affected parties have a right Interested parties may communicate their that includes working with the environmental to influence which questions should be concerns to risk managers about the community, industry, trade associations, risk the subject of analysis and can contribute environment, economics, cultural changes, assessor organizations, states, and bordering both to developing information and to the or other values potentially at risk from countries. In addition, the office’s periodic deliberative pars of the process.” environmental management activities. reviews of water quality standards and . . . In some cases, interested parties may other nonregulatory actions, such as health provide important information to risk advisories, are all open processes that allow assessors. Local knowledge, particularly for public input on various stages of the in rural communities, and traditional analysis. knowledge of native peoples can provide valuable insights about ecological “For risk assessments involving the characteristics of a place, past conditions, reregistration of pesticides, the Office of and current changes.” Pesticide Programs (OPP) established a process that provides several opportunities EPA 1997a Memorandum: Cumulative Risk for public participation. Depending on the Assessment Guidance—Phase I Planning potential health risks posed by a pesticide

and Scoping: In accordance with product, the public has anywhere from one recommendations from NRC 1996, the to four separate opportunities to comment. agency is engaged in several activities For example, if risk assessors estimate that that involve working with stakeholders. the product poses little risk to human health, Experience with these activities will provide the public will have one opportunity to a solid basis for engaging interested and comment before OPP decides whether to affected parties in risk-assessment and risk- approve the pesticide product. For higher- management issues. risk products, the public will have as many as four opportunities to comment. The first EPA 1997a Guidance on Cumulative Risk opportunity to comment occurs after OPP Assessment at 1, 2: “Directs each office has completed a preliminary risk assessment. to take into account cumulative risk This preliminary assessment contains all issues in scoping and planning major risk of the elements of a risk assessment and assessments and to consider a broader has undergone internal review, but is not scope that integrates multiple sources, yet finalized. Notice of the opportunity to effects, pathways, stressors and populations comment is distributed to people who have for cumulative risk analyses in all cases for elected to sign up for such notifications, which relevant data are available. . . .Our as well as through a ‘notice of availability’ goal is to ensure that citizens and other published in the Federal Register. The stakeholders have an opportunity to help public can also comment on risk assessments define the way in which an environmental prepared by the Office of Pesticide Programs or public health problem is assessed, to through the office’s Science Advisory understand how the available data are Panel—which holds periodic public meetings used in the risk assessment, and to see on pesticide-related risk assessment issues, how the data affect decisions about risk such as methods to assess skin sensitivity management.” to exposure to pesticides or models used to estimate dietary exposures.” GAO 2006 at 38: “EPA also changed how it sets priorities for which chemicals need new or updated IRIS assessments. Annually, EPA asks its program offices, regions, and the public to identify contaminants for which it should develop or revise IRIS assessments. EPA publishes the list in the Federal Register and requests the public and scientific community to submit any relevant data on substances undergoing review. EPA is currently reviewing ways to increase 349 continued

TABLE D-1  Continued 350 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc coordination with other governmental agencies that develop chemical assessments, outreach to stakeholders earlier in the development of IRIS assessments, and consultation with independent external reviewers.” Risk NRC 1983 at 20: “Risk characterization EPA 1984 at 14, Risk Assessment and Characterization is the process of estimating the incidence Management: Framework for Decision of a health effect under the various Making: “The final assessment should conditions of human exposure described display all relevant information pertaining in exposure assessment. It is performed by to the decision at hand, including such combining the exposure and dose-response factors as the nature and weight of assessments. The summary effects of the evidence for each step of the process, the uncertainties in the preceding steps are estimated uncertainty of the component described in this step.” parts, the distribution of risk across various sectors of the population, the assumptions contained within the estimates, and so forth.” EPA 1992b Agency-wide Policy Memorandum: “Well-balanced risk characterization presents information for other risk assessors, EPA decision-makers, and the public regarding the strengths and limitations of the assessment.” (NRC 1994, Appendix B). EPA 1995a Agency-wide Policy Memorandum: “Each risk assessment prepared in support of decision-making at EPA should include a risk characterization . . . that is clear, transparent, reasonable and consistent with other risk characterizations of similar scope prepared across programs in the agency. . . . To ensure transparency, risk characterizations

should include a statement of confidence in the assessment that identifies all major uncertainties along with comment on their influence on the assessment, consistent with [EPA 1995b, Guidance in the Risk Characterization Handbook.]” (Reprinted as Appendix A in Risk Characterization Handbook.) EPA 2000b Risk Characterization Handbook at 39: “At EPA, various risk assessment guidelines have been written to ensure a scientifically defensible and consistent approach to risk assessment. When you write the risk characterization portion of your assessment, indicate whether or not you followed the guidelines and describe the key assumptions you made during your assessment and the impact they have on the assessment outcome. . . . In years past, different EPA offices sometimes had different policies about how to assess risk (e.g., different uncertainty factors or different levels of regulatory concern). While the development of the various risk assessment guidelines and the establishment of the Science Policy Council have helped to eliminate such discrepancies, possibilities for policy choices affecting risk assessment outcomes still exist in EPA (i.e., different laws and their implementing regulations may still dictate divergent policies). Also, there may be important differences between EPA’s risk assessment policy choices and those of other agencies. To the extent you are aware of such information be sure to describe it in the risk characterization portion of your assessment and to let your manager know of the impact the alternative policy choices have on the outcome of your 351 assessment.” continued

TABLE D-1  Continued 352 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc NRC 1994 at 5: “Risk characterization EPA 1986 Guidelines for Carcinogen Risk combines the assessments of exposure and Assessment 51 FR 33999: “The section response under various exposure conditions of risk characterization should summarize to estimate the probability of specific harm the hazard identification, dose- response to an exposed individual or population. To assessment, exposure assessment, and the the extent feasible, this characterization public health estimates. Major assumptions, should include the distribution of risk in scientific judgments, and, to the extent the population. When the distribution of possible, estimates of the uncertainties risk is known, it is possible to estimate the embodied in the assessment are presented.” risk to individuals who are most exposed to the substance in question.” EPA 1996 Proposed Carcinogen Assessment Guidelines at 125: “The result of using NRC 1994 at 10: “EPA should continue straight line extrapolation is thought to be to use as one of its risk characterization an upper bound on low-dose potency to metrics, upper-bound potency estimates of the human population in most cases, but the probability of developing cancer due to as discussed in the major defaults section, lifetime exposure. Whenever possible, this it may not always be. Exploration and metric should be supplemented with other discussion of uncertainty of parameters descriptions of cancer potency that might in curve-fitting a model of the observed more adequately reflect the uncertainty data or in using a biologically based or associated with the estimates.” case-specific model is called for in the dose response assessment and characterization sections of these guidelines.” EPA 2005a Guidelines for Carcinogen Risk Assessment at 70 FR 17801: “Linear extrapolation should be used in two distinct circumstances: (1) When there are data to indicate that the dose-response curve has a linear component below the POD [point of departure] and (2) as a default for a tumor site where the mode of action is not established. . . . The slope of this line, known as the slope factor, is an upper-bound estimate of risk per increment of dose that can be used to estimate risk probabilities for different exposure levels.”

NRC 1994 at 27: “Risk characterization EPA 2005a Guidelines for Carcinogen should also include a full discussion of the Risk Assessment at 70 FR 17808: “The uncertainties associated with the estimates risk characterization presents an integrated of risk.” and balanced picture of the analysis of the hazard, dose-response, and exposure. The risk analyst should provide summaries of the evidence and results and describe the quality of available data and the degree of confidence to be placed in the risk estimates. Important features include the constraints of available data and the state of knowledge, significant scientific issues, and significant science and science policy choices that were made when alternative interpretations of data exist (EPA 1995a, 2000b). Choices made about using data or default options in the assessment are explicitly discussed in the course of analysis, and if a choice is a significant issue, it is highlighted in the summary. In situations where there are alternative approaches for risk assessment that have significant biological support, the decisionmaker can be informed by the presentation of these alternatives along with their strengths and uncertainties.” Risk NRC 1994 at 15: “When EPA reports EPA 1996 Proposed Carcinogen Risk GAO 2006 at 64: “Experts also said EPA Communication estimates of risk to decision-makers and Assessment Guidelines at 126: “In part risk assessments should clearly describe the in Relation the public, it should present not only point as a response to these recommendations, sufficiency of the data and the scientific to Risk estimates of risk, but also the sources and the Administrator of EPA issued guidelines basis for its choice of a default assumption, Management magnitudes of uncertainty associated with for risk characterization and required method, or model. Some experts pointed out these estimates.” implementation plans from all programs that risk assessments should identify and in EPA (EPA 1995a). The Administrator’s clearly discuss any data that are not available NRC 1994 at 13: “Risk managers should guidance is followed in these cancer for the analysis, including the form the data be given characterizations of risk that are guidelines. The assessments of hazard, need to be in and the most appropriate study both qualitative and quantitative, i.e., both dose response, and exposure will all have design or methodology to obtain the needed descriptive and mathematical.” accompanying technical characterizations data. In addition, several experts said EPA covering issues of strengths and needs to more explicitly communicate 353 continued

TABLE D-1  Continued 354 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc limitations of data and current scientific which default assumptions were used in understanding, identification of defaults a risk assessment and why the defaults utilized in the face of gaps in the former, were chosen. For example, one expert said discussions of controversial issues, and that even though a risk assessment may be discussions of uncertainties in both their perfect, if the public does not understand qualitative, and as practicable, their the rationale behind the agency’s choices, quantitative aspects.” the risk assessment might be seen as flawed. Furthermore, in individual risk assessments, EPA 1998 Ecological Risk Guidelines at the agency could more transparently identify 109-110: “When risk characterization is which critical studies would help the agency complete, risk assessors should be able avoid relying on default assumptions. to estimate ecological risks, indicate the Some experts also suggested that EPA use overall degree of confidence in the risk as case studies completed assessments for estimates, cite lines of evidence supporting which the agency had sufficient data to use the risk estimates, and interpret the models and other analytic tools rather than adversity of ecological effects. Usually default assumptions to more accurately this information is included in a risk assess risks. Finally, some experts said that characterization report . . . . EPA should more transparently consider alternate methods and models in each risk EPA 2005a Guidelines for Carcinogen assessment. For example, EPA should be more Risk Assessment at 71FR 17807: “The risk transparent about the judgments it makes characterization includes a summary for when it employs certain methods, such as the the manager in nontechnical discussion that benchmark dose method, which identifies the minimizes the use of technical terms. It is dose that produces a small increase in the risk an appraisal of the science that informs the of an adverse effect.” risk manager. . . . It also serves the needs of other interested readers. The summary is an information resource for preparing risk communication information, but . . . is not itself the vehicle for communication with every audience.” Uncertainty NRC 1994 at 185: “EPA should make EPA 1995a Agency-Wide Memorandum GAO 2006 at 43: “EPA’s 1997 policy states and Variability uncertainties explicit and present them at 5: “Particularly critical to full that probabilistic techniques, such as Monte Analysis and as accurately and fully as is feasible and characterization of risk is a frank and open Carlo analysis, can be viable statistical tools Character- needed for risk management decision- discussion of the uncertainty in the overall to analyze variability in risk assessments, ization making. To the greatest extent feasible, assessment and in each of its components. when they are based on adequate supporting

(see also Risk EPA should present quantitative, as The uncertainty discussion is important data and credible assumptions. The guidance Character- opposed to qualitative, representations for several reasons. 1. Information from presents a general framework and broad ization, of uncertainty. However, EPA should not different sources carries different kinds set of principles to ensure the use of Defaults) necessarily quantify model uncertainty (via of uncertainty and knowledge of these good scientific practices when conducting subjective weights or any other technique), differences is important when uncertainties probabilistic analyses of variability and but should try to quantify the parameter are combined for characterizing risk. uncertainty. In addition, the guidelines and other uncertainty that exists for 2. The risk assessment process, with present a new cancer characterization system each plausible choice of scientific model. management input, involves decisions consisting of five summary descriptors, to In this way, EPA can give its default regarding the collection of additional data be used in conjunction with narrative, to models the primacy they are due under (versus living with uncertainty); in the describe the extent to which available data its guidelines, while presenting useful, but risk characterization, a discussion of the support the conclusion that a contaminant distinct alternative estimates of risk and uncertainties will help to identify where causes cancer in humans and to justify the uncertainty. In the quantitative portions additional information could contribute summary descriptor selected.” of their risk characterizations (which will significantly to reducing uncertainties in serve as one important input to standard- risk assessment. 3. A clear and explicit setting and residual-risk decisions under the statement of the strengths and limitations Act), EPA risk assessors should consider of a risk assessment requires a clear and only the uncertainty conditional on the explicit statement of related uncertainties.” choice of the preferred models for dose- response relationships, exposure, uptake, EPA 1996 Proposed Guidelines on etc.” Carcinogenic Risk Assessment at 126: “In part as a response to these NRC 1994 at 13: “Quantitative recommendations [that EPA consider uncertainty characterizations conducted the limits of scientific knowledge], the by EPA should appropriately reflect Administrator of EPA issued guidelines the difference between uncertainty and for risk characterization and required interindividual variability.” implementation plans from all programs in EPA (EPA 1995b). The Administrator’s NRC 1994 at 185: “EPA should develop guidance is followed in these cancer uncertainty analysis guidelines—both a guidelines. The assessments of hazard, general set and specific language added to dose response, and exposure will all have its existing guidelines for each step in risk accompanying technical characterizations assessment (e.g., the exposure assessment covering issues of strengths and guidance). The guidelines should consider limitations of data and current scientific in some depth all the types of uncertainty understanding, identification of defaults (model, parameter, etc.) in all the stages of utilized in the face of gaps in the former, risk assessment. The uncertainty guidelines discussions of controversial issues, and should require that the uncertainties in discussions of uncertainties in both their models, data sets, and parameters and their qualitative, and as practicable, their 355 relative contributions to total uncertainty quantitative aspects.” continued

TABLE D-1  Continued 356 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc in a risk assessment be reported in a EPA 2000b Risk Characterization written risk-assessment document.” Handbook at A-3: “Key scientific concepts, data and methods (e.g., use of animal or human data for extrapolating from high to low doses, use of pharmacokinetics data, exposure pathways, sampling methods, availability of chemical-specific information, quality of data) should be discussed. To ensure transparency, risk characterizations should include a statement of confidence in the assessment that identifies all major uncertainties along with comment on their influence on the assessment, consistent with the Guidance on Risk Characterization.” (See “Risk Characterization” section above for other relevant policy statements in EPA risk-assessment guidelines and other sources.) NRC 1994 at 12: “EPA should conduct EPA 1997c Guiding Principles for Monte formal uncertainty analyses, which can Carlo Analysis at 1: “Such probabilistic show where additional research might analysis techniques as Monte Carlo resolve major uncertainties and where it analysis, given adequate supporting data might not.” and credible assumptions, can be viable statistical tools for analyzing variability NRC 1994 at 12: “EPA should consider in and uncertainty in risk assessments and its risk assessments the limits of scientific presents an initial set of principles to guide knowledge, the remaining uncertainties, the agency in using probabilistic analysis and the desire to identify errors of either tools.” overestimation or underestimation.” NRC 1994 at 12: “Despite the advantages EPA 1996 Proposed Guidelines on EPA did not adopt this recommendation in of developing consistent risk assessments Carcinogen Risk Assessment at 125: “The the 1996 guidelines. between agencies by using common rationale for adopting the oral scaling assumptions (e.g., replacing surface area factor of body weight to the 0.75 power

with body weight to the 0.75 power), EPA has been discussed above in the explanation should indicate other methods, if any, that of major defaults. The empirical basis might be more accurate.” is further explored in Federal Register 57(109): 24152 [1992]. The more accurate approach is to use a toxicokinetic model when data become available or to modify the default when data are available as encouraged under these guidelines. As the EPA [57 Fed. Reg. 24152 [1992] discussion explores in depth, data on the differences among animals in response to toxic agents are basically consistent with using a power of 1.0, 0.75, or 0.66. The Federal agencies chose the power of 0.75 for the scientific reasons given in the previous discussion of major defaults; these were not addressed specifically in the NRC report. It was also considered appropriate, as a matter of policy, for the agencies to agree on one factor. Again, the default for inhalation exposure is a model that is constructed to become better as more agent-specific data become available.” NRC 1994 at 12: “When ranking risks, EPA 2004b at 16: “Since uncertainty and EPA should consider the uncertainties in variability are present in risk assessments, each estimate, rather than ranking solely EPA usually incorporates a ‘high-end’ on the basis of point estimate value. Risk hazard and/or exposure level in order to managers should not be given only a single ensure an adequate margin of safety for number or range of numbers. Rather, they most of the potentially exposed, susceptible should be given risk characterizations that population, or ecosystem. EPA’s high- are as robust (i.e., complete and accurate) end levels are around 90% and above—a as can be feasibly developed.” reasonable approach that is consistent with the NRC discussion (NRC 1994). This policy choice is consistent with EPA’s legislative mandates (e.g., adequate margin of safety). Even with a high-end value, there will be exposed people or environments at greater risk and at lower 357 risk. In addition to the high-end values, continued

TABLE D-1  Continued 358 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc EPA programs typically estimate central tendency values for risk managers to evaluate. This provides a reasonable sense of the range of risk that usually lies on the actual distribution.” NRC 1994 at 242: “The distinction EPA 2000b Risk Characterization between uncertainty and individual Handbook at 40: “The risk assessor should variability ought to be maintained strive to distinguish between variability rigorously at the level of separate risk- and uncertainty to the extent possible assessment components (e.g., ambient (see 3.2.8 for a discussion of uncertainty). concentration, uptake, and potency) as Variability arises from true heterogeneity well as at the level of an integrated risk in characteristics such as dose-response characterization.” differences within a population, or differences in contaminant levels in the environment. The values of some variables used in an assessment change with time and space, or across the population whose exposure is being estimated. Assessments should address the resulting variability in doses received by members of the target population. Individual exposure, dose, and risk can vary widely in a large population. Central tendency and high end individual risk descriptors capture the variability in exposure, lifestyles, and other factors that lead to a distribution of risk across a population (e.g., see Guidelines for Exposure Assessment).” Variability and NRC 1994 at 11: “Federal agencies EPA 1997d Exposure Factors Handbook: GAO 2006 at 47: “Another way EPA Differential should sponsor molecular, epidemiological, Risk assessors have used the Exposure addresses variability is through research. One Susceptibility and other types of research to examine Factors Handbook to account for variation of ORD’s four strategic research directions the causes and extent of interindividual in exposure. The purposes of the handbook in its Human Health Research Strategy variability in susceptibility to cancer and are to “(1) summarize data on human is designed to improve the understanding the possible correlations between behaviors and characteristics which affect of why some people and groups are more exposure to environmental contaminants, susceptible and highly exposed than others.

susceptibility and such covariates as age, and (2) recommend values to use for these According to this strategy, ORD’s research race, ethnicity, and sex.” factors” (p. 1-1). The document includes on subpopulations will focus on three over 150 data tables with information factors—life stage, genetic factors, and pre- on exposure scenarios. It also discusses existing diseases—that have been identified by variability and attempts to characterize the a program office and the scientific community variability of each of the exposure factors as having a high priority for risk assessment. “(1) as tables with various percentiles In 2000, ORD released its Strategy for or ranges of values; (2) as analytical Research on Environmental Risks to Children distributions with specified parameters; to strengthen the scientific foundation of and/or (3) as a qualitative discussion” risk assessment and management decisions (p. 1-5). The handbook discusses how that affect children and guide EPA’s research risk assessors can identify the types of needs and priorities over the following 5 to variability and ways that variability can be 10 years. Approximately 75 percent of the analyzed. funding for this strategy will be dedicated to research grants under the STAR program, such as those designed to evaluate children’s exposure to pesticides.” GAO 2006 at 46: “To further its understanding of variability in exposure, EPA has undertaken a number of research projects. For example, one of ORD’s laboratories conducted the National Human Activity Pattern Survey to provide detailed human exposure information for specific populations and allow EPA to better understand actual human exposure to pollutants in real-world situations. The survey results are stored in the Consolidated Human Activity Database to help risk assessors estimate the time that exposed people spend in various environments and their inhalation, ingestion, and dermal absorption rates while in those environments. This laboratory also conducts research to define, quantify, and reduce the uncertainty associated with the exposure and risk assessments, to develop improved methods to more accurately measure exposure and dose, and to develop technical information 359 continued

TABLE D-1  Continued 360 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc and quantitative tools to predict the nature and magnitude of human exposures to environmental contaminants. A recent EPA study was designed to identify chemicals commonly used in homes or day care centers, and whether children in these environments encountered the chemicals in the course of their daily activities. The research sought to identify the major routes (i.e., breathing and ingestion) and sources (i.e., dust, food, air, soil, and water) through which children come into contact with chemicals. “Variability also exists with regard to susceptibility to adverse affects because of inherent differences among humans.” NRC 1994 at 11: “EPA should adopt EPA 1996 Proposed Guidelines for a default assumption for differences in Carcinogen Risk Assessment at 125: susceptibility among humans in estimating “The issue of a default assumption for individual risks.” human differences in susceptibility has been addressed under the major defaults discussion in section 1.3 with respect to margin of exposure analysis. The EPA has considered but decided not to adopt a quantitative default factor for human differences in susceptibility when a linear extrapolation is used. In general, the EPA believes that the linear extrapolation is sufficiently conservative to protect public health. Linear approaches (both LMS and straight line extrapolation) from animal data are consistent with linear extrapolation on the same agents from human data (Goodman and Wilson 1991; Hoel and Portier 1994). If actual data on human variability in sensitivity are

available they will, of course, be used.” EPA 2005a Guidelines for Cancer Risk Assessment at 17802: “The dose-response estimate strives to derive separate estimates for susceptible populations and lifestages so that these risks can be explicitly characterized. For a susceptible population, higher risks can be expected from exposures anytime during life, but this applies to only a portion of the general population. . . . In contrast, for a susceptible lifestage, higher risks can be expected from exposures during only a portion of the lifetime, but everyone in the population may pass through those lifestages.” Also at 17811: “As a default for oral exposure, a human equivalent dose for adults is estimated from data on another species by an adjustment of animal applied oral dose by a scaling factor based on body weight to the ¾ power. The same factor is used for children because it is slightly more protective than using children’s body weight (see sec. 3.1.3).” NRC 1994 at 11: “The distinction between EPA 2000b Risk Characterization GAO 2006 at 45: “All program offices uncertainty and individual variability Handbook at 40: “The risk assessor should address exposure variability in their risk should be maintained rigorously in each strive to distinguish between variability and assessments, although they do so in different component of risk assessment.” uncertainty to the extent possible.” ways. For example, risk assessors in the Office of Air Quality Planning and Standards EPA 2000b Risk Characterization who set certain air quality standards for Handbook at 40: “The risk assessor should six principal pollutants said they consider strive to distinguish between variability and individual activity patterns for sensitive uncertainty to the extent possible (see 3.2.8 populations like children or asthmatics in for a discussion of uncertainty). Variability exposure modeling by including a distribution arises from true heterogeneity in of breathing rates to reflect variability 361 continued

TABLE D-1  Continued 362 Topic NRC Report: Recommendationa EPA Response: Stated Policyb EPA Response: Implementation Activityc characteristics such as dose-response inherent in the population. Furthermore, by differences within a population, or modeling to protect the most sensitive or differences in contaminant levels in the at-risk groups, they are assured of protecting environment. The values of some variables the rest of the population. Variability in used in an assessment change with time exposure to the six principal pollutants is and space, or across the population whose generally described qualitatively in scientific exposure is being estimated. Assessments summaries for each pollutant. The Office of should address the resulting variability in Water includes an analysis of risks to various doses received by members of the target subpopulations and a narrative discussion of population. Individual exposure, dose, and the strengths and weaknesses of the studies risk can vary widely in a large population. it used to estimate exposure, but generally Central tendency and high end individual does not include a quantitative analysis. The risk descriptors capture the variability Office of Pesticide Programs considers 24 in exposure, lifestyles, and other factors different population subgroups in its exposure that lead to a distribution of risk across a estimates, including differences in age, gender, population (e.g., see Guidelines for Exposure ethnicity, and geographic dispersion. When Assessment).” data allow, the Office of Pesticide Programs develops a distribution of exposures and risks EPA 2003b Framework for Cumulative for its more refined risk assessments.” Risk Assessment at 65: “NRC (1994) notes a clear difference between uncertainty and variability and recommends that the distinction between these two be maintained: A distinction between uncertainty (i.e., degree of potential error) and interindividual variability (i.e., population heterogeneity) is generally required if the resulting quantitative risk characterization is to be optimally useful for regulatory purposes, particularly insofar as risk characterizations are treated quantitatively. The distinction between uncertainty and individual variability ought to be maintained rigorously at the level of separate risk assessment components (e.g., ambient concentration, uptake, and potency) as well as at the level of an integrated risk characterization.”

NRC 1994 at 220: “If there is reason EPA 2005b Supplemental Guidance for GAO 2006 at 46: “Legislation can also to believe that risk of adverse biological Assessing Susceptibility from Early-Life require EPA to consider potentially effects per unit dose depends on age, EPA Exposure to Carcinogens at 1: “The susceptible populations and life stages. should present separate risk estimates for National Research Council (NRC, 1994) For example, the Safe Drinking Water Act adults and children. When excess lifetime recommended that ‘EPA should assess risks Amendments mandate that EPA consider risk is the desired measure, EPA should to infants and children whenever it appears risks to groups within the general population compute an integrated lifetime risk, taking that their risks might be greater than those that are at greater risk of adverse health into account all relevant age-dependent of adults.’ This document focuses on cancer effects, including children, the elderly, and variables. risks from early-life exposure compared people with serious illnesses. In addition, with those from exposures occurring later the Food Quality Protection Act contains “EPA does not usually explore or consider in life. Evaluating childhood cancer and special provisions for the consideration of interindividual variability in key biologic childhood exposures resulting in cancer risks to children from pesticides. In 1995, parameters when it uses or evaluates later in life are related, but separable, EPA’s Science Policy Council called for EPA various physiologic or biologically based issues.” to consider the risks to infants and children risk-assessment models (or else evaluates consistently and explicitly as part of its risk some data but does not report on this EPA 2004b at 42: “Consideration of the assessments. In 1997, the White House issued in its final public documents). In some variability among humans is a critical an executive order that required EPA and other cases, EPA does gather or review aspect of risk assessment. It is the goal other federal agencies to identify and assess data that bear on human variability, but of EPA risk assessments to identify all environmental health and safety risks that tends to accept them at face value without potentially affected populations, including may disproportionately affect children and ensuring that they are representative of the human populations (e.g., gender, nutritional to ensure that policies, programs, activities, entire population. As a general rule, the status, genetic predisposition) and life- and standards address such disproportionate larger the number of characteristics with stages (e.g., childhood, pregnancy, old risks.” an important effect on risk or the more age) that may be more susceptible to toxic variable those characteristics are, the larger effects or are highly or disproportionately the sample of the human population needed exposed.” to establish confidently the mean and range of each of those characteristics.” Also at 43: “When data are available to describe toxicological differences for a susceptible population or life-stage, then those data are summarized and analyzed, and the decisions based on this information are presented. It is preferable to have population- and chemical-specific data to describe a susceptibility to toxic effects.” aExample of recommendation from NRC 1983, 1994, or 1996. bExample of EPA policy bearing on issues raised in the recommendation in the form of written guidelines, reports, or policy memoranda. cCommentary, practice, or activities related to issues raised in the National Research Council recommendation and related EPA guidance. 363 dThese guidelines were not specifically in response to the National Research Council report but reflect agency policy related to this topic.

364 SCIENCE AND DECISIONS: ADVANCING RISK ASSESSMENT References EPA (U.S. Environmental Protection Agency). 1984. Risk Assessment and Management: Framework for Decision Making. EPA 600/9-85-002. Office of the Administrator, U.S. Environmental Protection Agency, Washington, DC. EPA (U.S. Environmental Protection Agency). 1986. Guidelines for Carcinogen Risk Assessment. EPA/630/R-00/004. Risk Assessment Forum, U.S. Environmental Protection Agency, Washington, DC [online]. Available: http:// cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=54933 [accessed June 3, 2007]. EPA (U.S. Environmental Protection Agency). 1989a. Exposure Factors Handbook. EPA/600/8-89/043. NTIS PB90-106774/AS. Office of Health and Environmental Assessment, Office of Research and Development, U.S. Environmental Protection Agency, Washington, DC. EPA (U.S. Environmental Protection Agency). 1989b. Risk Assessment Guidance for Superfund: Volume I—Human Health Evaluation Manual (Part A). Interim Final. EPA-540/1-89/002. Office of Emergency and Remedial Response, U.S. Environmental Protection Agency, Washington, DC [online]. Available: http://www.epa.gov/ oswer/riskassessment/ragsa/pdf/rags-vol1-pta_complete.pdf [accessed Oct. 16, 2007]. EPA (U.S. Environmental Protection Agency). 1992a. Guidelines for Exposure Assessment. EPA/600/Z-92/001. Risk Assessment Forum, U.S. Environmental Protection Agency, Washington, DC. May 1992 [online]. Available: http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=15263 [accessed Oct. 10, 2007]. EPA (U.S. Environmental Protection Agency). 1992b. Guidance on Risk Characterization for Risk Managers and Risk Assessors. Memorandum to Assistant Administrators, and Regional Administrators, from F. Henry Habicht, Deputy Administrator, Office of the Administrator, Washington, DC. February 26, 1992 [online]. Available: http://www.epa.gov/oswer/riskassessment/pdf/habicht.pdf [accessed Oct. 10, 2007]. EPA (U.S. Environmental Protection Agency). 1994a. Model Validation for Predictive Exposure Assessments. U.S. Environmental Protection Agency, Washington, DC. July 4, 1994 [online]. Available: http://www.epa.gov/ord/ crem/library/whitepaper_1994.pdf [accessed Oct.15, 2007]. EPA (U.S. Environmental Protection Agency). 1994b. Report on the Workshop on Cancer Risk Assessment Guide- lines Issues. EPA/630/R-94/005a. Office of Research and Development, Risk Assessment Forum, Washington, DC. EPA (U.S. Environmental Protection Agency). 1994c. Peer Review and Peer Involvement at the U.S. Environmental Protection Agency. Memorandum to Assistant Administrators, General Counsel, Inspector General, Associate Administrators, Regional Administrators, and Staff Office Directors, from Carol M. Browner, Administrator, U.S. Environmental Protection Agency. June 7, 1994 [online]. Available: http://www.epa.gov/osa/spc/pdfs/ perevmem.pdf [accessed Oct. 16, 2007]. EPA (U.S. Environmental Protection Agency). 1995a. Policy for Risk Characterization at the U.S. Environmental Protection Agency. Memorandum from Carol M. Browner, Office of the Administrator, U.S. Environmental Protection Agency, Washington, DC. March 21, 1995 [online]. Available: http://64.2.134.196/committees/ aqph/rcpolicy.pdf [accessed Oct. 10, 2007]. EPA (U.S. Environmental Protection Agency). 1995b. Guidance for Risk Characterization. Science Policy Council, U.S. Environmental Protection Agency. February 1995 [online]. Available: http://www.epa.gov/osa/spc/pdfs/ rcguide.pdf [accessed Oct. 15, 2007]. EPA (U.S. Environmental Protection Agency). 1996. Proposed Guidelines for Carcinogen Risk Assessment. EPA/600/P-92/003C. Office of Research and Development, U.S. Environmental Protection Agency, Washing- ton, DC. April 1996 [online]. Available: http://www.epa.gov/ncea/raf/pdfs/propcra_1996.pdf [accessed Oct. 15, 2007]. EPA (U.S. Environmental Protection Agency). 1997a. Cumulative Risk Assessment Guidance—Phase I Planning and Scoping. Memorandum to Assistant Administrators, General Counsel, Inspector General, Associate Administrators, Regional Administrators, and Staff Office Directors, from Carol M. Browner, Administrator, and Fred Hansen, Deputy Administrator, Office of Administrator, U.S. Environmental Protection Agency, Washington, DC. July 3, 1997 [online]. Available: http://www.epa.gov/swerosps/bf/html-doc/cumulrsk.htm [accessed Oct. 15, 2007]. EPA (U.S. Environmental Protection Agency). 1997b. Guidance on Cumulative Risk Assessment. Part 1. Planning and Scoping. Science Policy Council, U.S. Environmental Protection Agency, Washington, DC. July 3, 1997 [online]. Available: http://www.epa.gov/osa/spc/pdfs/cumrisk2.pdf [accessed Oct. 10, 2007]. EPA (U.S. Environmental Protection Agency). 1997c. Guiding Principles for Monte Carlo Analysis. EPA/630/R- 97/001. Risk Assessment Forum, U.S. Environmental Protection Agency, Washington, DC [online]. Available: http://www.epa.gov/NCEA/pdfs/montcarl.pdf [accessed June 3, 2007].

APPENDIX D 365 EPA (U.S. Environmental Protection Agency). 1997d. Exposure Factors Handbook, Vol. 1. General Factors. EPA/600/P-95/002F. Office of Research and Development, National Center for Environmental Assessment, U.S. Environmental Protection Agency, Washington, DC [online]. Available: http://www.epa.gov/ncea/efh/ [accessed June 3, 2007]. EPA (U.S. Environmental Protection Agency). 1998. Guidelines for Ecological Risk Assessment. EPA/630/R-95/002F. Risk Assessment Forum, U.S. Environmental Protection Agency, Washington, DC [online]. Available: http:// cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=12460 [accessed June 3, 2007]. EPA (U.S. Environmental Protection Agency). 2000a. Supplementary Guidance for Conducting Health Risk As- sessment of Chemical Mixtures. EPA/630/R-00/002. Risk Assessment Forum, U.S. Environmental Protection Agency, Washington, DC. August 2000 [online]. Available: http://www.epa.gov/ncea/raf/pdfs/chem_mix/ chem_mix_08_2001.pdf [accessed Oct. 15, 2007]. EPA (U.S. Environmental Protection Agency). 2000b. Risk Characterization: Science Policy Council Handbook. EPA 100-B-00-002. Office of Science Policy, Office of Research and Development, U.S. Environmental Protec- tion Agency, Washington, DC [online]. Available: http://www.epa.gov/OSA/spc/pdfs/rchandbk.pdf. [accessed June 3, 2007]. EPA (U.S. Environmental Protection Agency). 2000c. Benchmark Dose Technical Guidance Document. EPA/630/ R-00/001. Risk Assessment Forum, U.S. Environmental Protection Agency, Washington, DC [online]. Avail- able: http://www.epa.gov/ncea/pdfs/bmds/BMD-External_10_13_2000.pdf [accessed June 3, 2007]. EPA (U.S. Environmental Protection Agency). 2000d. Per Review Handbook., 2nd Ed. EPA 100-B-00-001. Science Policy Council, Office of Science Policy, Office of Research and Development, U.S. Environmental Protection Agency, Washington, DC. December 2000 [online]. Available: http://www.epa.gov/osa/spc/pdfs/prhandbk.pdf [accessed Oct. 16, 2007]. EPA (U.S. Environmental Protection Agency). 2000e. EPA Quality Manual for Environmental Programs. 5360A1. Office of Environmental Information, U.S. Environmental Protection Agency, Washington, DC. May 5, 2000 [online]. Available: http://www.epa.gov/quality/qs-docs/5360.pdf [accessed Oct. 15, 2007]. EPA (U.S. Environmental Protection Agency). 2001. General Principles for Performing Aggregate Exposure and Risk Assessments. Office of Pesticide Programs, U.S. Environmental Protection Agency, Washington, DC. November 28, 2001 [online]. Available: http://www.epa.gov/pesticides/trac/science/aggregate.pdf [accessed Oct. 15, 2001]. EPA (U.S. Environmental Protection Agency). 2002a. Guidance on Cumulative Risk Assessment of Pesticide Chemicals That Have a Common Mechanism of Toxicity. Office of Pesticide Programs, U.S. Environmental Protection Agency, Washington, DC. January 14, 2002 [online]. Available: http://www.epa.gov/pesticides/trac/ science/cumulative_guidance.pdf [accessed Oct. 16, 2007]. EPA (U.S. Environmental Protection Agency). 2002b. Lessons Learned on Planning and Scoping for Environmental Risk Assessments. Prepared by the Planning and Scoping Workgroup of the Science Policy Council Steering Committee, U.S. Environmental Protection Agency, Washington, DC. January 2002 [online]. Available: http:// www.epa.gov/OSA/spc/pdfs/handbook.pdf [accessed Oct. 16, 2007]. EPA (U.S. Environmental Protection Agency). 2002c. Guidelines for Ensuring and Maximizing the Quality, Objec- tivity, Utility and Integrity of Information Disseminated by the Environmental Protection Agency. EPA/260R- 02-008. Office of Environmental Information, U.S. Environmental Protection Agency, Washington, DC [on- line]. Available: http://www.epa.gov/QUALITY/informationguidelines/documents/EPA_InfoQualityGuidelines. pdf [accessed Oct. 10, 2007]. EPA (U.S. Environmental Protection Agency). 2003a. Human Health Research Strategy. EPA/600/R-02/050. Office of Research and Development, U.S. Environmental Protection Agency, Washington, DC [online]. Available: http://www.epa.gov/nheerl/humanhealth/HHRS_final_web.pdf [accessed June 3, 2007]. EPA (U.S. Environmental Protection Agency). 2003b. Framework for Cumulative Risk Assessment. EPA/630/ P-02/001F. Risk Assessment Forum, U.S. Environmental Protection Agency, Washington, DC [online]. Avail- able: http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=54944 [accessed June 3, 2007]. EPA (U.S. Environmental Protection Agency). 2003c. A Summary of General Assessment Factors for Evaluating the Quality of Scientific and Technical Information. EPA 100/B-03/001. Science Policy Council, U.S. Environ- mental Protection Agency, Washington, DC. June 2003 [online]. Available: http://www.epa.gov/osa/spc/pdfs/ assess2.pdf [accessed Oct. 12, 2007]. EPA (U.S. Environmental Protection Agency). 2004a. Air Toxics Risk Assessment Reference Library, Vol. 1- Technical Resource Manual, Part III: Human Health Risk Assessment: Multipathway. EPA-453-K-04-001. Office of Air Quality Planning and Standards, U.S. Environmental Protection Agency, Research Triangle Park, NC [online]. Available: http://www.epa.gov/ttn/fera/risk_atra_vol1.html#part_iii [accessed Oct. 12, 2007]. EPA (U.S. Environmental Protection Agency). 2004b. An Examination of EPA Risk Assessment Principles and Prac- tices. EPA/100/B-04/001. Office of the Science Advisor, U.S. Environmental Protection Agency, Washington, DC [online]. Available: http://www.epa.gov/OSA/pdfs/ratf-final.pdf [accessed June 3, 2007].

366 SCIENCE AND DECISIONS: ADVANCING RISK ASSESSMENT EPA (U.S. Environmental Protection Agency). 2005a. Guidelines for Carcinogen Risk Assessment (Final). EPA/630/ P-03/001F. Risk Assessment Forum, U.S. Environmental Protection Agency, Washington, DC. March 2005 [online]. Available: http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=116283 [accessed Oct. 16, 2007]. EPA (U.S. Environmental Protection Agency). 2005b. Supplemental Guidance for Assessing Susceptibility from Early-Life Exposure to Carcinogens. EPA/630/R-03/003F. Risk Assessment Forum, U.S. Environmental Pro- tection Agency, Washington, DC [online]. Available: http://www.epa.gov/iris/children032505.pdf [accessed Oct. 19, 2007]. GAO (U.S. Government Accountability Office). 2006. Human Health Risk Assessment. GAO-06-595. Washington, DC: U.S. Government Printing Office [online]. Available: http://www.gao.gov/new.items/d06595.pdf [accessed Oct. 11, 2007]. Gilman, P. 2003. Statement of Paul Gilman, Assistant Administrator for Research and Development and EPA Science Advisor, U.S. Environmental Protection Agency, before the Committee on Transportation and Infra- structure, Subcommittee on Water Resources and the Environment, U.S. House of Representatives, March 5, 2003 [online]. Available: http://www.epa.gov/ocir/hearings/testimony/108_2003_2004/2003_0305_pg.pdf [accessed Feb. 9, 2007]. Goodman, G., and R. Wilson. 1991. Predicting the carcinogenicity of chemicals in humans from rodent bioassay data. Environ. Health Perspect. 94:195-218. Hoel, D.G., and C.J. Portier. 1994. Nonlinearity of dose-response functions for carcinogenicity. Environ. Health Perspect. 102(Suppl 1):109-113. NRC (National Research Council). 1983. Risk Assessment in the Federal Government: Managing the Process. Washington, DC: National Academy Press. NRC (National Research Council). 1994. Science and Judgment in Risk Assessment. Washington, DC: National Academy Press. NRC (National Research Council). 1996. Understanding Risk: Informing Decisions in a Democratic Society. Wash- ington, DC: National Academy Press.

Next: Appendix E: Environmental Protection Agency Program and Region Responses to Questions from the Committee »
Science and Decisions: Advancing Risk Assessment Get This Book
×
Buy Paperback | $65.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Risk assessment has become a dominant public policy tool for making choices, based on limited resources, to protect public health and the environment. It has been instrumental to the mission of the U.S. Environmental Protection Agency (EPA) as well as other federal agencies in evaluating public health concerns, informing regulatory and technological decisions, prioritizing research needs and funding, and in developing approaches for cost-benefit analysis.

However, risk assessment is at a crossroads. Despite advances in the field, risk assessment faces a number of significant challenges including lengthy delays in making complex decisions; lack of data leading to significant uncertainty in risk assessments; and many chemicals in the marketplace that have not been evaluated and emerging agents requiring assessment.

Science and Decisions makes practical scientific and technical recommendations to address these challenges. This book is a complement to the widely used 1983 National Academies book, Risk Assessment in the Federal Government (also known as the Red Book). The earlier book established a framework for the concepts and conduct of risk assessment that has been adopted by numerous expert committees, regulatory agencies, and public health institutions. The new book embeds these concepts within a broader framework for risk-based decision-making. Together, these are essential references for those working in the regulatory and public health fields.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!