Science and Decisions: Advancing Risk Assessment (2009)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

8 Improving the Utility of Risk Assessment The committee’s primary charge was to propose ways to improve risk assessment in the Environmental Protection Agency (EPA). As described in Chapter 1, we decided to focus on two broad criteria for improvement. The first criterion for improvement involves the technical content of risk assessment, which has been addressed in Chapters 4-7. The second concerned opportunities for making risk assessments more useful for informing risk-manage- ment decisions. Risk assessment in EPA is not an end in itself but a means to develop policies that make the best use of resources to protect the health of the public and of ecosystems. In Chapter 3, the committee demonstrated the importance of increased attention to risk-as- sessment planning and to ensuring that the levels and complexity of risk assessment (their “design”) are consistent with the goals of decision-making. Increased attention to planning and scoping and to problem formulation, referred to in EPA guidance for ecologic risk as- sessment (EPA 1998) and cumulative risk assessment (EPA 2003), was shown in Chapters 3 and 7 to provide opportunities for increasing the relevance, and hence the utility, of the products of risk assessment. Environmental problems arise in many forms, and new ones are always emerging. Some are large in scope, involving multiple sources of potential harm and many pathways from their sources to the creation of exposures of large human and ecologic populations. At the other extreme, a problem may involve a single source of harm and a single pathway of exposure, perhaps of relatively small populations (of production workers, for example). In some cases, a problem concerns the entire life cycle of a product or line of products; in oth- ers, it may concern approvability of a new pesticide by EPA or of a new food ingredient by the Food and Drug Administration, both driven by highly specific legislative requirements. Concerns raised by a community regarding emissions from nearby sources are increasingly common, as are concerns about the safety of various products moving in international commerce. All those problems have in common their origins in the environment and their potential to threaten human health or ecosystems; many involve not only chemicals but biologic, radiologic, and physical agents, and their potential interactions. The scope of environmental problems is increasingly enlarged to include the search for methods of re- 240 

IMPROVING THE UTILITY OF RISK ASSESSMENT 241 source use and product manufacture that are likely to be more sustainable—a criterion that includes health and environmental factors but others as well. Moreover, decisions in EPA often require consideration of difficult questions of costs, benefits, and risk-risk tradeoffs. Much of the discussion of Chapter 7, for example, revealed the difficulties encountered in current approaches as attempts are made to apply them to complex problems of cumulative and communitywide risks. As the complexities of the problems and of needed decisions faced by EPA increase, so do the challenges to risk assessment to provide evaluations of clear relevance to the ques- tions posed. That means, of course, that the questions posed to risk assessors must be both relevant to the problems and decisions faced and sufficiently comprehensive to ensure that the best available options for managing risks are given due consideration. This chapter provides guidance on the development and application of questions, methods, and decision processes to enhance the utility of risk assessment; although many elements of the guidance are applicable in the near term, our emphasis is on the longer-term future. Beyond the Red Book The model described in Risk Assessment in the Federal Government: Managing the Process (NRC 1983), referred to as the Red Book, was discussed in Chapters 1 and 2; in this model, risk assessment occupies a place between research and risk management. Risk assessment is seen as a framework within which complex and often inconsistent, and always incomplete, research information is interpreted and put into usable form for risk managers. The Red Book committee was concerned principally with defining risk assessment and iden- tifying the steps necessary to complete an assessment. It was also concerned with ensuring that risk characterization (the fourth and final step) is faithful to the underlying science and its uncertainties. Finally, and perhaps most important, the committee was concerned with protecting risk assessments from the inappropriate intrusions of policy-makers and other stakeholders, and from that concern came recommendations for the conceptual separation of assessment and management and for the development of risk-assessment guidelines and the elucidation and selection of “inference options” (defaults; see Chapters 2 and 6). Those and other recommendations of the Red Book have served for 25 years as sources of clar- ity and guidance for regulatory and public-health officials throughout the world and for stakeholders of many types. The present committee supports retention and advancement of the major recommenda- tions of the Red Book as they pertain to definitions, the content of risk assessment, the need for guidelines and defaults, and the conceptual separation of assessment from management. Many of our recommendations advance those aspects of the recommendations in the Red Book (and the National Research Council’s 1994 report Science and Judgment in Risk Assessment). To the extent that risk assessment is perceived as becoming less relevant to many im- portant decisions or as contributing to protracted scientific debate and regulatory gridlock, that perception may result from interpretations of the Red Book that take the conceptual distinctions and separations as representing the committee’s guide to a preferred decision- making process. In fact, the Red Book’s concern with “process” focused heavily on protecting the integrity of risk assessment, and the committee offered little discussion of how all the necessary elements of decision-making should be arranged to achieve good decisions. That   The term framework as used here refers to the entire decision process, of which risk assessment is one element. Risk assessment has its own framework, as described in Chapters 1 and 2 and the Red Book. 

242 SCIENCE AND DECISIONS: ADVANCING RISK ASSESSMENT committee did not discuss the process whereby risk assessment might achieve maximum relevance, how it might be tailored in scientific depth to match the decision-making context, or how various stakeholders might influence the question of specifically what risk assessment should focus on in specific decision contexts. Those were not central issues for the Red Book committee. They clearly are issues for today in the evolution of risk assessment. A Decision-Making Framework That Maximizes the Utility of Risk Assessment To ensure that risk assessments are maximally useful for risk-management decisions, the questions that risk assessments need to address must be raised before risk assessment is conducted and may need to be different from the questions that risk assessors have tra- ditionally been tasked with answering. The more complex and multifaceted the problem to be dealt with, the more important the need to operate in that fashion. As noted in the previous section, the Red Book framework was not oriented to identifying the optimal process for complex decision-making but rather to ensuring the conceptual separation of risk assessment and risk management. A framework for risk-based decision-making (Figure 8-1, “the framework”) is proposed here to provide the guidance that was missing from the Red Book. Its principal purpose, in the context of the present report, is to ensure that risk assessment is maximally useful for decision-making; as noted, this would fulfill the second of our two criteria for improving risk assessment. The framework is also intended to ensure that the methodologic changes recommended in Chapters 4-7 are put to the best use, given the repeated emphasis on analytic efforts that are appropriate to decision-making in scope and content. We offer some background on the framework in this section and then describe it more fully in the next section. Perhaps the easiest way to explain the basic difference between the framework and the traditional assessment-management relationship is to look first at the beginnings and ends of each process. We start with an assumption that in either model no analysis would be done and no decision would be needed unless some “signal” of potential harm had come to EPA’s attention. The signal can arrive in many forms, but it would generally involve a set of environmental conditions that appear to pose a threat to human or environmental health. The traditional process receives that signal and begins immediately with the question, What are the probability and consequence of one or more adverse health (or ecologic) effects posed by the signal? The framework (in Figure 8-1), in contrast, receives the signal and asks, What options are there to reduce the hazards or exposures that have been identified, and how can risk assessments be used to evaluate the merits of the various options? Beginning the inquiry with the latter type of question immediately focuses attention on the options for dealing with a potential problem—the risk-management options. The options are often thought of as possible interventions—actions designed both to provide adequate public-health and environmental protection and to satisfy the criterion of well-supported decision-making. We note that, in most cases, “no intervention required” is one of the op- tions to be considered explicitly. In the framework, the questions to be posed for risk assessment arise from early consider- ation of the types of assessments needed to judge the relative merits of the options considered. By examining both the options and the types of assessments available, one may expand the scope of the options considered to embrace other possible interventions. Risk management involves choosing among the options after the appropriate assessments have been undertaken and evaluated. Assessments of relevant risk-management factors other than risk—such as costs, technical feasibility, and other possible benefits—also require early planning. 

PHASE I: PHASE II: PHASE III: PROBLEM FORMULATION PLANNING AND CONDUCT RISK MANAGEMENT AND SCOPING OF RISK ASSESSMENT Stage 1: Planning • For the given decision context, what are the attributes of assessments necessary to characterize risks of existing conditions and the effects on risk of proposed options? What level of uncertainty and variability analysis is appropriate? • What problems are • What are the relative health or associated with existing environmental benefits of the environmental conditions? proposed options? Stage 2: Risk Assessment • If existing conditions appear • How are other decision- to pose a threat to human or • Hazard Identification making factors (technologies, environmental health, what costs) affected by the proposed options exist for altering those What adverse health or environmental effects options? conditions? are associated with the agents of concern? • What is the decision, and its • Under the given decision • Dose-Response Assessment • Risk Characterization justification, in light of benefits, context, what risk and other costs, and uncertainties in each For each determining adverse effect, what is the What is the nature and technical assessments are option? relationship between dose and the probability of magnitude of risk associated necessary to evaluate the with existing conditions? the occurrence of the adverse effect in the range • How should the decision be possible risk-management of doses identified in the exposure assessment? communicated? options? What risk decreases (benefits) are associated with each of the • Is it necessary to evaluate the options? effectiveness of the decision? • Exposure Assessment Are any risks increased? What • If so, how should this be done? are the significant uncertainties? What exposures/doses are incurred by each population of interest under existing conditions? How does each option affect existing conditions and resulting exposures/doses? Stage 3: Confirmation of Utility NO • Does the assessment have the attributes called for in planning? YES • Does the assessment provide sufficient information to discriminate among risk- management options? • Has the assessment been satisfactorily peer reviewed? FORMAL PROVISIONS FOR INTERNAL AND EXTERNAL STAKEHOLDER INVOLVEMENT AT ALL STAGES • The involvement of decision-makers, technical specialists, and other stakeholders in all phases of the processes leading to decisions should in no way compromise the technical assessment of risk, which is carried out under its own standards and guidelines. FIGURE 8-1  A framework for risk-based decision-making that maximizes the utility of risk assessment. 243 Figure 8-1.eps landscape 

244 SCIENCE AND DECISIONS: ADVANCING RISK ASSESSMENT Risk assessment, in the framework of Figure 8-1, would typically be asked to examine risks associated with the “no intervention” option in addition to examining risk reductions (and possible increases) associated with each of the proposed interventions. Questions arising from consideration of options need to be well formulated (including a sufficient precision and breadth of issues) to ensure that important risk issues are not inadvertently overlooked; this requires that the array of options not be unnecessarily restricted. As emphasized in Chapter 3 and elsewhere, without early and careful consideration of the decision-context, risk assessors cannot identify the types of assessments and the required level of their scientific depth necessary to support decisions (or, indeed, whether risk assess- ment is even the appropriate decision support tool, as shown in Figure 3-1). Without such a well-defined context, assessments will often lack well-defined stopping points and may yield ancillary analyses (for example, highly detailed quantitative uncertainty analyses) that are not essential for the decision at hand, prolonging the decision process unnecessarily (Chapter 4). By focusing on early and careful problem formulation and on the options for managing the problem, implementation of the framework can do much to improve the utility of risk assessment. Indeed, without such a framework, risk assessments may be addressing the wrong questions and yielding results that fail to address the needs of risk managers. The framework is based on a re-examination of one of the misinterpretations of the Red Book—that assessors should be shielded from the specific decision-making issues that their analyses are intended to support. Instead, it asserts that risk assessment is of little usefulness, and can even waste resources, if it is not oriented to help discriminate among risk-manage- ment options that have to be informed by risk (and often nonrisk) considerations. More important, the framework should ensure that decisions themselves will be improved if risk- assessment information is presented to demonstrate how it affects the worth of competing choices, not for how it sheds light on an isolated substance or “problem.” To be clear, the framework maintains the conceptual distinction between risk assessment and risk manage- ment articulated in the Red Book, and it remains intent on not allowing the manipulation of risk-assessment calculations to support predetermined policy choices. The conduct of risk assessments used to evaluate the risk-management options are in no way to be influenced by the preferences of risk managers. The proposed decision-making framework resembles the well-known decision-analytic process that has been used in diverse fields for many decades (Raiffa 1968; Weinstein et al. 1980; Lave and Omenn 1986; Lave et al. 1988; Clemen 1991), in which the utility of various concrete policy options is evaluated according to the benefits that each provides. Similarly, the need to ensure that the full range of policy options is considered for the analysis has been emphasized by others including Finkel (2003); Hattis and Goble (2003); and Ashford and Caldart (2008). The committee also recognizes that numerous previous reports and guidance documents, and EPA practice in some settings, have anticipated this framework to some extent. For example, Science and Judgment in Risk Assessment (NRC 1994) empha- sized that “risk assessment is a tool, not an end in itself,” and recommended that resources be focused on obtaining information that “helps risk managers to choose the best possible course of action among the available options.” The 1996 National Research Council report Understanding Risk: Informing Decisions in a Democratic Society (NRC 1996) emphasized that “risk characterization should be a decision driven activity, to inform choices in solv- ing problems.” The latter report also called for attention to problem formulation, with an explicit options-selection step, and representation of interested and affected parties from the earliest stages of the process. The framework also builds on but goes beyond the recommen- dations of the 1997 Presidential/Congressional Commission on Risk Assessment and Risk Management report (PCCRARM 1997) that called for a six-stage risk-management frame- 

IMPROVING THE UTILITY OF RISK ASSESSMENT 245 work: formulate the problem in broad context, analyze the risks, define the options, make sound decisions, take actions to implement the decisions, and perform an evaluation of the effectiveness of the actions taken. Yet another National Research Council report, Estimating the Public Health Benefits of Proposed Air Pollution Regulations (NRC 2002), focused on evaluating the benefits of air pollution regulations and emphasized that EPA should evalu- ate multiple regulatory options in any benefit-cost analysis to make best use of the insights available through quantitative risk assessment. However, none of those recommendations to think more systematically about risk-management options moves consideration of op- tions to the beginning of the assessment process in EPA, which is the key procedural change that we recommend. As articulated in more detail below, the present committee views the framework as a step beyond previous proposals and current practice—one that can possibly meet multiple objectives: • Systematically identify problems and options that risk assessors should evaluate at the earliest stages of decision-making. • Expand the range of effects assessed beyond individual end points (for example, cancer, respiratory problems, and individual species) to include broader questions of health status and ecosystem protection. • Create opportunities to integrate regulatory policy with other decision-making op- tions and strategies that expand environmental protection (for example, economic incentives, public-private partnerships, energy and other resource efficiencies, material substitution, public awareness, and product-stewardship programs). • Serve the needs of a greatly expanded number of decision-makers (for example, government agencies, private companies, consumers, and various stakeholder organiza- tions) whose individual and institutional roles in environmental decision-making continue to expand. • Increase understanding of the strengths and limitations of risk assessment by deci- sion-makers at all levels. We expand on some of those objectives in later sections. First, we present the framework and discuss its key elements. The Framework: An Overview Three broad phases of the framework are evident in Figure 8-1: enhanced problem for- mulation and scoping, planning and conduct of risk assessment, and risk management. Risk assessment and other technical and cost assessments necessary to evaluate risk-management options are carried out in the assessment phase of the process, although Figure 8-1 focuses on the planning and conduct of risk assessment given the charge of this committee. It is critical that those assessments be undertaken with assurance of their scientific integrity; technical guidelines are necessary to achieve this end, as are procedures to ensure they are followed. At the same time, it is important to recognize that risk assessments and other technical as- sessments are not undertaken simply because research data are available and assessments are possible; they are undertaken, in the proposed framework, only when the reasons for them are understood and the necessary level of their technical detail has been clarified. The utility of assessments will be enhanced if they are undertaken within the framework. The framework will have particular importance, given the potential complexity of our pro- posed unified approach for dose-response assessment (Chapter 5) or methods for cumulative risk assessment of chemical and nonchemical stressors (Chapter 7), in that it emphasizes that 

246 SCIENCE AND DECISIONS: ADVANCING RISK ASSESSMENT these methodologic advances should not occur in a vacuum and are most valuable if they are clearly linked to and can inform risk-management decisions. We emphasize that our promotion of the framework is focused on improving the util- ity of risk assessment to support better decision-making. As noted earlier, the framework is intended to provide guidance that was not provided by the Red Book. Elements of the Framework: A Process Map In this section, we outline the content of each of the elements of the framework. Each element involves a set of discrete activities, which are briefly suggested in Figure 8-1 and more fully described in Boxes 8-1 through 8-5. Some of the institutional issues associated with implementation of the framework are described in Chapter 9. BOX 8-1 Key Definitions Used in the Framework for Risk-Based Decision-Making PROBLEM: Any environmental condition (a method of product manufacture, residence near a manu- facturing facility, exposure to a consumer product, occupational exposure to a pesticide, exposure of fish to manufacturing effluents, a transboundary or global environmental challenge, and so on) that is suspected to pose a threat to human or ecosystem health. It is assumed that early screening-level risk assessments may sometimes be used to identify problems or to eliminate concerns. RISK-MANAGEMENT OPTION: Any intervention (a change of manufacturing process, imposition of an environmental standard, the development of warnings, use of economic incentives, voluntary initia- tives, and so on) that may alter the environmental condition, reduce the suspected threat, and perhaps provide ancillary benefits. Any given problem may have several possible risk-management options. In most cases, “no intervention” will be one of the options. LIFE-CYCLE ANALYSIS: A formal process for evaluating and managing problems associated with each stage of a product’s manufacture, distribution, uses, and disposal. It includes problems as defined above and can include evaluations of such issues as resource use and sustainability. POPULATION: Any group of general or occupational populations or populations of nonhuman organisms. AGENT: Any chemical (including pharmaceuticals and nutrients), biologic, radiologic, or other physical entity. MEDIA: Air, water, food, soils, or substances having direct contact with the body. RISK SCENARIO: A combination of agents, media, and populations in which risks to human or eco- system health can arise. BENEFITS: The changes (positive or negative) in health and environmental attributes that are associ- ated with an intervention. Typically, a risk assessment will estimate the number of cases of disease, in- jury, or death associated with a problem—which is equivalent to the benefits of eliminating the problem. Any intervention that reduces risk without eliminating it will have benefits estimated by the difference between the status quo and the risks remaining after the intervention. STAKEHOLDER: Any individual or organization that may be affected by the identified problem (de- fined above). Stakeholders may include community groups, environmental organizations, academics, industry, consumers, and government agencies. 

IMPROVING THE UTILITY OF RISK ASSESSMENT 247 BOX 8-2 Phase I of the Framework for Risk-Based Decision-Making (Problem Formulation and Scoping) Identification of Risk-Management Options and Required Assessments a. What is the problem to be investigated, and what is its source? b. What are the possible opportunities for managing risks associated with the problem? Has a full array of possible options been considered, including legislative requirements? c. What types of risk assessments and other technical and cost assessments are necessary to evaluate existing conditions, and how do the various risk-management options alter the conditions? d. What impacts other than health and ecosystem threats will be considered? e. How can the assessments be used to support decisions? f. What is the required timeframe for completion of assessments? g. What resources are needed to undertake the assessments? Scope of the Framework and Definitions of Key Terms The framework is intended for broad applicability, as can be discerned from the defini- tions (see Box 8-1) of terms used to describe activities in the elements of the framework. Phase I. PROBLEM FORMULATION and SCOPING Two types of activities are associated with Phase I of the risk-based decision-making framework (Figure 8-1): problem formulation and the simultaneous (and recursive) iden- tification of risk-management options and identification of the types of technical analyses, including risk assessments, that will be necessary to evaluate and discriminate among the options. The expected contents of Phase I are outlined in Box 8-2, as a series of questions to be pursued. Agency decisions related to premarket product approvals (for example, for new pesti- cides) depend on long-established requirements for toxicology and exposure data, and there are also well-established guidelines for risk assessments and criteria for premarket approv- ability. Those well-established requirements can be said to constitute Phase I planning for this type of decision-making, and the committee sees no need to alter the existing arrange- ment; but we do note that the proposed framework of Figure 8-1 accommodates this specific category of regulatory decision-making. Phase II. PLANNING AND CONDUCT OF ASSESSMENT Risk assessments designed to evaluate the risk-management options set out in Phase I are undertaken during Phase II. Phase II consists of three stages; planning, assessment, and confirmation of the utility of the assessment (see Box 8-3). The first stage of Phase II involves the development of a careful set of plans for the necessary risk assessments. Risk assessments should not be conducted unless it is clear that they are designed to answer specific questions, and that the level of technical detail and   he committee acknowledges that there may be cases following completion of appropriate problem formula- T tion and scoping in which it is determined that risk assessment is not needed. 

248 SCIENCE AND DECISIONS: ADVANCING RISK ASSESSMENT BOX 8-3 Phase II of the Framework for Risk-Based Decision-Making (Planning and Conduct of Risk Assessment) Stage 1: Planning for Risk Assessment a. What are the goals of the required risk assessments? b. What specific risk scenarios (agents, media, and populations, including possible consideration of background exposures and cumulative risks) are to be investigated? c. What scenarios are associated with existing conditions and with conditions after application of each of the possible risk-management options, and how should they be evaluated? d. What is the required level of risk quantification and uncertainty/variability analysis? e. Will life-cycle impacts be considered? f.  there critical data gaps that prevent completion of the required assessment? If so, what should Are be done? g. How are the risk assessments informed by the other technical analyses of options (technical feasibility, costs, and so on)? How will communication with other analysts be ensured? h. What processes should be in place to ensure that the risk assessments are carried out ef- ficiently and with assurance of their relevance to the decision-making strategy, including time requirements? i. What procedures are in place to ensure that risk assessments are conducted in accordance with applicable guidelines? j. What are the necessary levels and timing of peer review? Stage 2: Risk Assessment a. Hazard Identification: •  What adverse health or environmental effects are associated with each of the agents of potential interest? • What is the weight of scientific evidence supporting the classification of each effect? • What adverse effects are the likely risk determinants? b. Exposure Assessment: •  For the agents under study, what exposures and resulting doses are incurred by each relevant population under existing conditions? •  What do the technical analyses (Box 8-4) reveal about how existing conditions and resulting exposures/doses would be altered by each proposed risk-management option? c. Dose-Response Assessment: •  each determining adverse effect, what is the relationship between dose and the prob- For ability of the occurrence of the adverse effect in the dose region identified in the exposure assessment? d. Risk Characterization: •  For each population, what is the nature and magnitude of risk associated with existing conditions? • How are risks altered by each risk-management option (both decreases and increases)? •  What is the distribution of individual risks in the population and subpopulations of concern, and what is the distribution of benefits under each option? •  Considering the weight-of-evidence classification of hazards, the dose-response assess- ment, and the exposure assessment, what degree of scientific confidence is associated with the risk characterization? • What are the important uncertainties, and how are they likely to affect the risk results? Stage 3: Confirmation of the Utility of the Risk Assessment • Does the assessment have the attributes called for in planning? • Does the assessment provide sufficient information to discriminate among risk-manage- ment options? • Has the assessment been satisfactorily peer-reviewed, and have all peer-reviewer com- ments been explicitly addressed? 

IMPROVING THE UTILITY OF RISK ASSESSMENT 249 BOX 8-4 Other Technical Analyses Necessary for the Framework for Risk-Based Decision-Making • How does each of the proposed risk-management options alter existing conditions, and with what degree of certainty? • Are there important impacts other than those directly affecting existing conditions (as revealed, for example, by life-cycle analysis)? • What costs are associated with no intervention to alter existing conditions and with each of the proposed risk-management options? • What are the uncertainties in the cost assessments and the variabilities in the distribution of costs? • Do the assessments conform to the requirements set forth in the planning phase? uncertainty and variability analysis is appropriate to the decision context. Such attention to planning should ensure the most efficient use of resources and the relevance of the risk assessment to decision-makers. The typical questions addressed during the risk assessment planning process are set out in Box 8-3, Stage 1 (Planning). Other technical analyses are typically required to evaluate how specific interventions will alter existing conditions; the information developed through such technical analyses (see Box 8-4) must be communicated to risk assessors, so that the effects of these interventions on risk can be evaluated. Once the planning has been completed, risk assessments are conducted (Phase II, Stage 2). Risk assessments are conducted under agency guidelines. The guidelines should include defaults and explicit criteria for departures from defaults with other elements recommended in the present report, including those related to uncertainty assessment, unification of cancer and noncancer dose-response methods, and cumulative or community-based risk assessment (Chapters 4-7). Once risk assessments have been completed, the framework calls for an evaluation of the utility of what has been produced (Stage 3 of Phase II). Thus, an evaluation of whether the assessments have the attributes called for in planning, and of whether they allow dis- crimination among the risk-management options, is necessary to determine whether they are useful for decision-making. If the assessments are not determined to be adequate given the problem formulation and risk-management options, the framework calls for a return to the planning stage. If they are adequate, Phase III of the framework is entered. Phase III. RISK MANAGEMENT In Phase III of the framework, the relative health or environmental benefits of the pro- posed risk-management options are evaluated, as are other factors relevant to decisions. Legislative requirements are also critical to the decision process. The purpose of Phase III is to reach decisions, fully informed by the risk assessments. A justification for the decision, with full elucidation of the roles played by the risk informa- tion, and other pertinent factors, should be offered. A discussion of how uncertainties in all of the information used to develop decisions influenced those decisions is essential. Some of the questions that are central to risk management are set out in Box 8-5. 

250 SCIENCE AND DECISIONS: ADVANCING RISK ASSESSMENT BOX 8-5 Elements of Phase III of the Framework for Risk-Based Decision-Making (Risk Management) Analysis of Risk-Management Options •  hat are the relevant health or environmental benefits of the proposed risk-management op- W tions? How are other decision-making factors (technologies, costs) affected by the proposed options? • s it indicated, with a sufficient degree of certainty given the preference of risk managers, that I any of the options are preferred to a “no intervention” strategy? •  hat criteria are used to assess the relative merits of the proposed options (for example, does W the risk manager consider population benefits, reductions below a predefined de miminis level, or equity considerations)? Risk-Management Decisions • What is the preferred risk-management decision? • Is the proposed decision scientifically, economically, and legally justified? • How will it be implemented? • How will it be communicated? • Is it necessary to evaluate the effectiveness of the decision? How should this be done? Stakeholder Involvement A critical feature of the framework is related to stakeholder involvement. A continuing theme in earlier National Research Council and other expert reports on risk assessment, and loudly echoed in opinions offered to the present committee by many commenters, concerns the consistent failure to involve stakeholders adequately throughout the decision process. Without such involvement, the committee sees no way to ensure that the decision process will be satisfactory; indeed, without such involvement, it is inevitably deficient. Figure 8-1 emphasizes that point through the box on the bottom, which spans all three phases. In addition, the two-headed arrows are meant to represent the fact that adequate communication among analysts and stakeholders, which is necessarily two-way, is critical to ensure efficiency and relevance of the analyses undertaken to support decisions. Adequate stakeholder involvement and communication among those involved in the policy and tech- nical evaluations are difficult to achieve, but they are necessary for success. It is time that formal processes be established to ensure implementation of effective stakeholder participa- tion in all stages of risk assessment. For any given problem that requires EPA action, there are certain to be a number of af- fected parties seeking to influence the agency’s course. Some stakeholders may wish to ensure that particular problems come to the attention of the agency and that their formulations be adequate. Others will hope that the agency consider various possible management options, sometimes including options that have not traditionally been part of regulatory thinking. Still others will have proposals that they believe will improve the scientific strength of agency risk assessments. And, of course, many parties will seek to influence ultimate decisions. For cases in which agency actions will lead to regulations, formal procedural require- ments are in place to allow members of the public to offer comments on proposed regula- tions. That type of stakeholder involvement in agency activities is obviously important, but 

IMPROVING THE UTILITY OF RISK ASSESSMENT 251 it is insufficient in that it applies only to formal rule-making and typically comes only at the end of the process of decision-making. The present committee, like several that have come before it (Chapter 2), recommends that EPA make formal a process for gathering stake- holder views in each of the three broad phases of decision-making depicted in Figure 8-1; conflicts of interest will need to be considered in this process. It is critical that time limits for stakeholder involvement be well defined so that decision-making schedules can be met. In addition, effective stakeholder participation must consider incentives to allow for balanced participation including impacted communities and less advantaged stakeholders. Additional Improvements Offered by the Framework Operating under the framework can lead to improvements in the technical aspects of analysis (including economics and other nonrisk components) and can help to improve the basic research supporting risk assessment by allowing formal or informal value-of-informa- tion considerations (Chapter 3). But the major advances that the framework can bring about involve improving the quality of risk-based decision-making by raising the expectations for what risk assessments can provide. The framework could address the frustration among some that the current system channels substantial energy toward dissecting and comparing problems rather than advancing decisions that deal with problems. Other important advan- tages of the framework include the following: 1. It augments and complements related trends in risk-assessment practice. As de- scribed in Chapters 3 and 7, there is a need to design risk assessments to better inform the technical aspects of risk assessment and the ultimate decision context. EPA’s Framework for Cumulative Risk Assessment (EPA 2003) and Guidelines for Ecological Risk Assessment (EPA 1998) endorse this approach and emphasize that it would be impossible to determine the appropriate scope or level of resolution of an assessment in the absence of the risk-man- agement context. The framework takes the planning stage one step further by embedding the development of risk-management options as a formal step before the planning of the assessment, thereby encouraging the development of risk assessments that adequately capture important tradeoffs and cross-media exposures. In addition, the methodologic developments proposed in Chapter 5 and elsewhere are meant in part to provide greater insight for risk managers regarding the health-risk implications of specific management decisions, feeding directly into the proposed framework. A related trend involves the growth of life-cycle as- sessment, which includes many aspects of risk assessment but also evaluates a broader array of issues related to energy use, water consumption, and other characteristics of technolo- gies, industrial processes, and products that determine their propensity to consume natural resources or to generate pollution. The term life cycle refers to the need to include all stages of a business process—raw-material extraction, manufacturing, distribution, use, and dis- posal, including all intervening transportation steps—to provide a balanced and objective assessment of alternatives. A critical component in the planning of a life-cycle assessment is the “functional-unit determination,” in which various alternatives are compared on the basis of their ability to achieve a desired end point (for example, generation of a kilowatt-hour of electricity). The approach emphasizes the need to understand the objectives of the process or product under study, broaden the scope, and bring novel approaches and risk-management options to the forefront, including considering pollution prevention efforts. The framework builds on those important trends and emphasizes that risk assessments should be designed to provide risk managers with the necessary information to discriminate among risk-man- 

252 SCIENCE AND DECISIONS: ADVANCING RISK ASSESSMENT agement options and that life-cycle and functional-unit thinking (if not analysis itself) will facilitate the development of a wide array of options. 2. It makes it easier to discern “locally optimal” decisions. The framework helps to identify locally optimal decisions (for example, choices among strategies to reduce risks posed by a given compound) by making it more difficult to make the fundamental mathematical error of averaging the predictions of incompatible models together. If, for example, there is a default estimate (including parameter uncertainty, perhaps, but small with respect to the model uncertainty) that predicts a risk X for a particular substance and a credible alterna- tive model (with expert weight 1 – p assigned to it) that posits that the risk is zero, there is a temptation to declare that the “best estimate” of the risk is pX. In the traditional para- digm, if the risk assessment reports that the “best estimate” is pX, a decision-maker might be inclined to regulate as though the baseline risk is exactly pX. Following the framework in Figure 8-1 would bring the options to the fore and emphasize to all stakeholders that key uncertainties might imply that different options would be chosen, depending on key risk-assessment assumptions. In this setting, the risk characterization would more likely take the form of the statement “there is a probability p that the risk is X, in which case option B is preferred, and a probability 1 – p that the risk is zero, in which case option C is preferred.” Thus, operating with the framework can sometimes help to avoid confusing “expected-value decision-making” (a coherent although ethically controversial approach) with “decision-making by expected value” (an incorrect and precarious approach—see Box 4-5). Careful consideration of uncertainty is not precluded by the conventional framework, but the framework in Figure 8-1 helps to determine the degree to which key uncertainties influence decisions among risk-management options and orients the risk assessor and other stakeholders around such questions about uncertainty. 3. It makes it easier to identify and move toward “globally optimal” decisions. More broadly, the framework opens the prospect of moving beyond a choice among strategies to deal with a single substance to the development, evaluation, and selection of alternative strategies to fulfill the function with minimum net risk. As implied by the functional-unit definition above, this involves expanding the lens of current environmental decision-making from primarily a single-issue and incremental-risk focus to address issues of comparative and cumulative risk, benefits and costs, life-cycle risks, technologic innovation and public values. We believe that questions about the risks posed by industrial processes can often be answered better by considering risk-risk tradeoffs and evaluating risk-management options than by studying risks in isolation from the feasible means of control. Although the expanded scope may exceed the bounds of EPA decision-making (either in a practical sense, given current regulations, or in a theoretical sense, given the agency’s jurisdiction), functional-unit think- ing will help to avoid considering only local optima that represent the peaks within a valley, will encourage the development of agencywide initiatives and strategies, and will encourage EPA to cooperate with other federal agencies (and vice versa) to work on more sweeping interventions that increase efficiency and minimize untoward risk-risk tradeoffs. In short, the framework would allow EPA to compare options with appropriate use of knowledge about uncertainty and would allow it to broaden (within reason) the set of options under consideration. 4. It can provide the opportunity for improved public participation. The framework can broaden the focus of inquiry from studying the risk—which may be dominated by highly technical discussions of potency, fate and transport, mode of action, and so on—to develop- ing and evaluating alternative interventions, which should be a more accessible and interest- ing arena for affected stakeholders to participate in. Stakeholders (such as local communities) may also bring particular knowledge about the benefits, costs, and implementation of risk- 

IMPROVING THE UTILITY OF RISK ASSESSMENT 253 management options to a discussion. The process would recognize the roles, relationships, and capabilities of government and nongovernment decision-makers and would ensure that risk assessments serve their needs. The committee recognizes that effective implementation of the framework in many cases will not be possible without the involvement of other gov- ernmental agencies and other organizations. 5. It would make economics and risk-risk tradeoffs more central in the analysis. Although many regulatory, legislative, and logistical constraints complicate the simultane- ous consideration of costs of control and benefits, the framework would, where applicable and feasible, encourage the use of similar methods between disciplines (such as the explicit incorporation of uncertainty and variability and the development of default assumptions and criteria for departure in economic analyses) and would spur collaboration between risk assessors and regulatory economists. As articulated above, the framework would also make consideration of potential risk-risk tradeoffs central in the assessment, inasmuch as the initial planning and scoping steps and the development of risk-management options under study would lead to an explicit discussion of the array of exposures that could be influenced by each option. In Appendix F, the committee presents three case examples to demonstrate how the usefulness of risk assessments might be enhanced by implementation of the framework for risk-based decision-making. POTENTIAL CONCERNS RAISED BY the Framework The framework has many desirable attributes that can allow risk assessment to be maxi- mally informative for decision-making, but various concerns could be raised about it. Some of the concerns are misconceptions, and others are legitimate issues that would need to be addressed. We discuss various critiques and consider their potential implications below. Concern 1: There are many contexts in which EPA is constrained to a narrow set of options by the structure of regulations or in which it is unclear at the outset whether a problem is of sufficient magnitude to require an intervention or whether a potential intervention exists, so the framework may waste effort in producing needless evaluations. This concern has some legitimacy, but the framework does not preclude risk assessment solely to determine the potential magnitude of a problem or to compare the impacts of op- tions within a severely constrained solution set. As to the former, the framework is intended to keep one eye continually on problems and one on interventions, and choosing between one and the other is a false dichotomy. The committee believes that the current use of risk assessment has disproportionately emphasized dissecting risks rather than implementing possible interventions, but the pendulum does not need to (and should not) swing past a middle ground. As to the latter, in situations where the regulatory requirements preclude consideration of a wide array of risk-management options, EPA could both formally evalu- ate the options that can be considered and use the framework to determine the extent to which current constraints preclude a better risk-management strategy. At a minimum, the   The three case examples in Appendix F address electricity generation, decision support for drinking-water systems, and control of methylene chloride exposure in the workplace and general environment. These are stylized examples intended to illustrate how application of the framework for risk-based decision-making might lead to a process and outcome different from those of conventional application of risk assessment. 

254 SCIENCE AND DECISIONS: ADVANCING RISK ASSESSMENT framework would emphasize the need for EPA to consider risk tradeoffs and alternative strategies explicitly when devising risk-management options. Concern 2: The framework may exacerbate the problem of “paralysis by analysis,” both because the analytic burdens will increase with the need to evaluate numerous options and because risk assessments may show that uncertainties are too great to permit discrimination among various options. The committee proposed earlier that the framework will help risk assessments to come to closure by focusing on the information needed to discriminate among risk-management op- tions rather than focusing on the information needed to “get the number right.” However, it could be argued that the need to quantify benefits among multiple potential risk-management options, including tradeoffs and multimedia considerations, will greatly expand the ana- lytic requirements of a given assessment, especially given that the uncertainties in a simpler assessment may prove too large for discrimination among options. That is an important concern, but many of the more analytically complex components (for example, cumulative risk assessment and multimedia exposure) would be needed for any risk assessment with a similar scope, regardless of what risk-management options are under consideration, and the marginal time to evaluate multiple risk-management options should be relatively small once a model has been constructed to evaluate the benefit of one option appropriately. In addition, if the uncertainties are too large for discrimination among options on a risk basis, it would imply simply that other considerations are central in the risk-management decisions or that further research is required. Concern 3: The framework will not lead to better decisions and public-health protection, because the process does not provide for equal footing for competing interest groups. Although the committee proposes that the framework will enhance public participa- tion and will reduce asymmetries among stakeholders by focusing on early development of risk-management options, there would continue to be asymmetries in the ability of different stakeholders to get options “on the table,” given issues of political power and imbalance in available information. More generally, the framework could potentially be manipulated if the set of options evaluated were constrained inappropriately. In addition, the importance of risk assessment is not reduced in the framework, so the technical imbalance would remain. The concern is relevant, but it is not introduced by the framework, but rather is endemic to processes that bring together government, communities, and industries to debate decisions that will have serious economic and public-health effects. The framework could improve on the current practice provided there is substantive stakeholder involvement throughout the process, if stakeholder groups have sufficient technical expertise (which can be developed over time through efforts by EPA and others), and if EPA formally addresses all suggested options in writing (either by evaluating them quantitatively or by discussing qualitatively how they are strictly dominated by other options and therefore do not need to be consid- ered). The potential for manipulation is not created by the framework and in fact would be reduced by it: risk managers can now implicitly reduce the option set by asking risk assessors to evaluate the benefits of a preselected control scenario, and a public process to explicitly construct a wide-ranging set of options seems preferable. As a component of the development and implementation of the framework, EPA should propose guidelines for the options-development step of Phase I, focusing explicitly on stakeholder participation and formal processes for transparent selection of risk-management options to study. 

IMPROVING THE UTILITY OF RISK ASSESSMENT 255 Concern 4: The framework breaks down the firewall between risk assessment and risk man- agement, creating a potential for manipulation. That the framework allows assessors to see the choices facing the decision-maker does not imply that they would be involved in risk management, nor does it imply that deci- sion-makers would have license or opportunity to impose their will on the analysis. The framework empowers risk assessment to drive the engine that determines which options perform best in the presence of uncertainty, variability, and public preferences, but it does not empower risk assessors to impose their preferences on the analysis. It will remain important in the framework to have clear risk-assessment guidelines (see, for example, Chapters 3, 5, and 7) that can be used to conduct the assessments needed to evaluate options. Increasing the interaction between risk assessors and risk managers requires that there be further protection against the possibility that identified or preferred policy options will bias the evaluation of risks or, even more problematically, that risk managers will influence the content of the risk assessment to support preferred risk-management options. Ensuring the integrity of evaluations along the continuum of the risk-assessment–risk-management discussion fundamentally rests on maintaining an effective system of governance in EPA and other organizations applying risk assessment. The governance process should have the following elements: • Clarity and accountability of roles and responsibilities. The extent to which risk assessors and risk managers understand their roles and are evaluated on the basis of their fulfilling their responsibilities will assist in mitigating concerns about potential compromise of scientific or policy-related assessments. • Greater transparency of the process. Making information about the assumptions used and judgments reached in risk-assessment and policy deliberations more widely avail- able is itself an important safeguard against abuse. • Documentation of the process. There needs to be appropriate documentation of the rules and milestones of the process and of the relevant information base at all important stages of risk-assessor–risk-manager deliberations. • Oversight and periodic review. EPA should submit selected decisions each year for independent review to ensure the integrity of the risk-assessment–risk-management process. Independent reviewers should issue a public report on their findings. As mentioned above, the problems can occur with the current (conceptual or institu- tional) “firewall” between assessment and management. A risk manager who keeps analysts in the dark about the choices can still order them to “make the risk look smaller (bigger).” Safeguards against any form of manipulation of the risk-assessment process, whether related to the framework or not, must be in place; it seems to the committee that a process that emphasizes evaluation of risk-management options will by definition involve broader par- ticipation, which implies more “sunshine” and less opportunity for the type of manipulation that the Red Book committee was justifiably concerned with. Conclusions and Recommendations Some features of the framework may be evident in EPA programs, but its full imple- mentation will require a substantial transition period. The committee believes that the long-term utility of risk assessment as a decision-support tool requires that EPA operate in the proposed framework (or a very similar one) and so urges the agency to begin the 

256 SCIENCE AND DECISIONS: ADVANCING RISK ASSESSMENT transition. It is perhaps useful to conceive of the transition process as involving a period of experimentation and development of carefully selected “demonstration projects” to illustrate the application of the framework. Selection of a few important environmental problems to which the framework would be applied in full (with formal and time-limited stakeholder involvement at all stages) would constitute a learning period for agency assessors, managers, and stakeholders. Lessons from such demonstration projects could be recorded and used to improve the framework and its application. The committee believes strongly that gradual adoption of the framework will do much to improve the analytic power and utility of risk assessment and will reveal this power and utility to a much wider audience; its credibility and general acceptability will thereby be enhanced. In summary, we recommend the following: • The technical framework for risk assessment presented in the Red Book should remain intact but should be embedded in a broader framework in which risk assessment is used principally to help to discriminate among risk-management options. • The framework for risk-based decision-making (Figure 8-1) should have as its core elements a problem-formulation and scoping phase in which the available risk-management options are identified, a planning and assessment phase in which risk-assessment tools are used to determine risks under existing conditions and with proposed options, and a manage- ment phase in which risk and nonrisk information is integrated to inform choices among options. • EPA should develop multiple guidance documents relevant to the framework, includ- ing a more expansive development of the framework itself (with explicit steps to determine the appropriate scope of the risk assessment), formal provisions for stakeholder involvement at all stages, and methods for options development that ensure that a wide array of options will be formally evaluated. • EPA should phase in the use of the framework with a series of demonstration projects that apply the framework and that determine the degree to which the approach meets the needs of the agency risk managers, and how risk-management conclusions differ as a result of the revised orientation. REFERENCES Ashford, N.A. and C.C. Caldart. 2008. Environmental Law, Policy, and Economics: Reclaiming the Environmental Agenda. Cambridge: MIT Press. Clemen, R.T. 1991. Making Hard Decisions: An Introduction to Decision Analysis. Boston: PWS-Kent Pub. Co. EPA (U.S. Environmental Protection Agency). 1998. Guidelines for Ecological Risk Assessment. EPA/630/R- 95/002F. Risk Assessment Forum, U.S. Environmental Protection Agency, Washington, DC. April 1998 [online]. Available: http://oaspub.epa.gov/eims/eimscomm.getfile?p_download_id=36512 [accessed Feb. 9, 2007]. EPA (U.S. Environmental Protection Agency). 2003. Framework for Cumulative Risk Assessment. EPA/600/ P-02/001F. National Center for Environmental Assessment, Risk Assessment Forum, U.S. Environmen- tal Protection Agency, Washington, DC [online]. Available: http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm? deid=54944 [accessed Jan. 4, 2008]. Finkel A.M. 2003. Too much of the [National Research Council’s] ‘Red Book’ is still ahead of its time. Hum. Ecol. Risk Assess. 9(5):1253-1271. Hattis, D., and R. Goble. 2003. The Red Book, risk assessment, and policy analysis: The road not taken. Hum. Ecol. Risk Assess. 9(5):1297-1306. Lave, L.B., and G.S. Omenn. 1986. Cost-effectiveness of short-term tests for carcinogenicity. Nature 324(6092): 29-34. Lave, L.B., F.K. Ennever, H.S. Rosenkranz, and G.S. Omenn. 1988. Information value of rodent bioassay. Nature 336(6200):631-633. 

IMPROVING THE UTILITY OF RISK ASSESSMENT 257 NRC (National Research Council). 1983. Risk Assessment in the Federal Government: Managing the Process. Washington, DC: National Academy Press. NRC (National Research Council). 1994. Science and Judgment in Risk Assessment. Washington, DC: National Academy Press. NRC (National Research Council). 1996. Understanding Risk: Informing Decisions in a Democratic Society. Wash- ington, DC: National Academy Press. NRC (National Research Council). 2002. Estimating the Public Health Benefits of Proposed Air Pollution Regula- tions. Washington, DC: The National Academies Press. PCCRARM (Presidential/Congressional Commission on Risk Assessment and Risk Management). 1997. Frame- work for Environmental Health Risk Management - Final Report, Vol. 1. [online]. Available: http://www. riskworld.com/nreports/1997/risk-rpt/pdf/EPAJAN.PDF [accessed Jan. 4, 2008]. Raiffa, H. 1968. Decision Analysis: Introductory Lectures on Choices under Uncertainty. New York: Random House. Weinstein, M.C., H.V. Fineberg, A.S. Elstein, H.S. Frazier, D. Neuhauser, R.R. Neutra, and B.J. McNeil. 1980. ������ Clinical Decision Analysis. Philadelphia: W.B. Saunders.