Summary

The U.S. Environmental Protection Agency (EPA) is one of several federal agencies responsible for protecting Americans against significant risks to human health and the environment. As part of that mission, EPA estimates the nature, magnitude, and likelihood of risks to human health and the environment; identifies the potential regulatory actions that will mitigate those risks and protect public health1 and the environment; and uses that information to decide on appropriate regulatory action. Uncertainties, both qualitative and quantitative, in the data and analyses on which these decisions are based enter into the process at each step. As a result, the informed identification and use of the uncertainties inherent in the process is an essential feature of environmental decision making.

MULTIPLE SOURCES OF UNCERTAINTY

This task is critical because of the multiple sources of uncertainty in the decision-making process. EPA has a long record of producing risk assessments and guidance documents relating to the analysis of uncertainty in estimating human health risks. Similarly, advisory bodies commenting on the role of uncertainty in EPA risk assessments and regulatory decisions have focused on the health risk component. However, EPA takes many other factors—economic and technological factors in particular—into

___________________

1 Throughout this report the committee uses the term public health when referring to EPA’s mission. The committee includes in the use of that term the whole population and individuals or individual subgroups within the whole population.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
Summary T he U.S. Environmental Protection Agency (EPA) is one of several fed- eral agencies responsible for protecting Americans against significant risks to human health and the environment. As part of that mission, EPA estimates the nature, magnitude, and likelihood of risks to human health and the environment; identifies the potential regulatory actions that will mitigate those risks and protect public health1 and the environment; and uses that information to decide on appropriate regulatory action. Un- certainties, both qualitative and quantitative, in the data and analyses on which these decisions are based enter into the process at each step. As a result, the informed identification and use of the uncertainties inherent in the process is an essential feature of environmental decision making. MULTIPLE SOURCES OF UNCERTAINTY This task is critical because of the multiple sources of uncertainty in the decision-making process. EPA has a long record of producing risk as- sessments and guidance documents relating to the analysis of uncertainty in estimating human health risks. Similarly, advisory bodies commenting on the role of uncertainty in EPA risk assessments and regulatory deci- sions have focused on the health risk component. However, EPA takes many other factors—economic and technological factors in particular—into 1  Throughout this report the committee uses the term public health when referring to EPA’s mission. The committee includes in the use of that term the whole population and individuals or individual subgroups within the whole population. 1

OCR for page 1
2 ENVIRONMENTAL DECISIONS IN THE FACE OF UNCERTAINTY consideration when making its decisions, and the uncertainties in those other components are also worthy of attention. Unfortunately, the uncer- tainties in these areas receive much less attention than those in the area of human health, both from EPA and from advisory bodies. Social factors, such as environmental justice, and the political context also play a role in EPA’s decisions and can have inherent uncertainties that are difficult to quantify. This report strives to address this imbalance by giving attention to uncertainties in some of the factors that affect EPA’s decision making in ad- dition to the uncertainties in the estimates of human health risks. Although the committee distinguishes among the different factors in this report, the factors are not independent, and the lines between them are often blurred. Technological factors can affect an economic analysis in a number of ways. The cost of complying with a regulation might be estimated in a techno- logical assessment, for example, but typically it would also be discussed as part of an economic assessment. The consideration of susceptible popula- tions can affect estimates of health risks, and the socioeconomic status of a population affected by a regulation can affect estimates of a “willingness- to-pay” analysis conducted as part of an economic analysis. The political context can affect, explicitly or implicitly, the relative considerations given to the different factors in a decision. This increasingly complex set of issues requires agreed-upon principles and analytical tools for conducting the uncertainty analyses used in making environmental decisions. As developed in this report, the use of those new tools in the analysis of uncertainty poses new challenges and opportunities for EPA in making and communicating its environmental decisions. This summary opens with a description of EPA’s charge to the commit- tee and the committee’s approach to the charge, followed by an overview of three types of uncertainty. Focusing next on the multiple sources of un- certainty and their use in decision making, the summary presents highlights from each section of the report. This summary closes with the committee’s recommendations to EPA. APPROACH TO THE CHARGE Statement of Task EPA requested that the Institute of Medicine convene a committee to provide guidance to its decision makers and their partners in states and localities on approaches to managing risk in different contexts when uncertainty is present. It also sought guidance on how information on un- certainty should be presented to help risk managers make sound decisions and to increase transparency in its communications with the public about

OCR for page 1
SUMMARY 3 BOX S-1 Committee’s Statement of Task Based upon available literature, theory, and experience, the committee will provide its best judgment and rationale on how best to use quantitative information on the uncertainty in estimates of risk in order to manage environmental risks to human health and for communicating this information. Specifically, the committee will address the following questions: •  ow does uncertainty influence risk management under different public H health policy scenarios? •  hat are promising tools and techniques from other areas of decision W making on public health policy? What are benefits and drawbacks to these approaches for decision makers at EPA and their partners? • Are there other ways in which the EPA can benefit from quantitative characterization of uncertainty (e.g., value of information techniques to inform research priorities)? • What approaches for communicating uncertainty could be used to en- sure the appropriate use of this risk information? Are there communi- cation techniques to enhance the understanding of uncertainty among users of risk information like risk managers, journalists, and citizens? • What implementation challenges would EPA face in adopting these alternative approaches to decision making and communicating uncer- tainty? What steps should EPA take to address these challenges? Are there interim approaches that EPA could take? those decisions. The specific questions that EPA requested the committee to address are presented in Box S-1.2 Given that its charge is not limited to human health risk assessment and includes broad questions about managing risks and decision making, in this report the committee examines the analysis of uncertainty in those other areas in addition to human health risks. Types of Uncertainty All EPA decisions involve uncertainty, but the type of uncertainty can vary widely from one decision to another. For an analysis of uncertainty to be useful or productive and for decision makers to determine when to invest 2  Consistent with its charge, the committee focuses on “environmental risks to human health” in this report, and does not directly address ecological risk assessment. The committee notes, however, that many of the principles discussed and developed in this report would also apply to decision making related to ecological risks.

OCR for page 1
4 ENVIRONMENTAL DECISIONS IN THE FACE OF UNCERTAINTY resources to reduce the uncertainty, a first and critical step is identifying the types of key uncertainties that contribute to a particular decision prob- lem. The types of uncertainty also, in part, determine the best approaches for analyzing and communicating uncertainty. In this report, the com- mittee classifies the various types of uncertainty into three categories: (1) statistical variability and heterogeneity (also called aleatory or exogenous uncertainty),3 (2) model and parameter uncertainty (also called epistemic uncertainty), and (3) deep uncertainty (uncertainty about the fundamental processes or assumptions underlying a risk assessment). Variability and heterogeneity refer to the natural variations in the en- vironment, exposure paths, and susceptibility of subpopulations. They are inherent characteristics of a system under study, cannot be controlled by decision makers, and cannot be reduced by collecting more information. Empirical estimates of variability and heterogeneity can, however, be better understood through research in order to refine such estimates. Variability can often be quantified with standard statistical techniques, although it may be necessary to collect additional data. Model4 and parameter uncertainty include uncertainty due to the lim- ited scientific knowledge about the nature of the models that link the causes and effects of environmental risks with risk-reduction actions as well as uncertainty about the specific parameters of the models. There may be vari- ous disagreements about the model, such as which model is most appropri- ate for the application at hand, which variables should be included in the model, the model’s functional form (that is, whether the relationship being modeled is linear, exponential, or some other form), and how generalizable the findings based on data collected in another context are to the problem at hand (for example, the generalizability of findings based on experiments on animals to human populations). In theory, model and parameter uncer- tainty can be reduced by additional research. Deep uncertainty is uncertainty that is not likely to be reduced by additional research within the time period in which a decision must be 3  Although chemical risk assessors typically consider uncertainty and variability as separate and distinct, in the other areas uncertainty is seen as encompassing statistical variability and heterogeneity as well as model and parameter uncertainty. Because variability and heterogeneity can contribute to the uncertainty when a decision is being made, in this report the committee discusses them as a specific type of uncertainty. 4  A “model” is defined in the National Research Council’s Science and Decisions: Advancing Risk Assessment (2009, The National Academies Press) as a “simplification of reality that is constructed to gain insights into select attributes of a particular physical, biologic, economic, or social system. Mathematical models express the simplification in quantitative terms” (p. 96). Model parameters are “[t]erms in a model that determine the specific model form. For computational models, these terms are fixed during a model run or simulation, and they define the model output. They can be changed in different runs as a method of conducting sensitivity analysis or to achieve a calibration goal” (p. 97).

OCR for page 1
SUMMARY 5 made. Typically, deep uncertainty is present when underlying environmen- tal processes are not understood, when there is fundamental disagreement among scientists about the nature of the environmental processes, and when methods are not available to characterize the processes (such as the mea- surement and evaluation of chemical mixtures). When deep uncertainty is present, it is unclear how those disagreements can be resolved. In situations characterized by deep uncertainty, the probabilities associated with various regulatory options and associated utilities may not be known. Neither the collection and analysis of data nor expert elicitation to assess uncertainty is likely to be productive when key parties to a decision do not agree on the system model, prior probabilities, or the cost function. The task instead is to make decisions despite the presence of deep uncertainty using the available science and judgment, to communicate how those decisions were made, and to revisit those decisions when more information is available. UNCERTAINTY IN EPA’S ESTIMATES OF HEALTH RISK Uncertainty is inherent in the scientific information upon which health risk estimates are based. Uncertainties enter the health risk assessment process at every step and can be caused by the potential confounders in observational studies, by extrapolation from animal studies to human stud- ies, by extrapolation from high- to low-dose exposures, by interindividual variability, and by modeling the relationships between concentrations, hu- man exposures, and human health responses and evaluating the effect of interventions or risk control options on public health risk. A number of reports from the National Research Council (NRC) and other bodies discuss the need to evaluate, assess, and communicate the uncertainties in such estimates. Many of those reports emphasize the need to quantify the uncertainties inherent in human health risk estimates, rec- ommend moving away from the presentation of health risk as point esti- mates, detail the pitfalls of using defaults to capture uncertainties in those assessments, and urge the EPA to seek data that could supplant the use of defaults. To that end, EPA has been a leader in the development of quan- titative approaches for uncertainty analysis, such as applying Monte Carlo analysis and Bayesian approaches to environmental risk assessments. These types of uncertainty analysis have been used in a broad variety of EPA risk assessments, ranging from complex analyses for major chemicals such as arsenic, methylmercury, and dioxin to work done for the multiplicity of chemicals entering the IRIS database or found at Superfund sites. On the other hand, those analyses of and concerns about uncertainties have in some cases (such as in the agency’s work involving dioxin con- tamination) delayed rulemaking. Furthermore, some uncertainty analyses have not provided useful or necessary information for the decision at hand.

OCR for page 1
6 ENVIRONMENTAL DECISIONS IN THE FACE OF UNCERTAINTY Because of that, NRC and other organizations have cautioned against ex- cessively complex uncertainty analysis and have emphasized the need for such analyses to be decision driven; that is, they have recommended that the amount of uncertainty analysis matches the needs of the decision maker. The connection of information to decision making is a key feature in value- of-information analyses. This committee agrees that EPA often focuses on the analysis of un- certainty in human health risk estimates without considering the role of the uncertainty in the context of the decision, that is, without considering whether—or explaining how—the analysis influences the agency’s regula- tory decision. The magnitude of the uncertainty in risk estimates might not always be large enough to influence the decision, or the uncertainty in the estimates might be overshadowed by the uncertainty in the other factors that EPA considers in a decision (see below for discussion). UNCERTAINTY IN COMPONENTS OF DECISION MAKING OTHER THAN ESTIMATES OF HUMAN HEALTH RISK Data and analyses from fields other than human health risks play a role in EPA’s decisions, including technological and economic considerations.5 As with estimates of health risks, the three different types of uncertainty dis- cussed above can be present, and the presence of uncertainty is usually un- avoidable in those data and analyses. Some of EPA’s technological feasibility and cost–benefit analyses assess uncertainty, but many do not. Furthermore, the contribution of uncertainties in other factors, such as social factors (for example, environmental justice) and the political context receive little or no attention. With the exception of EPA’s new guidance on economic analysis, which includes a discussion of uncertainty analysis, the agency offers little guidance or information about how to assess uncertainty in factors other than health risks or about how it considers that uncertainty in its decisions. UNCERTAINTY: OTHER PUBLIC HEALTH SETTINGS Chapter 4 reviews the methods and processes used for uncertainty analysis at other public health agencies and organizations. Although a num- ber of agencies conduct complex analyses that use probabilistic techniques to assess uncertainties in health risks, the tools and techniques that those 5  Statutory requirements and constraints in the nation’s environmental laws shape the overall decision-making process, with general requirements relating to data expectations, schedules and deadlines, public participation, and other considerations. At the same time these laws allow EPA considerable discretion in the development and implementation of environmental regulations.

OCR for page 1
SUMMARY 7 organizations use are similar to those used by EPA. Thus, the committee did not identify promising tools and techniques for assessing uncertainty from other areas of public health that would present new guidance for EPA. There are some examples from those organizations, however, that illustrate important concepts in decision making in the face of uncertainty. For instance, an analysis of the effects of various regulatory options on the risks from bovine spongiform encephalopathy offers an example of an uncertainty analysis targeted to a regulatory decision (that is, a decision- driven assessment). An assessment of the risks of Listeria monocytogenes in different foods illustrates the importance of involving stakeholders and external experts early in the decision process in order to identify and decrease uncertainties; it also demonstrates how an assessment can help identify targeted risk-mitigation strategies. An assessment of the risks from melamine in infant formula shows how a simple risk assessment and uncer- tainty analysis can provide the information necessary to make a decision. FDA’s handling of its Avandia® (rosiglitazone) decision illustrates how dis- agreements among scientists about scientific evidence can be communicated so that the public can understand the rationale for the ultimate decision. The lessons learned from an often-criticized decision about vaccinating during the 1976 pandemic scare emphasize the importance of a systematic approach to incorporating uncertainty into a decision and of an iterative approach to decision making under deep uncertainty. INCORPORATING AND USING UNCERTAINTY IN DECISION MAKING The appropriate uncertainty analysis for a decision—and how to con- sider uncertainty in a decision—will depend on the types, source, and magnitude of the uncertainty as well as on the context of the decision (for example, the severity of the adverse effects and the time frame within which a decision is needed). Although uncertainty analysis needs to be designed on a case-by-case basis, there are general frameworks and processes that should be followed for determining uncertainty analyses and how to in- corporate uncertainty into a decision. The legal context of a decision will determine, in part, the degree of caution. In some contexts best estimates of risks will indicate the best action, whereas in others that require more caution, upper limits on risk will indicate the best action. In all cases, de- cision makers should explain why and how uncertainties were taken into account in their decisions. Systematically considering uncertainties and their potential to affect a decision from the onset of the decision-making process will improve the decision, focus uncertainty analyses on the decision at hand, facilitate the identification of uncertainties in factors in addition to health risk estimates, improve the planning of uncertainty analyses, and

OCR for page 1
8 ENVIRONMENTAL DECISIONS IN THE FACE OF UNCERTAINTY set the stage for the consideration of uncertainties in decisions. To that end it is important that uncertainty is considered and incorporated into each phase—problem formulation, risk assessment, and risk management—of a systematic decision-making process. Chapter 5 presents a framework for decision making that incorporates the planning, conducting, and consider- ing of uncertainty analyses into the decision-making process. The types and source of uncertainty are often key determinants of the appropriate type of uncertainty analysis. In Box S-2 the committee provides BOX S-2 Implications of Uncertainty Analysis for Decision Making Health Uncertainties Uncertainty analyses in human health risk estimates can help decision makers to • evaluate alternative regulatory options; • assess how credible extreme risk estimates are and how much to rely on them in decision making; • weigh the marginal decrease in risk against the effort made to reduce it; • clarify issues within a decision by using scenarios to characterize very different worlds; and • in the case of scenario analyses for deep uncertainty, identify regulatory solutions that are effective over a broad spectrum of scenarios. Uncertainties About Technology Availability Uncertainty analyses in technology availability can help decision makers to • differentiate between well-established technologies with reasonably well-known costs, and those that have not been used for the purposes at hand; and • consider which technology may be considered “best practicable” or “best available” by providing information about both the likelihood of success of the unproven technologies, the time frame for success, and the effectiveness if successful. Uncertainties About Cost and Benefits Given the highly uncertain estimates of both health benefits and costs, uncertainty analyses in cost–benefit analyses can inform decision makers about • how difficult it is to differentiate among different potential decisions; • the disagreement among experts about the way regulation affects the economy, even when using similar models; and • the ranges and sensitivity of estimates to different variables.

OCR for page 1
SUMMARY 9 guidance on how uncertainty analyses about health effects, technological availability, and cost can be used in the decison-making process. Decisions in the presence of deep uncertainty are particularly challenging. As dis- cussed in Chapter 5, scenario analysis, value-of-information methods, and robust decision methods that allow for adaptive management can be useful under those circumstances. There are no simple rules to translate uncertainty information into a de- cision. However, a decision maker should be informed about and appreciate the range of uncertainty when making a decision. How uncertainty analysis is used will differ depending on the type of uncertainty under consideration. Uncertainty analyses in human health risk estimates can help decision mak- ers to weigh the marginal decrease in risk against the effort made to reduce it. Uncertainty analyses in technology availability can help decision makers to differentiate between well-established technologies with reasonably well- known costs and those that have not been used for the purposes at hand. Uncertainty analyses in cost–benefit analyses can inform decision makers about the disagreement among experts about the way regulations affect the economy, even when using similar models. Box S-2 provides other examples of the ways in which uncertainty analyses can be used in decision making. The interpretation and incorporation of uncertainty into environmental decisions will depend on a number of characteristics of the risks and the decision. Those characteristics include the distribution of the risks, the deci- sion makers’ risk aversion, and the potential consequences of the decision. The quality of the analysis and recommendations following from the analysis will depend on the relationship between the analyst and the deci- sion maker. The planning, conduct, and results of uncertainty analysis should not be conducted in isolation, separated from the individuals who will eventually make the decisions. The success of a decision in the face of uncertainty depends on the analysts having a good understanding of the context of the decision and the information needed by the decision makers and also the decision makers having a good understanding of the evidence on which to base the decision, including understanding the uncertainty in that evidence. COMMUNICATING UNCERTAINTY IN DECISIONS Much of the research related to communicating uncertainty in en- vironmental decisions actually focuses on communicating uncertainty in estimates of health risks. Research on risk communication, which includes the communication of the uncertainty in health risks, highlights the impor- tance of using an interactive approach to communication. In other words, communication should not consist only of EPA providing information to others, but rather it should include an active exchange of information,

OCR for page 1
10 ENVIRONMENTAL DECISIONS IN THE FACE OF UNCERTAINTY with both EPA and stakeholders providing input to the conversation. Such interaction should occur from the onset of a decision—that is, during the problem-formulation phase of the decision-making process. Early commu- nication facilitates incorporation of stakeholder perspectives in the process and helps to identify uncertainties for consideration in the decision. In addition, discussing known and potential uncertainties from the start of the decision-making process can increase social trust, which is critical for effective decision making, especially for decisions made in the face of high levels of uncertainty or scientific disagreement. Uncertainty is typically expressed in terms of the probability or likeli- hood of an event and can be presented numerically, verbally, and graphi- cally; each approach has its advantages and disadvantages. Numeric presentations can communicate a large amount of information but are useful only with audiences that are knowledgeable and capable of interpret- ing them. Verbal presentations (that is, the use of such terms as “likely” or “unlikely”) can capture people’s attention and portray directionality, but they can be prone to different interpretations in different contexts and by different people. Graphic presentations of probabilistic information can capture and hold people’s attention, but individuals can vary in their ability to correctly interpret those presentations. EPA communicates with people with a broad range of expertise and interests under a broad range of circumstances, and the appropriate com- munication approach will depend on who the communication is with, the source and types of the uncertainty, the context of the decision, and the purpose of the communication. For example, because the approach should be geared toward the audience, a table that includes numerical presenta- tions of probabilities might be appropriate when communicating with a scientist or technical expert who is accustomed to thinking in those terms, but a graphical presentation might be more appropriate for a person with- out that background. Different people and groups of people, including scientists and regulatory decision makers, have biases (availability, confir- mation, confidence, and group bias, among others) that can affect both the interpretation and the framing of uncertainty. Acknowledging those biases at the beginning of the decision-making process is critical to successful com- munication about an issue. More communication is needed about the different sources of the uncertainty in EPA’s decisions and about how those sources of uncer- tainty compare and affect a decision. For example, if the uncertainty in an estimate of health risks contributes to the uncertainty in a decision less than does the uncertainty in cost estimates, that should be communicated. Documenting the nature and magnitude of uncertainty in a decision is not only important at the time of the decision, but it is also important when a decision might be revisited or evaluated in the future.

OCR for page 1
SUMMARY 11 FINDINGS AND RECOMMENDATIONS Uncertainties in the Characterization of Human Health Risks Finding 1 Decision documents6 prepared to explain specific decisions often lack a robust discussion of the uncertainties identified in the health risk assess- ments prepared by agency scientists. Although those documents and com- munications should be succinct, open, and transparent, they should also include information on what uncertainties are present, which uncertainties need to be addressed, and how those uncertainties affected a decision. It should be clear from agency communications that uncertainty is inherent in science, including the science that informs EPA decisions. In addition to contributing to full transparency, providing information and fostering dis- cussion of the existence of uncertainties, including unresolved uncertainty, could eventually lead to greater public understanding and appreciation of uncertainty in decision making. RECOMMENDATION 1 To better inform the public and decision makers, U.S. Environmental Protection Agency (EPA) decision documents and other communica- tions to the public should systematically • include information on what uncertainties in the health risk assess- ment are present and which need to be addressed; • discuss how the uncertainties affect the decision at hand; and • nclude an explicit statement that uncertainty is inherent in science, i including the science that informs EPA decisions. Uncertainty in Other Factors That Influence a Decision Finding 2 Although EPA decisions have included discussions and consideration of the uncertainties in the health risk assessment, the agency has generally given less attention to uncertainties in other contributors influencing the regulatory decision. Those contributors include economic and technologi- cal factors as well as other factors that are not easily quantified, such as environmental justice. A major challenge to decision making in the face of 6  Thecommittee uses the term “decision document” to refer to EPA documents that go from EPA staff to the decision maker and documents produced to announce an agency decision.

OCR for page 1
12 ENVIRONMENTAL DECISIONS IN THE FACE OF UNCERTAINTY uncertainty is the uncertainty in those other factors. Methods and processes should be available for situations in which such analyses are appropriate and helpful to a decision maker. In general, this might require a research program to develop methods for this new type of uncertainty analysis, changes in decision documents and other analyses, and a program for re- search on communicating uncertainties. RECOMMENDATION 2 The U.S. Environmental Protection Agency should develop methods to systematically describe and account for uncertainties in decision- relevant factors in addition to estimates of health risks—including technological and economic factors—in its decision-making process. When influential in a decision, those new methods should be subject to peer review. Finding 3 EPA has developed guidance about, and conducted in-depth analyses of, the costs and benefits of major decisions. EPA guidance contains appro- priate advice about the conduct of these analyses, including the discussion of some uncertainties. However, the committee noted a lack of transparency regarding uncertainty analyses in the cost–benefit assessments in some EPA decision documents. The information presented about these uncertainties is often arcane, hard to locate, and technically very challenging to non- experts. Those analyses often shape regulatory decisions; thus, they should be described in ways that are useful and interpretable for the decision maker and stakeholders. The needs of the two audiences—that is, techni- cal and non-expert audiences—differ, but a given set of decision documents and supporting analyses could include descriptions that explain the sources of uncertainties to the non-expert and provide links, either electronically or via text, to more detailed descriptions of the economic analyses that are appropriate for experts. RECOMMENDATION 3 Analysts and decision makers should describe in decision documents and other public communications uncertainties in cost–benefit analyses that are conducted, even if not required by statute for decision making, and the analyses should be described at levels that are appropriate for technical experts and non-experts.

OCR for page 1
SUMMARY 13 Finding 4 The role of uncertainty in the costs and benefits and availability and feasibility of control technologies is not well investigated or understood. The evidence base for those factors is not robust. Evaluating case studies of past rulemaking and developing a directed research program on assessing the availability of technologies might be the first steps toward understand- ing the robustness of technology feasibility assessments and economic as- sessments as well as the potential for technology innovation. RECOMMENDATION 4 The U.S. Environmental Protection Agency (EPA) should fund research, conduct research, or both to evaluate the accuracy and predictive ca- pabilities of past assessments of technologies and costs and benefits for rulemaking in order to improve future efforts. This research could be conducted by EPA staff or else by nongovernmental policy analysts, who might be less subject to biases. This research should be used as a learning tool for EPA to improve its analytic approaches to assessing technological feasibility. Finding 5 The committee did not find any specific guidance for assessing the un- certainties in the other factors that affect decision making, such as social factors (for example, environmental justice) and the political context. The committee also did not find examples of systematic consideration of those factors and their uncertainty when exploring the policy implications of strategies to mitigate harms to human health. In response to requirements in statutes or executive orders that require regulations to be based on the open exchange of information and the perspectives of stakeholders, some EPA programs (e.g., Superfund) work to address issues related to public (stakeholder) values and concerns. Ecological risk assessments7 have included contingent valuation to help inform policy development. Similarly, economists have explored the values people hold regarding specific health outcomes for the purposes of resource allocation or clinical guideline development. More research is needed into methods to appropriately characterize the uncertainty in those other factors and to communicate that uncertainty to decision makers and the public. 7  Ecological risk assessment is a “process that evaluates the likelihood that adverse ecological effects may occur or are occurring as a result of exposure to one or more stressors” (http:// www.epa.gov/raf/publications/pdfs/ECOTXTBX.PDF [accessed January 16, 2013]).

OCR for page 1
14 ENVIRONMENTAL DECISIONS IN THE FACE OF UNCERTAINTY RECOMMENDATION 5 The U.S. Environmental Protection Agency should continue to work with stakeholders, particularly the general public, in efforts to iden- tify their values and concerns in order to determine which uncertain- ties in other factors, along with those in the health risk assessment, should be analyzed, factored into the decision-making process, and communicated. Finding 6 The nature of stakeholder participation in and input to a decision depends on the type of stakeholder. The regulated industry, local business communities, and environmental activists (including those at the local level, if they exist) are more likely to be proactively engaged in providing input on pending regulations. The general public, without encouragement or assistance from EPA (or local environmental regulatory departments), is less likely to participate effectively or at all in such activities. One means to bridge the gap in understanding the values of the public is a formal research program. RECOMMENDATION 6 The U.S. Environmental Protection Agency should fund or conduct methodological research on ways to measure public values. This could allow decision makers to systematically assess and better explain the role that public sentiment and other factors that are difficult to quantify play in the decision-making process. Framework for Incorporating Uncertainty in Decision Making Finding 7 Uncertainty analysis must be designed on a case-by-case basis. The choice of uncertainty analysis depends on the context of the decision, in- cluding the nature or type of uncertainty, and the factors that are considered in the decision (that is, health risk, technological and economic factors, public sentiment, and the political context), as well as on the data that are available. Most environmental problems will require the use of multiple approaches to uncertainty analysis. As a result, a mix of statistical analyses and expert judgments will be needed. A sensible, decision-driven, and resource-responsible approach to un- certainty analyses that includes decision makers and stakeholders is needed. Such a process will help ensure that the goals of the uncertainty analysis are consistent with the needs of the decision makers and the values and

OCR for page 1
SUMMARY 15 concerns of stakeholders, and it will help define analytic endpoints and identify population subgroups and heterogeneity and other uncertainties. The committee believes that quantitative uncertainty analyses should only be undertaken when they are important and relevant to a given de- cision. Whether further quantitative uncertainty analysis is needed will depend on the ability of these analyses to affect the environmental decision at hand. One way to gauge this is to inquire whether perfect information would be able to change the decision, for example, whether knowing the exact dose–response function would change the regulatory regime. Clearly, if an environmental decision would stay the same for all states of infor- mation and analysis results, then it would not be worth conducting the analysis. RECOMMENDATION 7 Although some analysis and description of uncertainty is always im- portant, how many and what types of uncertainty analyses are carried out should depend on the specific decision problem at hand. The effort to analyze specific uncertainties through probabilistic risk assessment or quantitative uncertainty analysis should be guided by the ability of those analyses to affect the environmental decision. Communication Finding 8 A structured format for the public communication of the basis of EPA’s decisions would facilitate transparency and subsequent work with stake- holders, particularly community members. EPA decision documents should make it clear that the identified uncertainties are in line with reasonable expectations presented in EPA guidelines and other sources. This practice would facilitate the goals of the first recommendation of the committee in this report—that EPA decision documents should make it clear that uncer- tainty is inherent in agency risk assessments. The committee intends that the recommendations in this report support full discussion of the difficulties of decision making, including—and possibly particularly—when social fac- tors (such as environmental justice and public values) and political context play a large role. RECOMMENDATION 8.1 U.S. Environmental Protection Agency (EPA) senior managers should be transparent in communicating the basis of the agency’s decisions, in- cluding the extent to which uncertainty may have influenced decisions.

OCR for page 1
16 ENVIRONMENTAL DECISIONS IN THE FACE OF UNCERTAINTY RECOMMENDATION 8.2 U.S. Environmental Protection Agency decision documents and com- munications to the public should include a discussion of which uncer- tainties are and are not reducible in the near term. The implications of each to policy making should be provided in other communication documents when it might be useful for readers. Finding 9 Given that decision makers vary in their technical backgrounds and experience with highly mathematical depictions of uncertainty, a variety of communication tools should be developed. The public increasingly wants, and deserves, the opportunity to understand the decisions of appointed officials in order to manage their own risk and to hold decision makers ac- countable. With respect to which uncertainties or aspects of uncertainties to communicate, attention should be paid to the relevance to the audience of the uncertainties, so that the uncertainty information is meaningful to the decision-making process and the audience(s). Those efforts should include different types of decisions and should include communication of uncer- tainty to decision makers and to stakeholders and other interested parties. RECOMMENDATION 9.1 The U.S. Environmental Protection Agency (EPA), alone or in collabo- ration with other relevant agencies, should fund or conduct research on communication of uncertainties for different types of decisions and to different audiences, develop a compilation of best practices, and systematically evaluate its communications. RECOMMENDATION 9.2 As part of an initiative evaluating uncertainties in public sentiment and communication, U.S. Environmental Protection Agency senior manag- ers should assess agency expertise in the social and behavioral sciences (for example, communication, decision analysis, and economics) and ensure it is adequate to implement the recommendations in this report. In summary, the committee was impressed by the technical advances in uncertainty analysis used by EPA scientists in support of EPA’s human health risk assessments, which form the foundation of all EPA decisions. The committee believes that EPA can lead the development of uncertainty analyses in economic and technological assessment that are used for regu- latory purposes as well as the development of ways to characterize and

OCR for page 1
SUMMARY 17 account for public sentiment and political context. Leading in this way will require a targeted research program as well as disciplined attention to how those uncertainties are described and communicated to a variety of audiences, including the role that uncertainties have played in a decision.

OCR for page 1