Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
5 Impediments to a Successful Application of the Risk-Informed Approach The risk-informed approach outlined in Chapter 4 can be useful in an exemption process, but the committee is concerned that this process not leave the impression that the risk-informed approach will be easy to per- form successfully. In fact, the committee wishes to emphasize that risk analysis cannot be performed in âcookbook fashionâ if it is to be mean- ingful and useful in decision making. In this chapter, the committee re- views some of the unavoidable and irreducible complexities in perform- ing risk analysis and other important sources of concern with risk as- sessments. In order to apply the risk-informed approach successfully, the Department of Energy (DOE) must be fully aware of the complexities that a good risk analysis will encounter and be prepared to address them directly with exceptional intellectual and analytical effort. Additionally, DOE must accept that the results of the risk analysis should be only a part of a decision-making process, not the sole basis for the final deci- sion. The first section of this chapter discusses the limits and uncertainties of science in modeling long-term environmental processes and future risks. These complexities need not prevent progress toward making sound decisions about the disposition of waste streams, as long as the uncertain- ties are acknowledged, addressed where possible, and incorporated into 113
114 RISK AND DECISIONS ABOUT TRU AND HLW decision-making deliberations. However, doing so may require a change in oneâs perspective about the goals of the risk analysis. The goal of the risk analysis in the risk-informed approach described in this report is to inform DOE, regulators, American Indian nations, local governments, and the public of the full range of outcomes that might occur by accounting as much as possible for the limitations of our knowledge of the systems in- volved. The analysis should inform the process participants of the ways in which undesirable outcomes may occur and provide insight into how to better engineer for or manage those risks. The committee cautions against using risk analysis to generate a specific quantitative estimate of risk (or risk ranges) that would then be deemed âacceptableâ or âunacceptableâ for the purpose of arguing for an exemption. Such strictly quantitative interpretations of risk estimates likely will fail in the face of the complexi- ties and unknowns of the science underlying them. The committee de- scribes a structured process, but does not provide decision criteria. The purpose of the risk analysis is not to produce a de facto decision, but rather to inform. Sections 5.2 to 5.4 highlight a range of other issues that also could be impediments to the successful application of a risk-informed approach. These include stakeholder and decision-maker concerns and impedi- ments in institutional culture. Again, the process recommended in this report and the set of steps outlined in Chapter 4 are designed to address and manage these concerns. Nevertheless, in this chapter each issue is discussed individually to emphasize that DOE must grapple directly with these complexities if the proposed risk-informed approach is to be ap- plied successfully. The National Research Council (NRC, 1996) provides a more in-depth exploration of these and other issues. That reference can 1 serve to supplement the material in this chapter. 1 The âanalytic-deliberative processâ described in NRC (1996) is consistent with and, indeed, a foundation for the risk-informed approach recommended here. This report describes an analytic-deliberative process that has been tailored specifically to support decision making on disposition of high-level and tran- suranic wastes, and it focuses its discussion on those concerns that appear to present the greatest impediments in this particular application.
IMPEDIMENTS TO THE RISK-INFORMED APPROACH 115 5.1 RISK MODELING ISSUES At the core of the risk-informed exemption process recommended in this report is the estimation of the risk that results from different disposi- tion options associated with a given waste type at a specific location. In most cases, this will be an analysis of the risk posed by waste, nominally classified as HLW or TRU waste, that is left at a site rather than being removed and disposed of in a geologic repository, as currently required by law. Once calculated, the risk can be weighed against other relevant factors, such as cost and worker impacts, and this information becomes a part of the risk-informed decision-making process. Properly done, risk assessment is a powerful tool for systematically organizing the information and understanding the behavior and impacts of radioactive waste at a particular location. Although the âriskâ associ- ated with disposition of different types of radioactive waste may be judged initially by the properties of the waste (e.g., amount of radioactiv- ity, chemical form, toxicity, half-life, complexity of composition), the actual risk assessment that is used to evaluate a disposition strategy will involve models of the interaction of the waste components with the near- field and far-field environments, including interactions with engineered barriers or other containment mechanisms and the natural geochemical and hydrological systems. The site analysis provides the basic data used to calculate the risk that results from the many different exposure path- ways in the biosphere. Thus, the risk assessment is composed of a series of coupled models used to describe the relevant processes. The quality of the risk analysis that would inform exemption decisions will rest on the quality of the data and on the scientific understanding of the underlying physical, chemical, and behavioral phenomena, and how well that under- standing is reflected in the analysis. Risk-informed approaches have been endorsed by several previous committees (e.g., NRC, 1990, 1996) to support decision making on complex social policy problems. However, actual applications of risk assessment by DOE to its waste management issues have brought out 2 the difficulties of achieving scientific and technical consensus. Further, risk assessment in general is not accepted by everyone (Tal, 1997). The 2 An illustration of the challenges involved in such a risk assessment when long time horizons are in question is the experience with the performance as- sessments of the proposed geologic repository at Yucca Mountain for spent nu- clear fuel and high-level waste (e.g., Budnitz et al., 1999; Long and Ewing, 2004).
116 RISK AND DECISIONS ABOUT TRU AND HLW committee is aware of these difficulties, and its recommendations have been prepared with these concerns in mind. In particular, the committee has recommended a risk analysis process that emphasizes sensitivity analysis to understand how lack of knowledge can affect decisions on waste disposition, rather than blindly adding model complexity without first demonstrating its necessity or value to making a better decision. Additionally, the committee has recommended a risk-informed process rather than decision making based solely on the quantitative output of 3 the risk analysis itself. This being said, it is useful to explore the objec- tions to risk analysis that have been expressed and their historical un- derpinnings. The following discussion emphasizes risk analysis of geo- logic systems, but similar concerns relate to other environmental sys- tems and their pathways, such as air and surface contaminant exposures. Scientists may object to the results of a risk analysis because the models used may not capture the actual system behavior of the radionu- clides at a specific site. The calculation of risk over long periods of time or to future generations is especially problematic. The agency or institu- tion responsible for completing the risk assessment may have a real or perceived bias that compromises the credibility of the results. Stake- holders may be excluded from the process and skeptical of results that become the basis for changing previous agreements, particularly when the decision is to leave the waste in place. Decision makers may be baf- fled by the complexity of the analysis and uncomfortable with equivocal results that, in fact, may have a high uncertainty. Rechard (1999) reviewed the history of the use of risk analysis and performance assessments of geologic systems. Performance assessments, and their supporting methodologies, were first used to evaluate the reli- ability of nuclear weapons delivery systems. In the early 1970s, probabil- istic risk assessments (PRAs) were used to evaluate the risk and conse- quences of nuclear power reactor accidents. The WASH-1400 report by Rasmussen and colleagues was the first comprehensive, probabilistic analysis of the health risks from a large and complicated technological system, a nuclear power plant (Rasmussen, 1975). The crossover point for the use of risk assessments for geologic systems came in 1976 when the Energy Research and Development Administration sponsored two 3 This philosophy is also central to the analytic-deliberative process recom- mended in the NRC report (1996). The committee believes that some of the dif- ficulties in DOEâs risk assessment track record are attributable to a failure to reflect such a philosophy in practice.
IMPEDIMENTS TO THE RISK-INFORMED APPROACH 117 workshops that brought together nuclear engineers who were conversant with the recently developed PRA technique and earth scientists involved in modeling geologic systems. Much of the faith in risk assessments of geologic systems is based on a belief that the methods of risk analysis, as developed for analyzing nuclear reactor safety, can be used to analyze geologic systems over long periods (Garrick and Kaplan, 1995). How- ever, as early as 1978, John Bredehoeft and colleagues had clearly out- lined the challenges to geosciences in attempting to model the long-term behavior of a geologic system used for the disposal of radioactive waste (Bredehoeft et al., 1978). One of their conclusions is worth quoting: In summary, predictive models are an essential step in the selection and imple- mentation of a radioactive-waste repository and a radioactive-waste treatment system. They are invaluable tools for analyzing the problem and for identifying factors that are likely to have the greatest effect on radionuclide migration. However, some components of the models are inherently unpredictable at pre- sent and are likely to change at different times. In no sense, therefore, will these models give a single answer to the question of the fate of radioactive waste in geologic repositoriesâ¦. A quarter of a century later, there is an ongoing debate about the value and limitations of risk assessments (Ewing et al., 1999). At the heart of the discussion are the clear limitations in the predictive capabili- ties of models in principle (Oreskes et al., 1994) and in practice (Oreskes and Belitz, 2001). Models of radioactive waste in geologic systems are inevitably limited for the following reasons: 1. They are simplified, hence, incomplete descriptions of the sys- tem. 2. The governing equations of the models may not provide unique solutions. 3. They cannot be calibrated to data sets that include the relevant spatial and temporal scales to be modeled. 4. Boundary conditions may change over time in ways that are un- foreseeable. 5. Relevant, critical processes may have been omitted from the analysis.
118 RISK AND DECISIONS ABOUT TRU AND HLW Each of these five issues is inherent in any risk assessment and may never be completely eliminated by additional work or research. The fifth issue, conceptual model uncertainty, is probably the greatest source of uncertainty and yet the most difficult to estimate or characterize (NRC, 1995b, 2001b). The committee returns to the issue of conceptual model uncertainty below. In addition to the fundamental limitations in the use of models for risk assessments, there is ample opportunity for additional, but more manageable, sources of error to enter into the analysis, which add to the uncertainty of conclusions that may be drawn from it: â¢ Input data may be wrong. â¢ The governing equations may have been solved incorrectly. â¢ Parameter distribution functions may be wrong. â¢ Estimated probabilities of critical events may be wrong. Analysts must make judgments about how to set parameters and how to characterize their variability or uncertainty. A known difficulty in this step is that such judgments are subject to a number of heuristic biases, the net effect of which is that the true uncertainty is frequently underes- timated (Morgan and Henrion, 1990; NRC, 1996). Analysts may some- times make judgments that are subconsciously (or even consciously) an- chored in values that can systematically drive the results towards their personal prior view of the risk endpoint. There are methods to correct for judgmental biases (Morgan and Henrion, 1990), and it is important that they be applied in risk analysis. Other elements of the analysis process in Chapter 4 also can help reduce these errors, such as quality assurance programs, review committees, public input, and having an external au- thority to judge the merits of the evidence in favor of an exemption. Even when the above issues are addressed carefully and the input data are as good as the current state of knowledge allows, uncertainty in the analysis will remain. The final risk assessment will still be model dependent, and conceptual model uncertainty will be a concern. This un- certainty will be exacerbated by the fact that the risk assessment will en- tail a coupling of many models (e.g., waste form degradation, geochemi- cal interactions and speciation, hydrology, biosphere pathways), and the connections between these models may have complicated feedback rela-
IMPEDIMENTS TO THE RISK-INFORMED APPROACH 119 tions. New emergent behaviors that come about through complex interac- tions of different components of nonlinear systems may be extremely difficult or nearly impossible to predict. This means that understanding of behavior hinges upon resolving complexity, a feature often found in nonlinear systems, yet there are no general rules for recognizing and re- solving such complexities. Thus, despite great advances in science dur- ing the past 50 years in describing, modeling, and even predicting the behavior of systems, the types of errors described above remain substan- tial. Examples of remaining difficulties in modeling hydrological and geochemical systems are provided in Sidebar 5.1; but similar issues are important for all of the physical, chemical, and behavioral systems that must be considered in a complete risk assessment, especially if the analy- sis will attempt to project outcomes thousands of years in the future. In sum, uncertainty is an inherent aspect of risk assessment, emanat- ing from several sources (see Sidebar 5.2). The challenge is not to dimin- ish the role of uncertainty, but rather to properly and fully reflect it in information that decision makers will be asked to consider. Often ana- lysts believe that this means that the risk assessment must be performed probabilistically. However, probabilistic analysis can increase the com- plexity and opacity of the analysis, yet distract from the most important 4 form of uncertainty: conceptual model uncertainty. NRC (1996, pp. 113â114) acknowledged the limitations of probabilistic uncertainty analysis in the face of this general source of uncertainty: 4 For example, a Monte Carlo analysis may incorporate explicit representa- tion of uncertainty on dozens or hundreds of parameters of a model, yet never address the possibility that the model itself could be inappropriate. A more use- ful âuncertainty analysisâ might be accomplished with a just few model runs from two alternative conceptual models.
120 RISK AND DECISIONS ABOUT TRU AND HLW Sidebar 5.1: Uncertainties in Hydrologic, Geologic, and Geochemical Components of Modeling The heterogeneous, near-surface earth environment has been termed the âcritical zone,â because it is the region where the hydro- sphere, biosphere, geosphere, and atmosphere interact with one another and with the human sphere in a highly complex manner (see, e.g., NRC, 2001c). The complexity of the critical zone presents special challenges to the development of long-term predictive models. During the past 50 years, earth scientists have made substantial strides in describing, mod- eling, and even predicting the behavior of critical zone systems. Never- theless, our knowledge, understanding, and ability to model critical zone processes are strained by the demands of risk assessments extending for many thousands of years into the future. Modelers are trying to predict waste form and contaminant behavior at DOE sites, including some of the most complicated and as yet poorly understood types of critical zone hydrogeologic systems, including va- dose zones, regions of groundwater-surface-water interactions (including the poorly understood hyperheic zone, which is the geologic material immediately surrounding and underlying rivers and streams), arid re- gions, and fractured flow regimes. Konikow and Bredehoeft conducted postaudits of a number of hydrologic systems, comparing predicted be- havior to actual behavior over periods of decades (Konikow, 1986, 1992, 1995; de Marsily et al. 1992; Konikow and Bredehoeft, 1992; Bredehoeft and Konikow, 1993). A particularly important conclusion was that for models based on a history match (i.e., a calibration to previous hydro- logic data), the predictive capability diminishes rapidly for periods longer than that of the historical data (Bredehoeft, 2003). This limits confidence in the predictive capabilities of hydrologic models to no more than some hundreds of years. One might argue that for some of the most highly complex and poorly understood hydrogeologic settings, the limits of con- fidence could be considerably shorter. Geochemical systems are subject to the same types of uncertainty as hydrologic systems, particularly conceptual model uncertainty (Bethke, 1996; Nordstrom, 2004). Areas of ongoing research, such as the role of nanoparticles, bacteria, and organic matter in the fate and transport of radionuclides, highlight some of the current challenges in understanding geochemical interactions. Geochemical models may have non-unique solutions (Bethke, 1992), can be highly sensitive to very small changes in fundamental input parameters (Ewing et al., 1999) and may produce greatly differing results at different spatial scales and for
IMPEDIMENTS TO THE RISK-INFORMED APPROACH 121 changing boundary conditions (Duro et al., 2000; MadÃ© et al., 2000; Jen- sen et al., 2002; Wang et al., 2003). The most widely used types of models are reactive transport codes that are meant to describe coupled thermal-hydrologic-geochemical processes (Lichtner et al., 1996), although such models often do not adequately address biogeochemical or nanoscale processes. Browning et al. (2003) reviewed a reactive transport model for the ambient unsatu- rated hydrogeochemical system at Yucca Mountain. They found model predictions to be particularly sensitive to assumptions about percolation flux, fracture-matrix interactions, groundwater compositions, and the rate of volcanic glass dissolution; each of these processes is knowable to a greater or lesser degree. However, the coupling of these processes pre- sents challenges to reliable predictions of radionuclide transport. Hugh- son et al. (2000) tested risk analysis models by using them to describe the migration of trace elements from metallic artifacts preserved in a vol- canic tuff some 3600 years ago by a volcanic eruption on Aktori, Greece. They found that different conceptual models, matching the site data, pro- duced different predictions attributed to underdetermined parameters and changing boundary conditions. These are just a few examples of the challenge presented in model- ing hydrogeologic systems, but they are representative of the types of limitations that one should expect of a risk assessment of a hydro- geologic system. The key to success in modeling highly coupled geologic systems is the careful integration of experimental results with field obser- vations (Bethke, 1996; Nordstrom, 2004). The initial risk assessments are likely to reveal the need for additional site characterization, experi- mental data, and perhaps development of better multiscale modeling approaches. Unfortunately, experience shows that it is often these unknown circumstances and surprise events that shake risk analyses and topple expectations, rather than the factors (important though they might be) that have been recognized and in- corporated into formal analysesâ¦ Also, formal uncertainty analysis may not help if the uncertainty in the fundamental understanding of the basic processes that drive the riskâ¦ is so large that a quantitative estimate can only lead to ob- fuscationâ¦ In such cases, identification of important issues and perhaps some selected analysis of scenarios (without assigning probabilities to these scenar- ios), is the best that can be accomplished. Consistent with this earlier NRC report, the approach outlined in Chapter 4 emphasizes use of sensitivity analysis (which includes âsce- nario analysisâ) to understand the role of uncertainties. Sensitivity analy- ses are important in testing the potential inaccuracy of a risk model and
122 RISK AND DECISIONS ABOUT TRU AND HLW determining which of the many forms of remaining uncertainty are im- portant enough to affect decision-making deliberations. As noted in Chapter 4, if there are competing conceptual models, sensitivity analyses should start by considering the impacts of the alternative models on risk estimates. This stands in direct contrast to a procedure of first selecting a single conceptual model to simplify the analysis demands and then per- forming sensitivity analyses on that modelâs parameters. The latter ap- proach will only exacerbate the potential for controversies among scien- tists on the merits of the risk analysis results, and this, in turn, can only further deepen the widespread skepticism and distrust that members of the general public express for risk analyses. Sidebar 5.2: Sources of Uncertainty Uncertainty of mutual estimate increases as models based on labo- ratory experiments or limited field data are extrapolated to larger spatial and temporal scales. The uncertainty has three main sources: (1) data uncertainty; (2) scenario uncertainty; and (3) conceptual model uncer- tainty (Andersson and Grundteknik, 1999). Data uncertainty can enter the analysis at any point. There will be uncertainties about the composi- tion and form of the radioactive waste, the composition of the groundwa- ter, groundwater flow paths, and so forth. Scenario uncertainty will have more to do with the boundary conditions that will affect the site, such as climate change, recharge rates, and seismicity. Boundary conditions also change over time and space, for example, along the groundwater flow path. Conceptual model uncertainty is related to whether the relevant processes have been included in the models. As an example, a number of different conceptual models have been used to describe fluid flow and transport in the unsaturated zone (above the water table). In previous studies, initial estimates of groundwater travel timesâthat is, the time it takes for a contaminant to travel to some reference pointâhave de- creased by as much as four orders of magnitude as the conceptual mod- els have been changed (see Sidebars 6.1 and 6.2 in NRC, 2001b). Fi- nally, the theoretical and mathematical foundations of multiscale model- ing are themselves currently incomplete. In addition to the shift in emphasis toward exploring conceptual model uncertainty, risk analysis results may be viewed and communi- cated most effectively as relative indicators of potential impacts under
IMPEDIMENTS TO THE RISK-INFORMED APPROACH 123 different disposition alternatives. In fact, it would be misleading to con- sider such a risk assessment as yielding âquantitativeâ results, in the sense that the absolute results have reliable magnitudes. The uncertain- ties in the results remain large and increase with time. A risk assessment is quantitative only in the sense that it provides a numerical result, but the meaningfulness of the result is actually more qualitative. By treating the results as if they have quantitative significance, one can mislead deci- sion makers into concentrating on numerical comparisons rather than on evaluation of the adequacy of the strategy for the safe disposal of a par- ticular type of radioactive waste. Despite these enumerated limitations, risk assessment remains a powerful tool in organizing our understanding of the behavior of a physi- cal and behavioral system. However, a risk assessment is only one ele- ment in a properly made risk-informed decision (Apostolakis, 2004) and is not, by itself, a sufficient basis for determining that a site or strategy is âsafe.â 5.2 PUBLIC PARTICIPATION AND STAKEHOLDER ISSUES Participation by members of the interested public (commonly referred to as stakeholders) in risk analysis and risk managementâespecially those who will be directly affected by the decisions madeâhas changed consid- erably in the past two decades. At its most unsophisticated level, public participation has been thought to mean âeducatingâ the public about the âtruth and wisdomâ of expert analysis or informing the public about a final decision. As is discussed in greater detail in Section 5.4, when DOE was actively involved in decision making against the backdrop of the Cold War, it often addressed public participation as an afterthought or item of secon- dary importance. Under this management model, deficiencies in public participation were marked by the following common features: â¢ Stakeholders are presented with the risk assessment at the end of the process. â¢ The risk assessment is opaque and difficult to understand.
124 RISK AND DECISIONS ABOUT TRU AND HLW â¢ Those who present the risk assessment may not have been inti- mately involved with the work and are unable to answer impor- tant questions. â¢ Stakeholder comment periods are too short. â¢ Stakeholder comments and questions are dismissed or not ad- dressed adequately, particularly in ways that visibly affect the re- sults of the risk assessment. â¢ New knowledge often does not result in a change in the conclu- sions. â¢ There are so many âknobs,â that is assumptions and choices made during the analysis, that there is a perception that adjust- ments are made to arrive at âwantedâ results. It is not surprising, then, that citizen groups have asked the question, What environmental decision-making tools do we have other than risk assessment? (Raffensperger and Tickner, 1999). As the use of unilateral approaches has receded, and as risk assess- ment and risk management have evolved to recognize that stakeholder input is necessary early in and throughout the risk analysis process, fed- eral, state, and local agencies have adopted an analytic-deliberative model for risk-informed decision making. The analytic-deliberative model calls for continuous stakeholder input. A more complete descrip- tion of this model is set out in Appendix A. Although the important role of stakeholders in risk-informed deci- sion making is now widely acknowledged, past practices have made stakeholders suspicious and skeptical of the results of risk analyses, no matter how well conceived and executed. In a survey of environmental groups in the United States, Tal (1997) found that â[a]lmost unani- mously, environmentalists resent the technocratic, exclusionary nature of risk assessments that undermine democratic participation in local envi- ronmental decisions.â The risk analysis is seen as a political tool that uses science as a rationalization for decisions that have already been made. Agendas that are as much political as scientific, such as deregula- tion, are closely tied to risk analyses. Finally, the focus on risk inevitably emphasizes excess cancers as the final criterion or end point of the analysis, but not every stakeholder
IMPEDIMENTS TO THE RISK-INFORMED APPROACH 125 group shares this perspective or value system. In particular, American Indians are often concerned with the availability of natural resources, which are essential for the maintenance of a tribal life-style and commu- nity health (NEJAC, 2004). A group or a culture may legitimately place value on what it perceives as broader environmental and ecological con- sequences. As discussed previously in this report, risk-informed decisionmaking is a process for making choices, not merely an exercise for drafting a final stand-alone report on risk. Unless equal or greater attention is paid to the process, the final conclusions of the risk analysis may not be ac- cepted by the affected parties (NRC, 1996). Studies of stakeholder involvement and effective decision making suggest that there are ways to allay stakeholder concerns: â¢ Involve stakeholders at the earliest stages of initiating a risk as- sessment. â¢ Define the problem that is to be addressed by the risk analysis. â¢ Define the measures that are used to judge the final outcome of the analysis. â¢ Develop a process that shows and records the response to the im- portant issues that are raised by stakeholders. As risk practice has evolved, success in decision making has increas- ingly come to be characterized by success in meaningfully involving stakeholders in the risk assessment. Many models incorporate early and frequent stakeholder interaction. By using such a model, stakeholders may actually participate in the risk assessment (van den Belt, 2004), or they may be provided resources that allow them to hire their own team of experts who participate in the risk assessment or review it throughout the entire process (see Sidebar 4.5). Some readers will respond that the DOE process already involves stakeholders, uses risk assessment effectively, and reaches credible decisions through environmental impact statements under the National Environmental Policy Act (NEPA) and through the Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA) process. As noted in Chapter 4, the committeeâs recommended approach is compatible with the CERCLA process. Although the committee did
126 RISK AND DECISIONS ABOUT TRU AND HLW not examine the NEPA process in detail, it believes the risk-informed approach probably is compatible with that process too. It is a strength of the recommended approach that it does not require a major overhaul of the regulatory system in order to be implemented. But there is a difference between the recommended approach and that used in the environmental impact statements and the records of decision the committee examined. The difference is in how the parties go about implementing the approach. The risk-informed approach recommended in this report requires a cooperative approach to framing the decisions and carrying out the risk assessment. Even the appearance that DOE decides first and then does a pro forma risk assessment to support its decisionâa view the committee heard several times from some participants at its meetingsâdelegitimizes DOEâs actions in the eyes of some regulators and other interested parties. Even more aggravating to these parties, however, was the perception that DOE tries to circumvent CERCLA and the tri-party agreements. The committee noted in its site visits that there was substantial disaf- fection among stakeholders with DOEâs Risk-Based End States (RBES) project (DOE, 2003), which was initiated about the same time as this committee was formed. The RBES project is an outgrowth of the DOE Office of Environmental Management (EM) Top-to-Bottom Review (DOE, 2002d), which was undertaken when Assistant Secretary Roberson was appointed to run the EM. DOE announced that each site must develop a site-wide vision document describing for each cleanup activity an end state that is consistent with the anticipated future land use at the site. The guidance required sites to identify variances to seek from current plans, regulations, and agreements. Deadlines for the vision documents occurred over the span of the committeeâs site visits, and the committee heard presentations on the RBES project at each site. DOEâs announcement and initial implementation of the RBES pro- ject triggered negative responses from the U.S. Environmental Protection Agency (EPA), host states, local governments, American Indian nations, and environmental groups. DOE did not articulate well the problem it was simply trying to address by instituting the project, which led many outside DOE to suspect that its purpose was simply to enable DOE to do less cleanup. DOE developed the first drafts of the vision documents with virtually no input from outside. Thereafter, very little time was al- lowed for public input and even when deadlines were extended, people who were not brought on board at the beginning were reluctant to con- tribute to the project for fear of legitimizing it. The deadlines for the vi- sion documents were extended further and DOE acknowledged some of
IMPEDIMENTS TO THE RISK-INFORMED APPROACH 127 the mistakes it made in implementing the project. To try to move beyond the hostility DOEâs implementation of the project fomented, DOE worked with the National Governors Association to hold a meeting with many of the interested and affected parties. The meeting was held on Oc- tober 6 and 7, 2004. Many at the meeting said that the RBES project had undermined trust between DOE and the other parties, and DOE represen- tatives acknowledged errors made in implementing the policy. At the same time, DOE still sees the RBES project as valuable and wants to pursue it. At the time of preparation of this report, it was not clear whether DOE had created enough buy-in from other parties to make the project effective. If DOE is to implement successfully a risk-informed process for re- questing exemptions from requirements for deep geologic disposal of HLW and TRU waste, DOE must strive to avoid these pitfalls. Some stakeholders may view the process with suspicion, charging that DOE is changing the rules to make legal an unacceptable outcome. The goal of the risk-informed process is to find mutually acceptable outcomes and, where necessary, change the rules to make an illegal, mutually accept- able outcome into a legal, acceptable outcome. Failure to follow dili- gently and seriously the stakeholder involvement aspects of the process in DOEâs first attempt to implement the recommended risk-informed process not only would engender the same degree of stakeholder con- cern, but would likely doom the chances of the committeeâs proposed risk-informed exemptions process to be used at all. The step-by-step process in Chapter 4 was designed with this concern in mind. 5.3 DECISION-MAKER ISSUES Once a risk analysis is complete it becomes an important part of the final decision. However, even in the best circumstance, some disagree- ments and uncertainty inevitably will remain. One cannot expect that every scientific uncertainty will have been eliminated or that all stake- holders will accept the results of the analysis. Moreover, science and sci- entific analysis cannot answer every question necessary to a decision. Policy, political, and value choices remain. These questionsâsometimes called âtransscientificâ (Weinberg, 1971, 1972)âare informed by scien- tific analysis but are not resolved by such analysis. Perhaps the best ex- ample of a transscientific question is the level of acceptable risk. Techni- cal analysis can provide estimates of risk, describe timing and receptors
128 RISK AND DECISIONS ABOUT TRU AND HLW of risk, and point out uncertainties, but it cannot identify the level of risk that society should tolerate. Transscientific decisions such as the level of acceptable risk are among the most difficult facing public officials in Congress and adminis- trative agencies. They can look for guidance to the risks apparently or explicitly accepted in other activities (assuming they are analogous), and they can look at cost and other factors. Ultimately, however, it is a bal- ancing of many factorsâscientific and nonscientificâthat results in a decision (Jasanoff, 1990; Wagner, 1995). Sometimes the technical analy- ses all seem to point in one direction, making the decision easier (or at least easier to justify), but often this is not the case. It is therefore to be expected that public officials will look to science in the hope that it will give a clear direction for hard choices. In addition, science has an allure, as one writer puts it, of precision and objectivity that can provide much needed support for politically unpopular decisions (Applegate, 1995; Hornstein, 1992). (The cynic may find this use to be no more than seek- ing political cover; but more charitably, it is perfectly reasonable to seek a firm basis for decisions that hurt some peopleâs interests.) In both cases (that is, hoping for clear direction and looking for objective justification), however, the tendency is to expect or claim more than science can le- gitimately deliver. The National Research Council has repeatedly warned of the overre- liance on science in the context of risk analysis. As early as the Red Book (NRC, 1983), it called for a distinction, as far as possible, between the risk- and impact-assessment aspect of the process and the policy- oriented, risk-management aspect (NRC, 1983, 1994a, 1996). Scientific analyses should take care (1) to disclose fully limitations and uncertain- ties in their conclusions, and (2) to distinguish carefully between science 5 and nonscientific judgments. Ultimately, a decision maker cannot avoid making hard decisions in the face of the uncertainty in the analysis or continued disagreements among the stakeholders. The dilemma has been well summarized by Her- rick and Sarewitz (2000): 5 This is not to say that scientists should shrink from judgments or recom- mendations that are not purely scientific in character. On the contrary, it is im- perative that the scientific community be actively involved in the important is- sues of the day, especially those with a technical aspect. Rather, scientists should be clear as to the basis forâand, if appropriate, the limitations ofâtheir judgments.
IMPEDIMENTS TO THE RISK-INFORMED APPROACH 129 Unreasonable expectations about the nature and character of scientific knowl- edge support the widespread political assumption that predictive scientific as- sessments are a necessary precursor of environmental decision making. All too often, the practical outcome of this assumption is that scientific uncertainty be- comes a ready-made dodge for what is in reality just a difficult political deci- sion. Interdisciplinary assessments necessary to address complex environmental policy issues invariably result in findings that are inherently contestable, espe- cially when applied in the unrestrained realm of partisan politics. (Italics are the committeeâs.) Although the âinherently contestableâ quality of a risk or perform- ance assessment cautions against overreliance on science, it does not jus- tify disregarding the results of such an analysis. A rigorous scientific analysis provides a logically coherent organization to the many factors in a decision, provides essential information for the decision, and clarifies areas of remaining uncertainty and remaining (nonscientific) choices. Scientific analysis, therefore, deserves to be a core part of decision mak- ing on questions of public significance, but it will not ultimately relieve the public decision maker of the weighty responsibility of hard choices. To continue a theme in Chapter 4, the scientific information captured in a risk assessment, or any risk assessment process, is a tool for decision makers that is most useful when taken as an integrated whole. Summariz- ing an entire risk process in a single point estimate or even risk range is not helpful for decision makers or those who need to understand how a decision was made. 5.4 ISSUES OF INSTITUTIONAL CULTURE The above discussion of scientific issues has used examples from some of the most difficult problems that DOE faces, such as the pro- posed Yucca Mountain repository for HLW. Here, the problem is with predicting the behavior of a complex natural and engineered system ex- pected to contain long-lived radionuclides over extremely long time pe- riods. The Yucca Mountain problem is being addressed by use of a com- plex integrated performance assessment model, but according to a white paper of the Board on Radioactive Waste Management, this complexity 6 is viewed as virtually incomprehensible to nonspecialists (NRC, 1999). 6 âThere are many indications that publics neither understand nor trust the expert community on radioactive waste issues. For the non-specialists, the array
130 RISK AND DECISIONS ABOUT TRU AND HLW DOE faces many other technical problems of a simpler nature. These less complex problems would appear to be more amenable to efforts to gain public confidence by virtue of their being addressed by a simpler risk-informed process. To the committeeâs knowledge, however, even these simple problems are not currently being addressed by anything re- sembling the risk-informed process recommended here. It seems that nearly every new assistant secretary for the Office of Environmental Management has come into office with the expressed de- sire to reform the process in order to make it focus more on risk and achieve cleanup âfaster, better, and cheaper.â The committee is also im- pressed by the number of other committees (see Appendix B) that have recommended virtually the same thing: the implementation of a risk- based or risk-oriented process to focus attention and money on the prob- lems that really do require action and not to spend unnecessarily large amounts of money on problems that are not, in fact, a source of signifi- cant risk to either humans or the environment. Some of these committees were specifically charged at the time to help DOE determine how to use the risk assessment and management process effectively. DOE has asked other groups to examine the environmental management problem and those groups have come up with similar recommendations. In spite of these multiple recommendations and the apparent open- ness of persons high within the DOE management structure, and in spite of this committeeâs attempts to find examples of a risk-based or risk- informed decision process at work, it appears that, with rare exception, the process of risk assessment is not being utilized effectively by the na- tion. Virtually all of the so-called risk assessment activities presented to this committee were being performed principally because they were re- quired by the U.S. EPA under the CERCLA or the Resource Conserva- tion and Recovery Act (RCRA). (These requirements typically followed on from the listing of the various sites on the National Priorities List [NPL].) It is the committeeâs impression that these risk assessment ac- tivities were not being regarded within DOE as particularly significant or effective input into the decision-making process. While in public presen- tations some site personnel stated that risk currently is the driving factor for decisions through the regulatory process, in private some of the same people indicated that factors other than risk fixed many decisions. For example, many decisions were made in the federal facility agreements, of applicable scientific data and proposed methods is so complex as to be virtu- ally incomprehensibleâ (NRC, 1999).
IMPEDIMENTS TO THE RISK-INFORMED APPROACH 131 which were established before most of the risk assessments were con- ducted. Thus, one of the fundamental questions that the committee asks is Why is risk assessment not already one of the foundations of DOEâs en- vironmental management program? What are the impediments? Some members of this committee have participated over periods of approxi- mately 15 years on other committees or review groups examining these questions. None of the members of this committee has actually worked for DOE, so committee members can examine these fundamental ques- tions only as outsiders with the information furnished from DOE, regula- tory agencies, and members of the public. Some or perhaps many im- pediments may lie beyond DOE in the larger institutional or societal structure. Although U.S. EPAâs use of risk assessment in some of its programs has been criticized severely, U.S. EPA has, in comparison, used the risk- assessment process relatively effectively at many of the sites it regulates under RCRA and CERCLA, and at WIPP. Risk analyses are accepted as an important part of the process of regulatory decisions and cleanups conducted by the U.S. EPA. Russell (2000) attributes U.S. EPAâs use of risk assessment to the efforts of Administrator William Ruckleshaus to change the U.S. EPAâs culture from a ââ¦view of its goal of eliminating pollution to one of assessing and then managing risk in order to reduce harm to people and the environment.â DOE has never had a similarly influential risk management champion. Some of the fundamental differences between U.S. EPA and DOE relate to the differences in the history and structure of the two organiza- tions. The U.S. EPA is a relatively new organization with a clear, nar- rowly defined mission. DOE has its roots in the Manhattan Project, and DOE has a variety of different missions, some of which may inherently conflict with each other. The organization of DOE (actually its predeces- sor agencies, the Manhattan Engineering District of the Army Corps of Engineers and the Atomic Energy Commission) was unique from its very first days. The first mission of the original predecessor agency was to develop and build a few deliverable nuclear weapons. This was a sophis- ticated scientific and engineering problem for which the members of the usual government bureaucracies had no experience or expertise. Thus, from the beginning there was an unusual dependence of DOE upon pri- vate contractors of one kind or another. Urgency and secrecy were also fundamental to the early days of DOE. The urgency led to unprecedented steps to achieve the fundamental mission with little importance attached to environmental considerations, although considerable attention was
132 RISK AND DECISIONS ABOUT TRU AND HLW always focused on the health and safety of workers. The secrecy led to islands of knowledge that were known in detail only to persons outside of the main bureaucracies. Over time, the DOE management style also came to depend on field organizations. Some of these organizations, such as the Richland (Han- ford), Oak Ridge, and Savannah River Field Offices, became very pow- erful. Persons at these three field offices oversaw large enterprises, run by management and operations (M&O) contractors, whose financial con- tributions to the local and regional economies became very substantial, if not dominant. This economic influence did not escape the notice of local and regional politicians, and each major DOE site inevitably built up its own base of political influence based on its economic power. Enormous changes have occurred within DOE during the last two decades. The major mission, the production of special nuclear materials and the fabrication of nuclear weapons, has virtually been eliminated. Some of the smaller DOE sites have been, or are being, decommissioned and dismantled. At about the same time that this major structural shift within DOE was occurring, the powerful trend of environmental protec- tion and/or remediation was developing. As the production of special nuclear materials was winding down, the concern about (and the legal liabilities from) the residues of contamination from decades of urgent production came to the fore. This concern led to the establishment at DOE of an Office of Environmental Management and, at each DOE site still run by M&O contractors, of a new organization devoted to environ- mental management or cleanup. At many of the remaining DOE sites, this environmental management organization was about the same size and had about the same budget as the former production organizations. In spite of this fundamental shift, DOE tried to manage as it had pre- viously by using a decision-making process that was rather closed. One of the sea changes that eventually swamped this previous methodology was the loss of DOEâs exclusive self-regulation. One of the early changes was the empowerment of U.S. EPA to regulate airborne emis- sions from DOE facilities under the Clean Air Act; eventually, it became clear that the U.S. EPA had the power to regulate DOE under the terms of other environmental laws, notably CERCLA and RCRA. The real meaning of these changes was brought home to DOE by the realization that U.S. EPA had the power to levy fines and even enforce personal criminal penalties. This was illustrated dramatically by the raid by Fed- eral Bureau of Investigation and U.S. EPA Enforcement Police on the Rocky Flats Plant; one result of the raid and other investigations was criminal conviction of M&O officials at the Rocky Flats Plant.
IMPEDIMENTS TO THE RISK-INFORMED APPROACH 133 During this period of the late 1980s and early 1990s the possibility of DOE sites being listed on the NPL was viewed by personnel at most sites as threatening and negative. After the first DOE site was listed, however, it received a large amount of funding to deal with its environmental prob- lems. This flow of funds was not lost on environmental protection per- sonnel at other sites, and a form of subtle competition soon evolved in which listing on the NPL was accompanied by receiving larger amounts of money. Eventually, most significant DOE sites become listed on the NPL, and while fines have been assessed, no additional high profile criminal convictions have occurred. An additional activity carried out during this period was the negotia- tion of triparty compliance agreements among DOE, U.S. EPA, and local (usually state) regulators. These agreements typically contained binding statements on what would be done, along with a time scale over which the specified work would be done. Persons negotiating these agreements on behalf of DOE were concerned mainly with legal, managerial, and political issues. A result was that many of these agreements contained commitments to perform work that was technically or economically in- feasible and might pose substantial risk to workers using present tech- nologies. This eventually led to the need to renegotiate many agreements and to some loss of credibility for all participating agencies. Another im- portant aspect in terms of the deliberation of this committee was that these agreements were not based on an analysis of the risks posed by the considered wastes. In some cases, agreements were reached to perform cleanups that not only were expensive, but also would have resulted in little or no reduction in risk (DOE, 2002d). In essence, this process of negotiating compliance agreements without thorough and detailed con- sideration of risk, cost, or technical feasibility represents an enormous lost opportunity for the nation to insert discipline into the environmental management process. There have been several attempts by Congress and high-level DOE management personnel over the last several years to insert such disci- pline at DOE sites; these activities are discussed in Appendix B. Such activities have not been particularly successful and may have been re- sisted by a variety of ad hoc consortia at the various sites. Members of such consortia might include persons from DOE field offices, manage- ment and operations contractors, state and local regulators, environ- mental groups, state governors, and members of the U.S. Congress. A strong theme binding such diverse groups together has been the thou- sands of jobs and billions of dollars at stake. The emotions and issues are
134 RISK AND DECISIONS ABOUT TRU AND HLW similar to those that periodically surface when the Department of De- fense contemplates closure of military bases. Another important theme is that local persons feel that top-down im- position of discipline, whether through risk assessment or other means, would be a violation of the content and spirit of the agreements that have been negotiated. In some cases, risk-based approaches are also seen to be in conflict with national or local laws. A persistent problem is that the nation has permitted and perhaps even encouraged a lack of focus on cleanup as a result of viewing some site cleanups as regional economic entitlement programs (see, e.g., Greenberg et al., 2002; Probst and Lowe, 2000; Russell, 1997). DOE has adapted itself to the environment in which it operates. The complexity, constraints, and politicization make these problems very challenging and bear some responsibility for the situation. Stated bluntly, there is a conflict between the wishes of the nation to limit the costs and time of the DOE cleanup program (and the attempts of the various assistant secretaries of energy for environmental management to implement these wishes) and the incentive to persons at local sites to ensure a large and sustained influx of funds. This is a fundamental institutional barrier to the use of a risk-based approach to environ- mental management at DOE sites. Local persons have frequently stated that they could be comfortable with the use of risk assessment to allocate priorities among subsites (or âoperable unitsâ in the official jargon) at their overall site, but they state a strong resistance to the use of a risk-based approach, or any other approach, to set priorities among sites. Such statements are apparently motivated by fears of the possible loss of funding for their site in favor of another site. These fears may be accompanied by a sense that if funding is lost, cleanup programs could be curtailed or even abandoned, leaving cleanups un- resolved and local stakeholders totally marginalized. The problems of DOE legacy wastes are most severe at Hanford, Idaho, and Savannah River. There are two important reasons for this. The more obvious is that these sites have most of the wastes within the system. The other less obvious factor is that these three sites were pur- posely located in rural areas to place their hazards farther from large populations. The DOE contribution to the regional economy in these areas has been dominant. Thus, the abrupt withdrawal of DOEâs con- tribution would be catastrophic to these local economies. However, a short-term view is dangerous and only postpones the inevitable. Rus- sell (1997) has argued that these communities are entitled to transi- tional economic assistance, but that hijacking the DOE environmental
IMPEDIMENTS TO THE RISK-INFORMED APPROACH 135 management program is not a productive avenue to achieve this rea- sonable goal. Rather, Russell has argued that the cleanup and transi- tional assistance programs should be divorced. Probst and Lowe (2000) have made similar arguments. In summary, the institutional barriers to the implementation of a risk- based approach to cleanup at DOE sites have been demonstrated to be effective and deeply entrenched. Although Congress has repeatedly criti- cized the DOE process, members of Congress representing the individual sites have sometimes contributed to the problem by the long-established method of introducing special legislation in favor of a particular site. Nearly every new assistant secretary for environmental management has tried to reform the system, but it does not appear that any assistant secre- tary has sufficient power to impose discipline on the system. Finally, the coalitions of interests at each site are extremely resistant to any priority- setting activity that might remotely result in a loss of funding for that site. These factors, coupled with the legally binding agreements that have been made at each site, present a formidable barrier to the implementa- tion of the risk-based process advocated in this report. Is there a solution? The answer is not clear, but it does not seem likely that a solution will be forthcoming without strong instructions from the Congress to the highest level of management at DOE. The DOEâs Top-to-Bottom Review Team made the following state- ment, âThe results of the Teamâs review make clear that there is a sys- temic problem with the way EM has conducted its activities: the EM programâs major emphasis has been on managing risk, rather than actu- ally reducing risk to workers, the public, and the environmentâ (DOE, 2002d, pp. ES-1-ES-2). As a consequence of this conclusion, DOE redi- rected its program toward reducing the long-term risks posed by legacy wastes with initiatives such as closing tanks, pursuing supplemental technologies for tank waste processing, and pursuing risk-based end states. However, these efforts have apparently run afoul of EMâs lack of use of risk management in the sense that it is normally understood (see NRC, 1983)âthat is, as a process that defines the character and extent of the problems to be addressed, helps identify options to manage risks, and engages regulators and other stakeholders to balance risk considerations with other factors. As subsequent events (e.g., successful litigation con- cerning on-site disposition of tank heels; a RBES process that appears to be effectively stalled) have amply demonstrated, the absence of a real risk management process has been a substantial impediment to the cleanup and consequent risk reduction sought by EM.
136 RISK AND DECISIONS ABOUT TRU AND HLW Thus, the key ingredient of a needed program is precisely risk man- agement, as defined by an earlier committee (NRC, 1983). Recognition must be given to the fact that some wastes cannot be cleaned up, that transgenerational risks will occur, and that long-term stewardship will be required into the indefinite future. These are the costs for this nationâs choices during World War II and the Cold War. Finding 10: The DOE risk assessments and decision processes exam- ined by the committee do not exhibit all of the characteristics of an effective and credible risk-informed decision-making process, listed in Finding 8. Other bodies have made similar recommendations on how DOE should incorporate risk into environmental decision mak- ing, and DOE has made progress, but institutional factors appear to have interfered and perhaps undermined attempts to implement these approaches. This implies that changes at DOE are needed to address internal and external impediments to the risk-informed ap- proach. In its site visits and after, the committee requested that DOE present its best examples of risk assessment informing waste disposition or cleanup decisions. Through DOEâs presentations to the committee and the committeeâs review of documents, the committee examined many risk assessments and decision processes. DOE and its contractors have performed technically complex risk assessments, and in many cases have performed risk assessments as part of regulatory processes that lead to cleanup decisions with stakeholder input. Yet the cases examined by the committee do not meet the needs identified and described in this report for the following reasons. The complex analyses were not decision ori- ented and were not carried out in a transparent manner needed for mean- ingful participation by those outside DOE. The actions supporting regu- latory decisions in many cases also were lackingâthe steps in the proc- esses appeared to have been performed simply to meet procedural re- quirements and most did not appear to have taken the kind of cooperative approach that the committee sees as essential to reach credible decisions and to foster buy-in by other relevant parties. That the risk assessments examined by the committee do not exhibit all of the characteristics of an effective and credible risk-informed deci- sion-making process does not imply that DOE has been derelict. These are technically difficult cleanup problems being addressed in a complex political and social environment. DOE has stabilized into safe, although
IMPEDIMENTS TO THE RISK-INFORMED APPROACH 137 temporary, conditions the dangerous wastes and facilities at its sites, and in most cases has an enviable safety record in its cleanup program. Working toward effective and credible risk-informed decisions on these issues is very difficult. Further, many of the risk assessments examined by the committee were addressing smaller although significant problems, and so may not have warranted the effort recommended in this report. Also, the risk assessments were not necessarily aimed to fill the role de- scribed in this report. But on the latter point, the committee notes that numerous studies summarized in Appendixes A and B make recommen- dations consistent with those made in this report on how to incorporate risk into environmental decision making. DOE has made progress, but approaches such as the one recom- mended by the committee still have not permeated DOEâs decision- making apparatus. It appears that institutional factors both inside and outside DOE have impeded attempts to implement risk-informed ap- proaches. These factors include a tradition of internal rather than open decision making, incentive structures that favor distorting or ignoring risk, and a public wariness or mistrust of DOEâs use of risk assessment to justify proposed actions. The committeeâs role is to help DOE to bring the best practices to bear on the challenges DOE is addressing on the nationâs behalf. DOEâs difficulty in adopting risk-based or risk-informed approaches recommended previously by other committees and observers implies that DOE needs to make changes. Moreover perhaps changes are needed more broadly in the nationâs approach toward managing risks at DOE sites. Recommendation 5: To address the challenges of implementation and acceptance, DOE should form an authoritative, credible, and reasonably independent group to revamp the way it goes about im- plementing risk-informed approaches applied to waste disposition decisions. These are enormously complex problems with numerous parties involved and a great deal of institutional inertia (as evidenced by un- successful previous attempts to change). The committee sees a need to break out of old approaches. To this end DOE needs an action-oriented group that provides advice and identifies alternatives, but also assists with implementation and draws in major stakeholders to get buy-in. The group must be credible, and to be credible the group must be au-
138 RISK AND DECISIONS ABOUT TRU AND HLW thoritative on the issues it addresses and independent so as to be unbi- ased and free of conflicts of interest. Before implementing this rec- ommendation, it would be useful to consider the extensive experience of a variety of federal agencies with outside advisory committees, in- cluding the committeesâ roles and effectiveness.