5
Impediments to a Successful Application of the Risk-Informed Approach

The risk-informed approach outlined in Chapter 4 can be useful in an exemption process, but the committee is concerned that this process not leave the impression that the risk-informed approach will be easy to perform successfully. In fact, the committee wishes to emphasize that risk analysis cannot be performed in “cookbook fashion” if it is to be meaningful and useful in decision making. In this chapter, the committee reviews some of the unavoidable and irreducible complexities in performing risk analysis and other important sources of concern with risk assessments. In order to apply the risk-informed approach successfully, the Department of Energy (DOE) must be fully aware of the complexities that a good risk analysis will encounter and be prepared to address them directly with exceptional intellectual and analytical effort. Additionally, DOE must accept that the results of the risk analysis should be only a part of a decision-making process, not the sole basis for the final decision.

The first section of this chapter discusses the limits and uncertainties of science in modeling long-term environmental processes and future risks. These complexities need not prevent progress toward making sound decisions about the disposition of waste streams, as long as the uncertainties are acknowledged, addressed where possible, and incorporated into



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 113
Risk and Decisions: About Disposition of Transuranic and High-Level Radioactive Waste 5 Impediments to a Successful Application of the Risk-Informed Approach The risk-informed approach outlined in Chapter 4 can be useful in an exemption process, but the committee is concerned that this process not leave the impression that the risk-informed approach will be easy to perform successfully. In fact, the committee wishes to emphasize that risk analysis cannot be performed in “cookbook fashion” if it is to be meaningful and useful in decision making. In this chapter, the committee reviews some of the unavoidable and irreducible complexities in performing risk analysis and other important sources of concern with risk assessments. In order to apply the risk-informed approach successfully, the Department of Energy (DOE) must be fully aware of the complexities that a good risk analysis will encounter and be prepared to address them directly with exceptional intellectual and analytical effort. Additionally, DOE must accept that the results of the risk analysis should be only a part of a decision-making process, not the sole basis for the final decision. The first section of this chapter discusses the limits and uncertainties of science in modeling long-term environmental processes and future risks. These complexities need not prevent progress toward making sound decisions about the disposition of waste streams, as long as the uncertainties are acknowledged, addressed where possible, and incorporated into

OCR for page 113
Risk and Decisions: About Disposition of Transuranic and High-Level Radioactive Waste decision-making deliberations. However, doing so may require a change in one’s perspective about the goals of the risk analysis. The goal of the risk analysis in the risk-informed approach described in this report is to inform DOE, regulators, American Indian nations, local governments, and the public of the full range of outcomes that might occur by accounting as much as possible for the limitations of our knowledge of the systems involved. The analysis should inform the process participants of the ways in which undesirable outcomes may occur and provide insight into how to better engineer for or manage those risks. The committee cautions against using risk analysis to generate a specific quantitative estimate of risk (or risk ranges) that would then be deemed “acceptable” or “unacceptable” for the purpose of arguing for an exemption. Such strictly quantitative interpretations of risk estimates likely will fail in the face of the complexities and unknowns of the science underlying them. The committee describes a structured process, but does not provide decision criteria. The purpose of the risk analysis is not to produce a de facto decision, but rather to inform. Sections 5.2 to 5.4 highlight a range of other issues that also could be impediments to the successful application of a risk-informed approach. These include stakeholder and decision-maker concerns and impediments in institutional culture. Again, the process recommended in this report and the set of steps outlined in Chapter 4 are designed to address and manage these concerns. Nevertheless, in this chapter each issue is discussed individually to emphasize that DOE must grapple directly with these complexities if the proposed risk-informed approach is to be applied successfully. The National Research Council (NRC, 1996) provides a more in-depth exploration of these and other issues. That reference can serve to supplement the material in this chapter.1 1   The “analytic-deliberative process” described in NRC (1996) is consistent with and, indeed, a foundation for the risk-informed approach recommended here. This report describes an analytic-deliberative process that has been tailored specifically to support decision making on disposition of high-level and transuranic wastes, and it focuses its discussion on those concerns that appear to present the greatest impediments in this particular application.

OCR for page 113
Risk and Decisions: About Disposition of Transuranic and High-Level Radioactive Waste 5.1 RISK MODELING ISSUES At the core of the risk-informed exemption process recommended in this report is the estimation of the risk that results from different disposition options associated with a given waste type at a specific location. In most cases, this will be an analysis of the risk posed by waste, nominally classified as HLW or TRU waste, that is left at a site rather than being removed and disposed of in a geologic repository, as currently required by law. Once calculated, the risk can be weighed against other relevant factors, such as cost and worker impacts, and this information becomes a part of the risk-informed decision-making process. Properly done, risk assessment is a powerful tool for systematically organizing the information and understanding the behavior and impacts of radioactive waste at a particular location. Although the “risk” associated with disposition of different types of radioactive waste may be judged initially by the properties of the waste (e.g., amount of radioactivity, chemical form, toxicity, half-life, complexity of composition), the actual risk assessment that is used to evaluate a disposition strategy will involve models of the interaction of the waste components with the near-field and far-field environments, including interactions with engineered barriers or other containment mechanisms and the natural geochemical and hydrological systems. The site analysis provides the basic data used to calculate the risk that results from the many different exposure pathways in the biosphere. Thus, the risk assessment is composed of a series of coupled models used to describe the relevant processes. The quality of the risk analysis that would inform exemption decisions will rest on the quality of the data and on the scientific understanding of the underlying physical, chemical, and behavioral phenomena, and how well that understanding is reflected in the analysis. Risk-informed approaches have been endorsed by several previous committees (e.g., NRC, 1990, 1996) to support decision making on complex social policy problems. However, actual applications of risk assessment by DOE to its waste management issues have brought out the difficulties of achieving scientific and technical consensus.2 Further, risk assessment in general is not accepted by everyone (Tal, 1997). The 2   An illustration of the challenges involved in such a risk assessment when long time horizons are in question is the experience with the performance assessments of the proposed geologic repository at Yucca Mountain for spent nuclear fuel and high-level waste (e.g., Budnitz et al., 1999; Long and Ewing, 2004).

OCR for page 113
Risk and Decisions: About Disposition of Transuranic and High-Level Radioactive Waste committee is aware of these difficulties, and its recommendations have been prepared with these concerns in mind. In particular, the committee has recommended a risk analysis process that emphasizes sensitivity analysis to understand how lack of knowledge can affect decisions on waste disposition, rather than blindly adding model complexity without first demonstrating its necessity or value to making a better decision. Additionally, the committee has recommended a risk-informed process rather than decision making based solely on the quantitative output of the risk analysis itself.3 This being said, it is useful to explore the objections to risk analysis that have been expressed and their historical underpinnings. The following discussion emphasizes risk analysis of geologic systems, but similar concerns relate to other environmental systems and their pathways, such as air and surface contaminant exposures. Scientists may object to the results of a risk analysis because the models used may not capture the actual system behavior of the radionuclides at a specific site. The calculation of risk over long periods of time or to future generations is especially problematic. The agency or institution responsible for completing the risk assessment may have a real or perceived bias that compromises the credibility of the results. Stakeholders may be excluded from the process and skeptical of results that become the basis for changing previous agreements, particularly when the decision is to leave the waste in place. Decision makers may be baffled by the complexity of the analysis and uncomfortable with equivocal results that, in fact, may have a high uncertainty. Rechard (1999) reviewed the history of the use of risk analysis and performance assessments of geologic systems. Performance assessments, and their supporting methodologies, were first used to evaluate the reliability of nuclear weapons delivery systems. In the early 1970s, probabilistic risk assessments (PRAs) were used to evaluate the risk and consequences of nuclear power reactor accidents. The WASH-1400 report by Rasmussen and colleagues was the first comprehensive, probabilistic analysis of the health risks from a large and complicated technological system, a nuclear power plant (Rasmussen, 1975). The crossover point for the use of risk assessments for geologic systems came in 1976 when the Energy Research and Development Administration sponsored two 3   This philosophy is also central to the analytic-deliberative process recommended in the NRC report (1996). The committee believes that some of the difficulties in DOE’s risk assessment track record are attributable to a failure to reflect such a philosophy in practice.

OCR for page 113
Risk and Decisions: About Disposition of Transuranic and High-Level Radioactive Waste workshops that brought together nuclear engineers who were conversant with the recently developed PRA technique and earth scientists involved in modeling geologic systems. Much of the faith in risk assessments of geologic systems is based on a belief that the methods of risk analysis, as developed for analyzing nuclear reactor safety, can be used to analyze geologic systems over long periods (Garrick and Kaplan, 1995). However, as early as 1978, John Bredehoeft and colleagues had clearly outlined the challenges to geosciences in attempting to model the long-term behavior of a geologic system used for the disposal of radioactive waste (Bredehoeft et al., 1978). One of their conclusions is worth quoting: In summary, predictive models are an essential step in the selection and implementation of a radioactive-waste repository and a radioactive-waste treatment system. They are invaluable tools for analyzing the problem and for identifying factors that are likely to have the greatest effect on radionuclide migration. However, some components of the models are inherently unpredictable at present and are likely to change at different times. In no sense, therefore, will these models give a single answer to the question of the fate of radioactive waste in geologic repositories…. A quarter of a century later, there is an ongoing debate about the value and limitations of risk assessments (Ewing et al., 1999). At the heart of the discussion are the clear limitations in the predictive capabilities of models in principle (Oreskes et al., 1994) and in practice (Oreskes and Belitz, 2001). Models of radioactive waste in geologic systems are inevitably limited for the following reasons: They are simplified, hence, incomplete descriptions of the system. The governing equations of the models may not provide unique solutions. They cannot be calibrated to data sets that include the relevant spatial and temporal scales to be modeled. Boundary conditions may change over time in ways that are unforeseeable. Relevant, critical processes may have been omitted from the analysis.

OCR for page 113
Risk and Decisions: About Disposition of Transuranic and High-Level Radioactive Waste Each of these five issues is inherent in any risk assessment and may never be completely eliminated by additional work or research. The fifth issue, conceptual model uncertainty, is probably the greatest source of uncertainty and yet the most difficult to estimate or characterize (NRC, 1995b, 2001b). The committee returns to the issue of conceptual model uncertainty below. In addition to the fundamental limitations in the use of models for risk assessments, there is ample opportunity for additional, but more manageable, sources of error to enter into the analysis, which add to the uncertainty of conclusions that may be drawn from it: Input data may be wrong. The governing equations may have been solved incorrectly. Parameter distribution functions may be wrong. Estimated probabilities of critical events may be wrong. Analysts must make judgments about how to set parameters and how to characterize their variability or uncertainty. A known difficulty in this step is that such judgments are subject to a number of heuristic biases, the net effect of which is that the true uncertainty is frequently underestimated (Morgan and Henrion, 1990; NRC, 1996). Analysts may sometimes make judgments that are subconsciously (or even consciously) anchored in values that can systematically drive the results towards their personal prior view of the risk endpoint. There are methods to correct for judgmental biases (Morgan and Henrion, 1990), and it is important that they be applied in risk analysis. Other elements of the analysis process in Chapter 4 also can help reduce these errors, such as quality assurance programs, review committees, public input, and having an external authority to judge the merits of the evidence in favor of an exemption. Even when the above issues are addressed carefully and the input data are as good as the current state of knowledge allows, uncertainty in the analysis will remain. The final risk assessment will still be model dependent, and conceptual model uncertainty will be a concern. This uncertainty will be exacerbated by the fact that the risk assessment will entail a coupling of many models (e.g., waste form degradation, geochemical interactions and speciation, hydrology, biosphere pathways), and the connections between these models may have complicated feedback rela-

OCR for page 113
Risk and Decisions: About Disposition of Transuranic and High-Level Radioactive Waste tions. New emergent behaviors that come about through complex interactions of different components of nonlinear systems may be extremely difficult or nearly impossible to predict. This means that understanding of behavior hinges upon resolving complexity, a feature often found in nonlinear systems, yet there are no general rules for recognizing and resolving such complexities. Thus, despite great advances in science during the past 50 years in describing, modeling, and even predicting the behavior of systems, the types of errors described above remain substantial. Examples of remaining difficulties in modeling hydrological and geochemical systems are provided in Sidebar 5.1; but similar issues are important for all of the physical, chemical, and behavioral systems that must be considered in a complete risk assessment, especially if the analysis will attempt to project outcomes thousands of years in the future. In sum, uncertainty is an inherent aspect of risk assessment, emanating from several sources (see Sidebar 5.2). The challenge is not to diminish the role of uncertainty, but rather to properly and fully reflect it in information that decision makers will be asked to consider. Often analysts believe that this means that the risk assessment must be performed probabilistically. However, probabilistic analysis can increase the complexity and opacity of the analysis, yet distract from the most important form of uncertainty: conceptual model uncertainty.4 NRC (1996, pp. 113–114) acknowledged the limitations of probabilistic uncertainty analysis in the face of this general source of uncertainty: 4   For example, a Monte Carlo analysis may incorporate explicit representation of uncertainty on dozens or hundreds of parameters of a model, yet never address the possibility that the model itself could be inappropriate. A more useful “uncertainty analysis” might be accomplished with a just few model runs from two alternative conceptual models.

OCR for page 113
Risk and Decisions: About Disposition of Transuranic and High-Level Radioactive Waste Sidebar 5.1: Uncertainties in Hydrologic, Geologic, and Geochemical Components of Modeling The heterogeneous, near-surface earth environment has been termed the “critical zone,” because it is the region where the hydrosphere, biosphere, geosphere, and atmosphere interact with one another and with the human sphere in a highly complex manner (see, e.g., NRC, 2001c). The complexity of the critical zone presents special challenges to the development of long-term predictive models. During the past 50 years, earth scientists have made substantial strides in describing, modeling, and even predicting the behavior of critical zone systems. Nevertheless, our knowledge, understanding, and ability to model critical zone processes are strained by the demands of risk assessments extending for many thousands of years into the future. Modelers are trying to predict waste form and contaminant behavior at DOE sites, including some of the most complicated and as yet poorly understood types of critical zone hydrogeologic systems, including vadose zones, regions of groundwater-surface-water interactions (including the poorly understood hyperheic zone, which is the geologic material immediately surrounding and underlying rivers and streams), arid regions, and fractured flow regimes. Konikow and Bredehoeft conducted postaudits of a number of hydrologic systems, comparing predicted behavior to actual behavior over periods of decades (Konikow, 1986, 1992, 1995; de Marsily et al. 1992; Konikow and Bredehoeft, 1992; Bredehoeft and Konikow, 1993). A particularly important conclusion was that for models based on a history match (i.e., a calibration to previous hydrologic data), the predictive capability diminishes rapidly for periods longer than that of the historical data (Bredehoeft, 2003). This limits confidence in the predictive capabilities of hydrologic models to no more than some hundreds of years. One might argue that for some of the most highly complex and poorly understood hydrogeologic settings, the limits of confidence could be considerably shorter. Geochemical systems are subject to the same types of uncertainty as hydrologic systems, particularly conceptual model uncertainty (Bethke, 1996; Nordstrom, 2004). Areas of ongoing research, such as the role of nanoparticles, bacteria, and organic matter in the fate and transport of radionuclides, highlight some of the current challenges in understanding geochemical interactions. Geochemical models may have non-unique solutions (Bethke, 1992), can be highly sensitive to very small changes in fundamental input parameters (Ewing et al., 1999) and may produce greatly differing results at different spatial scales and for

OCR for page 113
Risk and Decisions: About Disposition of Transuranic and High-Level Radioactive Waste changing boundary conditions (Duro et al., 2000; Madé et al., 2000; Jensen et al., 2002; Wang et al., 2003). The most widely used types of models are reactive transport codes that are meant to describe coupled thermal-hydrologic-geochemical processes (Lichtner et al., 1996), although such models often do not adequately address biogeochemical or nanoscale processes. Browning et al. (2003) reviewed a reactive transport model for the ambient unsaturated hydrogeochemical system at Yucca Mountain. They found model predictions to be particularly sensitive to assumptions about percolation flux, fracture-matrix interactions, groundwater compositions, and the rate of volcanic glass dissolution; each of these processes is knowable to a greater or lesser degree. However, the coupling of these processes presents challenges to reliable predictions of radionuclide transport. Hughson et al. (2000) tested risk analysis models by using them to describe the migration of trace elements from metallic artifacts preserved in a volcanic tuff some 3600 years ago by a volcanic eruption on Aktori, Greece. They found that different conceptual models, matching the site data, produced different predictions attributed to underdetermined parameters and changing boundary conditions. These are just a few examples of the challenge presented in modeling hydrogeologic systems, but they are representative of the types of limitations that one should expect of a risk assessment of a hydrogeologic system. The key to success in modeling highly coupled geologic systems is the careful integration of experimental results with field observations (Bethke, 1996; Nordstrom, 2004). The initial risk assessments are likely to reveal the need for additional site characterization, experimental data, and perhaps development of better multiscale modeling approaches. Unfortunately, experience shows that it is often these unknown circumstances and surprise events that shake risk analyses and topple expectations, rather than the factors (important though they might be) that have been recognized and incorporated into formal analyses… Also, formal uncertainty analysis may not help if the uncertainty in the fundamental understanding of the basic processes that drive the risk…is so large that a quantitative estimate can only lead to obfuscation… In such cases, identification of important issues and perhaps some selected analysis of scenarios (without assigning probabilities to these scenarios), is the best that can be accomplished. Consistent with this earlier NRC report, the approach outlined in Chapter 4 emphasizes use of sensitivity analysis (which includes “scenario analysis”) to understand the role of uncertainties. Sensitivity analyses are important in testing the potential inaccuracy of a risk model and

OCR for page 113
Risk and Decisions: About Disposition of Transuranic and High-Level Radioactive Waste determining which of the many forms of remaining uncertainty are important enough to affect decision-making deliberations. As noted in Chapter 4, if there are competing conceptual models, sensitivity analyses should start by considering the impacts of the alternative models on risk estimates. This stands in direct contrast to a procedure of first selecting a single conceptual model to simplify the analysis demands and then performing sensitivity analyses on that model’s parameters. The latter approach will only exacerbate the potential for controversies among scientists on the merits of the risk analysis results, and this, in turn, can only further deepen the widespread skepticism and distrust that members of the general public express for risk analyses. Sidebar 5.2: Sources of Uncertainty Uncertainty of mutual estimate increases as models based on laboratory experiments or limited field data are extrapolated to larger spatial and temporal scales. The uncertainty has three main sources: (1) data uncertainty; (2) scenario uncertainty; and (3) conceptual model uncertainty (Andersson and Grundteknik, 1999). Data uncertainty can enter the analysis at any point. There will be uncertainties about the composition and form of the radioactive waste, the composition of the groundwater, groundwater flow paths, and so forth. Scenario uncertainty will have more to do with the boundary conditions that will affect the site, such as climate change, recharge rates, and seismicity. Boundary conditions also change over time and space, for example, along the groundwater flow path. Conceptual model uncertainty is related to whether the relevant processes have been included in the models. As an example, a number of different conceptual models have been used to describe fluid flow and transport in the unsaturated zone (above the water table). In previous studies, initial estimates of groundwater travel times—that is, the time it takes for a contaminant to travel to some reference point—have decreased by as much as four orders of magnitude as the conceptual models have been changed (see Sidebars 6.1 and 6.2 in NRC, 2001b). Finally, the theoretical and mathematical foundations of multiscale modeling are themselves currently incomplete. In addition to the shift in emphasis toward exploring conceptual model uncertainty, risk analysis results may be viewed and communicated most effectively as relative indicators of potential impacts under

OCR for page 113
Risk and Decisions: About Disposition of Transuranic and High-Level Radioactive Waste different disposition alternatives. In fact, it would be misleading to consider such a risk assessment as yielding “quantitative” results, in the sense that the absolute results have reliable magnitudes. The uncertainties in the results remain large and increase with time. A risk assessment is quantitative only in the sense that it provides a numerical result, but the meaningfulness of the result is actually more qualitative. By treating the results as if they have quantitative significance, one can mislead decision makers into concentrating on numerical comparisons rather than on evaluation of the adequacy of the strategy for the safe disposal of a particular type of radioactive waste. Despite these enumerated limitations, risk assessment remains a powerful tool in organizing our understanding of the behavior of a physical and behavioral system. However, a risk assessment is only one element in a properly made risk-informed decision (Apostolakis, 2004) and is not, by itself, a sufficient basis for determining that a site or strategy is “safe.” 5.2 PUBLIC PARTICIPATION AND STAKEHOLDER ISSUES Participation by members of the interested public (commonly referred to as stakeholders) in risk analysis and risk management—especially those who will be directly affected by the decisions made—has changed considerably in the past two decades. At its most unsophisticated level, public participation has been thought to mean “educating” the public about the “truth and wisdom” of expert analysis or informing the public about a final decision. As is discussed in greater detail in Section 5.4, when DOE was actively involved in decision making against the backdrop of the Cold War, it often addressed public participation as an afterthought or item of secondary importance. Under this management model, deficiencies in public participation were marked by the following common features: Stakeholders are presented with the risk assessment at the end of the process. The risk assessment is opaque and difficult to understand.

OCR for page 113
Risk and Decisions: About Disposition of Transuranic and High-Level Radioactive Waste of risk, and point out uncertainties, but it cannot identify the level of risk that society should tolerate. Transscientific decisions such as the level of acceptable risk are among the most difficult facing public officials in Congress and administrative agencies. They can look for guidance to the risks apparently or explicitly accepted in other activities (assuming they are analogous), and they can look at cost and other factors. Ultimately, however, it is a balancing of many factors—scientific and nonscientific—that results in a decision (Jasanoff, 1990; Wagner, 1995). Sometimes the technical analyses all seem to point in one direction, making the decision easier (or at least easier to justify), but often this is not the case. It is therefore to be expected that public officials will look to science in the hope that it will give a clear direction for hard choices. In addition, science has an allure, as one writer puts it, of precision and objectivity that can provide much needed support for politically unpopular decisions (Applegate, 1995; Hornstein, 1992). (The cynic may find this use to be no more than seeking political cover; but more charitably, it is perfectly reasonable to seek a firm basis for decisions that hurt some people’s interests.) In both cases (that is, hoping for clear direction and looking for objective justification), however, the tendency is to expect or claim more than science can legitimately deliver. The National Research Council has repeatedly warned of the overreliance on science in the context of risk analysis. As early as the Red Book (NRC, 1983), it called for a distinction, as far as possible, between the risk- and impact-assessment aspect of the process and the policy-oriented, risk-management aspect (NRC, 1983, 1994a, 1996). Scientific analyses should take care (1) to disclose fully limitations and uncertainties in their conclusions, and (2) to distinguish carefully between science and nonscientific judgments.5 Ultimately, a decision maker cannot avoid making hard decisions in the face of the uncertainty in the analysis or continued disagreements among the stakeholders. The dilemma has been well summarized by Herrick and Sarewitz (2000): 5   This is not to say that scientists should shrink from judgments or recommendations that are not purely scientific in character. On the contrary, it is imperative that the scientific community be actively involved in the important issues of the day, especially those with a technical aspect. Rather, scientists should be clear as to the basis for—and, if appropriate, the limitations of—their judgments.

OCR for page 113
Risk and Decisions: About Disposition of Transuranic and High-Level Radioactive Waste Unreasonable expectations about the nature and character of scientific knowledge support the widespread political assumption that predictive scientific assessments are a necessary precursor of environmental decision making. All too often, the practical outcome of this assumption is that scientific uncertainty becomes a ready-made dodge for what is in reality just a difficult political decision. Interdisciplinary assessments necessary to address complex environmental policy issues invariably result in findings that are inherently contestable, especially when applied in the unrestrained realm of partisan politics. (Italics are the committee’s.) Although the “inherently contestable” quality of a risk or performance assessment cautions against overreliance on science, it does not justify disregarding the results of such an analysis. A rigorous scientific analysis provides a logically coherent organization to the many factors in a decision, provides essential information for the decision, and clarifies areas of remaining uncertainty and remaining (nonscientific) choices. Scientific analysis, therefore, deserves to be a core part of decision making on questions of public significance, but it will not ultimately relieve the public decision maker of the weighty responsibility of hard choices. To continue a theme in Chapter 4, the scientific information captured in a risk assessment, or any risk assessment process, is a tool for decision makers that is most useful when taken as an integrated whole. Summarizing an entire risk process in a single point estimate or even risk range is not helpful for decision makers or those who need to understand how a decision was made. 5.4 ISSUES OF INSTITUTIONAL CULTURE The above discussion of scientific issues has used examples from some of the most difficult problems that DOE faces, such as the proposed Yucca Mountain repository for HLW. Here, the problem is with predicting the behavior of a complex natural and engineered system expected to contain long-lived radionuclides over extremely long time periods. The Yucca Mountain problem is being addressed by use of a complex integrated performance assessment model, but according to a white paper of the Board on Radioactive Waste Management, this complexity is viewed as virtually incomprehensible to nonspecialists (NRC, 1999).6 6   “There are many indications that publics neither understand nor trust the expert community on radioactive waste issues. For the non-specialists, the array

OCR for page 113
Risk and Decisions: About Disposition of Transuranic and High-Level Radioactive Waste DOE faces many other technical problems of a simpler nature. These less complex problems would appear to be more amenable to efforts to gain public confidence by virtue of their being addressed by a simpler risk-informed process. To the committee’s knowledge, however, even these simple problems are not currently being addressed by anything resembling the risk-informed process recommended here. It seems that nearly every new assistant secretary for the Office of Environmental Management has come into office with the expressed desire to reform the process in order to make it focus more on risk and achieve cleanup “faster, better, and cheaper.” The committee is also impressed by the number of other committees (see Appendix B) that have recommended virtually the same thing: the implementation of a risk-based or risk-oriented process to focus attention and money on the problems that really do require action and not to spend unnecessarily large amounts of money on problems that are not, in fact, a source of significant risk to either humans or the environment. Some of these committees were specifically charged at the time to help DOE determine how to use the risk assessment and management process effectively. DOE has asked other groups to examine the environmental management problem and those groups have come up with similar recommendations. In spite of these multiple recommendations and the apparent openness of persons high within the DOE management structure, and in spite of this committee’s attempts to find examples of a risk-based or risk-informed decision process at work, it appears that, with rare exception, the process of risk assessment is not being utilized effectively by the nation. Virtually all of the so-called risk assessment activities presented to this committee were being performed principally because they were required by the U.S. EPA under the CERCLA or the Resource Conservation and Recovery Act (RCRA). (These requirements typically followed on from the listing of the various sites on the National Priorities List [NPL].) It is the committee’s impression that these risk assessment activities were not being regarded within DOE as particularly significant or effective input into the decision-making process. While in public presentations some site personnel stated that risk currently is the driving factor for decisions through the regulatory process, in private some of the same people indicated that factors other than risk fixed many decisions. For example, many decisions were made in the federal facility agreements,     of applicable scientific data and proposed methods is so complex as to be virtually incomprehensible” (NRC, 1999).

OCR for page 113
Risk and Decisions: About Disposition of Transuranic and High-Level Radioactive Waste which were established before most of the risk assessments were conducted. Thus, one of the fundamental questions that the committee asks is Why is risk assessment not already one of the foundations of DOE’s environmental management program? What are the impediments? Some members of this committee have participated over periods of approximately 15 years on other committees or review groups examining these questions. None of the members of this committee has actually worked for DOE, so committee members can examine these fundamental questions only as outsiders with the information furnished from DOE, regulatory agencies, and members of the public. Some or perhaps many impediments may lie beyond DOE in the larger institutional or societal structure. Although U.S. EPA’s use of risk assessment in some of its programs has been criticized severely, U.S. EPA has, in comparison, used the risk-assessment process relatively effectively at many of the sites it regulates under RCRA and CERCLA, and at WIPP. Risk analyses are accepted as an important part of the process of regulatory decisions and cleanups conducted by the U.S. EPA. Russell (2000) attributes U.S. EPA’s use of risk assessment to the efforts of Administrator William Ruckleshaus to change the U.S. EPA’s culture from a “…view of its goal of eliminating pollution to one of assessing and then managing risk in order to reduce harm to people and the environment.” DOE has never had a similarly influential risk management champion. Some of the fundamental differences between U.S. EPA and DOE relate to the differences in the history and structure of the two organizations. The U.S. EPA is a relatively new organization with a clear, narrowly defined mission. DOE has its roots in the Manhattan Project, and DOE has a variety of different missions, some of which may inherently conflict with each other. The organization of DOE (actually its predecessor agencies, the Manhattan Engineering District of the Army Corps of Engineers and the Atomic Energy Commission) was unique from its very first days. The first mission of the original predecessor agency was to develop and build a few deliverable nuclear weapons. This was a sophisticated scientific and engineering problem for which the members of the usual government bureaucracies had no experience or expertise. Thus, from the beginning there was an unusual dependence of DOE upon private contractors of one kind or another. Urgency and secrecy were also fundamental to the early days of DOE. The urgency led to unprecedented steps to achieve the fundamental mission with little importance attached to environmental considerations, although considerable attention was

OCR for page 113
Risk and Decisions: About Disposition of Transuranic and High-Level Radioactive Waste always focused on the health and safety of workers. The secrecy led to islands of knowledge that were known in detail only to persons outside of the main bureaucracies. Over time, the DOE management style also came to depend on field organizations. Some of these organizations, such as the Richland (Hanford), Oak Ridge, and Savannah River Field Offices, became very powerful. Persons at these three field offices oversaw large enterprises, run by management and operations (M&O) contractors, whose financial contributions to the local and regional economies became very substantial, if not dominant. This economic influence did not escape the notice of local and regional politicians, and each major DOE site inevitably built up its own base of political influence based on its economic power. Enormous changes have occurred within DOE during the last two decades. The major mission, the production of special nuclear materials and the fabrication of nuclear weapons, has virtually been eliminated. Some of the smaller DOE sites have been, or are being, decommissioned and dismantled. At about the same time that this major structural shift within DOE was occurring, the powerful trend of environmental protection and/or remediation was developing. As the production of special nuclear materials was winding down, the concern about (and the legal liabilities from) the residues of contamination from decades of urgent production came to the fore. This concern led to the establishment at DOE of an Office of Environmental Management and, at each DOE site still run by M&O contractors, of a new organization devoted to environmental management or cleanup. At many of the remaining DOE sites, this environmental management organization was about the same size and had about the same budget as the former production organizations. In spite of this fundamental shift, DOE tried to manage as it had previously by using a decision-making process that was rather closed. One of the sea changes that eventually swamped this previous methodology was the loss of DOE’s exclusive self-regulation. One of the early changes was the empowerment of U.S. EPA to regulate airborne emissions from DOE facilities under the Clean Air Act; eventually, it became clear that the U.S. EPA had the power to regulate DOE under the terms of other environmental laws, notably CERCLA and RCRA. The real meaning of these changes was brought home to DOE by the realization that U.S. EPA had the power to levy fines and even enforce personal criminal penalties. This was illustrated dramatically by the raid by Federal Bureau of Investigation and U.S. EPA Enforcement Police on the Rocky Flats Plant; one result of the raid and other investigations was criminal conviction of M&O officials at the Rocky Flats Plant.

OCR for page 113
Risk and Decisions: About Disposition of Transuranic and High-Level Radioactive Waste During this period of the late 1980s and early 1990s the possibility of DOE sites being listed on the NPL was viewed by personnel at most sites as threatening and negative. After the first DOE site was listed, however, it received a large amount of funding to deal with its environmental problems. This flow of funds was not lost on environmental protection personnel at other sites, and a form of subtle competition soon evolved in which listing on the NPL was accompanied by receiving larger amounts of money. Eventually, most significant DOE sites become listed on the NPL, and while fines have been assessed, no additional high profile criminal convictions have occurred. An additional activity carried out during this period was the negotiation of triparty compliance agreements among DOE, U.S. EPA, and local (usually state) regulators. These agreements typically contained binding statements on what would be done, along with a time scale over which the specified work would be done. Persons negotiating these agreements on behalf of DOE were concerned mainly with legal, managerial, and political issues. A result was that many of these agreements contained commitments to perform work that was technically or economically infeasible and might pose substantial risk to workers using present technologies. This eventually led to the need to renegotiate many agreements and to some loss of credibility for all participating agencies. Another important aspect in terms of the deliberation of this committee was that these agreements were not based on an analysis of the risks posed by the considered wastes. In some cases, agreements were reached to perform cleanups that not only were expensive, but also would have resulted in little or no reduction in risk (DOE, 2002d). In essence, this process of negotiating compliance agreements without thorough and detailed consideration of risk, cost, or technical feasibility represents an enormous lost opportunity for the nation to insert discipline into the environmental management process. There have been several attempts by Congress and high-level DOE management personnel over the last several years to insert such discipline at DOE sites; these activities are discussed in Appendix B. Such activities have not been particularly successful and may have been resisted by a variety of ad hoc consortia at the various sites. Members of such consortia might include persons from DOE field offices, management and operations contractors, state and local regulators, environmental groups, state governors, and members of the U.S. Congress. A strong theme binding such diverse groups together has been the thousands of jobs and billions of dollars at stake. The emotions and issues are

OCR for page 113
Risk and Decisions: About Disposition of Transuranic and High-Level Radioactive Waste similar to those that periodically surface when the Department of Defense contemplates closure of military bases. Another important theme is that local persons feel that top-down imposition of discipline, whether through risk assessment or other means, would be a violation of the content and spirit of the agreements that have been negotiated. In some cases, risk-based approaches are also seen to be in conflict with national or local laws. A persistent problem is that the nation has permitted and perhaps even encouraged a lack of focus on cleanup as a result of viewing some site cleanups as regional economic entitlement programs (see, e.g., Greenberg et al., 2002; Probst and Lowe, 2000; Russell, 1997). DOE has adapted itself to the environment in which it operates. The complexity, constraints, and politicization make these problems very challenging and bear some responsibility for the situation. Stated bluntly, there is a conflict between the wishes of the nation to limit the costs and time of the DOE cleanup program (and the attempts of the various assistant secretaries of energy for environmental management to implement these wishes) and the incentive to persons at local sites to ensure a large and sustained influx of funds. This is a fundamental institutional barrier to the use of a risk-based approach to environmental management at DOE sites. Local persons have frequently stated that they could be comfortable with the use of risk assessment to allocate priorities among subsites (or “operable units” in the official jargon) at their overall site, but they state a strong resistance to the use of a risk-based approach, or any other approach, to set priorities among sites. Such statements are apparently motivated by fears of the possible loss of funding for their site in favor of another site. These fears may be accompanied by a sense that if funding is lost, cleanup programs could be curtailed or even abandoned, leaving cleanups unresolved and local stakeholders totally marginalized. The problems of DOE legacy wastes are most severe at Hanford, Idaho, and Savannah River. There are two important reasons for this. The more obvious is that these sites have most of the wastes within the system. The other less obvious factor is that these three sites were purposely located in rural areas to place their hazards farther from large populations. The DOE contribution to the regional economy in these areas has been dominant. Thus, the abrupt withdrawal of DOE’s contribution would be catastrophic to these local economies. However, a short-term view is dangerous and only postpones the inevitable. Russell (1997) has argued that these communities are entitled to transitional economic assistance, but that hijacking the DOE environmental

OCR for page 113
Risk and Decisions: About Disposition of Transuranic and High-Level Radioactive Waste management program is not a productive avenue to achieve this reasonable goal. Rather, Russell has argued that the cleanup and transitional assistance programs should be divorced. Probst and Lowe (2000) have made similar arguments. In summary, the institutional barriers to the implementation of a risk-based approach to cleanup at DOE sites have been demonstrated to be effective and deeply entrenched. Although Congress has repeatedly criticized the DOE process, members of Congress representing the individual sites have sometimes contributed to the problem by the long-established method of introducing special legislation in favor of a particular site. Nearly every new assistant secretary for environmental management has tried to reform the system, but it does not appear that any assistant secretary has sufficient power to impose discipline on the system. Finally, the coalitions of interests at each site are extremely resistant to any priority-setting activity that might remotely result in a loss of funding for that site. These factors, coupled with the legally binding agreements that have been made at each site, present a formidable barrier to the implementation of the risk-based process advocated in this report. Is there a solution? The answer is not clear, but it does not seem likely that a solution will be forthcoming without strong instructions from the Congress to the highest level of management at DOE. The DOE’s Top-to-Bottom Review Team made the following statement, “The results of the Team’s review make clear that there is a systemic problem with the way EM has conducted its activities: the EM program’s major emphasis has been on managing risk, rather than actually reducing risk to workers, the public, and the environment” (DOE, 2002d, pp. ES-1-ES-2). As a consequence of this conclusion, DOE redirected its program toward reducing the long-term risks posed by legacy wastes with initiatives such as closing tanks, pursuing supplemental technologies for tank waste processing, and pursuing risk-based end states. However, these efforts have apparently run afoul of EM’s lack of use of risk management in the sense that it is normally understood (see NRC, 1983)—that is, as a process that defines the character and extent of the problems to be addressed, helps identify options to manage risks, and engages regulators and other stakeholders to balance risk considerations with other factors. As subsequent events (e.g., successful litigation concerning on-site disposition of tank heels; a RBES process that appears to be effectively stalled) have amply demonstrated, the absence of a real risk management process has been a substantial impediment to the cleanup and consequent risk reduction sought by EM.

OCR for page 113
Risk and Decisions: About Disposition of Transuranic and High-Level Radioactive Waste Thus, the key ingredient of a needed program is precisely risk management, as defined by an earlier committee (NRC, 1983). Recognition must be given to the fact that some wastes cannot be cleaned up, that transgenerational risks will occur, and that long-term stewardship will be required into the indefinite future. These are the costs for this nation’s choices during World War II and the Cold War. Finding 10: The DOE risk assessments and decision processes examined by the committee do not exhibit all of the characteristics of an effective and credible risk-informed decision-making process, listed in Finding 8. Other bodies have made similar recommendations on how DOE should incorporate risk into environmental decision making, and DOE has made progress, but institutional factors appear to have interfered and perhaps undermined attempts to implement these approaches. This implies that changes at DOE are needed to address internal and external impediments to the risk-informed approach. In its site visits and after, the committee requested that DOE present its best examples of risk assessment informing waste disposition or cleanup decisions. Through DOE’s presentations to the committee and the committee’s review of documents, the committee examined many risk assessments and decision processes. DOE and its contractors have performed technically complex risk assessments, and in many cases have performed risk assessments as part of regulatory processes that lead to cleanup decisions with stakeholder input. Yet the cases examined by the committee do not meet the needs identified and described in this report for the following reasons. The complex analyses were not decision oriented and were not carried out in a transparent manner needed for meaningful participation by those outside DOE. The actions supporting regulatory decisions in many cases also were lacking—the steps in the processes appeared to have been performed simply to meet procedural requirements and most did not appear to have taken the kind of cooperative approach that the committee sees as essential to reach credible decisions and to foster buy-in by other relevant parties. That the risk assessments examined by the committee do not exhibit all of the characteristics of an effective and credible risk-informed decision-making process does not imply that DOE has been derelict. These are technically difficult cleanup problems being addressed in a complex political and social environment. DOE has stabilized into safe, although

OCR for page 113
Risk and Decisions: About Disposition of Transuranic and High-Level Radioactive Waste temporary, conditions the dangerous wastes and facilities at its sites, and in most cases has an enviable safety record in its cleanup program. Working toward effective and credible risk-informed decisions on these issues is very difficult. Further, many of the risk assessments examined by the committee were addressing smaller although significant problems, and so may not have warranted the effort recommended in this report. Also, the risk assessments were not necessarily aimed to fill the role described in this report. But on the latter point, the committee notes that numerous studies summarized in Appendixes A and B make recommendations consistent with those made in this report on how to incorporate risk into environmental decision making. DOE has made progress, but approaches such as the one recommended by the committee still have not permeated DOE’s decision-making apparatus. It appears that institutional factors both inside and outside DOE have impeded attempts to implement risk-informed approaches. These factors include a tradition of internal rather than open decision making, incentive structures that favor distorting or ignoring risk, and a public wariness or mistrust of DOE’s use of risk assessment to justify proposed actions. The committee’s role is to help DOE to bring the best practices to bear on the challenges DOE is addressing on the nation’s behalf. DOE’s difficulty in adopting risk-based or risk-informed approaches recommended previously by other committees and observers implies that DOE needs to make changes. Moreover perhaps changes are needed more broadly in the nation’s approach toward managing risks at DOE sites. Recommendation 5: To address the challenges of implementation and acceptance, DOE should form an authoritative, credible, and reasonably independent group to revamp the way it goes about implementing risk-informed approaches applied to waste disposition decisions. These are enormously complex problems with numerous parties involved and a great deal of institutional inertia (as evidenced by unsuccessful previous attempts to change). The committee sees a need to break out of old approaches. To this end DOE needs an action-oriented group that provides advice and identifies alternatives, but also assists with implementation and draws in major stakeholders to get buy-in. The group must be credible, and to be credible the group must be au-

OCR for page 113
Risk and Decisions: About Disposition of Transuranic and High-Level Radioactive Waste thoritative on the issues it addresses and independent so as to be unbiased and free of conflicts of interest. Before implementing this recommendation, it would be useful to consider the extensive experience of a variety of federal agencies with outside advisory committees, including the committees’ roles and effectiveness.