National Academies Press: OpenBook

Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles (2013)

Chapter: 3 Utility of Proliferation Assessments

« Previous: 2 Key Proliferation Policy Questions
Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×

3

UTILITY OF PROLIFERATION ASSESSMENTS

The preceding chapter focused on the first task by identifying the types of policy questions that technical analysis of host-state proliferation risk could inform. This chapter focuses on the second charge of the study task:

TASK 2: Assess the utility for decision makers of existing and historical methodologies and metrics used by DOE and others (such as the International Atomic Energy Agency [IAEA]) for assessing proliferation risk, both for considering the deployment of these facilities domestically as well as the implications of deployment outside the United States.

The committee describes the general characteristics of a selected set of predefined framework methodologies, discusses their strengths and weaknesses, and then comments on their utility to policy makers and decision makers. The first part of the chapter describes the characteristics of each of the methodologies used by the Department of Energy (DOE) and others to assess technical aspects of proliferation risk, recognizing that the methodologies were originally developed to assess proliferation resistance. The committee then evaluates the methodologies and their execution, and identifies shortcomings and applications for which the methodologies are well-suited. The chapter concludes with the committee’s evaluation of the utility of predefined frameworks for decision makers.

In Chapter 2 the committee noted that technical assessments related to proliferation risk are made using a variety of approaches and often are designed to address a question about a particular nuclear technology or capability in the context of a specific country or region. Such assessments are frequently performed on a case-by-case basis by multidisciplinary teams of subject matter experts, often in close collaboration with the intelligence community. As far as the committee was able to determine, these assessments were systematic but did not follow predefined, structured frameworks or use probabilistic risk assessment.

Within the area of nonproliferation policy, questions concerning technical analysis of host-state proliferation risk of a given nuclear fuel cycle represent a relatively small subset. Technical assessments of proliferation resistance address an even smaller subset because, as noted previously, proliferation resistance is contained within the

Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×

analysis of proliferation risk.12 Regardless, “predefined framework methodologies” have been developed specifically for assessing proliferation resistance. Such predefined frameworks provide a generic structured approach for comparing the technical features of different generalized nuclear fuel cycles by breaking the full fuel cycle into individual processing steps and analyzing each step against a threat using a set of metrics (or attributes). These predefined framework methodologies will be the focus of this chapter because they are used by both the Department of Energy Office of Nuclear Energy (DOE-NE) and National Nuclear Security Administration (NNSA) and are being developed by others such as the International Atomic Energy Agency (IAEA) (ANS 2012, Herczeg 2012, Lockwood 2012). The committee has not included other methodologies for assessing proliferation resistance in the structured evaluation in this chapter. However, the committee did consider other analyses in Chapter 2, such as case-by-case analyses, and in Chapter 4 in which other methods for assessing risk are reviewed.

BACKGROUND OF THE DEVELOPMENT OF PREDEFINED FRAMEWORKS

Numerous methods for analyzing technical aspects of proliferation risk of the nuclear fuel cycle have been developed over the decades (GENIV Forum 2011a; Charlton 2012). The Technical Opportunities to Increase the Proliferation Resistance of Global Civilian Nuclear Power Systems (TOPS; NERAC TOPS Task Force, 2001) report formulated a set of qualitative attributes relevant to the intrinsic (materials and inherent properties of the fuel cycle) and extrinsic (institutional and application of safeguards) aspects of a nuclear fuel cycle. The TOPS report was also used as a basis for identifying R&D priorities for DOE and recommended, “Development of improved methodologies for assessing the proliferation resistance of different systems, including those that further the understanding of the trade-offs between intrinsic and extrinsic measures.” Most predefined frameworks under consideration today were developed in response to the TOPS report and the R&D priorities that it identified.

In 2001, a program plan for development of a nonproliferation assessment methodology was prepared for NNSA’s Office of Nonproliferation and International Security (NA-24) (NNSA 2001). The goal of the program and eventual working group (DOE 2002, Mladineo et al. 2003; DOE 2002) was to:

• Develop a standardized methodology or set of methodologies for assessing proliferation resistance of different reactor and fuel cycle systems, and other nuclear systems and processes.

pag The methodology must be capable of measuring trade-offs among concepts and systems.

pag The methodology must include quantitative tools to ensure technical rigor.

_______________

12 As discussed in the Introduction, “proliferation resistance” focuses on the engineering aspects of a particular nuclear fuel. “Proliferation risk” is a broader concept and includes analysis of country-specific issues such as the probability that an adversary will choose to proliferate along a particular pathway, and the probability of success along that path. For more details, see Chapter 1.

Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×

In 2002, NA-24 issued functional requirements for development of a nonproliferation assessment methodology (NPAM) (DOE 2002) which identified the need for the methodology to:

• address a broad range of questions to be addressed by the methodology including country-specific analyses;

• address uncertainties and sensitivity analysis;

• incorporate weighting techniques to vary priorities; and facilitate use of expert knowledge

On the basis of the NPAM documents discussed above, NNSA and DOE-NE and several countries within the Generation IV International Forum (GIF) embarked on a joint effort to develop a methodology for use in diverse applications including country-specific applications, which resulted in the Proliferation Resistance and Physical Protection (PR&PP) methodology.

OVERVIEW OF PREDEFINED FRAMEWORKS

Predefined frameworks are methodologies designed to consistently and transparently evaluate proliferation resistance through a standardized set of predetermined attributes (or metrics) that are evaluated throughout the individual processing steps of a given fuel cycle (Mendez et al. 2006, Ford 2010).They provide a framework to gather and organize data to score the attributes, which requires expert knowledge and experiential data. Such data are gathered and combined into higher-level scores to give the fuel cycle an overall measure of proliferation resistance (usually high, medium, or low or numerical equivalents). This “black box” approach can mask the underlying reliance on subject-matter experts and lead to assumptions that the frameworks are objective models or simulations. They are not.

A typical set of metrics of proliferation resistance includes

• technical difficulty of proliferation,

• cost required to overcome barriers to proliferation,

• time needed to proliferate,

• type of material to be acquired for proliferation purposes, and

• the probability of detection (or transparency).

The committee judges these metrics to be reasonable, and many are cited in the proposed key policy questions found in Chapter 2.

The committee identified six predefined framework methodologies for more thorough review based on their variety of approaches for combining assessments at each processing step (e.g., multi-attribute utility analysis or event-tree logic). Several were selected on the basis of their use internationally and their use most commonly in the United States. This list and the committee’s assessment of the methodologies were reviewed by a set of practitioners of various predefined frameworks (see Appendix B). The following frameworks were selected:

Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×

• TOPS methodology,

• Japan Atomic Energy Agency (JAEA) methodology,

• Simplified Approach for Proliferation Resistance Assessment (SAPRA) methodology,

• Texas A&M University Multi-Attribute Utility Analysis (TAMU MAUA) methodology,

• Risk-Informed Proliferation Analysis (RIPA) methodology, and

• Generation IV International Forum Proliferation Resistance & Physical Protection (GIF PR&PP) methodology.

Perhaps noteworthy by its absence is the IAEA’s International Project on Innovative Reactors and Nuclear Fuel Cycles (INPRO). Initial efforts to use INPRO precepts to assess proliferation resistance led by the Republic of Korea under IAEA auspices (IAEA 2011b; Lee et al. 2012) have been published, for example, the Proliferation Resistance: Analysis/Diversion Pathway Analysis (PRADA) project on Direct Use of Pressurized Water Reactor Spent Fuel in CANDU (DUPIC). However, in the most recent guidance on use of the INPRO assessment methodology (IAEA 2008 volume 5, Annex A, p. 39) it is stated:

The goal of a PR [proliferation resistance] evaluation is to provide guidance for the nuclear energy system development groups that will develop the proliferation resistant technology, and to present results, showing how the non-proliferation goals will be met, to institutions responsible for deciding which nuclear concepts to pursue.

and

At present, the [INPRO] evaluation method in the field of proliferation resistance is not complete. The group of consultants working in this area did not come to a common conclusion on scales for some indicators as well as on acceptance limits of criteria. Thus, final presentation of the evaluation results is not yet defined.

Thus, the Korea-led effort which involved developing detailed metrics within high-level INPRO basic principles of proliferation resistance and five general “user requirements” has not yet been adopted by the IAEA. Rather than further developing a framework for INPRO, the international community is pursuing an approach of harmonizing the relatively broad INPRO methodology with the more detailed PR&PP methodology (ANS 2012) in the Proliferation Resistance and Safeguardability Assessment (PROSA) project which began in February 2012. Therefore, the committee chose not to review a work in progress. Assessment of the PR&PP methodology can be seen below.

Below, the committee provides high-level descriptions of these six methodologies. Additional details on their characteristics are also noted in a summary table in Appendix B.

Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×

Technical Opportunities to Increase the Proliferation Resistance of Global Civilian Nuclear Power Systems

The TOPS task force was established by DOE-NE’s Nuclear Energy Research Advisory Committee (NERAC). The task force was charged “to identify near- and long-term technical opportunities to further increase the proliferation resistance of global civilian nuclear power systems.” This included an R&D focus on methods for assessing proliferation resistance by evaluating “the relative proliferation resistance of specific fuel cycles in terms of a generic set of ‘attributes.’ The attributes are derived by first defining the barriers to proliferation inherent in the design of the system, its materials and facilities, and its modes of operation” (NERAC TOPS Task Force 2001). The approach followed a few basic steps for each fuel cycle under consideration:

• Define the different processing steps.

• Identify potential proliferation pathways in each processing step.

• Assign values to the attributes based on the threat (e.g. host state or subnational threat).

Attributes describe the relationship between the elements of a fuel cycle, the threats to those elements, and the effectiveness of barriers to inhibit these threats. Intrinsic attributes are technical features such as material properties (isotopic or chemical barriers, amount of material, detectability) and extrinsic attributes are institutional and operational barriers such as safeguards and inspections. The evaluation of the attributes relies on subject matter expertise and existing data.13 Although the 2001 TOPS report made an initial attempt at comparative assessment of a set of fuel cycles following its proposed attributes methodology, it is not currently a methodology that is used. It is included in this list as a historic methodology.

The proliferation resistance attributes for a proposed fuel cycle (Fuel Cycle A) against a generic threat (sophisticated host state, covert diversion) are shown in a notional example of a TOPS-like assessment in Table 3.1. The table includes the list of intrinsic and extrinsic barrier attributes, the weighting factors for aggregation, a list of processing steps, and values of each attribute for each processing step. The values within this particular table are purely randomly assigned because it is a notional example; in a real assessment, the values would be determined by subject matter experts. Other fuel cycle

_______________

13 It should be noted that among the quantitative measures, some require underlying judgments made by experts. For example, the determination of threshold values for seemingly technical attributes, such as the 100-rem/h dose rate at 1 m below which nuclear material is assumed to not be self-protecting (NRC 2011a) may be an important demarcation in a proliferation risk assessment, but it has been questioned by other experts who believe a value of 500-rem/h at 1 m is more appropriate (Bathke et al. 2009).

Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×
Fuel Cycle A Against Sophisticated State, Covert
Process Step Intrinsic Barrier Extrinsic Barrier
Weighting factors Isotopic Chemical Radiological Mass & Bulk Detect-ability Facility access- Facility unattract-iveness Detectability of diversion Technical capability Time
0.2 0.2 0.4 0.5 0.8 0.5 0.3 0.2 0.8 0.2
Front-end 1 Enrichment low low low med med low low low med med
2 Fresh Fuel Fabrication low low low med med low med med med med
3 Fresh Fuel Storage low low low med med high high med med med
Reactor Site 1 Fresh Fuel Storage low high med med med med med med high med
2 Fuel Loading/Irradiation low high med high high med low high high high
3 Spent Fuel Storage low high high high med low low high high high
4 Spent Fuel Transportation low high high high med low low high med high
Back-end 1 Spent Fuel Storage low high high high med med low med med med
2 Reprocessing low med high low low med med high low high
3 TRU Waste Storage low med high low med low med high low high
4 Recovered NM Storage low high high low med low low high low med
5 Fuel Fabrication low high med low med med med med med med
6 Fuel Storage low high med med high low med med low med
7 Fuel Transportation low high med med high med med med med med
8 HLW Disposal low high high med high high med high med high

TABLE 3.1 Notional Example of a TOPS-Like Assessment of Proliferation Resistance Attributes for a Proposed Fuel Cycle A Against a Generic Threat (Sophisticated Host State, Covert Diversion). SOURCE: Modified from Inoue et al (2004).

and/or threat options would have a similar spreadsheet developed to allow for relative comparison at this or fully aggregated levels. The values within the table can be combined through weighting functions so that a final, single value of proliferation resistance is determined and can be compared with other fuel cycles.

Japan Atomic Energy Agency Methodology

The Japan Atomic Energy Agency (JAEA) developed an assessment methodology based primarily on the TOPS attributes methodology to provide a qualitative relative assessment of the proliferation resistance of systems, processes, and nuclear facilities as part of its strategy to commercialize fast-reactor technology. Two threats are considered: a covert diversion by a state and theft by a subnational group. The methodology defines material and technical barrier attributes of mass and volume of material, radiation fields and isotopic and chemical composition and, like TOPS, primarily relies on qualitative expert knowledge to evaluate the attributes. Sensitivity analyses have been performed against specific attributes, but the characterization of uncertainties of each attribute and how the uncertainties are carried through the analysis are not discussed or included in the results.

Simplified Approach for Proliferation Resistance Assessment

SAPRA was developed by the French Working Group on Proliferation Resistance and Physical Protection, which includes representatives from the Foreign Affairs and Industry ministries, French Safety Institute (IRSN), Atomic Energy and Alternative Energies Commission (CEA), Electricity of France (EDF) and AREVA, Inc. SAPRA

Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×

further expands TOPS and JAEA (Greneche et al 2007) by delineating the steps to proliferation as four stages: diversion, transportation, transformation and nuclear weapons fabrication. SAPRA also introduces several different attributes (e.g. “dangerousness” instead of isotopic and chemical barriers to include other factors, such as reactivity of a chemical to water) but the approach to assigning values using experts and combining the values to a single result is consistent with the TOPS approach. The result of the analysis is a quantitative result that includes an assessment of each of the four defined stages of proliferation. At each stage, the values of the attributes are aggregated and normalized to 1. SAPRA considers only the case of state proliferation, not acquisition of material or a weapon by a subnational group. Uncertainty is not included in the assessment of the attributes nor is it reported in the final results. Sensitivity of the barriers to different threats is considered but an analysis of which attributes are most affected was not reported.

Texas A&M University Multi-Attribute Utility Analysis

A methodology for computing proliferation resistance was developed at Texas A&M University (TAMU) using a multi-attribute utility analysis (MAUA) method that assigns utility functions to each proliferation attribute (Charlton 2007, 2012). The result of this method is a numeric ”nuclear security measure” on a scale from 0 to 1, where “0” implies complete vulnerability to proliferation and “1” complete proliferation resistance or proliferation-proof. The method assigns weighting factors to a set of critical material and facility attributes of which there are 14 (further expanding the list of attributes beyond the SAPRA method for the purpose of decoupling interdependencies between previous attribute lists). The weighting factors are based on input from various experts in nuclear proliferation–related fields and contain both objective and subjective information.

The methodology focuses the proliferation resistance assessment on the flow of material through the fuel cycle as a function of time—from its initial input into the fuel cycle to its eventual disposal. The TAMU MAUA methodology limits the threat to diversion of nuclear material by a host state but does not address other threats such as theft or terrorism. It accounts for the intent of the host state to proliferate by assigning it a maximum and constant value in the analysis.

Uncertainty is not included in most of the utility functions that are established for each of the attributes, although “measurement uncertainty” is included as part of the higher-level metric of “Difficulty of evading detection by the accounting system.” There is no mention of sensitivity analysis.

Risk-Informed Probabilistic Analysis

The RIPA methodology assesses the most likely paths for proliferators to acquire nuclear weapons, including the cost and time required for each using various risk-informed assessment techniques (Blair et al. 2002, Rochau et al. 2012). The goal of RIPA was to create a set of separable components of the proliferation risk problem that could be analyzed by experts and reused as needed for future analysis. RIPA includes (1) influence diagrams and resulting proliferation pathways, (2) proliferation scenarios, and (3) the proliferation measures. The influence diagrams present the steps (nodes) that must

Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×

be followed to succeed in building a nuclear weapon. There were no examples of RIPA being used for specific applications. The documentation reviewed by the committee did not specify how uncertainties would be analyzed or reported nor did it mention sensitivity analysis.

Generation IV International Forum Proliferation Resistance & Physical Protection

The PR&PP methodology was developed as an approach for assessing the proliferation resistance of advanced nuclear energy systems and is the outcome of the NPAM documents described earlier. It is one of the more developed methods for analyzing proliferation resistance in the current literature and is used by GIF and others.14 For a given system, the goal of the method is to define a set of challenges, analyze the system response to those challenges, and assess the outcomes (see Figure 3.1). The challenges to the nuclear energy systems are the threats posed by potential proliferant states and by subnational adversaries. The response to these challenges is determined by evaluating the technical and institutional attributes of the proposed Generation IV nuclear energy systems. The outcomes of the system response are expressed in terms of proliferation resistance and physical performance measures.

The PR&PP approach considers multiple facility types and activities to enable the proliferation pathways from acquisition and processing of material to fabrication of a nuclear explosive device as concealed and overt misuse of nuclear facilities. Types of proliferation considered are overt and concealed diversion, concealed breakout from treaty agreements and misuse, and clandestine facilities. For proliferation resistance, the top-level metrics are technical difficulty, proliferation cost, proliferation time, fissile material type, detection probability, and detection resource efficiency. The final steps in the methodology are the integration of the findings of the analysis and the interpretation of the results. The form of the results includes best estimates from subject matter experts for numerical and linguistic descriptors that characterize the overall proliferation resistance of the fuel cycles (GENIV Forum 2007, 2011b).

The PR&PP studies to date have focused on comparing the proliferation resistance of existing and future fuel cycles. The challenges (threats) are defined by descriptions of generic (not specific) states and adversaries, and limited details are included on the fuel cycle facilities.15

_______________

14For example, safeguards design for new CANada Deuterium Uranium (CANDU) reactors, and accelerator-driven nuclear systems, MYRRHA, in Belgium.

15 General assumptions and categories have been made regarding the technical capabilities of the proposed host state. More information can be found at http://www.gen-4.org/Technology/horizontal/proliferation.htm.

Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×

pag

FIGURE 3.1. PR&PP’s methodology for assessment of proliferation resistance and physical protection. SOURCE: GENIV (2011a).

EVALUATION OF METHODOLOGIES

The methodologies were evaluated against a set of characteristics developed by the committee. Appendix B describes the evaluation and provides a summary table of the results.

The committee found that none of the methodologies currently capture specific host-state factors and therefore none currently assess proliferation risk, for example, the probability that a specific host state will choose to proliferate along a particular pathway, the probability of success along that path, and the consequences of proliferation summed over all possible pathways (Takakai et al. 2005, Pomeroy et al. 2008). The methodologies do assess proliferation resistance to allow for relative comparison between a set of given fuel cycles. There are some attempts to capture state-specific aspects via generic descriptions (e.g., a non–nuclear weapons state with nuclear energy systems, technically competent, and a signatory to the Treaty on the Non-Proliferation of Nuclear Weapons (NPT, GENIV Forum2011b or by assuming the intent to proliferate is maximized and constant but there are no examples of a specific host state or an existing nuclear energy facility.16

The general approach to assessing proliferation resistance is the same for all of the methodologies: to outline the detailed steps of the given nuclear fuel cycle and to evaluate a set of metrics (or attributes) that characterize barriers to proliferation against a given threat. The number and details of the attributes and how they are combined differ between methodologies but the results are consistent with each other and are in agreement with general accepted judgments on proliferation barriers within nuclear fuel

_______________

16 Breakout scenarios could be assessed by considering only the intrinsic resistance (and removing any barriers introduced by extrinsic measures such as safeguards).

Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×

cycles (e.g., enrichment and reprocessing steps have the lowest proliferation barriers).. The committee judges this predefined framework approach to be sound in defining a set of steps in a complex system for analysis and organization of data to allow for comparisons between fuel cycles. However, the committee determined that there were shortcomings in the execution of the assessments and inherent limitations on their application.

Shortcomings in Execution

Expert Elicitation

At their core, all predefined frameworks rely on expert knowledge. Information is obtained from subject matter experts leading to a score for each attribute and weighting factors for aggregation. Because expert knowledge is combined into higher-level outputs, the details of these practices and their impact on the results can be obscured.

The committee found that two important processes related to gathering data from experts were found to be poorly implemented: selection of experts and expert elicitation.

The U.S. Nuclear Regulatory Commission (USNRC) has published a technical position (NUREG-1563) on expert elicitation that “describe[s] acceptable procedures for conducting expert elicitation when formally elicited judgments are used to support a demonstration of compliance” (Kotra et al. 1996). The U.S. Environmental Protection Agency has also developed clear procedures for gathering expert judgment and knowledge (EPA 2009). These practices when implemented have produced results of higher quality than other methods and are therefore considered “best practices” for expert elicitation. The guidance from the USNRC describes how formal expert elicitations should be performed:

Expert elicitation is a formal, highly structured, and well-documented process whereby expert judgments, usually of multiple experts, are obtained. Although informal expert judgment involves only subject-matter experts, formal expert elicitations usually involve normative experts, generalists, and subject-matter experts. (Kotra et al. 1996, p. 3)

A normative expert is one trained in decision analysis, statistics, and probability; a generalist has broad training across the entire elicitation subject area; and subject matter experts are specialists concerning subparts of the subject area. In none of the six methodologies was the selection process of experts identified nor was there mention of the use of a normative expert (GENIV Forum 2011a) or a generalist. The most recent PR&PP documentation (GENIV Forum 2011a) does provide some general guidance on criteria for selection of experts, although the need for a normative expert or generalist is not part of the guidance.

The USNRC and EPA guidance also describes methods for eliciting information from the experts and the importance of documenting the results. In most of the methodologies, the process for collecting expert information was not stated. Some of the methodologies did indicate that expert knowledge was collected through the use of questionnaires and surveys (TAMU MAUA and PR&PP). Additionally, the committee notes that surveys and questionnaires have been found to be problematic in the methods

Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×

of expert elicitation. The impact of poor execution of expert elicitation was recently recognized by the PR&PP GIF working group:

informal expert elicitations often provide demonstrably biased or otherwise flawed answers to problems. Without a formal process and strong controls, experts may be asked to provide judgments on issues that go beyond their expertise, or their estimates might be combined in misleading ways which distort the results (GENIV Forum 2011a, p.67; Budlong-Sylvester et al. 2006).

However, the committee has found the proposed changes to the process of expert elicitation in PR&PP Revision 6 to be insufficient to effectively address these specific shortcomings (GENIV Forum 2011a) because experts are still solicited through a survey form, for example.17

Uncertainty Analysis

In technical assessments utilizing expert knowledge, the aggregate uncertainty of the results to changes in assumptions and information is an important factor in determining the confidence level of the results. It is important to define the uncertainties at the expert elicitation phase and to properly account for the uncertainties through their aggregation to the final result. In the predefined framework methodologies that were reviewed, the TAMU MAUA methodology accounts for measurement uncertainties as part of one high-level attribute but does not otherwise include them in its analysis. The PR&PP methodology notes that the results contain uncertainties because the measurement ranges are gross (high, medium, low).

The PR&PP, TAMU MAUA, and RIPA methodologies are, in theory, capable of being extended to doing quantitative uncertainty analyses. To date, these methodologies, in practice, do not provide a mechanism for addressing or quantifying uncertainty, although guidance for using the PR&PP framework indicates that this should be done (see PR&PP overview above).

Sensitivity Analysis

Sensitivity analysis of the final results (which may not be a single number or qualitative rating) to changes in the information gathered and the underlying assumptions by the experts often provides significant insight and may be more important than the final result per se. Only the SAPRA and JAEA methodologies reported sensitivity analyses against a subset of attributes and highlighted the effect on the results. As noted above for uncertainty analyses, the PR&PP, TAMU MAUA, and RIPA methodologies, in theory, can perform quantitative sensitivity analyses but did not include these analyses in the examples seen by the committee.

_______________

17 In best practices for expert elicitation established by the EPA and USNRC, a discussion leader requires the experts to explain the basis for their evaluation in front of the other experts. This leads to (a) making sure that each expert has the same understanding of the question, and (b) questions and challenges from other experts and modification of views in response to new facts and assumptions. The reason surveys are undesirable is that this interaction of the experts is lost or is not as efficient.

Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×

Limited Applicability

Facility details (e.g. technical design features, operational modalities, institutional arrangements and safeguards measures) are not known for deployments of future fuel cycles and may not be known during and throughout construction, especially for some countries in which the United States may have proliferation concerns. Country-specific and facility-specific factors will strongly affect the final proliferation resistance of a particular fuel cycle facility at the time of fuel cycle deployment and for several decades thereafter. Therefore, proliferation resistance assessments of future fuel cycles are limited to the known engineering details related to the intrinsic attributes of the proposed fuel cycle. Expert-guided assumptions and estimates are made for extrinsic attributes (which should have well-documented uncertainties associated with them). These assessments have a “shelf life” because the proliferation resistance will change as the barriers to proliferation are further defined throughout the development of the fuel cycle and the design of the facility. Assessments should not be considered final and should be periodically updated as more details of the specific fuel cycle or facility are determined.

Unfortunately, decision makers and policy makers need to make informed decisions related to future fuel cycles, nuclear exports, and peaceful nuclear agreements, as discussed in Chapter 2. This leads to an inherent limitation for the predefined frameworks because in these applications, the facility and host-state details are not defined. Of course, the case-by-case and other analyses are similarly limited by lack of data.

To summarize, the committee considered a set of predefined frameworks and found the following:

shortcomings in their execution because of

pag poor and/or undocumented expert elicitation processes and

pag lack of sensitivity and uncertainty analyses;

inherent limitation of applicability because of

pag unknown facility and host-state details for future fuel cycles and

pag limited shelf life of assessments.

UTILITY OF PREDEFINED FRAMEWORKS FOR DECISION MAKERS

In addressing this task, the committee notes that the “utility” of a methodology is subjective and dependent on the individual and/or organization. The committee considered the frequency of use as an indirect measure of utility and discussed the apparent effect of the methodologies’ results with policy makers and decision makers to determine their subjective opinions.

The committee considered the extent to which predefined frameworks are used and have been applied to various types of decisions. Only two examples were provided despite requests to both sponsors and to many of the policy and decisions makers who provided briefings or information to the committee (see Appendix D). These two examples involve the use of the PR&PP methodology: (1) during the process of

Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×

BOX 3.1
Examples of the Use of Predefined Framework Methodologies

When asked for examples in which predefined framework analysis played a part in policy or decision making within the U.S. government, the following two examples were cited by NNSA.

GNEP: During the process of formulating plans for the Global Nuclear Energy Partnership (GNEP) initiative, the Unites States used a predefined framework methodology (PR&PP) to comparatively assess the resistance to proliferation of different future fuel cycle alternatives in three categories: once-through, full actinide recycle, and partial actinide recycle (DOE 2008b). Results highlighted strengths and weaknesses of the different fuel cycles and summarized the results as follows: “Because the alternatives present complementary risks and benefits, this assessment does not identify a preferred alternative or alternatives” (DOE 2008b, p. xv). This assessment was performed in parallel with preparation of the draft GNEP Programmatic Environmental Impact Statement (PEIS) (DOE 2008b) which stated that change to a closed fuel cycle represented DOE’s preferred option.

Pyroprocessing: In the context of discussions with South Korea about renewing its Nuclear Cooperation Agreement with the United States, South Korea has asked the United States for its consent to use pyroprocessing for its U.S.-origin spent fuel. When pyroprocessing was first considered by South Korea, the United States made a decision that pyroprocessing was not considered to be reprocessing and was acceptable. A more recent technical analysis using the PR&PP methodology (DOE 2008b, p. 68) determined that there was negligible difference in proliferation resistance between pyroprocessing (now further developed) and PUREX as compared to the once-through fuel cycle. This study was used in part to justify a change in policy with respect to pyroprocessing, which the South Koreans continue to contest. The two countries have subsequently undertaken a joint 10-year study on spent fuel management, one part of which will consider the proliferation resistance of pyroprocessing. This example highlights that proliferation resistance assessments made on technologies under development should be performed throughout the development cycle as further details are determined and new information that may have significant impact on proliferation resistance becomes known.

formulating plans for GNEP and (2) to compare pyroprocessing with traditional approaches to recycling. Details are provided in Box 3.1.

In its meetings with DOE and its contractors this committee heard that NNSA did not usually use any predefined framework methodology to guide nonproliferation decisions, especially in cases involving country-specific considerations. Reasons given for this include:

Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×

• not wanting to rely on a single rolled-up measure,18

• concern that the framework will make (instead of inform) a decision or might “box the decision maker in,”

• concern that “proliferation resistance” may be misinterpreted as “proliferation-proof”

As noted in Chapter 2, the committee found that nuclear nonproliferation policy makers and decision makers use their own knowledge, coupled with technical analysis by multidisciplinary teams of subject matter experts established on a case-by-case basis to provide insight into nuclear proliferation risk, and have shown little interest in multidisciplinary teams using predefined frameworks or formal risk-based approaches for such decision making. Based on the results to date, there is little expectation that predefined framework methodologies will provide additional insight, a view supported by previous findings including a brief, initial proliferation resistance assessment referenced in the TOPS report (NERAC TOPS Task Force 2001, p.15): “In some respects, most of the findings from the analyses were not new and simply reinforced the judgments that had arisen over the years.”

The committee found several examples in other domains within the U.S. government in which decision makers utilize predefined framework-like tools to inform decisions. Examples include the Office of Cooperative Threat Reduction within the Department of State in which an assessment tool is used to inform the prioritization of countries for engagement on nuclear, chemical, and biological security (Dolliff 2012); and the Domestic Nuclear Detection Office within the Department of Homeland Security, which uses a risk-based tool to guide the optimization of architectures for global nuclear detection (Streetman 2012). Such methodologies are apparently useful to policy makers and decision makers dealing with complex problems and with a willingness to engage in the analysis process and not simply the results. In these cases, the users recognize that the tools are not predictive and that they are not beholden to the results. Rather, the tools provide a structure for organizing complex problems with a large number of variables and assessing which factors are most important to the results that inform the final decisions.

While recognizing the limitations of predefined frameworks, the committee judges that those frameworks can have value if well executed. They provide a structured approach that causes the analyst to explore a range of possibilities in assessing proliferation resistance. They also have value as a way to consistently compare the attributes of potential future nuclear fuel cycles and for identifying where safeguards can be most effective in raising barriers to proliferation. They provide a common lexicon and structure for communicating with international partners about nuclear energy decisions. In addition, they provide a valuable structure for education of the next generation of experts on nuclear energy and nonproliferation.

_______________

18 In the committee’s review of the frameworks, it was noted that many predefined framework approaches do not yield only a single output and none of them have to do so.

Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×

FINDING 2.1: Predefined frameworks have been developed and used to assess the proliferation resistance of partial or full nuclear fuel cycles. These methods provide a useful framework for comparing the intrinsic metrics or “attributes” of existing and potential future nuclear fuel cycles and for identifying where safeguards can be most effective in raising barriers to proliferation. However, these comparisons address a small subset of the wider range of issues faced by policy makers and the committee was able to determine that the frameworks have rarely been used to inform policy decisions. Additionally, there have been shortcomings in their execution.

Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×
Page 26
Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×
Page 27
Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×
Page 28
Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×
Page 29
Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×
Page 30
Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×
Page 31
Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×
Page 32
Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×
Page 33
Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×
Page 34
Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×
Page 35
Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×
Page 36
Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×
Page 37
Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×
Page 38
Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×
Page 39
Suggested Citation:"3 Utility of Proliferation Assessments." National Research Council. 2013. Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles. Washington, DC: The National Academies Press. doi: 10.17226/18335.
×
Page 40
Next: 4 Applicaiton of Other Risk Methodologies »
Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles Get This Book
×
Buy Paperback | $40.00 Buy Ebook | $32.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The material that sustains the nuclear reactions that produce energy can also be used to make nuclear weapons—and therefore, the development of nuclear energy is one of multiple pathways to proliferation for a non-nuclear weapon state. There is a tension between the development of future nuclear fuel cycles and managing the risk of proliferation as the number of existing and future nuclear energy systems expands throughout the world. As the Department of Energy (DOE) and other parts of the government make decisions about future nuclear fuel cycles, DOE would like to improve proliferation assessments to better inform those decisions.

Improving the Assessment of the Proliferation Risk of Nuclear Fuel Cycles considers how the current methods of quantification of proliferation risk are being used and implemented, how other approaches to risk assessment can contribute to improving the utility of assessments for policy and decision makers. The study also seeks to understand the extent to which technical analysis of proliferation risk could be improved for policy makers through research and development.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!