In the case of the Stockpile Stewardship Program (SSP) described in Section 6.4, similar types of decisions must be made. The SSP has developed its own framework, known as Quantification of Margins and Uncertainty (QMU), that produces a quantity known as the margin-to-uncertainty (M/U) ratio (Goodwin and Juzaitis, 2006). If the M/U ratio is “large,” diligence is required, but the safety, security, and reliability of the weapon system are assured. If the M/U ratio approaches unity, decision makers are presented with a variety of options that involve trade-offs between two broad categories: increasing the margin or reducing the uncertainty. Each of these choices involves decisions that must take into account computational models and uncertainties as well as stochastic variables.
In many cases (and in the two examples referred to above), there is a close analogy to several areas of optimization that can play an important role in the mathematical foundation for decision making based on VVUQ. For example, the field of multiobjective optimization (Miettinen, 1999) is focused on the development of methods and algorithms for the solution of problems that involve multiple objectives that must be simultaneously minimized. This leads to approaches that can be used to trade off among different options. Stochastic optimization (Ermoliev, 1988; Heyman and Sobol, 2003) is another relevant area, in which some of the design parameters or constraints are described by random variables. The theory for these types of problems could be used to develop better bounds on the uncertainties associated with each decision. The simulation optimization field has other approaches. One alternative approach is that of robust optimization. Here, one seeks to find optimal solutions over a broad range of nonstochastic but uncertain input parameters (Ben-Tal and Nemirovski, 2002). In this case, a robust solution is one that remains “optimal” under the entire range given for the uncertain input parameters (Taguchi et al., 1987). These types of solutions are desirable if available, because decision makers can be assured that whatever option they choose, the consequences of uncertain input parameters will not generate large changes from the optimal solutions. All of these examples indicate that optimization will be a central component of the mathematical foundation for decision making under uncertainty.
A summary of the body of information that enables an assessment of the appropriateness of a model and its ability to predict the relevant quantities of interest (QOIs), as well as inclusion of the key assumptions used to make the prediction, is a necessary part of reporting model results. This information will allow decision makers to understand better the adequacy of the model as well as the key assumptions and data sources on which the reported prediction and uncertainty rely. In addition, the finding regarding transparency and documentation stated in Section 6.2 should be made available to decision makers and peer reviewers.
It is important to recognize that a UQ study will often be an ongoing effort, with decision making happening throughout the study, with respect to both the study itself and the external applications. The climate-modeling case study discussed in Section 2.10 in Chapter 2 is an example of such an ongoing effort—only limited UQ information is currently available for use in policy decisions. This example highlights the need for the development of decision-making platforms that can be based on only partial or very limited UQ information. It also highlights the need to identify situations for which more detailed UQ characterization would give additional clarity for decision making.
When the moratorium on nuclear testing was begun in 1992, the Department of Energy (DOE) established alternative means for maintaining and assessing the nation’s nuclear weapons stockpile. The SSP was created to “ensure the preservation of the core intellectual and technical competencies of the United States in nuclear weapons” (U.S. Congress, 1994). The SSP must assess, on an annual basis, the safety, security, and reliability of the nuclear weapons stockpile in the absence of nuclear testing. A key product of the annual assessment process is a report on the state of the stockpile, issued by the directors of the three national security laboratories—Lawrence Livermore National Laboratory, Los Alamos National Laboratory, and Sandia National Laboratories1—to the Secretary of Energy, the Secretary of Defense, and the Nuclear Weapons Council. With this report in hand, the Secretary of Energy, the Secretary of Defense, and the Commander of the U.S. Strategic Command each write a letter to
1 Sandia National Laboratories is a multiprogram laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000.