• Uncertainty quantification (UQ). How do the various sources of error and uncertainty feed into uncertainty in the model-based prediction of the quantities of interest?
Computational scientists and engineers have made significant progress in developing these processes and using them to produce not just a single predicted value of a physical quantity of interest (QOI) but also information about the range of values that the QOI may have in light of the uncertainties and errors inherent in a computational model. However, there remain many open questions, including questions about the mathematical foundations on which various processes and methods are based or could be based.
In recognition of the importance of computational simulations and the need to understand uncertainties in their results, the Department of Energy’s (DOE’s) National Nuclear Security Administration, the DOE’s Office of Science, and the Air Force Office of Scientific Research requested that the National Research Council study the mathematical sciences foundations of verification, validation, and uncertainty quantification (VVUQ) and recommend steps that will lead to improvements in VVUQ capabilities. The statement of task is as follows:
• A committee of the National Research Council will examine practices for VVUQ of large-scale computational simulations in several research communities.
• The committee will identify common concepts, terms, approaches, tools, and best practices of VVUQ.
• The committee will identify mathematical sciences research needed to establish a foundation for building a science of verification and validation (V&V) and for improving the practice of VVUQ.
• The committee will recommend educational changes needed in the mathematical sciences community and mathematical sciences education needed by other scientific communities to most effectively use VVUQ.
KEY PRINCIPLES AND PRACTICES
The Committee on Mathematical Foundations of Verification, Validation, and Uncertainty Quantification views its charge as emphasizing the mathematical aspects of VVUQ and, because of the breadth of the subject overall, has limited it focus to physics-based and engineering models. However, much of its discussion applies more broadly. Although the case studies presented in this report include physics or engineering considerations, they are meant to illuminate mathematical aspects of the associated VVUQ analysis. The committee noted several key VVUQ principles: As a first step toward identifying best practices,
• VVUQ tasks are interrelated. A solution-verification study may incorrectly characterize the accuracy of a code’s solution if code verification was inadequate. A validation assessment depends on the assessment of numerical error produced by solution verification and on the propagation of model-input uncertainties to computed QOIs.
• The processes of VVUQ should be applied in the context of an identified set of QOIs. A model may provide an excellent approximation to one QOI in a given problem while providing poor approximations to other QOIs. Thus, the questions that VVUQ must address are not well posed unless the QOIs have been defined.
• Verification and validation are not yes/no questions with yes/no answers, but rather are quantitative assessments of differences. Solution verification characterizes the difference between a computational model’s solution and that of the underlying mathematical model. Validation involves quantitative characterization of the difference between computed QOIs and true physical QOIs.
Specific to verification, the committee identified several guiding principles and associated best practices. The main text discusses all of these and provides supporting detail. Some of the more important principles and practices are summarized here:
• Principle: Solution verification is well defined only in terms of specified quantities of interest, which are usually functionals of the full computed solution.