In this report, “quantifying uncertainty” in a prediction for a QOI means making a quantitative statement about the values that the QOI for the physical system may take, often in a new, unobserved setting. The statement could take the form of a bounding interval, a confidence interval, or a probability distribution, perhaps accompanied by an assessment of confidence in the statement. Much more is said on this topic throughout this Introduction and the rest of the report.
There is wide but not universal agreement on the terms, concepts, and definitions described above. These and other terms, many of which are potentially confusing terms of art, are discussed in the Glossary (Appendix A).
Mathematical models for the computational simulation of complex real-world processes are a crucial ingredient in virtually every field of science, engineering, medicine, and business. The focus of this report is on physics-based and engineering models, which often provide a strong basis for producing useful extrapolative predictions.
There is a wide range of models, but the science and engineering models on which this report focuses are most commonly composed of integral equations and partial and ordinary differential equations. Each modeling scenario has unique issues and characteristics that strongly affect the implementation of VVUQ. Relevant issues include the following:
• The level of empiricism versus physical laws encoded in the model,
• The availability and relevance of physical data to the predictions required by the scenario,
• The extent of interpolation versus extrapolation needed for the required predictions,
• The complexity of the physical system being modeled, and
• The computational demands of running the computational model.
The modeling framework assumed throughout most of this report is common in science and engineering: a complex physical process or structure is modeled using applied mathematics, typically with a mathematical model consisting of partial differential equations and/or integral equations and with a computational model that solves a numerical approximation of the mathematical model. Referring to the issues listed above, this report considers scenarios in which:
• Models are strongly governed by physical constraints and rules,
• The availability of relevant physical observations is limited,
• Predictions may be required in untested and/or unobserved physical conditions,
• The physical system being modeled may be quite complex, and
• The computational demands of the model may limit the number of simulations that can be carried out.
Of course many of these scenarios are found in other simulation and modeling applications. To this extent, the topics covered in this report are applicable to other modeling applications.
The focus on mathematical foundations of VVUQ leads this committee to omit important concepts of model evaluation that are more qualitative in nature. The NRC report Models in Environmental Regulatory Decision Making (NRC, 2007a) considers a much broader perspective on model evaluation, including topics such as conceptual model formulation, peer review, and transparency that are not considered in this report. Behavioral Modeling and Simulation: From Individuals to Societies (NRC, 2007b) considers VVUQ for behavioral, organizational, and societal models.