application, the appropriately defined QOI should be sensitive to large-scale but not small-scale behavior. In this case the VVUQ analysis may find that the model is sufficiently accurate (e.g., uncertainties in the predicted QOI are sufficiently small) to provide actionable information. However, if small-scale details are important, the QOI should be defined accordingly, and the VVUQ analysis (of the same model applied to the same physical system) may find that the model is too inaccurate to be of value.

Leveraging work from previous VVUQ analyses should be done with caution. Since VVUQ results are specific to particular QOIs in particular settings, transferring results to new QOIs and settings can be difficult to justify. However, one can consider applying VVUQ to a model over a broad set of conditions and QOIs if physical data are available to support such wide-ranging assessments of model accuracy and there is a firm theoretical understanding of the physical phenomena being modeled. It can be argued that an example of such a situation is the Monte Carlo N-Particle transport code,1 a particle-transport code that incorporates a large body of knowledge and has been tested against measurements derived from thousands of experiments spanning many particle types and a broad range of conditions.

Within the VVUQ enterprise, the level of rigor employed should be commensurate with the importance and needs of the application and decision context. Some applications involve high-consequence decisions and therefore require a substantial VVUQ effort; others do not.

7.1.1 Verification Principles and Best Practices

Here the committee summarizes key verification principles, along with best practices associated with each principle. Chapter 3 provides more detail.

•  Principle: Solution verification is well defined only in terms of specified quantities of interest, which are usually functionals of the full computed solution.

—Best practice: Clearly define the QOIs for a given VVUQ analysis, including the solution verification task. Different QOIs will be affected differently by numerical errors.

—Best practice: Ensure that solution verification encompasses the full range of inputs that will be employed during UQ assessments.

•  Principle: The efficiency and effectiveness of code and solution verification can often be enhanced by exploiting the hierarchical composition of codes and mathematical models, with verification performed first on the lowest-level building blocks and then on successively more complex levels.

—Best practice: Identify hierarchies in computational and mathematical models and exploit them for code and solution verification. It is often worthwhile to design the code with this approach in mind.

—Best practice: Include in the test suite problems that test all levels in the hierarchy.

•  Principle: Verification is most effective when performed on software developed under appropriate software quality practices.

—Best practice: Use software configuration management and regression testing and strive to understand the degree of code coverage attained by the regression suite.

—Best practice: Understand that code-to-code comparisons can be helpful, especially for finding errors in the early stages of development but that in general they do not by themselves constitute sufficient code or solution verification.

—Best practice: Compare against analytic solutions, including those created by the method of manufactured solutions—a technique that is helpful in the verification process.

•  Principle: The goal of solution verification is to estimate, and control if possible, the error in each QOI for the problem at hand. (Ultimately, of course, one would want to use UQ to facilitate the making of decisions in the face of uncertainty. So it is desirable for UQ to be tailored in a way to help identify ways to reduce uncertainty, bound it, or bypass the problem, all in the context of the decision at hand. The use of VVUQ for uncertainty management is discussed in Section 6.2, “Decisions Within VVUQ Activities”.)


1 See Accessed September 7, 2011.

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement