• Development of scalable methods for constructing emulators that reproduce the high-fidelity model results at training points, accurately capture the uncertainty away from training points, and effectively exploit salient features of the response surface.
• Development of phenomena-aware emulators, which would incorporate knowledge about the phenomena being modeled and thereby enable better accuracy away from training points.
• Development of methods for characterizing rare events, for example by identifying input configurations for which the model predicts significant rare events, and estimating their probabilities.
• Development of methods for propagating and aggregating uncertainties and sensitivities across hierarchies of models. (For example, how to aggregate sensitivity analyses across micro-scale, meso-scale, and macro-scale models to give accurate sensitivities for the combined model remains an open problem.)
• Research and development in the compound area of (1) extracting derivatives and other features from large-scale computational models and (2) developing UQ methods that efficiently use this information.
• Development of techniques to address high-dimensional spaces of uncertain inputs.
• Development of algorithms and strategies across the spectrum of UQ-related tasks that can efficiently use modern and future massively parallel computer architectures.
• Development of methods and strategies to quantify the effect of subject-matter judgments, which necessarily are involved in validation and prediction, on VVUQ outcomes.
• Development of methods that help to define the “domain of applicability” of a model, including methods that help quantify the notions of near neighbors, interpolative predictions, and extrapolative predictions.
• Development of methods or frameworks that help with the important problem of relating model-to-model differences, among models in an ensemble, to the discrepancy between models and reality.
• Development of methods to assess model discrepancy and other sources of uncertainty in the case of rare events, especially when validation data do not include such events.
Computational modeling and simulation will continue to play key roles in research in engineering and physical sciences (and in many other fields). It already aids scientific discovery, advances understanding of complex physical systems, augments physical experimentation, and informs important decisions. Future advances will be determined in part by how well VVUQ methodology can integrate with the next generation of computational models, high-performance computing infrastructure, and subject-matter expertise. This integration will require that students in these various areas be adequately educated in the mathematical foundations of VVUQ. The committee observes that students in VVUQ-dependent fields are not as well prepared today as they could be to deal with uncertainties that invariably affect problem formulation, software development, and interpretation and presentation of results. As requested by its tasking, the committee identified several actions that could help to address this.
Recommendation: An effective VVUQ education should encourage students to confront and reflect on the ways that knowledge is acquired, used, and updated.
Recommendation: The elements of probabilistic thinking, physical-systems modeling, and numerical methods and computing should become standard parts of the respective core curricula for scientists, engineers, and statisticians.
Recommendation: Researchers should understand both VVUQ methods and computational modeling to more effectively exploit synergies at their interface. Educational programs, including research programs with graduate-education components, should be designed to foster this understanding.
Recommendation: Support for interdisciplinary programs in predictive science, including VVUQ, should be made available for education and training to produce personnel that are highly qualified in VVUQ methods.