between uncertainty quantification for climate change and ISI prediction is that hindcasts play a more important role in the latter; models that perform better in hindcast are more likely to perform better in forecast on shorter time scales when the effects of nonstationarity are more minor, for example, as in ISI versus decadal to century time scales. In this sense, optimal selection and weighting of models can be an important piece of an overall strategy not only for quantification, but also for reduction, of uncertainty, leading to improvement in ISI prediction skill.
Uncertainty in weather and climate model parameterizations of subgrid-scale physical processes is being addressed through stochastic parameterization methods, which have been reported to improve the probabilistic reliability of seasonal forecasts by some climate models (see Chapter 4).
There are nascent efforts to reduce the climatological biases of models through multivariate optimization of uncertain parameters. Stainforth et al. (2005) randomly perturbed a set of uncertain parameters in a version of the UKMO climate model and compared 2,017 resulting models against a suite of climatological error metrics; the best of the perturbed models had a modest 15 percent error reduction over the control model. Jackson et al. (2008) used a more systematic multivariate sampling and optimization approach on the CAM3 atmospheric general circulation model, finding 6 configurations of more than 500 tested that improved an overall measure of climatological error by 7 percent compared to the regular model. These improvements are significant but modest, and the parameter optimization needs to be repeated each time a new model version or a change in grid resolution is introduced. This experience suggest that, as models get more complex, periodic automatic parameter optimization may be valuable, but perhaps more as a device to save human effort involved in trial-and-error optimization (at the cost of more computer time) rather than as a method to make a model of substantially higher fidelity. Furthermore, it suggests that the systematic errors related to uncertain parameters in climate models are heavily compensating, such that improvements in one field are balanced by degradation in another so that the overall result is something of a wash.
Hence, it seems likely that structural errors in parameterizations or inadequacies in grid resolution not correctable by parameter tuning are probably a larger driver of systematic errors and projection uncertainty than suboptimal choices of existing uncertain parameters. In this environment, there is a tradeoff between maintaining fluidity of the model development process and the huge investment of computer time needed to apply the rigorous principles of uncertainty quantification and optimization. Some modeling groups, such as the Geophysical Fluid Dynamics Laboratory, are experimenting with some automatic parameter tuning as a routine part of model