Review of the U.S. Climate Change Science Program's Synthesis and Assessment Product 5.2, "Best Practice Approaches for Characterizing, Communicating, and Incorporating Scientific Uncertainty in Climate Decision Making" (2007)

1. Types of model-related uncertainty: In discussing the types of scientific uncertainty, particularly in the context of model-based estimation and prediction, it would be useful to exploit the conventional formulation involving observable quantities, models, and parameters. This formulation provides a convenient way to distinguish among non-modeled variability in observables, uncertainty over parameter values, and uncertainty over model specification.

2. Support for conclusions: There is not always a clear connection between the material presented and the findings, recommendations, and conclusions. This comment pertains to conclusions stated within the body of the draft and also to the recommendations listed in Chapter 8 of the draft. Concerning the latter, the committee finds that it is difficult to distinguish which of these statements are the opinions of the author and which are conclusions supported by the material discussed within the report. The committee suggests that the authors take steps to ensure that findings and conclusions in the final chapter are clearly grounded in material that is fully discussed in the chapters.

3. Certain versus debatable: Similarly, there are instances within the main body of the material where matters are stated with certitude that the committee consider to be debatable. As an example, consider on Page 4, Lines 32-33, where the authors claim that “Barring [sic] a few exotic exceptions, uncertainty about model functional form is inherently epistemic.” The committee suggests that model functional form is often epistemic, but this is not inherently true except in a few exotic circumstances.

4. Policy prescription: The authors of SAP 5.2 are charged with recommending ways to incorporate discussion of uncertainty into decision making, and this task is inherently policy prescriptive. However, the committee believes that the authors should recognize that in certain cases direct policy prescription should be avoided. For example, line 16 on Page 38 (“it is generally best to adopt robust strategies”) could be interpreted as recommending a specific course of action for policymakers, in this case to address the middle range of uncertainties, when policymakers may be better served by addressing less probable but extremely negative impacts.

5. Stylistic issues: The committee notes several stylistic issues, which, if addressed, could significantly improve the overall accessibility of the document for a wider audience and improve the coherence of the document. Specific instances will be

noted in the sections of this report that provide reviews of individual chapters of the draft. Broadly, these issues are as follows:

5a. Jargon and definitions: The language suffers from excessive use of jargon, overly complex explanations for concepts that could have been stated more clearly, and a lack of definitions of terms that may have multiple meanings to multiple readers. Some of these issues are related to the question of the target audience for this Synthesis and Assessment Product (SAP). If the final product, or some portion of it, is intended to serve as a practical guide for decision makers and non-scientific users of uncertainty information, then the language should be appropriate for that audience. Notwithstanding this question of audience, the committee believes that even if the document is intended for the narrower subset of the Climate Change Science Program (CCSP) researchers and assessors, there is benefit in stating difficult concepts in straightforward language. The authors might also consider adding a glossary for less commonly understood terms and phrases.

5b. Use of examples: It would be very helpful if the authors would consider providing one or two “real-world” examples of a climate-decision making scenario, and then carrying those examples through the rest of the document as points of reference. In each chapter, relating the technical material in question to these examples would not only elucidate that material for the uninitiated reader, but would also help to shore up the framework of the document.

5c. Use of quotes: The committee believes that the authors make excessive use of long, direct quotes from the literature. In most cases the point the authors wish to make could be conveyed adequately by paraphrasing these quotes into simpler statements.

5d. Content arrangement: The committee recommends that the content arrangement of the chapters be reconsidered once the needed material has been incorporated into the document. One approach might begin with a framing of the issue, followed by discussion of the sources and types of uncertainty, followed by a synthesis of the various methods for estimating uncertainty (with a progression from conventional methods to expert elicitation), with later material providing information on methods of communicating uncertainty and concluding with recommendations for best practices. An enhanced emphasis on communication might also include material from both Chapters 2 and 6. The material in Chapter 7 could be placed earlier in the document and amended slightly to provide the recommended addition of contextual information early in the document. Arranging the discussion of techniques to begin with the outlining of objective and then more subjective methods could also enhance readability. In its current form, the discussion somewhat confusingly alternates between the two types of methodologies.