Like the other synthesis and assessment products, SAP 5.3 is being produced with independent oversight and review from the wider scientific and stakeholder communities, as recommended by the National Research Council (NRC) review of the U.S. CCSP Strategic Plan (National Research Council, 2004b). NOAA as the lead agency asked the NRC to perform the independent review of SAP 5.3, and our panel was created to perform the task.
The Panel to Review CCSP Draft Synthesis and Assessment Product 5.3: Decision-Support Experiments and Evaluations Using Seasonal to Interannual Forecasts and Observational Data was asked to address the following review criteria:
Are the goals, objectives and intended audience of the product clearly described in the document? Does the product address all questions outlined in the prospectus?
Are any findings and/or recommendations adequately supported by evidence and analysis? In cases where recommendations might be based on expert value judgments or the collective opinions of the authors, is this acknowledged and supported by sound reasoning?
Are the data and analyses handled in a competent manner? Are statistical methods applied appropriately?
Are the document's presentation, level of technicality, and organization effective? Are the questions outlined in the prospectus addressed and communicated in a manner that is appropriate and accessible for the intended audience?
Is the document scientifically objective and policy neutral? Is it consistent with the scientific literature?
Is there a summary that effectively, concisely and accurately describes the key findings and recommendations? Is it consistent with other sections of the document?
What other significant improvements, if any, might be made in the document?
The panel received a draft of SAP 5.3 report “Decision-Support Experiments and Evaluations Using Seasonal to Interannual Forecasts and Observational Data” when it was completed on July 5, 2007. A table of contents for the draft appears in Box 1-2. The draft was prepared by a team of authors working over a 6-month period. We met on July 17 with three of the lead authors—Helen Ingram, the product lead author, and Nathan Mantua and David Feldman, two of the convening lead authors—to ask questions about the authoring team’s research and formulation of the draft document. During this meeting, we also interacted with NOAA personnel, who outlined their expectations for SAP 5.3.
Nancy Beller-Simms of NOAA described the document as “very much a first draft” and asked us for comments that would help the authors make revisions to strengthen it. Ingram said that the authoring group recognized the draft as needing reorganization—for one thing, it was written in subgroups that separated the natural scientists from the social scientists, so that integration was as yet unsatisfactory to the authoring group. She and the other authors present mentioned some of their ideas for reorganization. They also said that they expected that, in revision, the concluding chapter would expand and that it would address the idea of a climate service. The authors did not see the draft as requiring major additional writing, however.