A CREDIBLE AND INDEPENDENT REVIEW PROCESS

Assessments differ from more standard scientific publications, and therefore the typical science peer-review process needs modification. Assessments build on prior knowledge; identify recent advances and research needs; attempt to reach consensus on scientific debates; and in some cases, provide response options, including policy options. Because assessments may include policy-relevant information and even some value judgments in the case of impact assessments, assessment reviews need to be conducted to achieve salience, legitimacy, and credibility. In contrast, typical science peer review focuses solely on scientific credibility. Therefore, the review process should be consistent with the goals of the assessment and the type of assessment. For process assessments that focus only on the scientific understanding of the process, an expert peer review may suffice. However, assessments providing decision support or policy options, such as impact and response assessments, may require a broader review, involving stakeholders and decision makers, in particular.

Many assessments that the committee examined have well-established review mechanisms. For the IPCC, the review process includes a peer review, followed by an expert and government review, and finally a review by the governments who are party to the United Nations Framework Convention on Climate Change. The report and the various summaries are then subject to acceptance by the governments, which meet in plenary for this approval process. This government review process has raised some issues regarding credibility due to the potential for government interference with the scientific conclusions. In contrast, the ACIA provides a model of how a government review can be conducted successfully without the resulting perception that governments influenced the scientific consensus. The government review process for the ACIA involved scientists, who had the ultimate authority over the scientific conclusions, and government representatives, who were given the editorial authority over policy options. In the example of the TEAP, no external review process was undertaken because all the key players were already involved and the assessment contained proprietary information.

Because a well-designed review process has the potential to greatly enhance broad stakeholder ownership and the quality of the outcome, it is essential to the credibility, salience, and legitimacy of the assessment. As previously mentioned, an assessment review is distinct from a peer review in that it cannot be undertaken solely from the perspective of scientific credibility, but must also focus on issues of salience and legitimacy. The committee found that an effective approach is a staged review, such as employed in the ACIA, beginning with the scientific community, with subsequent involvement of governments and other relevant stakeholders. To ensure that legitimacy and credibility can be enhanced simultaneously, an approach should



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement