4
Case Studies of Global Change Assessments

The careful examination of past assessments can provide important lessons to steer future efforts. This committee was asked to look at past assessments that had objectives similar to those of the U.S. Climate Change Science Program (CCSP), evaluate the strengths and weaknesses of the selected past assessments, and identify lessons learned that might guide future CCSP assessment activities. The committee could not conduct an exhaustive review of all past assessments, so it limited itself to assessments with objectives similar to the CCSP and where adequate information was available. Because the charge was to provide advice on how to conduct future U.S. assessments, the analysis focuses on issues relevant to the decision-making context in the United States. It is important to acknowledge, however, that the literature shows that assessments of global environmental change are viewed differently in other countries and are less important for decision makers in developing countries than in industrialized countries (Biermann 2006). Also, both global environmental problems and their solutions are perceived and assessed differently by industrialized and developing countries (Gupta 1997).

This chapter summarizes the committee’s review of the eight assessments listed in Table 4.1. For each, the committee describes the science and policy context and stated purpose of the assessment, examines design issues and other elements, and then provides an analysis of strengths and weaknesses. Upon closer examination, the committee found that each of the assessments evaluated has strengths and weaknesses: none is a failure but none is without limits. This provides the variation in form and outcome



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 63
Analysis of Global Change Assessments Lessons Learned 4 Case Studies of Global Change Assessments The careful examination of past assessments can provide important lessons to steer future efforts. This committee was asked to look at past assessments that had objectives similar to those of the U.S. Climate Change Science Program (CCSP), evaluate the strengths and weaknesses of the selected past assessments, and identify lessons learned that might guide future CCSP assessment activities. The committee could not conduct an exhaustive review of all past assessments, so it limited itself to assessments with objectives similar to the CCSP and where adequate information was available. Because the charge was to provide advice on how to conduct future U.S. assessments, the analysis focuses on issues relevant to the decision-making context in the United States. It is important to acknowledge, however, that the literature shows that assessments of global environmental change are viewed differently in other countries and are less important for decision makers in developing countries than in industrialized countries (Biermann 2006). Also, both global environmental problems and their solutions are perceived and assessed differently by industrialized and developing countries (Gupta 1997). This chapter summarizes the committee’s review of the eight assessments listed in Table 4.1. For each, the committee describes the science and policy context and stated purpose of the assessment, examines design issues and other elements, and then provides an analysis of strengths and weaknesses. Upon closer examination, the committee found that each of the assessments evaluated has strengths and weaknesses: none is a failure but none is without limits. This provides the variation in form and outcome

OCR for page 63
Analysis of Global Change Assessments Lessons Learned TABLE 4.1 The Eight Assessments Included in the Comparative Analysis Assessment Brief Description Stratospheric Ozone Assessments Prior to the 1987 Montreal Protocol, there were several national (including NRC) and international assessments analyzing ozone-depleting chemicals and the state of the stratospheric ozone layer (e.g., WMO 1982, 1986a). Following the treaty, a system of expert advisory panels was established to periodically assess the atmospheric science of the ozone layer (WMO 1990a,b, 1992, 1995, 1999, 2003, 2007), the impacts of ozone loss (UNEP 1991a, 1994a, 1998a, 2002a), and the technology and economics of alternatives to ozone-depleting chemicals (UNEP 1991b, 1994b, 1998b, 2002b). Intergovernmental Panel on Climate Change (IPCC) IPCC analyzes scientific and socioeconomic information on climate change and its impacts, and assesses options for mitigation and adaptation. It provides scientific, technological, and socioeconomic findings to the Conference of the Parties to the United Nations Framework Convention on Climate Change (IPCC 1990a,b,c, 1995a,b,c, 2001a,b,c). Global Biodiversity Assessment (GBA) GBA provides a synthesis and analysis of available science on biodiversity to support the work of the UN Convention on Biological Diversity (GBA 1995). National Assessment of Climate Change Impacts (NACCI) Undertaken in response to the Global Change Research Act (1990) to evaluate the impacts of climate change on the United States (NAST 2001). Arctic Climate Impact Assessment (ACIA) Primary objectives were to evaluate and synthesize knowledge and indicators of climate variability, climate change, and ultraviolet radiation in the region; to assess possible impacts of future changes in climate and radiation; and to provide reliable information to both governments and peoples of the region to support policy-making processes (ACIA 2004). Millennium Ecosystem Assessment (MA) MA was designed to answer questions fundamental to various UN conventions dealing with natural resource issues, in particular the consequences of diverse environmental changes on the functioning of ecosystems, including their continuing capacity to deliver services essential to human well-being (MA 2005a,b). German Enquete Kommission on “Preventive Measures to Protect the Earth’s Atmosphere” The Enquete Kommission brings scientists and policy makers together to assess the importance and consequences of stratospheric ozone depletion and climate change for Germany among other dimensions of global environmental change (Enquete Kommission, 1988, 1991).

OCR for page 63
Analysis of Global Change Assessments Lessons Learned Assessment Brief Description Synthesis and Assessment Products by the U.S. Climate Change Science Program (CCSP) The 21 assessment products were designed to address the mandate of the Global Change Research Act, by considering science and policy issues spanning the range of topics addressed by the CCSP. The first product, on temperature trends in the lower atmosphere, was released in April 2006 (CCSP 2006). that is essential in comparative case analysis. Of the eight assessments examined as case studies, the committee selected two of the examples—the National Assessment of Climate Change Impacts (NACCI) and the CCSP Synthesis and Assessment Products—because they are the assessment efforts of the agencies of the U.S. federal government sponsoring this report. The NACCI is an interesting example of a large-scale assessment based largely on regional and sectoral analyses, as well as an interesting experiment in stakeholder participation and multisponsor coordination. Because most of the CCSP Synthesis and Assessment Products are still under way, that assessment effort is considered last, as a “work in progress.” The the Stratospheric Ozone Assessments, Intergovernmental Panel on Climate Change (IPCC) assessments, Global Biodiversity Assessment (GBA), and the Millennium Ecosystem Assessment (MA) were selected because they are the largest and best known global assessment efforts and thus provide the most extensive experiential base from which to learn. The committee also examined the recent Artic Climate Impact Assessment (ACIA) because it was a regional rather than a global assessment (albeit for a large, international region), and because it attempted to learn from the other cases examined here and deployed some innovative procedures based on that learning. The German Enquete Kommission was included because it provides a different model for linking science and decision makers than any of the other assessments, and thus provides a point of comparison on that critical issue. STRATOSPHERIC OZONE ASSESSMENTS OZONE ASSESSMENTS Concerns about anthropogenic destruction of stratospheric ozone first appeared in the 1960s, initially based on emissions of nitrogen oxides and hydrogen oxides from aviation, bombs, and rockets. Subsequently, ozone loss caused by chlorofluorocarbons (CFCs) was first proposed by Molina and Rowland (1974). Early research validated the key qualitative points in their hypothesis, but more authoritative and quantitative resolution required large research advances in reaction kinetics, atmospheric trace-gas measure-

OCR for page 63
Analysis of Global Change Assessments Lessons Learned ments, stratospheric modeling, and computational power. In the 1980s the scientific foundation for these concerns was solidified and then amplified by the discovery of the Antarctic ozone hole, leading to an intensive two-year effort to explain this observation. Atmospheric observations and laboratory studies eventually confirmed ozone-depleting substances as the cause of the ozone hole and also linked these substances to the global stratospheric ozone loss (WMO 1990a). After articulation of the CFC risk, multiple assessments were conducted through the 1970s and early 1980s, organized by the U.S. National Academy of Sciences (NAS), the U.S. National Aeronautics and Space Administration (NASA), the U.K. Department of Environment, the United Nations Environment Programme (UNEP) and World Meteorological Organization (WMO) Coordinating Committee on the Ozone Layer, the European Commission, and others. These were all conventional, small-panel assessments. The U.S. (NAS and NASA) and U.K. assessments were populated by scientists from their own nations. However, none of these efforts had the global legitimacy and credibility required to support an international agreement. Eventually, the separate efforts converged, with increasingly broad participation, into a single, authoritative international assessment process. The first two international ozone assessments (WMO 1982, 1986a) were produced independently of and prior to the Montreal Protocol. Participation in ozone assessments was broadened greatly by the 1985 assessment, which included hundreds of scientists from many countries and secured cosponsorship by WMO, UNEP, the U.K. Department of the Environment, and the German Umweltbundesamt, as well as three U.S. government agencies (WMO 1986a). This document provided the scientific basis for the 1987 Montreal Protocol. As a result of major changes in international politics and support from U.S. industry for international controls, during 1986-1987, negotiations led to the first Montreal Protocol, which represents an unprecedented international agreement to cut production and consumption of CFCs by half and to freeze the consumption of halons. Early assessments were mostly atmospheric process assessments with a couple of attempts to assess impacts or response options (such as technological alternatives to ozone-depleting chemicals). Under the Montreal Protocol, the following consistent structure of three parallel periodic assessments evolved: (1) an assessment of atmospheric processes (WMO 1990a,b, 1992, 1995, 1999, 2003, 2007); (2) an assessment of the effects (human health, agricultural, ecological, materials, later added air quality) (UNEP 1991a, 1994a, 1998a, 2002a); and (3) an assessment of the technology and economics of reducing the use of ozone-depleting substances by the Technology and Economics Assessment Panel (TEAP) (UNEP 1991b, 1994b, 1998b, 2002b).

OCR for page 63
Analysis of Global Change Assessments Lessons Learned Since the initial negotiation of the Montreal Protocol, there has been a rapid series of reductions of major ozone-depleting substances, led by private industry, with parallel tightening of regulatory restrictions on multiple occasions. Consequently, world emissions of ozone-depleting substances have been reduced significantly, and the only remaining issues are methyl bromide, developing-country phase-out schedule for hydrochlorofluorocarbons, and exemptions. Establishing Clear Rationales and Appropriate Institutional Structures. Leadership of the three assessment panels has been relatively stable over time, with a core group of experts leading the assessments since 1981 and the impacts process and TEAP assessments since 1989. It is these leaders, particularly for the process assessments and TEAP, who provided the interface between the policy process and the assessment panels. Through dialogue with the Parties to the Protocol, the assessment panels have focused on key policy questions, ensuring the relevance of results. The assessment panels then organized chapter lead authors to develop detailed report outlines and time lines. Although funding was provided under the Montreal Protocol for travel support for developing-country participants in the assessments, individual governments provided support for the participation of their scientists as well as for publication of the documents. Individual companies provided support for their experts to participate. Except for the inclusion of scientists and technical experts from developing countries, there was no focused effort at capacity building. Designing and Scheduling Assessment Activities. This assessment used the approach of providing an authoritative summary of the state of scientific research, addressed primarily to scientists working in the field. However, its scale and prominence, as well as its technical authority, comprehensiveness, and detail, brought coherence to the public debates over stratospheric ozone science that had never been attained before. In part because of the widely recognized contribution of the 1985 ozone assessment (WMO 1986a), the Montreal Protocol included authorization to establish similar expert assessment panels. Although, with few exceptions, the prior assessments had been exclusively atmospheric science assessments (process assessments), the Montreal Protocol authorized the establishment of four expert assessment panels addressing: atmospheric science (process assessment), impacts of ozone depletion (impact assessment), technology, and economics. The last two were subsequently merged to a single response assessment. The protocol mandates the assessments to be repeated in time to be available to the Parties in advance of their meetings. Over the last two decades, these assessments have been conducted every four years. According

OCR for page 63
Analysis of Global Change Assessments Lessons Learned to their original mandate these assessments are only supposed to update the Parties with the latest and newly available information, not provide a comprehensive review. Nevertheless, these assessments continue to be conducted at a large scale despite the fact that the main objective of these assessments, particularly the process assessment, has been fulfilled. The added value of each new assessment seems to be relatively small compared with the effort and resources committed to the process. Involving the Scientific Community and Other Relevant Experts. The 1985 stratospheric ozone assessment helped develop the approach referred to here and elsewhere as a process assessment. The assessment engaged a critical mass of highly respected scientists worldwide in an intensive process of critically reviewing current advances and later in an intensive scientific peer review. This approach established credibility and legitimacy. The impact assessment panel followed a model similar to that of the process assessment, but at a smaller scale. This panel lacked the same accumulated experience, and faced more difficult problems in that it had to consider multiple areas of impacts with non-overlapping fields of expertise. It also often worked with information from less mature fields. The technology assessment panel merged with the economics panel to form the TEAP, which adopted a strikingly different model. Organizers recognized from the start that success depended on private sector technical participation. Therefore, they designed the process to be easy and attractive for private-sector representatives to participate by focusing on solving problems identified by the private-sector and assessment organizers. As a result, confidential working groups of high-level technical experts from firms facing similar problems (e.g., how to reduce the use of CFCs in producing insulating foams) were convened; a rapid, informal, results-oriented process was established; and the process or product was not peer-reviewed. Instead of peer review, organizers trusted that participants with overlapping levels of expertise and a balance of material interests would police each other for self-serving claims. In almost all cases, the working groups were able to reach consensus without a formal process, simply by deliberating with internal norms of evidence-based argument and basic confidence in each other’s good faith. In a very few cases, groups were unable to come to agreement and included dissenting or minority text in their reports. The inclusion of experts from the firms directly affected by the phase-out of ozone-depleting substances, along with academic and government experts, ensured relevance, credibility, and legitimacy for the process. The committee considers the assessment processes conducted under the Montreal Protocol’s TEAP as representing the single area of conspicuous success in this regard.

OCR for page 63
Analysis of Global Change Assessments Lessons Learned Communicating Scientific Knowledge Accurately and Effectively. Communication of the results of the ozone assessments was simpler than was the case for many other global change assessments because the decision makers and key stakeholders were well defined and limited in number. Inclusion of most of the scientists directly involved relevant research in the process and impact assessments and of the technology leaders in the TEAP ensured communication with scientists and companies. Adequate communication of findings to regulators was ensured by distributing the reports to government decision makers and following up with presentations by scientists leading the processes at international meetings and before government bodies. Environmental organizations and the media helped communicate the results to the general public. The presentation of assessment conclusions has grown increasingly sophisticated over time. The 1985 stratospheric ozone assessment did not even have an executive summary whereas recent assessment reports have carefully prepared summaries, viewgraphs, talking points, and associated nontechnical publications, such as “Common Questions about Ozone,” that summarize current knowledge in commonsense terms and implicitly address any current attempts to mislead or obscure the consensus. These assessments have continued to exercise substantial influence over policy discussions, influencing multiple revisions of the treaty. One important flaw in the stratospheric ozone assessments is that there has been no consistent treatment of uncertainties across the assessment panels or even within individual panels. Perhaps the most important advance in avoiding the political pitfalls associated with characterizing uncertainties was the development of a measure called “effective equivalent stratospheric chlorine” (EESC, a weighted combination of anthropogenic chlorine and bromine) that can serve as the key metric for monitoring progress in ozone protection. After establishing the links between ozone-depleting substances and ozone depletion and between ozone depletion and health and environmental impacts, the scientific assessment panel used the level of EESC above a designated threshold as a measure of risk. With this metric, they could present policy options in the form of EESC curves illustrating a series of possible regulatory options. Guiding Plans for Future Activities. The character of the most important questions related to ozone has shifted over time, as the policy regime and the state of knowledge have advanced. First, it was critical to demonstrate authoritatively the seriousness of the issue by projecting the magnitude of future ozone loss under a wide range of emissions scenarios without control (WMO 1986a). Then, the emphasis shifted to presenting more precise quantitative projections of future impacts of specific alternative decisions that policy makers were considering. And recently, the major objective has

OCR for page 63
Analysis of Global Change Assessments Lessons Learned been to track observable environmental consequences of actions taken. This evolution demonstrates that past assessments under the Montreal Protocol have successfully guided the next generation of assessments to address the shift in issues and relevant questions. Creating Valued Products. The process assessments produced by the first panel have exercised decisive influence over the policy debate and made key contributions to the changes in certain policy actors’ positions. The 1985 assessment of the scientific understanding of stratospheric ozone depletion was essential to breaking ten years of policy deadlock and led to concrete international action in 1986 and 1987 (Benedick 1998; Parson 2006). Although the impact assessment was important for reasons of completeness, there is no evidence that the substance of these products ever mattered in policy debates, either before or after the Montreal Protocol. The TEAP assessments achieved unprecedented success in providing high-quality technical advice to the Montreal Protocol parties regarding the available technical alternatives. In addition, these assessments also served to promote problem solving in deploying alternatives and to disseminate information among relevant industry sectors. In other words, TEAP has not only provided highly credible assessments of present technical capabilities, but repeatedly helped to advance those capabilities. Like the atmospheric science assessments, the technical judgments of the TEAP have exercised substantial influence over policy decisions. Key Strengths and Weaknesses of the Stratospheric Ozone Assessment. Compared to many aspects of global change, the stratospheric ozone issue is confined to a relatively small number of stakeholders. This simplifies the process of developing salience, credibility, and legitimacy of the assessments. The stability of the structure and leadership of the panels ensured improvements in efficiency of the process and value of the products. Inclusion of scientists from laboratories around the world who were actually involved in research on the issues developed the necessary credibility and legitimacy. The skill of the leadership in communicating the results ensured relevance and recognition. Strengths: All panels enjoyed extreme autonomy, with leaders of each panel having unusual authority over the participation, process, and specific mandate covered, while still maintaining a close enough relationship with decision makers to ensure the content would be relevant. The process assessment benefited from excellent leadership, which was effective in forcing often reluctant participants to render synthetic

OCR for page 63
Analysis of Global Change Assessments Lessons Learned scientific judgments, even when going beyond what is already published in the literature. The TEAP addressed key decisions of private-sector actors regarding research, development, and investment. Weaknesses: The process assessment continues to be conducted in a comprehensive manner despite that its mandate is only to provide an update of new findings after early assessments succeeded in demonstrating the seriousness of the problem. Each new assessment has made a smaller incremental contribution to the decision-making process. In the TEAP process, the interests of key industrial participants gradually diverged from the remaining questions of concern to policy makers. The remarkable early successes of this process were based on the precise alignment of these interests. Now it has become harder to attract critical masses of participation to address the remaining implementation questions. The TEAP process succeeded for so long that it has attracted backlash from those seeking to reimpose political control on the process. Exclusion of economic or cost judgments in considering technical options initially proved to be a strength because it allowed the process to get started. However, it limits the ability to generalize the model, because technical assessments on other issues cannot necessarily ignore costs by assuming that rapid innovation will make them low enough. INTERGOVERNMENTAL PANEL ON CLIMATE CHANGE The IPCC was established in 1988 by the WMO and the UNEP to conduct assessments of the scientific basis for human-induced climate change, its likely impacts, and opportunities for adaptation and mitigation. Several international meetings that took place in 1985-1988 (Bulkeley and Betsill 2003; Torrence 2006), along with a series of unusual weather events in the summer of 1988 (Paterson 1996), helped move climate change to the central stage and served as the impetus to initiate the IPCC. The idea behind the IPCC was grounded both in the experience of the Montreal Protocol negotiations and in TEAP. Since its inception, IPCC has initiated four rounds of assessments, the last being completed over the course of the year 2007 (IPCC 1990a,b,c, 1995a,b,c, 2001a,b,c). The IPCC assessments have been important for informing formal negotiations of an international climate change treaty, a process which started in December 1990. The United Nations Framework Convention on Climate Change (UNFCCC) was adopted at the Earth Summit, held in Rio de Janeiro in May 1992; it was subsequently signed by more than 150 nations

OCR for page 63
Analysis of Global Change Assessments Lessons Learned and entered into force in March 1994. The UNFCCC established a goal of stabilizing greenhouse gases at a level that would prevent dangerous anthropogenic interference with the climate system (United Nations 1992). In 1997, despite some opposition, participants established a framework for negotiating the Kyoto Protocol. This protocol was signed at the Conference of the Parties 3 held that year in Kyoto and would enter into force with the ratification of at least 55 countries, in February 2005. The specific rules for its implementation were agreed at Conference of Parties 7 in 2001, and post-Kyoto negotiations have been very contentious. Establishing Clear Rationales and Appropriate Institutional Structures. The stated goal of the IPCC is to “assess the scientific, technical and socioeconomic information relevant for the understanding of the risk of human-induced climate change, its potential impacts and options for adaptation and mitigation.” Within IPCC, however, there is debate about whether its role is to review and assess existing literature, to conduct integrated assessments, or both. Major decisions of the IPCC are made by a plenary group of government representatives. The plenary elects a chair, three vice-chairs, two co-chairs for each of three working groups and a task force on greenhouse gas emissions inventories, and six vice-chairs for each of the three working groups. These 30 elected chairs and vice-chairs make up the IPCC bureau. The bureau oversees the organization and preparation of IPCC products including assessments, special reports, and technical papers. Each working group is supported administratively by a technical support unit; the overall bureau is supported by a secretariat. Designing and Scheduling Assessment Activities. IPCC’s work is divided among three working groups. Working Group I assesses scientific information on the climate system and the potential for human-induced climate change. Working Group II assesses scientific, socioeconomic, and technical information on the vulnerability of humans, ecological systems, and socioeconomic sectors to climate change, and evaluates information on their adaptive capacity and adaptation practices and options. Working Group III assesses scientific, technical, and socioeconomic information on options to mitigate climate change. In addition, the Task Force on National Greenhouse Gas Inventories provides guidelines for methodologies and practices for preparation of the inventories. The structure, including chapter outlines, and work plans for each assessment is prepared by the bureau and approved by the plenary. A team of lead authors is assigned to each chapter of each assessment, with one or two coordinating lead authors, usually representing a developed and a developing country. These teams are accountable for organizing the work

OCR for page 63
Analysis of Global Change Assessments Lessons Learned of their team and delivering the chapter to the Working Group Bureau. Ultimately the ownership and responsibility for a chapter lies with its lead author team. For each working group, a team of lead authors, consisting of coordinating lead authors and additional experts, prepares a technical summary based on the underlying chapters. The technical summary of each Working Group then provides the basis for preparation of a draft summary for policy makers. Once all three Working Group reports have been accepted and their summaries for policy makers approved, the IPCC bureau and selected coordinating lead authors prepare a synthesis report with the objective of providing a comprehensive and coherent summary of the three reports. These assessments are conducted on a regular interval and are expected to produce a report every five years. The next generation of assessments is initiated shortly after the completion of the previous due to the extensive effort and time required to conduct such a comprehensive effort. Thus, it has become almost a continuous process engaging a significant size of the research community and stakeholders placing a considerable strain on the participants’ time. Involving the Scientific Community and Other Relevant Experts. Governments and other organizations are involved in nominating lead authors. Based on the scientific and technical expertise and credentials of each of those nominated, the bureau selects lead authors for each chapter of the assessments. These authors can be from government, academia, industrial organizations, or environmental organizations and are expected to represent their individual knowledge and expertise, and not a preconceived organizational perspective, in the preparation of the assessments. When the IPCC was first established, participation was strikingly unbalanced. In the second IPCC assessment report for instance, “the percentage of Southern Hemisphere coordinating lead authors, lead authors, and contributing authors in working groups ranged from 5.1 percent (for Working Group I) to 25 percent (for Working Group III)” (Biermann 2006). The IPCC has taken different actions to increase the participation of scientists from developing countries and to remove obstacles that impair their involvement (e.g., current rules require each working group to be chaired by one developed- and one developing-country scientist). This has been perceived as a successful “learning” of an assessment institution (Biermann 2006). Nevertheless, lack of financing and resources still constrains the research and time that scientists from developing countries can devote to assessment activities (Biermann 2006). Engaging and training the next generation of scientists to participate in these assessments might also become a challenge due to the difficulty for young scientists to justify the opportunity cost to advancing their research. Thus, ensuring a continuity in highly quali-

OCR for page 63
Analysis of Global Change Assessments Lessons Learned end of the global assessment. This was due to inadequate funding, because most of the subglobal assessments were supported by local resources. Thus, dissemination of the findings from the subglobal assessments could not be included in the dissemination efforts of the global working groups. Involving the Scientific Community and Other Relevant Experts. The MA followed the IPCC model of using a large pool of leading experts from around the world to conduct the assessment. Selection of these experts was accomplished through wide consultation, including calls to governments and other sponsors. The goal was to get scientists of the highest credentials and at the same time, achieve a balance between social and natural scientists plus among countries and between genders. In addition to engaging internationally recognized leaders, fellowships were offered to young scientists so they could learn and assist in the assessment process. Guidelines restricted the information to be included mainly to the published literature. However, they made provision for the subglobal assessments to incorporate traditional knowledge as long as it was traceable. The review of the work also followed the IPCC model by including two rounds of outside review, with the responses to reviews in turn being reviewed by an independent group of experts. Engaging the Potential Users of Assessment Products. Having learned from previous assessments, the MA took great care to engage the user community at both the beginning and end of the process. The board of the MA consisted of representatives from the UN environmental conventions, who were the primary target audience of the assessment, as well as representatives from many international environmental, resource, and health organizations (governmental and nongovernmental), foundations, academia, the business community, and indigenous communities. Members of the board were active in scoping the project as well as in transmitting the findings to their constituencies. At the end, targeted summaries of results were directed at many of the major user communities. Communicating Scientific Knowledge Accurately and Effectively. A series of technical reports and synthesis documents were released, starting with the Framework of Assessment in 2003 (MA 2003), and the remainder in 2005 (MA 2005a,b). The MA secretariat distributed thousands of free copies of the framework document to target audiences and hundreds of free copies of the technical reports. The MA web site (http://www.maweb.org/) has all of these documents as well as slide show presentations and other outreach materials. The results of the MA were announced, with simultaneous press releases and seminars in major cities (London; Washington, D.C.; Tokyo; Beijing; Delhi; Cairo; Nairobi; Rome; Paris; Stockholm; Lisbon;

OCR for page 63
Analysis of Global Change Assessments Lessons Learned Brasilia; and Sao Paulo), resulting in coverage in major newspapers. The briefings were well attended. Each of the five synthesis reports (biodiversity, desertification, business and industry, wetlands, and health), which were targeted to specific audiences, also had separate launch events in which prime target audiences were involved. Guiding Plans for Future Activities. The MA was not designed to be a continuing activity yet it produced new datasets on trends that will serve as a baseline for measuring change in subsequent similar efforts. It also provided new analytical tools for analyzing environmental change in terms that are of direct interest to society (i.e., ecosystem services), and for doing so in a way that crosses geographic scales. Interest in the findings of the MA has stimulated possible follow-on assessments by various groups (Loreau et al. 2006). Creating Valued Products. The MA produced a large set of products designed for diverse communities. These communities ranged from decision makers involved in international environmental conventions to those working at the local levels where subglobal assessments were performed. Specific products were also designed for decision makers in the business community and the health community. An entire volume (MA 2005b) is devoted to policy options and decision support, including analyses of the past success of response options and an explicit chapter on “choosing responses.” Technical publications served the science community by providing carefully referenced information as well as information on future research needs. An active web site was established that provided all publications and outreach material. Thousands of copies of the various publications were provided free to focal points of the conventions and other crucial parties. Key Strengths and Weaknesses of the MA. The MA seems to have incorporated lessons learned from previous assessments such as the GBA and IPCC (Kaiser 2000), contributing to the fact that many design issues have been addressed appropriately. Strengths: Stakeholders from business, industry, academia, NGOs, UN agencies, and indigenous groups were part of a governing board. Developed and executed using a conceptual model that centered on ecosystem trends that affect human well-being. Focused on global trends but also targeted a sample of subglobal regions and localities. In the latter, traditional knowledge was incorporated. Leadership of all of the major components of the assessment had equal representation between social and natural scientists.

OCR for page 63
Analysis of Global Change Assessments Lessons Learned Working groups were interactive. IPCC approaches were utilized throughout the assessment, including evaluation of uncertainty, two rounds of review, and use of an independent review board. Strong communication strategy with specific products designed for a variety of stakeholders; wide distribution of products as well as web availability. Events publicizing program results held simultaneously in a host of major cities around the world. The prime audiences were all of the environmentally based UN conventions, which provided a stimulus for greater interaction among them. Weaknesses: There was no direct government involvement beyond interaction with the Conference of the Parties. There was no plan for follow-up activities. THE GERMAN ENQUETE KOMMISSION The German Parliament (Bundestag) has the opportunity to create parliamentary investigation committees called “Enquete Kommissions” to address specific subjects of societal interest. As a rule, half of the members who serve on these committees are elected members of the Bundestag, while the other half are experts in the field of the study. This model is notably different from other assessment processes, which in the United States normally do not include politicians. The members work jointly to address the questions under deliberation, so that in this institutional arrangement there is no “firewall” between scientists and policy makers. The rationale for composing an Enquete Kommission with both policy makers and scientists is that scientific findings can be integrated much more rapidly and comprehensively into the parliamentary deliberations. In October 1987, the Bundestag established, for example, an Enquete Kommission to recommend to the executive branch “Preventive Measures to Protect the Earth’s Atmosphere.” It was to assess the importance and consequences to the country of stratospheric ozone depletion and of climate change in a comprehensive manner. It advised both the Eleventh and Twelfth Deutsche Bundestag from 1988 to 1994. As of November 1988, this particular Enquete Kommission included 11 members of the Parliament (five Christian Democrats, three Social Democrats, one from the Free Democratic Party, and one from the Green Party) and nine members from the academic and scientific world. The secretariat, which was providing technical help, included eight members in addition to a study director.

OCR for page 63
Analysis of Global Change Assessments Lessons Learned Establishing Clear Rationales and Appropriate Institutional Structures. The terms of reference for this particular Enquete Kommission were established following a motion presented to the German Parliament by two political groups and approved by the Parliamentary Assembly, hence providing a clear mandate. The task of the committee was to collect evidence on global change in the Earth’s atmosphere, current scientific knowledge of the cause-effect relationships involved in changes taking place, and to propose national and international measures of prevention and control in the interest of protecting both humans and the environment. Designing and Scheduling Assessment Activities. The Enquete Kommission determines how it conducts its activities. Reports are expected at stated periods and are presented for discussion to the plenary Parliamentary Assembly. Visits of the committee to specialized institutions and participation of committee members in national and international scientific conferences are scheduled. A list of potential experts who will be invited to testify is established. The topics for expert reports and the related schedule for publication are established by the committee. Involving the Scientific Community and Other Relevant Experts. External experts originating from the academic, scientific, and industrial sectors are invited to testify before the committee. In the case of the Enquete Kommission on “Preventive Measures to Protect the Earth’s Atmosphere,” industry representatives as well delegates from the public sector were invited to provide input. Engaging the Potential Users of Assessment Products. The primary user of this assessment was involved in the deliberations throughout the process. Communicating Scientific Knowledge Accurately and Effectively. Each month, the Enquete Kommission organized a press conference to highlight the latest findings of the committee and to communicate the content of the presentations made by external experts. Topics discussed by the committee were often featured in the German press. Guiding Plans for Future Activities. The reports of the Enquete Kommission provide specific recommendations regarding future observation campaigns, laboratory experiments, and model development. Creating Valued Products. The Kommission produced a series of comprehensive documents that assessed the scientific knowledge on stratospheric ozone depletion and the protection of the tropical forest, and climate

OCR for page 63
Analysis of Global Change Assessments Lessons Learned change. These reports were prepared by the secretariat on the basis of debates that took place in committee. For certain specific topics, dissenting reports prepared by a fraction of the committee may be included. In addition to chapters that described the state of the science, the Kommission report included sections that recommended specific actions and strategies to address the problems. For example, a 1987 report provided a list of possible actions to protect the ozone layer at the national and international levels (Enquete Kommission 1988). The report called for a drastic reduction in the production and consumption of ozone-depleting substances. The 1991 report (Enquete Kommission 1991) focused on climate change and provided several recommendations for new energy policy. It proposed a 30 percent reduction in carbon dioxide emissions by the year 2005 and called for specific decisions by government bodies, industry, and the public to reach this recommended target. Key Strengths and Weaknesses of the Enquete Kommission. This assessment provides a strikingly different model for how to provide policy-relevant information to the target audience by directly involving them in the assessment process. Strengths: Strong engagement of political decision makers because the Parliamentary Committee was composed of equal numbers of representatives of the different political parties and the scientific community. Direct education of elected members of the Parliament who were members of the Committee. Extensive involvement of experts from all regions and different disciplines. Considerable involvement of a broad range of stakeholders. Deliberate, well-planned communication strategy, carried out through periodic press conferences. Weaknesses: Difficulties in conducting scientific discussions within a Kommission that included some elected parliamentarians with little expertise on the subject. Agreeing on specific resolutions was difficult due to political disagreements between members of different political parties. Experts selected by the factions of Parliament were not nominated by scientific bodies but directly appointed by a political party, which could have some significant ramifications in terms of the credibility and legitimacy of the process.

OCR for page 63
Analysis of Global Change Assessments Lessons Learned SYNTHESIS AND ASSESSMENT PRODUCTS OF THE U.S. CLIMATE CHANGE SCIENCE PROGRAM The CCSP oversees and coordinates research on climate and associated global change at 13 federal agencies and is responsible for responding to the GCRA of 1990. The GCRA mandates periodic assessment of global change impacts on the United States (see Appendix B). Therefore, the CCSP proposed in its 2003 strategic plan to conduct the assessment by producing 21 synthesis and assessment reports, each addressing a specific part of the five main goals identified by the program (CCSP 2003; see also Box 1.2). Three objectives of assessments were identified in the strategic plan: (1) to help shape the research agenda in climate change science, (2) to inform efforts for adaptation to climate change, and (3) to support decision making and policy formulation. Eleven of the reports are intended to address specific unresolved issues related to the understanding and simulation of the climate system. Four reports focus on impacts of climate change on ecosystems and three address direct human impacts (i.e., health, energy, transportation). Three reports deal with decision support (see Appendix C). These reports tend to be narrowly focused on specific issues and thus can be characterized as process and impact assessments. At the time of this writing there is no plan to integrate across the 21 synthesis and assessment products or to produce an integrated assessment of impacts similar to that of the U.S. National Assessment in terms of scope or sectoral and geographic focus. To date, only the first report, on temperature trends in the lower atmosphere, has been completed (CCSP 2006). A number of others are in review, are available in draft form, and should be officially released in the coming months. The remaining reports are scheduled for release in late 2007 and 2008. Because only one of the products has been completed, the committee has included a description of the approach and some of its strengths and weaknesses, but considers it premature to evaluate its effectiveness. Nevertheless, some valuable lessons can be learned from this approach. The CCSP developed guidelines for the production of its assessment and synthesis reports (Appendix D). The guidelines call for using an “open and transparent process for soliciting user input, author nomination and selection, expert peer review and public comment, as well as publication and release” (CCSP 2004). Oversight for report preparation, release, and publication rests with the CCSP Interagency Committee. The initial stage in the process involves the development and approval of a prospectus. The lead agency is responsible for drafting and finalizing the prospectus, which must be approved by the CCSP Interagency Committee. Experts and stakeholders are provided an opportunity to comment on the prospectus in an open process involving an announcement in the Federal Register and posting of the prospectus on the web.

OCR for page 63
Analysis of Global Change Assessments Lessons Learned Preparation of the actual report is carried out by lead and contributing authors selected for their technical expertise appropriate to the assessment topic. The lead authors have ultimate responsibility for the drafts. Although users and stakeholders are not specifically included in the report-writing process, the lead authors have the option of soliciting input from users and stakeholders; this solicitation is required to be open and consistent with the report prospectus. The review of the report is an iterative process initially involving comments from experts and stakeholders in an open process that includes posting of drafts on the web. The guidelines state that “the scientific judgment of the lead authors will determine responses to the comments.” Once the authors have responded to the review comments, the report is submitted to the CCSP Interagency Committee for approval, production, and ultimate release. Publication and release of the report by the Interagency Committee cannot occur until it is reviewed and cleared by the National Science and Technology Committee (NSTC). Approval of the NSTC in turn requires the written release of all members of the Committee on Environment and Natural Resources (CENR). The CENR is comprised of officials from the Executive Office of the President and the 15 federal agencies that have significant programs focused on the environment and natural resources. After approval by the NSTC, the report is published and disseminated using both printing and posting on the web. In addition to the report itself, the comments received during the review process are also posted. Establishing Clear Rationales and Appropriate Institutional Structures. The CCSP was intended to address the goals of both President George W. Bush’s 2001 Climate Change Research Initiative (CCRI) and the GCRA of 1990. This has created an inherent conflict because the goals of the initiative and the 1990 act differ. The CCRI tends to be more narrowly focused on near-term decisions and resolving specific scientific issues than the GCRA. CCSP assessments essentially deconstruct climate issues into many separate, narrow questions that are addressed in individual assessments without integration. Consequently, the structure and approach are not strong in integrating research across sectors and regions or in interdisciplinary science. For this reason the assessment process adopted by the CCSP has been criticized by the NRC (2004) and the Government Accountability Office (GAO 2005) for not including the kinds of integrated analysis intended by the GCRA. One way of addressing this concern without producing a full-blown integrated assessment would be for the CCSP to produce periodic overview reports that summarize the findings of individual reports, place them in a larger context, and discuss policy implications. A complex hierarchical institutional structure has been imposed on the CCSP assessment process with approval of the report requiring sign-off by

OCR for page 63
Analysis of Global Change Assessments Lessons Learned multiple agencies and departments as well as the Executive Office of the President. There are two concerns with this structure: (1) the complexity of the approval process may delay release of the reports; (2) the requirement that all reports be approved by all members of the CENR appears to give veto power over the report to diverse components of the executive branch. This raises the possibility that nonexpert government officials could attempt to influence the technical content of the report, which has the potential to reduce the perception of legitimacy and credibility. This concern is ameliorated to some extent by the fact that the initial draft reports and reviewers’ comments are publicly available on the web. Designing and Scheduling Assessment Activities. The design of the assessment activity is outlined in guidelines for the scoping, preparation, review, and dissemination of the reports. This design incorporates many of the elements that this committee recommends for assessment reports, including appropriate scoping, transparency, both expert and other stakeholder participation, and an open review process. The strategic plan calls for the completion of the reports over a three-year period. Due to administrative difficulties, related particularly to FACA requirements, some of the products are behind the schedule for completion outlined in the CCSP Strategic Plan (CCSP 2003). More generally, the scheduling of the assessment reports has been criticized by the GAO (2005) for not meeting the requirements of the GCRA for the completion of a scientific assessment every four years. Involving the Scientific Community and Other Relevant Experts. The guidelines call for appropriate participation of the expert community in the writing and review of the report. Of particular note is the explicit control given the expert authors over the technical content of the report in the drafting phase. Engaging Potential Users of Assessment Products. The assessment products planned for the CCSP are process and impact assessments, and the actual preparation of the report is essentially led by the expert community with input from government officials. The level of stakeholder engagement is left to the discretion of each assessment leader. Nevertheless the user and other stakeholder communities have the opportunity to comment and provide input on the report prospectus as well as the report itself. Communicating Scientific Knowledge Accurately and Effectively. The guidelines on the preparation of assessment reports do not specify communication approaches, and most importantly do not seem to provide guidance on how to characterize uncertainty and confidence limits. However,

OCR for page 63
Analysis of Global Change Assessments Lessons Learned the strategic plan sets forth as one of its five principles that uncertainties require explicit treatment. One of the assessment products focuses on this topic. Thus, it would appear that the specifics with regard to these decisions are left to the authors of each report. It remains to be seen how this critical issue will be addressed in each report. Guiding Plans for Future Activities. These issues are addressed largely in the overall CCSP mandate and strategic plan. How the individual reports will feed into this process remains to be seen. It is relevant to note that the National Research Council (NRC 2004) found that “CCSP should develop a more comprehensive strategy for implementing and sustaining a global climate observing system.” This recommendation was based on its perspective that not only climate observations, but also societal and ecosystem impacts, needed to be monitored more carefully. The lack of integrated assessments in the CCSP plan appears to be consistent with the NRC’s critique. Creating Valued Products. Only one of the 21 products has been released, so it is not possible to comprehensively assess the degree to which the products will be valued by their target audiences. The first product (Temperature Trends in the Lower Atmosphere, CCSP 2006) addressed a crucial, contentious, long-standing discrepancy in the scientific community between global temperature trends of the past few decades reported by surface thermometer record and those produced by analyses of the Microwave Sounding Unit satellite instrument. This assessment, by supporting detailed critical examination of methods used to produce alternative trends, reduced remaining discrepancies—both among alternative reductions of the satellite record and between them and the surface record—within the errors of the measurements. Because the assessment reports themselves tend to focus narrowly on specific questions and short-term goals, it is likely that the products will be of use to specific segments of the stakeholder community rather than the entire community. For example, the first product has been of high relevance to those most interested in understanding the physical characteristics of the current warming and its attribution, but of little relevance to those concerned with impacts and adaptation. Key Strengths and Weaknesses of CCSP’s Assessment Products. The CCSP assessment process is still in its formative phase, making it premature to comment extensively on its strengths and weaknesses. At the time of this writing only one of the 21 planned assessments has been completed and released. This first report appears to have been effective in meeting its objectives, having authoritatively resolved a key policy-relevant scientific question. The report’s conclusions have been disseminated widely and

OCR for page 63
Analysis of Global Change Assessments Lessons Learned well received by the relevant user, stakeholder, and expert communities. Although the individual products have the potential to result in effective assessments and achieve their individual goals, the overall approach differs from the schedule called for in GCRA (which calls for an assessment every four years) and it is not clear that the collection of assessment products will provide an integrated view of climate change impacts and possible response options.

OCR for page 63
Analysis of Global Change Assessments Lessons Learned This page intentionally left blank.