Climate Data Records from Environmental Satellites: Interim Report (2004)

In the 1950s, while still a postdoctoral fellow, Charles David Keeling designed and built the first highly accurate instrument to measure atmospheric carbon dioxide (CO₂) concentrations. In 1958, as part of the International Geophysical Year, he began measurements at the Mauna Loa Observatory in Hawaii, a site where CO₂ levels were thought to be characteristic of the unpolluted global atmosphere. These measurements are continued today under international agreements at several locations and by many investigators.

The graph of rising CO₂ concentration from 1958 to the present (Figure 1-1) is now known as the “Keeling curve.” This time series illustrates the qualities of an outstanding climate data record (CDR). Keeling insisted from the start on impeccable quality control. In addition to revealing the increase in carbon dioxide caused by human activities, the exceptionally high accuracy of the measurements has made possible many investigations into the carbon cycle. The importance of these fundamental observations was not always widely recognized (Keeling, 1998), especially early on, and so the ultimate success of this early CDR illustrated the value of carefully planned, long-term commitments to data collection and analysis.

The sustained effort to maintain the atmospheric carbon dioxide record is valuable for its implications as a paradigm for CDR development. Today the Keeling curve, documenting the power of human beings to alter the chemical composition of the entire atmosphere, has iconic status as the single discovery most responsible for motivating research on anthropogenic climate change.

Unfortunately, the CO₂ record is an atypical CDR: many climate records are deficient in length, stability, or accuracy. The ability to understand, predict, and adapt to climate change and variability, however, necessitates high-quality, long-term, and stable measurements of Earth’s environment. As noted by the National Research Council (NRC, 2001),

FIGURE 1-1 “The Keeling Curve.” Time series of annual departures from the 1961 to 1990 base period mean of 334 ppmv using direct measurements from Mauna Loa. SOURCE: Compiled by J. Hurrell, National Center for Atmospheric Research.

Long-term observations sustained over decades are a critical first step in providing the climate data necessary for scientists, decision makers, and stakeholders to make adaptive choices that could improve resiliency to climate change and vulnerability, and maintain economic vitality. Many international and national activities and reports concur on the need for

long-term climate observations. Internationally the 2002 Johannesburg World Summit on Sustainable Development called for strengthened cooperation and coordination among global observing systems and research programs, and the 2003 G8 Summit in Evian, France, appealed for strengthened international cooperation on global observations of Earth’s environment. The UN Framework Convention on Climate Change (UNFCC), through the Subsidiary Board on Scientific and Technical Assessment (SBSTA), has adopted the Second Adequacy Report on Global Climate Observations (GCOS, 2003), which outlines the needed observations, networks, and climate variables.

Nationally, the U.S. Global Change Research Act of 1990 specifically highlighted the climate data needs for “… global measurements, establishing worldwide observations necessary to understand the physical, chemical, and biological processes responsible for changes in the Earth system on all relevant spatial and time scales.” The NRC report on global environmental change (NRC, 1999a) also emphasized the critical nature of high-quality, long-term observations of the Earth system from both a public policy and a scientific perspective. More recently, the Climate Change Science Plan (CCSP, 2003), which integrates activities from the U.S. Global Change Research Program (USGCRP) and the Climate Change Research Initiative (CCRI), continues to emphasize the need for long-term, high-quality observations. A specific component of the CCSP plan addresses the following question:

NOAA’s mission for the next century includes a bold new mandate to “understand climate variability and change to enhance society’s ability to plan and respond,” and NOAA plans to create a global observing and data management system to help to achieve this goal (Box 1-1). With climate science now a high priority, NOAA is creating a CDR program to help to fulfill the climate mandate. The functions of the CDR program include:

• monitoring observing performance for long-term applications;

• generating authoritative long-term records from multiple observing platforms;

• assessing the state of atmospheric, oceanic, land, cryospheric, and space environments; and

• properly archiving and providing timely access to data and metadata.

The creation of high-quality, long-term data of global atmospheric, oceanic, and terrestrial satellite observations is a key component of NOAA’s strategy for achieving its new climate mandate (Box 1-1). Generation of these data also could be a pivotal aspect of the CCSP goal for observations and monitoring. Observations from both operational and research satellite programs are a primary information source for studying climate variability and change, in part because they uniquely provide global or near-global data. Yet, as noted by the NRC (2000a),

To generate the best possible plan for creating climate-quality data, NOAA asked the National Academies to assist in developing a plan for creating CDRs using satellites that monitor environmental conditions (see Box 1-2). The National Academies formed the Committee on Creating Climate Data Records from NOAA Operational Satellites and charged it with providing a comprehensive and practical evaluation of the NOAA CDR plan, including conclusions and recommendations. This interim report is the first phase of a two-phase process. It provides NOAA with general advice on the elements needed in a successful CDR generation process. NOAA will then use these recommendations to develop a plan to guide generation of satellite-based CDRs. In the second phase, the committee will review the NOAA CDR plan and make specific comments.

The committee’s 13 members are experts in the creation, use, and maintenance of CDRs (see Appendix B); they met four times in generating this interim report. The first meeting was a large community workshop, attended by over 40 scientists familiar with CDRs (see Appendix A for the workshop agenda and participant list). The committee also solicited community input by distributing a short questionnaire. In creating this report the committee relied upon the expertise of its members and the opinions of the community as discussed in the workshop and the surveys.

To assist NOAA in its planning process, the committee first had to agree on a definition of a climate data record. The idea of a “climate data record” has surfaced numerous times in recent literature, yet comments from workshop participants and community surveys indicated that the climate community has yet to settle on a consistent definition. For this report the following definition is used:

This report focuses on CDRs that are derived from satellite observations, which combined with ancillary data, potentially resolve the time and space scales of climate change and variability. In general, production of CDRs involves long time series of data from a variety of sensors, with quantified error characteristics. Data life cycles are long in relation to a human life span and are definitely longer than any single mission or set of missions.

These data will need to be reprocessed from the beginning of the series as new information is obtained; the quality, spatial resolution, and temporal resolution of the product may also improve through the time span covered by the data. Ultimately the CDRs should be consistent, continuous records of a climate system variable that do (or will) span at least a multidecadal period.

Not all time series of climate-related variables are designated as CDRs. Variables chosen for CDR development should address key questions about the climate system and lead to clear improvements in (1) scientific understanding of the climate system; (2) projections for future climate states; (3) regional, national and international climate assessments; and (4) the nation’s ability to respond to climate variations. The CDRs should be based on the best scientific research and measurement capability available, and they should represent a consensus within the scientific community regarding what is to be monitored and measured over time.

The committee further defines a hierarchy of CDRs (see Figure 1-2). Fundamental CDRs (FCDRs) are sensor data (e.g., calibrated radiances, brightness temperatures, radar backscatter) that have been improved and quality controlled over time, together with the ancillary data used to calibrate them. Thematic CDRs (TCDRs) are geophysical variables derived from the FCDRs, specific to various disciplines, and often generated by blending satellite observations, in situ data, and model output.

Plans for the National Polar-Orbiting Operational Satellite System (NPOESS) call for the generation of sensor data records (SDRs) and environmental data records (EDRs). The SDRs are time tagged, geolocated, and calibrated antenna signals, but they will not be created for long-term stability and reliability, and they will therefore not be suitable for climate purposes without reprocessing into FCDRs. Algorithms for TCDRs change over time as new scientific discoveries prompt changes; however, the FCDRs will eventually become fixed as our ability to improve calibrations of past satellite sensors will diminish over time. No one can know which theories, processes, or applied products will emerge as critical to scientists, decision makers, or stakeholders in future decades. Therefore, the generation, preservation, and maintenance of the FCDRs is vitally important for ensuring the success of NOAA’s program. The FCDRs will be the ultimate legacy that the long-term satellite programs leave to the next generation. The EDRs are time tagged and geolocated parameters produced from the SDRs, but they also are not calibrated and validated for long-term studies, unlike the CDRs.

Although CDRs can be created with multiple satellite platforms and in situ data, this committee was asked to focus mainly on the steps necessary

FIGURE 1-2 Thematic CDRs (TCDRs) related to different themes will be generated from the fundamental climate data records (FCDRs); for example, the calibrated antenna signals from a series of satellites (e.g., AVHRR, MODIS, VIIRS) will be used to generate a variety of TCDRs. A major effort should focus on creating and managing the FCDRs. The process of calibrating the FDCR generally involves the use of in situ measurements and critical feedback resulting from assessments of the TCDRs. Arrows might be shown in two directions. SOURCE: J. Campbell, University of New Hampshire.

to create and maintain state-of-the-art CDRs with polar-orbiting satellites. As a result the committee did not refer specifically to generation of CDRs primarily with geostationary platforms or in situ data, such as the CO₂ record. In comparison with CDRs generated solely with in situ data, satellite-based CDRs present some unique challenges:

• the need to manage extremely large volumes of data;

• restrictions of spatial sampling and resolution;

• accounting for orbit drift and sensor degradation over time;

• temporal sampling (aliasing);

• difficulty of calibrating after launch (e.g., vicarious or onboard calibration); and

• the need for significant computational resources for reprocessing.

Concern over the future availability of satellite-based climate-quality data led the National Research Council and several other bodies to issue reports on ensuring the climate record from satellites (e.g., NRC, 1999a,b, 2000a,b,c,d,e; NOAA, 2001; GCOS, 2003). In addition to highlighting the need for climate data records, many of the reports recommend steps for the long-term creation and preservation of climate data from the NPOESS and the NPOESS Preparatory Project (NPP). Although NPOESS and NPP were originally envisioned for serving civilian and defense needs for environmental data, the climate community quickly realized that these platforms also would be the primary information sources for any satellite CDRs in the coming decades. The committee viewed these reports as stepping-stones for this project; NRC (1999a) outlined the state of the observing system relative to the USGCRP and discussed several elements of a climate observing system, while NRC (1999b) assessed the adequacy of the climate observing system and endorsed the now well-known 10 climate monitoring principles. NRC (2000b) provided a short overview for creating and maintaining climate data specifically for NPP and NPOESS, and NRC (2000a,c) outlined in greater detail the science, design, and implementation of a potential program for creating climate-quality data for NPOESS. NRC (2000d) examined atmospheric temperature trends near the surface and in the lower to middle troposphere to reconcile disagreements in the observed trends. NRC (2000e) built upon the recommendations from NRC (2000d) and discussed strategies for NOAA to develop long-term monitoring capability of upper air temperature CDRs; as such, NRC (2000e) is particularly relevant for NOAA to refer to in addition to this report. NOAA (2001) was written by the NOAA Science Advisory Board, and this report suggested the creation of a new program for climate monitoring within NOAA, including but not limited to satellites. The GCOS (2003) report examined the state of the global climate observing system and suggested various methods to address inadequacies.

This report builds on the wealth of information available, giving specific attention to creating CDRs useful to NOAA’s new climate mandate. It also provides practical advice to help NOAA to create CDRs from operational

satellite data that respond to the needs of the climate science community as well as policy makers and other stakeholders, utilize the best scientific practices in the creation of CDRs, and are properly archived and disseminated to the user community.

This report is organized into six chapters. The following chapter (Chapter 2) discusses lessons learned from previous attempts at creating climate-quality data that NOAA should consider in developing its plan. Based on the historical lessons, committee expertise, community surveys, and the workshop, Chapter 3 outlines the key elements needed for a successful CDR generation program, beginning with identification of an appropriate organizational framework, continuing with suggested steps for creating the CDRs, and ending with comments on sustaining the program. Since data management is an integral component of the CDR legacy left to the next generation, Chapter 4 provides comments on data storage, archiving, and dissemination. Finally, with a realization that creating effective CDRs for every possible variable is a task that NOAA could never hope to achieve alone, Chapter 5 discusses the importance of partnerships. Chapter 6 summarizes the committee’s recommendations, beginning with an overarching recommendation and six supporting recommendations.