REQUIREMENTS FOR DATA MANAGEMENT OF CLIMATE DATA RECORDS

The preceding chapter highlighted 14 elements that contribute to a successful CDR generation program. An underlying requirement of many of these elements is a sound data plan for stewardship, management, access, and dissemination of CDRs; for instance, fundamental CDR (FCDR) and thematic CDR (TCDR) data obtained from satellites will involve huge volumes of data, but those who need the CDRs will generally not utilize large datasets. A balanced suite of TCDR products will meet the needs of most users, although there will be occasions when portions of FCDR time series, or even raw data, will be needed for independent research; for instance, they also will require other information, such as guides to the data, explanatory metadata based on community standards, fact sheets, frequently asked questions (FAQ) lists, browse images, and searchable archives (by location, time, and phenomenon). To preserve the integrity of the data series and the flexibility needed to constitute new CDRs from the same underlying data, the original data must be stored and available for scientific reanalysis over time. This requires full documentation, including instrument documentation (e.g., CDR information, hardware documentation, firmware documentation, engineering models, and computer models), platform documentation (e.g., overview), and algorithm documentation (e.g., Algorithm Theoretical Basis Developments, “gray” books). Long-term success for the CDRs also will depend critically on sufficient metadata, in standard formats, including metadata fully describing the product line and metadata to discuss CDR limitations and to aid in data management (dataset lineage, version control, and unique identification parameters). The committee cautions that the cost of metadata generation and maintenance can be a significant part of the overall data management costs.

SYSTEM DESIGN

A carefully designed, efficient data system is fundamental for ensuring success of the CDR program. Since CDRs will be stored, analyzed, and reprocessed in an environment of changing technology and user requirements, the system design should focus on simplicity and endurance. The more complex the system, the more difficult, time-consuming, and costly system upgrades will be. The lessons from Earth Observing System Data and Information System (EOSDIS) and reports from the Standish Group (1999) also suggest that large, complex systems are more prone to failure than



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement