TABLE 4.1 Data Set Processing Levels

Data Level NASA/NOAA

Description

Level 1A:

Raw data records

Reconstructed unprocessed instrument or payload data at full resolution, time referenced, and annotated with ancillary information, including radiometric and geometric calibration coefficients and georeferencing parameters (i.e., platform ephemeris and orientation) computed and appended but not applied to the level 0 data

Level 1B:

Sensor data records

Level 1A data that have been processed to sensor units

Level 2:

Environmental data records

Derived geophysical variables at the same resolution and location as the Level 1 source data

(NASA) and the Integrated Program Office (IPO), the NPP will provide an opportunity to benefit from the progress NASA is making in data system development.

In this chapter, the committee first establishes the need for an NCDS separate from the operational system. Some essential elements of an NCDS are then discussed. These elements include a long-term archive for the lowest-level NPOESS data (raw data records or sensor data records) and a system architecture in which science teams are primarily responsible for the development of the algorithms needed to generate geophysical products, which are then archived and distributed by innovative data centers. The need for the NCDS to accommodate algorithm and sensor evolution, reprocessing, and multiple versions of data sets is also described. Finally, the importance of innovation and competition in the emerging NCDS is stressed.

OPERATIONAL VERSUS RESEARCH NEEDS

Operational processing and research have different requirements. Operational processing, as it is now envisioned for NPOESS, will be done at a number of centralized sites, with each site using a common data processing system provided by the NPOESS prime contractor. The emphasis will be on generating products very quickly for weather applications. The centralized, no-archiving, one-time-processing architecture of the operation centers is totally different from that required by the research community, in which scientific algorithms are continuously evolving and reprocessing is routine. For research, the requirement of timeliness can be relaxed, thereby allowing for the implementation of complex algorithms using diverse ancillary data. As understanding of sensor calibration issues and radiative transfer from Earth improves, algorithms can be improved, and better products can be generated via reprocessing.

To be more specific, NCDS has the following basic requirements over and above what is needed for operational processing:

  • A long-term archiving system that can fully support the needs of the climate research community. This entails easy, affordable, and timely access for a large number of scientists in many different fields. The data must be supported by metadata that carefully document sensor performance history and data processing algorithms.

  • The ability to reprocess large data sets as understanding of sensor performance, algorithms, and Earth science improves. Examples of new information that would warrant reprocessing are detection of sensor calibration drift and the availability of better ancillary data sets, better geophysical models, and errors in previous processing.

  • Use of standard formats and interfaces, so that researchers, data producers, and archives can be closely linked. Research data systems tend to be less centralized and more distributed than operational processing systems.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement