Findings and Recommendations
Environmental satellite data have grown in volume, complexity, and information content over the last 10 years, and there is every indication that this trend will continue and will accelerate exponentially. In the upcoming decade, growth in that volume may equal that over the last two or three decades. Our nation’s next generation of polar-orbiting (2009) and geostationary (2012) satellite systems will deliver improved operational data products to an increasingly diverse environmental satellite data user community. The increase in the volume of data delivered to NOAA, and from NOAA to end users, will be orders of magnitude above the present volume. The committee strongly recommends that action be taken to prepare for that increased volume of data and to support the growing diversity of users that attends the data becoming more accessible and a more pervasive and essential part of everyday life. The increases in complexity and volume of environmental satellite data, and corresponding increases in user expectations, must be anticipated and accommodated by all the systems that support the operational, research, and other end-user environmental communities. Particular attention is recommended to the components that support data production, storage, access, and distribution, and to the interfaces with users. By 2014, many of today’s technologies will be obsolete. To obtain the maximum taxpayer value from the U.S. investment in environmental satellite data requires an approach that is as aggressive and proactive in developing and managing our data access and distribution systems as in developing and launching new satellite systems.
THE VALUE OF AND NEED FOR ENVIRONMENTAL SATELLITE DATA IN ADDRESSING SPECIFIC USER NEEDS
Finding: Improved and continuous access to environmental satellite data is of the highest priority for an increasingly broad and diverse range of users. Their needs include real-time imagery for decision making in response to events such as forest fires, floods, and storms; real-time data for assimilation into numerical weather prediction models; recent imagery for assessment of crops and determination of impacts on the environment resulting from diverse human activities such as marine and land transportation; and data coverage spanning many years that allows assessment of patterns and long-term trends in variables, such as sea-surface temperature, land use, urbanization, and soil moisture. Users of environmental satellite data include individuals; federal government agencies; state and local managers, planners, and governments; commercial producers of added-value products; and Web, print, and TV/radio broadcasters.
Recommendation 1: To best serve the diverse user communities and to meet growing demand, the committee recommends that, as soon as is practical, agencies providing environmental satellite data and products collaborate, with NASA and NOAA taking the lead, to develop an explicit strategy and implementation plan for data distribution systems, user interfaces, and increased user engagement and education. The goals of this plan should be to facilitate access to current, historical, and future environmental satellite data and products in ways that acknowledge the range of skills and evolving needs of the user communities and to support these users by providing appropriate supporting information and educational material.
Finding: The national and individual user requirements for multiyear climate system data sets from operational environmental satellites, as currently delineated in the Climate Change Science Program strategic plan,1 are placing special demands on current and future data archiving and utilization systems. These demands include more stringent requirements for accurate cross-platform radiometric calibration, new combinations of multiple satellite and instrument data, and algorithms for generating advanced biophysical variables. Detecting climate change trends often involves evaluating data at the limits of
U.S. Climate Change Science Program, Strategic Plan for the U.S. Climate Change Science Program: A Report by the Climate Change Science Program and the Subcommittee on Global Change Research, available at http://www.climatescience.gov/Library/stratplan2003/default.htm, accessed July 12, 2004.
measurement precision, and so periodic, absolutely consistent reprocessing of climate data records is a fundamental requirement.
Recommendation 2: Creating climate data records (CDRs),2 which quantify subtle but important global change trends, is not a task that can be accomplished solely in routine operational environments (such as with the National Polar-orbiting Operational Environmental Satellite System (NPOESS) and Geostationary Operational Environmental Satellite (GOES)). The committee recommends that NASA, along with NOAA, select multidisciplinary, research-oriented, end-to-end science teams that will select those NPOESS, GOES, and other systems’ data products and variables that are scientifically important and technologically feasible for long-term CDR development. These science teams will design and maintain a proactive strategy for the stewardship and multidecadal production of the selected CDRs.
Finding: NOAA has limited experience with land data sets because historically its mission has focused on the oceans and atmosphere. Major advances in land remote sensing have occurred in the last decade, fostered primarily by the Earth Observing System developed by NASA, that are not reflected in NPOESS planning. The committee found that NOAA has so far not effectively utilized current satellite technologies and data sets for vegetation science, management, or applications. For example, of 58 environmental data records (EDRs) defined for NPOESS, only 6 are specifically for land, and of these only 2 are vegetation oriented. For the 2012 flight of GOES-R, only 20 of the approximately 170 environmental observation requirements (EORs) are land-surface related; of these, only 4 are vegetation related.
Recommendation 3: NOAA should convene an intergovernmental committee with NASA, the U.S. Department of Agriculture, the Department of the Interior, the Environmental Protection Agency, and other interested parties to select the variables for land vegetation data for generation from NPOESS, GOES, and other operational systems that will have high utility for both land management and the hydroecological sciences.
THE DISTRIBUTION OF ENVIRONMENTAL SATELLITE DATA
Finding: The constellations of satellites now in space and planned for the future include platforms launched by several nations, and more complete and comprehensive coverage of environmental data fields can be achieved by combining the data from these different national efforts.
Recommendation 4: The U.S. Environmental Satellite Data Program should work to facilitate user access to data from other nations’ satellites as well as its own and to facilitate synthesis of data across platforms by providing supporting metadata.
Finding: The Comprehensive Large Array-data Stewardship System is being designed by NOAA to catalog, archive, and disseminate all NOAA environmental satellite data produced after 2006. Given the magnitude of this effort—and considering the growing volume, types, and complexity of environmental satellite data; the increasingly large and diverse user base; and expectations for wider and more effective use of the data—the committee emphasizes the importance of NOAA’s (1) having a comprehensive understanding of the full scope of the technical requirements for data cataloging, archiving, and dissemination and (2) ensuring implementation based on that knowledge. Key to successful implementation of a strong system that will serve operational users and the nation well are detailed planning, proactive follow-through, and NOAA’s incorporation of lessons learned from previously developed, similarly scaled initiatives with similar systems requirements.
NOAA should conduct an immediate review of the entire Comprehensive Large Array-data Stewardship System (CLASS) program. This review should aggressively solicit and incorporate recommendations from the designers, builders, operators, and users of similar systems, particularly those systems comprised by the Earth Observing System Data and Information System.
CLASS should be designated and developed as NOAA’s primary data archive system for environmental satellite data and other related data sets. NOAA should ensure that CLASS is designed to adequately serve the full spectrum of potential environmental satellite data users. In addition to end users, CLASS should be designed to disseminate data to the broadest possible community of data brokers and value-added providers. The CLASS architec-
ture should explicitly include the public programmatic (e.g., Web services) interfaces that these third parties require.
NOAA should plan for and identify resources required for an increased CLASS effort to fulfill the needs outlined in a and b above.
Finding: NOAA does not appear to be effectively leveraging the substantial and growing third-party resources available for creating, archiving, and distributing environmental satellite data products. In particular, the current CLASS effort appears to include end-user services (such as Web ordering, e-commerce, and product customization) that could just as easily be provided by third parties, while ignoring the lower-level programmatic interfaces that value-added providers require.
NOAA should consider both centralized and decentralized approaches to managing the generation and distribution of environmental satellite data products to ensure cost-effective and efficient utilization of existing human and institutional expertise and resources. Centralized handling should be provided for operationally critical core products and should include the acquisition, processing, distribution, archiving, and management of calibrated, navigated radiances and reflectances at the top and bottom (atmospherically corrected) of the atmosphere, as well as for selected key products and metadata. Specialized higher-level environmental data products could be handled (processed, reprocessed, and distributed) in a physically and organizationally distributed (and diverse) manner.
NOAA should take maximum advantage of the exponentially decreasing costs of computing resources and allow for distributed implementations by third parties.
NOAA should consider mutually beneficial partnerships and partnering models with the private sector (e.g., commercial value-added data and product services providers) that have the twin objectives of ensuring user-oriented open access to the data and providing the best value to end users.
Finding: Over the life of a project the cost of ownership of online (disk) storage is competitive with, and decreasing more rapidly than, that of offline (tape or optical) storage. The ability to store and process large volumes of satellite data online will thus become ubiquitous. More than any physical medium, Internet connections to these online data sources will prove a stable, economical, and widely available mechanism for data transfer.
NOAA’s default policy should be to maintain all public satellite data online, in archives that can be accessed (partitioned) to maximize throughput and replicated (mirrored) to ensure survivability.
NOAA should transition to exclusively online access to satellite data. Distribution on physical media should be provided as a custom service by third parties.
NOAA should plan for and identify resources to support handling of the anticipated increase in archival and dissemination requirements beyond 2010.
DATA ACCESS AND UTILIZATION
Finding: Data from diverse satellite platforms and for different environmental variables must often be retrieved from different sources, and these retrievals often yield data sets in different formats with different resolution and gridding. The multiple steps currently required to retrieve and manipulate environmental satellite data sets are an impediment to their use.
Recommendation 8: Data access and distribution should be designed, and associated products tailored, to be compatible with users’ processing, storage, distribution, and communications resources and their information requirements.
NOAA should improve access to its data by allowing users to focus searches by geographic region, dates, or environmental variables, thus helping provide the means to search from one user interface across all environmental satellite data held by U.S. agencies. Tailored subsets of data products should be made available for routine distribution and/or in response to a specific request.
Further, NOAA’s user interfaces should allow stored environmental satellite data sets and/or images to be retrieved in a common data format and with geolocated gridding selected from a list of options by the user. Subsetting and subsampling should be combined to provide a continuum of data products from broad-area, low/moderate-resolution products to regional, high-resolution products.
NOAA should concentrate on ensuring the commonality, ease, and transparency of access to environmental satellite data and providing no-cost data streams in a few standardized, user-friendly formats selected primarily to maximize ease of translation into community-specific formats.
NOAA should support the development of third-party format translation services and the adaptation of existing community-standard tools to NOAA-standard formats.
The data that NOAA provides to users should be accompanied by metadata that documents data quality, discusses possible sources of error, and includes a complete product “pedigree” (algorithm theoretical basis, sensor and calibration, ancillary data, processing path, and validation status and component uncertainties).
Finding: Some major segments of the user community currently do not have the resources to fully utilize all of the environmental satellite data available to them. The principal obstacles to expanded use have been inadequate and/or discontinuous funding for applied research as a part of data utilization programs, the lack of support for education and outreach programs, and the lack of trained professional brokers and facilitators available to work with the various bidirectional interfaces between users and providers within the environmental satellite data utilization system.
Recommendation 9: A continuous level of adequate resources, especially for applied research and education of the work force in the use of environmental satellite data, is needed to exploit the huge investments already made in the satellite system. Satellite data providers and the scientific research community should also take a leading role in facilitating collaboration with their end-user partners. These efforts should include outreach, training, and technical assistance for the more sophisticated user communities as well as for the rapidly emerging nonscientific, nongovernmental user groups, with the ultimate goal being to enable straightforward and effortless user access to environmental satellite data and data products.
Finding: Early and ongoing cooperation and dialogue among users, developers of satellite remote sensing hardware and software, and U.S. and international research and operational satellite data providers is essential for the rapid and successful utilization of environmental satellite data. Active research and development is required to achieve operational sustainability—today’s research anticipates and underpins the satisfaction of tomorrow’s operational requirements. Many of the greatest environmental satellite data utilization success stories (see, e.g., the case study on the European Centre for Medium-range Weather Forecasts in Appendix D) have a common theme: the treatment of research and operations as a continuum, with a relentless team focus on excellence with the freedom to continuously improve and evolve.
Recommendation 10: To ensure the ongoing development of future operational environmental satellite data products that have high quality and value requires an ongoing evaluation of the U.S. effort to collect and provide environmental satellite data. An integrated, sustainable basis for the stewardship of future operational systems, sensors, and algorithms should be fostered by establishing close cooperation between the research and operational agencies responsible for the utilization of environmental satellite data (including their development, collection, processing and reprocessing, validation, distribution, and exploitation), with research and operations viewed as a continuum and not as two independent areas of effort. To meet evolving customer requirements, this cooperation between research and operational agencies should be coordinated in close partnership with the user community. Only a fully funded, end-to-end system, from satellite sensor design to data assimilation/utilization, can fully optimize the investments that have been made.