Letter to NOAA/NESDIS
April 27, 2001
Dr. Gregory W. Withee
NESDIS Exec Route: E
Building SSMC1—Room: 8338 1335 East West Highway Silvery Spring, MD 20910-3284
It was a pleasure to work with you and your staff during the NRC-NOAA workshop on Opportunities for NOAA’s Environmental Satellite Program. Your attendance at essentially the entire workshop was very much appreciated by the participants. In the following I will outline my recollection of the principal points of discussion that could lead to one or more formal NRC studies in the areas of interest to NOAA/NESDIS.
As you observed, we followed the background presentations by dividing our participants into splinter groups devoted to system architectures (including ground systems), data utilization, and technology infusion. However, when we brought the groups together to deliver their reports it became evident that the separation, while convenient to carry out the workshop effort, was artificial in terms of offering possible
studies for your consideration. Therefore, the studies listed below reflect the overall workshop discussions and have not been categorized according to splinter group.
Potential Study: An Unencumbered Vision for the Future
All three splinter groups recommended the development of a largely unencumbered vision for the future architecture of NESDIS’s satellite and ground systems. In side discussions, several participants quoted the familiar, “Form follows function.” NOAA/NESDIS is the de facto and de jure national agent for the conduct of operational Earth observations from space and for the archiving of the resulting data; this places NOAA in an extraordinarily important position in the federal structure.
In a formal sense, NOAA’s roles are assigned through legislation and administration policy, but NOAA’s roles are also shaped by the agency’s aspirations and the needs of society. Advances in technology and emerging societal needs lead to requirements or “desirements” for enhancements to existing measurements and entirely new measurements as well. The stimulus for such improvements is as often through visions and inspirations as it is from the formal requirements tabulation processes so attractive to systems engineers. Therefore, the workshop participants believe that it would be productive to articulate a vision for NOAA/NESDIS in the 2015-2020 era that begins with the core responsibilities of the agency and then expansively looks at what the future might yield. It is often said that, while the future cannot be predicted, it can be invented. The workshop participants believe that a well-crafted study in the near future can advance the process of inventing NESDIS’s future. Given the time scale of space system planning, development, and deployment, the period 2015-2020 is closer than we might wish. NESDIS’s near-term concerns are inevitably dominated by the current GOES program, POES program, DMSP, and the evolution of the latter two low-altitude systems into NPOESS. While many of the core functions will remain in any future architecture, a broad study should not be limited to only those systems or their present architectures. A more unencumbered, but realistic, view is needed of both the satellite architecture and its accompanying ground system, and of the role that NOAA will play in the future.
Participants suggested various ways to visualize NOAA’s future role that ranged from concentric circles to Venn diagrams. The fundamental idea is that NOAA has a present set of core responsibilities to carry out, but that advancing societal needs will cause NOAA to accrete new responsibilities and the related observations. Examples were cited in oceanic, solar, and ecological disciplines as being areas of likely accretion, and it was noted that the NESDIS customer base should be considered in all of its dimensions (government, commercial, civilian, and scientific).
The study should likewise be unencumbered in terms of the satellite orbits and the satellite configurations. Without prejudging what the outcome might be, Low-
Earth Orbit (LEO), Medium-Earth Orbit (MEO), Geostationary (GEO), and shared opportunities on other unmanned satellites or the International Space Station should be considered. Likewise, large, mid-size, and smaller satellites should be considered to meet cost effectiveness, performance, and flexibility needs. International opportunities should be sought where appropriate partners can be secured. Beyond the obvious programs of Europe and Japan, the participants noted the environmental satellite program of China as one deserving of consideration for possible partnership. It was assumed by the participants that current political tensions will subside, and normal relations resume. Several people noted the favorable climate for cooperation that exists in China at the science and applications agency level.
The NESDIS vision could, in the view of the participants, encompass the full end-to-end milieu within which the NOAA measurements fit. A study could reexamine the present requirements on the system to assure their continued validity, project potential evolved NPOESS, GOES, and possibly other satellite configurations, and also seek revolutionary architectures going beyond the incremental evolution of the current plans. An examination of the space segment could address satellite size, but also on-board processing, “bent-pipe” data communication, the role of direct broadcast, space cross-links, spacing on orbit, and the scheduled or unscheduled use of excess capacity on launch vehicles such as the Atlas-5 and Delta-4. A sense of the workshop was that a growing launch vehicle capacity exists that may outstrip user needs. Access to space will always be a major concern in terms of NESDIS’s overall space architecture and its sustenance, and in providing for the flight of research and test articles in the next study to be suggested. Aspects beyond the space segment that could be addressed include data processing, distribution, modeling, the Internet and its derivatives, and flexible ground architectures. Needless to say, the issue of transition from the present architecture to some future architecture would need to be considered.
Potential Study: The Transition from Research to Operations to Further Operations
A continuing theme which was expressed by all of the splinter groups was the need for NOAA to have a systematic approach to transitioning new measurement techniques, research instruments, data types, and software to initial operational use (usually within the agency or its DOD partner) and then on to a wider set of operational users outside the agency. Considerable success can be observed in collaborations that have occurred between NASA and NOAA, and there is much to be commended in the largely ad-hoc efforts of recent years. Nevertheless, nothing has replaced the Operational Satellite Improvement Program (OSIP) conducted and funded by NASA from the 1960’s through the early 1980’s. Budget pressures forced the cancellation of the program, and the effect has been the transfer of risk and cost
from NASA to NOAA without an accompanying transfer of funds, facilities, and capabilities. NOAA long relied upon NASA to develop “first-of-a-kind” spacecraft and instruments at NASA expense, with NOAA then procuring subsequent copies without the burden of paying nonrecurring engineering and development costs. This no longer takes place. At best, very successful research instruments flown by NASA serve as exemplars and conceptual test beds for NOAA’s later consideration, but they produce little or no reduction of NOAA’s nonrecurring engineering and development costs in moving the instruments to operations.
A related element to the disappearance of OSIP affects the existence of what has been termed a “spannable distance,” which is a measure of the cultural and knowledge gaps between R&D and operations. The gap between the two is reduced and made manageable when the R&D personnel, often in another agency as in the case of NASA, are intimately aware of present and potential future operations. Likewise, the gap is reduced when personnel in the operational entity understand thoroughly the technology being transferred. The technology transfer process needs both a smart provider and a smart buyer. This concept extends to the need for skilled inhouse personnel in both NASA and NOAA. The agencies rely upon the private sector, but the sensors and their applications are not things that can be “bought by the yard” in a routine fashion. Numerous examples exist that demonstrate the importance of in-house capability, and that capability should be an element in the study of the technology transition process.
There is another new source of risk that has been injected into the NOAA satellite program. For many years, NOAA shared a production facility with the Defense Meteorological Satellite Program at the then RCA Astroelectronics. Risk was shared between the two programs, with each serving as a precursor for the other in various aspects of the program. Trained, expert personnel moved from one set of spacecraft to the other and provided numerous efficiencies. Corporate reorganizations and the merger of the two programs have eliminated this advantage, and the NPOESS effort must stand on its own—without sharing of development costs and without an easy means to smooth fluctuations in workload as occurred between the separate POES and DMSP activities. None of this can be resurrected, and it would not be a purpose of a study to attempt to do so, but the loss of capability and efficiency does add further urgency to the need to carefully manage the transition from research (or simply conception) to operations.
NASA has conducted a very successful Earth observations program that has greatly benefited NOAA, and indeed the entire world. NASA and NOAA will likely continue to collaborate in a very successful manner in the future. However, the systematic evolution of sensors and spacecraft from research to operations is not assured, and must be regularly renegotiated based on changes in one or the other of the agencies. A study could be conducted to examine in detail the heritage of the
current sensors (and to capture the lessons that can be gleaned from that history) and to review possible approaches that would smooth and speed the path from the conception of a research sensor to its eventual deployment in an operational satellite.
A major aspect of such a study is the infusion of new technology into the operational system. Workshop participants cited a number of issues deserving of scrutiny that ranged from mutual planning and shared R&D efforts to the flight of proof-of-concept instruments and spacecraft. The matrix of possible approaches could include still closer collaborations between NASA and NOAA, increased resources within NOAA for the conduct of instrument and satellite development, and the enlistment of international partners in a wide variety of possible roles. In the last of these, modestly capable international collaborators might, for example, provide small satellites to carry less complex, but important, instruments in support of either research or operational goals. The partners might then continue to provide such measurements, or simply provide a demonstration of technological readiness for an instrument to be incorporated into a U.S. operational satellite. At the other end of the possible spectrum of cooperation, international partners might provide highly capable sensing satellites as permanent additions to the operational constellation. Naturally, the current difficulties surrounding the International Trafficking in Arms Regulation (ITAR) and the problems it creates in the exchange of hardware, software, and data will have to be conquered. At some point natural self-interest seems likely to lead to more rather than less cooperation, but participants cited examples of where the current ITAR procedures are interfering with the productive activities of universities, companies, and government agencies.
Naturally, beyond the ITAR issues, the engagement of international partners in the overall NESDIS effort may be complicated by the POES/DMSP merger, and by the DOD’s understandable requirement to be assured of the availability of its data sources. The workshop participants did not offer responses to this concern, but one can imagine the DOD users welcoming the availability of a richer set of information than they would otherwise have. The degree to which military users would incorporate the additional information based on the contributions of international partners into DOD’s operational needs would likely vary from situation to situation.
The workshop participants noted that there are many intriguing possibilities, should NOAA have the resources to carry them out. As mentioned in the preceding discussion on system architectures, there are a large number of possible new measurements that would be logical for NOAA to adopt and sustain. Many of these involve new instruments or radically modified existing instruments requiring a new flight test. The participants noted that emerging smaller satellite technology (and, perhaps, small satellite technology) would likely offer the possibility to fly instruments in tandem, constellation, cluster, or tailored orbits at relatively modest cost. For example, it could be imagined that oceanic measurements might be carried out,
either initially or permanently, by an adjunct set of satellites with tailored Equator-crossing times or in non-Sun-synchronous orbits that would avoid tidal aliasing. Naturally these comments intersect with the international cooperation issues mentioned in the preceding paragraph. Likewise, these comments would also be consistent with NASA providing first-of-a-kind satellites and instruments for such tailored orbits for NOAA’s later replication and continuation.
Intertwined with all of the above threads is the desire for the timely implementation of program changes. The ten-year (or more) cycle time for major program changes is inhibiting. While this time period may be rooted in system economics that are difficult to overcome, a more systematic transition process might aid in making the system more responsive to user wishes. At the same time, the use of auxiliary satellites (domestic or international) may improve responsiveness as well.
The title for this potential study was made consciously redundant to point out that the extension of new measurements into operations can involve layers of users. NOAA will always have a certain core of major users around which its activities tend to revolve, but the agency’s expanding roles and missions can bring NOAA into support of varied users of global, coastal, and regional information. A significant part of transitioning research to operations is the entraining of new classes of users through their joint involvement in pilot tests and other collaborations. An essentially unbounded set of user communities can be imagined. Particular mention was made of the U.S. Navy’s oceanographic work at Bay St. Louis, Mississippi. It is feasible to imagine analogs to this effort in the civil sector, and not only in oceanography, but in many areas. Several workshop participants stressed the regional nature of many societal problems—flooding being an example that was cited. Several participants observed that one cannot move from global models in successive steps all the way down to a local application. No global model can be expected to carry within it all of the details that affect local data and forecast needs. On the other hand, some participants also noted how future enhancements in weather forecasting would involve not only the atmosphere but also the integration of still further ocean and land data into the forecasting models, so the complexity of those models will continue to grow despite not encompassing all local needs. Extending the time over which effective weather forecasts can be made will require a better characterization of all of the boundaries of the models, and of the internal variables of the models as well. While there will be vital interfaces with the private sector and other government agencies (federal, state, and local), there seems to be a permanent and highly productive role for NOAA to play. This leads naturally into the third of the potential studies that the NRC might undertake.
Potential Study: Data Utilization
Just as the first two possible studies noted above are related to one another, a study of data utilization will be tied to both system architecture and technology infusion. In addition, data utilization can be thought of as having a 360-degree, bidirectional interface. In the analysis of conventional data networks, an engineer describes the various linkages using drawings and computers, and examines the capacities and blockages that may occur. It is an appealing picture, and it is tempting to seek to apply this concept to data utilization in NOAA’s data world. While some of the same principles would apply, NOAA’s world is infinitely complicated, and this is a good thing.
Environmental data collected by NOAA’s satellites are applied to numerous disciplines: research, operations, meteorology, oceanography, rivers, coasts, fisheries, hydrology, agriculture, solar-terrestrial interactions, etc. Orthogonal, related, and parallel uses abound, hence a 360-degree interface. The collected data are not blindly delivered to a passive user community, but may (should) be shaped by the needs of the user community. Hence, the interface must be bi-directional. Furthermore, it is not simply the individual data bits that must flow in all directions and be tailored according to their destination, it is likewise the overall conception of the system and its segments that must also serve multiple uses and respond to the special requirements of those uses. Particular mention should be made of the third-party “added-value” commercial and non-profit users who broker applications by converting the data to more usable form. Such groups can play an important role at the bi-directional interface.
User communities served by NOAA include intragovernmental (at all levels), international, regional, researchers, for-profit, non-profit, and educational entities and consortia. The increasing sophistication of requirements spawns increasing complexity in applications. The need to lessen adverse consequences demands greater timeliness in responses to observed phenomena. NOAA’s satellite and data systems must be designed to meet the nation’s and—in many instances—the world’s environmental information needs. Those information needs will expand as noted in the preceding two study descriptions.
However, expanding information needs carries with it serious implications for ready accessibility of high-quality, stable data in each of the research and applications areas. Users will require education on the nature and quality of the data, but the providers of the data will require an equally intense education on the needs of the user community and the data forms that will best meet those needs. Even in advance of the new needs that will inevitably emerge in the 2020s, the crest of an enormous wave of data is already approaching NOAA/NESDIS and its data centers. The sheer quantity of data is vastly greater than what the agency has accommodated
in the past, and the rate at which the data will flow into and through the NOAA system is unprecedented. Estimates of increases by factors of 100 and more were offered in the workshop discussions, and they were based only on systems already in the pipeline.
At the same time, user needs will evolve and change. The production of real-time, high-resolution data products involving as little as a single observation will coexist in the overall information system with the development of synthesized, derived products involving data taken over decades, with all of the concomitant data quality issues. Long-term archiving of environmental data is an accepted NOAA responsibility, but the impending data crest will make new demands for data stream transparency, traceability, access, and characterization. The archives will have to provide carefully documented descriptions of algorithms and models that are employed with the data. Changes in understanding will require extensive reprocessing of data. Modifications in interpretive algorithms will be common, and especially for so-called “difficult” variables, e.g., all-weather atmospheric soundings over land and soil moisture. Such modifications will lead to a need to re-process massive amounts of data.
The NOAA environmental data system includes the community conceiving of a sensor, manufacturer building a sensor, data and information system that processes the sensor data, distribution system delivering data to users, short-term active archive, long-term permanent data archive, and the myriad users who will employ the data in various ways. In some instances those users may be principally concerned with relative measurements, e.g., a time series showing how a particular phenomenon evolves where the desired information is in the changes rather than their absolute values. In other instances, as in the measurement of climate change, issues of data continuity, calibration, and long-term stability will dominate as researchers examine data over the extended lifetime of an individual sensor or from several sensors on multiple space platforms. The ubiquitous presence of the Internet also shapes the way that we think of data availability and distribution.
A study could be conducted of end-to-end data utilization involving all of the above issues, or some selected subset. Regarding the engagement of the science and applications community, and mentioned at the workshop, an element of the study could particularly address the issue of enhancing the utilization of both active short-term and long-term archives and the conduct of pilot studies involving NOAA and visiting scientists. Pilot studies can be used to develop and transfer knowledge from NOAA to the user community and from the user community back to NOAA on the most efficacious data forms and products. The two-way flow of information can foster the greater utilization of the NOAA real-time and archival data. While the workshop participants were skeptical of the 15% data utilization number cited for satellite data (believing it to be understated), the target of 500% utilization that you
cited would certainly be a worthy target, and would require extensive use and re-use of NOAA’s environmental data. The workshop participants strongly believe that there is much more of value that can be mined from NOAA’s archived and real-time data.
I have chosen to package the possible work that the NRC might do for NOAA in three large studies, with each possibly taking the better part of two years at a cost of the order of $500-700K. There are undoubtedly other ways in which the work could be arranged. The NRC staff will be glad to explore those with anyone you designate. As I said at the workshop, NOAA occupies an extraordinarily important role in the federal hierarchy. We all perceive that the NOAA role will be a growing and increasingly important one. We look forward to participating in developing that role in whatever manner is appropriate.
John H. McElroy