Satellite Observations of the Earth's Environment: Accelerating the Transition of Research to Operations (2003)

Several previous National Research Council (NRC) studies have addressed the issue of transitioning research results into operations, or technology transfer (see Box 2.1). Appendix A presents some of the findings and recommendations from these studies, which are generally consistent and emphasize the importance of improving the technology-transfer process. In addition, they recognize that there is no unique way to effectively transition research results into operations; the process is complex, often inconsistent, not formalized, and consequently depends very much on the individuals in various leadership positions during the process.

This chapter describes the mission and roles of NASA, NOAA, and DOD and then discusses research-to-operations transition pathways in general.

In order to meet the present and future needs of the growing number of users of satellite observations of the Earth environment, it is necessary to continually develop satellites that can reliably provide the observations necessary to support the information database for the envisioned Earth Information System and the growing capability for predictions. Satellites and the data products that are developed in conjunction with them have historically been outgrowths of research programs that develop new sensor technology and make new kinds of observations. In the federal government, these research programs are usually the responsibility of NASA. The operational use

of remote sensing satellites, as well as the dissemination of useful information associated with them, is a responsibility of NOAA. Another important agent in the research-to-operations process is the DOD, which has a legacy of infusing advanced research and technology into operational programs and which actively participates with NOAA in weather and climate monitoring.

The mission of NASA’s Earth Science Enterprise (ESE) is to “develop a scientific understanding of the Earth System and its response to natural or human-induced changes to enable improved prediction capability for climate, weather, and natural hazards.”¹ NASA spends approximately $1.5 billion annually on developing satellites to observe and learn about the Earth system, and NASA’s contributions to the understanding of Earth over the years have been substantial (see NRC, 2001d, and its bibliography).² In addition, many of the NASA research satellites and instruments have provided advanced measurement and satellite technologies that contribute to operational monitoring programs and satellite systems at NOAA and DOD. The transition of NASA research and technology capabilities (satellites, instruments, data-assimilation techniques, and scientific understanding) into NOAA programs has led to major improvements in weather forecasting, warning, and climate monitoring (Hertzfeld and Williamson, 2002). However, some of the transitions have taken many years to complete, and given the increasing importance of weather and climate to society, there is a strong desire to increase the rate of transitioning from research to operations.

The mission of NOAA includes describing and predicting Earth’s environment in order to protect lives and property and to contribute to the nation’s economic and environmental health. According to the Report to Congress on the Status and Challenges for NOAA’s Environmental Data Systems (NOAA, 2001a), NOAA spends almost $1 billion each year collecting environmental data from around the world and space in support of this mission. NOAA’s data products and information help the agency fulfill its mission in various ways: by supporting decisions that save lives and protect property, by assisting in the development of public policy for environmental stewardship, by helping in the management and conservation of marine resources, and in general by enhancing the economic prosperity and quality of life in the United States. Colgan and Weiher (2003) describe how NOAA’s role as a provider of environmental information supports the “knowledge economy” and the creation of economic wealth across a very broad range of U.S. sectors.

The Department of Defense invests approximately $1 billion per year in operations, research, and development related to the collection of environmental data and the use of these data in support of national security. Geostationary and polar-orbiting satellites are critical platforms for this data acquisition. The U.S. Air Force and U.S. Navy (and their respective support to the U.S. Army and U.S. Marine Corps) apply meteorological and oceanographic products to a full range of activities, including mission planning (e.g., force prepositioning), operational support (e.g., air tasking orders), and tactics (e.g., target selection).

Operational agencies such as NOAA have traditionally been conservative in the introduction of new sensors and sensor technologies (NRC, 1997, 2001e), an understandable approach given the consequences of failed operational systems or flawed data. Research agencies such as NASA, however, tolerate and indeed embrace greater risk. The previous NASA administrator (Goldin, 2000) made statements encouraging NASA to accept a 30 percent failure rate for space missions in order to balance acceptable risk with technology innovation. A research instrument with the potential for operational use must be designed so that residual risk in the research configuration can be effectively mitigated in a low-risk transition to the operational version. The research design phase may require planning in order to specifically accommodate the operational transition.

The delineation between research and operations is sometimes blurred, making it important in this report to clarify the terms and definitions (see Box 2.2).

Both weather and climate have their own particular operational data requirements. To illustrate the terms in Box 2.2, “operational weather” data must be acquired globally and processed in real time and often do not have the need for consistency in measurement characteristics that is required for climate observations. “Operational” in the climate context, however, includes a long-term commitment to certain key observations, ensuring that there are few gaps in the data record, that the data are acquired and preserved, and that acquisition costs remain affordable. Although long time series are generally used to detect climate change, many processes, such as ocean circulation, are only revealed on decadal scales. These long-term, systematic measurements may need to be “operationalized” within either NASA or NOAA in order to provide a secure, consistent observation base. Previous NRC reports (NRC 2000c, 2001e,f) have focused on these issues.

The end-to-end set of processes for transitioning research results into operations is called a transition pathway. Pathways may vary depending on the type of research and the potential application. A complete pathway begins with basic research and ends with operational application of the research results and the generation of

products that meet the needs or requirements of end users. Ideally, the research-to-operations process related to observational technologies includes (1) the demonstration that useful measurements can be acquired, quantitatively calibrated, and characterized; and (2) the development and implementation of the observing, data processing, modeling, and dissemination systems, allowing the measurements to be routinely obtained and used.

A full transition requires that the measurements be acquired routinely and reliably by an operational observing system. While it is possible and even desirable to use data obtained by research systems in operations, the production of operational products and services cannot rely on measurements obtained by research systems, which generally lack the mechanisms needed to ensure timeliness, stability, and longevity.

Transition pathways and challenges depend on whether the fundamental impetus originates from operational needs (pull) or research capabilities (push). In the most straightforward but rare cases of pull transitions, the operational entity determines, through numerical simulations or some other means, that measurements with speci-

fied characteristics could contribute to improvements in operational products or services. The operational agency then communicates specific and complete measurement requirements to the research community, including the observing system resources that would be needed for acquiring the data. The operational entity also evaluates and controls optimization across all requirements, including measurement accuracies, resolution, and costs. The research community has clear responsibility for developing the science and technologies within the requirements and constraints established by the operational entity.

The transition pathway is significantly more difficult when the initial transitioning impetus results from research and new capabilities—the push side of the process. In this scenario, the measurement or analysis capability is often demonstrated with little or no regard for specific operational utility or for the availability of resources in the operational data-acquisition-and-analysis system. The operational entity must plan, fund, and carry out the demonstration of utility before the new technology can be considered for inclusion in the operational system. The European Centre for Medium-Range Weather Forecasts³ (ECMWF) has done an excellent job in aggressively and rapidly demonstrating the utility of measurements developed in the research community. The NASA-NOAA Joint Center for Satellite Data Assimilation (JCSDA)⁴ promises to be a substantial improvement over the existing approach for demonstration of the use of environmental research data in the United States.

Managing multiple and often conflicting time scales for transitions is one of the most difficult pathway challenges. In planning a transition, time scales may include 1-year budget cycles, 2-/4-/8-year political cycles, 3- to 8-year satellite lifetimes (with accompanying uncertainty), lengthy procurement cycles, obsolescence cycles, the doubling of computational power every 18 months, and many other issues. A well-defined transition pathway carefully balances these time scales in establishing schedules and planning for resources.

When the time scale for transition is short (for example, some months to a couple of years), the processes and interfaces are relatively straightforward. The transition process can be managed much as a “product improvement” process, during which research and operations work closely together. The requirements are generally well understood, and the resources necessary to support the improved product can be identified and acquired.

For transitions occurring on a medium time scale (for example, 2 to 5 years), the technical and administrative issues become more complicated. The process is analogous to “product development,” during which a research and development (R&D) center is attempting to develop and refine a potential product and bring it to market. The product requirements may not be well understood, and the operational/ production side may be skeptical of the actual costs that may be incurred. Time schedules may be poorly defined, and the project may run late, thus incurring more costs. Continuous evolution in the research requirements may also add to complexity and uncertainty. The transfer of programmatic responsibility for a research satellite mission to an operational agency is an example of such a midrange transition.

At long time scales (5 to 15 years), the transition process resembles “market development,” during which needs and capabilities are poorly known but often evolve rapidly as research proceeds and new ways are discovered to use the research data.

Borrowing from the language of technology transfer, there needs to be a “spannable distance” between the research and operational communities, as well as a mechanism that bridges this distance and connects the researchers to the end users of the data and technologies (NRC, 2001b). The gap between research and operations is often called the valley of death (NRC, 2000a).

Closely related to the valley of death, the valley of lost opportunities represents the gap between research and unknown or unrecognized applications. The valley of lost opportunities represents the chasm between what might have been possible and what is actually realized. It represents the missing of opportunities that are presented by the push side of the transition process. For example, the operational community may be slow to recognize the potential of new technologies because it cannot foresee uses for them, their impact on operations, or new users created by these technologies (the example of the Global Positioning System is discussed in Chapter 3). The valley of death represents the barrier between research and recognized operational needs (the pull impediment), whereas the valley of lost opportunities represents the gap between research and unknown or unrecognized applications (the push impediment). The combined impediments are referred to here as the

valley of death and lost opportunities. There may be more than one such valley in the long transition pathway from research to operations.

Figure 2.1 illustrates the valleys of death and lost opportunities—the gap between researchers and operational satellite data providers and the gap between the satellite data providers and end users. Each valley can be measured in several dimensions, including those of organizational structure and culture, mission objective and design, planning and coordination, communication, and financial and human resources, as well as that of the limitations associated with scientific understanding and technical capability.

Ideally, the organizational structure of the research and operational agencies would be efficiently aligned to support technology transfer across the transition pathway. Staff would be highly educated, trained, and motivated. The cultures would be open to new ideas from other organizations, as well as being cooperative, team-oriented, and supportive of the technology-transfer process. Planning for technology transfer would occur from the very beginning of a research mission and would be updated continuously as the mission progressed. Communication and coordination would occur between the researchers and the operational personnel throughout the mission. The necessary scientific and technological underpinnings of the mission would be solid, including the scientific understanding required to use the mission results (e.g., observations) effectively to improve operational products and services (e.g., numerical weather forecasts). And finally, adequate financial and human resources would be available, not only to the research and operational sides but also to the transitioning process that bridges the gap.

Given the complexity of the transitioning process, there are many ways in which the pathway can be slow and inefficient, or it might break down altogether. The absence of ab initio plans to transfer research results into operations will delay the process when the research results emerge, or it may prevent them from ever being used. Even with good plans from the beginning, poor communication and coordination or cultural issues may prevent the implementation of the plans and their modification as necessary throughout the mission. If the scientific and technological underpinnings are weak, the research/technology part of a mission can fail altogether; or, the operational community may not have the scientific basis to use the research results and observations when they become available. Of course, the success of the entire end-to-end process depends on adequate financial and human resources. In general, financial resources can be invested to obtain the necessary human resources, but this is not guaranteed. It takes time to develop the necessary human resources through education and training, so the human resource needs must be anticipated explicitly throughout the process: “Well trained people are, therefore, one of the most important components of the remote sensing technology transfer process” (NRC, 2001b).

As has been recognized in previous studies (NRC, 2000a, 2001b), the needs and issues that drive the development and use of research satellites are in many ways distinctly different from those that drive their operational counterparts. These differences constrain transition pathways in important but often subtle ways. For example, while systematic research and operational measurements are commonly viewed as being synonymous, in reality, resolving their differences (see Table 2.1) is critical to the success of the research-to-operations transition. Operational measurements are driven by the need for uninterrupted availability; systematic research measurements are driven by the need for long-term stability and are less sensitive to occasional data loss.

In spite of their differences, research and operations often experience strong synergies and positive feedbacks. While research is a key to advancing operational capability, operational data in turn provide essential support for research. For more than a hundred years, observations collected primarily for the purpose of operational weather forecasting have been used by scientists to study the structure and behavior of the atmosphere and to monitor climate. Evidence of the real-time access of environmental data by the research community can be seen in the Web sites of many university departments and research laboratories. Improved understanding of the atmosphere-Earth system from this research has been as important to the progress of operational weather forecasting as the data themselves.

The quality of research data sets must be based on calibrated and validated data, as their impact on theories, models, and algorithms may last indefinitely. Because the type of data required for research is generally the same type required for operations, the research data sets will often come from the same sensors used to support operational applications, or from experimental prototypes of future operational

instruments. Thus, a validation and quality-assurance approach should be adopted that enables the near-real-time generation of research-quality data, which can be archived for future research. At the same time, research data should be made available to operational users to test and evaluate their potential value in operations. The involvement of operational centers in the early testing and use of research data sets has provided invaluable feedback to mission researchers. Early and continual data-management planning is essential if NASA and NOAA are to satisfy the multitudes of research and operational users.

The relationship involving dynamic, positive feedback between research and operations highlights the importance of a strong cooperative relationship between NASA, NOAA, and DOD. As the rate of technological change increases and society becomes more dependent on the products and information provided by satellite remote sensing, these agencies’ ability to quickly transfer new satellite and sensor technologies, models and algorithms, and scientific knowledge into new applications and user products becomes even more critical (NRC, 2003).