2
The Research-to-Operations Context: Bridging the Valleys of Death and Lost Opportunities

FIGURE 2.1 Illustration of a transition pathway from research to operations and the “push-pull” dynamic. Currently the valleys of death and lost opportunities are spanned by relatively ineffective bridges. The goal is to strengthen these bridges and their supporting infrastructure, making the transition pathways quicker and more efficient.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 12
2 The Research-to-Operations Context: Bridging the Valleys of Death and Lost Opportunities FIGURE 2.1 Illustration of a transition pathway from research to operations and the “push-pull” dynamic. Currently the valleys of death and lost opportunities are spanned by relatively ineffective bridges. The goal is to strengthen these bridges and their supporting infrastructure, making the transition pathways quicker and more efficient.

OCR for page 12
BOX 2.1 From Research to Operations in Weather Satellites and Numerical Weather Prediction: Crossing the Valley of Death—A Brief Summary A recent NRC report, From Research to Operations in Weather Satellites and Numerical Weather Prediction: Crossing the Valley of Death (NRC, 2000a), is highly relevant to the present study. The earlier report discusses the need for improved transitioning of research and development in the areas of numerical weather prediction and environmental satellites. It notes the increasing sensitivity of many sectors of society to weather and climate and therefore the increasing value of accurate weather and climate information. It discusses the difficulties of transitioning research results to operations across the “valley of death,” which is a metaphor for the barriers and obstacles separating research results and operational applications. The report notes that successful transitions require an understanding of the importance and risks of transition, the development of appropriate transition plans, adequate resources for the transitions, and continuous communication and feedback between the research and operational communities. The 2000 report makes a number of important recommendations (see Appendix A in the present report) to NASA and NOAA with respect to improving the transitioning process. The Committee on NASA-NOAA Transition from Research to Operations supports those recommendations, and builds upon them. Several previous National Research Council (NRC) studies have addressed the issue of transitioning research results into operations, or technology transfer (see Box 2.1). Appendix A presents some of the findings and recommendations from these studies, which are generally consistent and emphasize the importance of improving the technology-transfer process. In addition, they recognize that there is no unique way to effectively transition research results into operations; the process is complex, often inconsistent, not formalized, and consequently depends very much on the individuals in various leadership positions during the process. This chapter describes the mission and roles of NASA, NOAA, and DOD and then discusses research-to-operations transition pathways in general. MISSION AND ROLES OF NASA, NOAA, AND DOD In order to meet the present and future needs of the growing number of users of satellite observations of the Earth environment, it is necessary to continually develop satellites that can reliably provide the observations necessary to support the information database for the envisioned Earth Information System and the growing capability for predictions. Satellites and the data products that are developed in conjunction with them have historically been outgrowths of research programs that develop new sensor technology and make new kinds of observations. In the federal government, these research programs are usually the responsibility of NASA. The operational use

OCR for page 12
of remote sensing satellites, as well as the dissemination of useful information associated with them, is a responsibility of NOAA. Another important agent in the research-to-operations process is the DOD, which has a legacy of infusing advanced research and technology into operational programs and which actively participates with NOAA in weather and climate monitoring. The mission of NASA’s Earth Science Enterprise (ESE) is to “develop a scientific understanding of the Earth System and its response to natural or human-induced changes to enable improved prediction capability for climate, weather, and natural hazards.”1 NASA spends approximately $1.5 billion annually on developing satellites to observe and learn about the Earth system, and NASA’s contributions to the understanding of Earth over the years have been substantial (see NRC, 2001d, and its bibliography).2 In addition, many of the NASA research satellites and instruments have provided advanced measurement and satellite technologies that contribute to operational monitoring programs and satellite systems at NOAA and DOD. The transition of NASA research and technology capabilities (satellites, instruments, data-assimilation techniques, and scientific understanding) into NOAA programs has led to major improvements in weather forecasting, warning, and climate monitoring (Hertzfeld and Williamson, 2002). However, some of the transitions have taken many years to complete, and given the increasing importance of weather and climate to society, there is a strong desire to increase the rate of transitioning from research to operations. The mission of NOAA includes describing and predicting Earth’s environment in order to protect lives and property and to contribute to the nation’s economic and environmental health. According to the Report to Congress on the Status and Challenges for NOAA’s Environmental Data Systems (NOAA, 2001a), NOAA spends almost $1 billion each year collecting environmental data from around the world and space in support of this mission. NOAA’s data products and information help the agency fulfill its mission in various ways: by supporting decisions that save lives and protect property, by assisting in the development of public policy for environmental stewardship, by helping in the management and conservation of marine resources, and in general by enhancing the economic prosperity and quality of life in the United States. Colgan and Weiher (2003) describe how NOAA’s role as a provider of environmental information supports the “knowledge economy” and the creation of economic wealth across a very broad range of U.S. sectors. 1   See the NASA ESE Web site at <http://www.earth.nasa.gov/>. Accessed February 3, 2003. 2   The total budget for NASA’s Office of Earth Science is provided for developing satellites, research and technology, mission operations, education, and other investments. See the Web site at < http://ifmp.nasa.gov/codeb/budget2003/>. Accessed February 3, 2003.

OCR for page 12
The Department of Defense invests approximately $1 billion per year in operations, research, and development related to the collection of environmental data and the use of these data in support of national security. Geostationary and polar-orbiting satellites are critical platforms for this data acquisition. The U.S. Air Force and U.S. Navy (and their respective support to the U.S. Army and U.S. Marine Corps) apply meteorological and oceanographic products to a full range of activities, including mission planning (e.g., force prepositioning), operational support (e.g., air tasking orders), and tactics (e.g., target selection). Operational agencies such as NOAA have traditionally been conservative in the introduction of new sensors and sensor technologies (NRC, 1997, 2001e), an understandable approach given the consequences of failed operational systems or flawed data. Research agencies such as NASA, however, tolerate and indeed embrace greater risk. The previous NASA administrator (Goldin, 2000) made statements encouraging NASA to accept a 30 percent failure rate for space missions in order to balance acceptable risk with technology innovation. A research instrument with the potential for operational use must be designed so that residual risk in the research configuration can be effectively mitigated in a low-risk transition to the operational version. The research design phase may require planning in order to specifically accommodate the operational transition. RESEARCH, OPERATIONS, AND THE “PUSH-PULL” DYNAMIC The delineation between research and operations is sometimes blurred, making it important in this report to clarify the terms and definitions (see Box 2.2). Both weather and climate have their own particular operational data requirements. To illustrate the terms in Box 2.2, “operational weather” data must be acquired globally and processed in real time and often do not have the need for consistency in measurement characteristics that is required for climate observations. “Operational” in the climate context, however, includes a long-term commitment to certain key observations, ensuring that there are few gaps in the data record, that the data are acquired and preserved, and that acquisition costs remain affordable. Although long time series are generally used to detect climate change, many processes, such as ocean circulation, are only revealed on decadal scales. These long-term, systematic measurements may need to be “operationalized” within either NASA or NOAA in order to provide a secure, consistent observation base. Previous NRC reports (NRC 2000c, 2001e,f) have focused on these issues. The end-to-end set of processes for transitioning research results into operations is called a transition pathway. Pathways may vary depending on the type of research and the potential application. A complete pathway begins with basic research and ends with operational application of the research results and the generation of

OCR for page 12
BOX 2.2 Definition of Terms For the purposes of this report, the following definitions of research, operations, and transitioning activities are adopted: Research activities—develop scientific understanding of important processes and/or demonstrate the capabilities of new analysis, modeling techniques, or measurement technologies, typically through acquiring, calibrating, and characterizing a specific set of measurements. Operations activities—routinely and reliably generate specific services and products that meet predefined accuracy, timeliness, and scope/format requirements, as well as disseminating or making them available to a variety of users in the public, private, and academic sectors. Transitioning activities, or processes—transfer new or improved scientific knowledge or technologies produced by research to operations. The end-to-end set of processes for transitioning research results into operations is a transition pathway. products that meet the needs or requirements of end users. Ideally, the research-to-operations process related to observational technologies includes (1) the demonstration that useful measurements can be acquired, quantitatively calibrated, and characterized; and (2) the development and implementation of the observing, data processing, modeling, and dissemination systems, allowing the measurements to be routinely obtained and used. The transition process should involve a “push-pull” dynamic in which research and technology development programs respond to the requirements (pull) of the operational user and the operational system takes advantage of new research results and technologies (push) that emerge as a result of science and technology evolution. A full transition requires that the measurements be acquired routinely and reliably by an operational observing system. While it is possible and even desirable to use data obtained by research systems in operations, the production of operational products and services cannot rely on measurements obtained by research systems, which generally lack the mechanisms needed to ensure timeliness, stability, and longevity. Transition pathways and challenges depend on whether the fundamental impetus originates from operational needs (pull) or research capabilities (push). In the most straightforward but rare cases of pull transitions, the operational entity determines, through numerical simulations or some other means, that measurements with speci-

OCR for page 12
fied characteristics could contribute to improvements in operational products or services. The operational agency then communicates specific and complete measurement requirements to the research community, including the observing system resources that would be needed for acquiring the data. The operational entity also evaluates and controls optimization across all requirements, including measurement accuracies, resolution, and costs. The research community has clear responsibility for developing the science and technologies within the requirements and constraints established by the operational entity. Efficient transition pathways are needed to enable the continuous infusion of new scientific results and technologies into the operational environmental satellite observing system. The transition pathway is significantly more difficult when the initial transitioning impetus results from research and new capabilities—the push side of the process. In this scenario, the measurement or analysis capability is often demonstrated with little or no regard for specific operational utility or for the availability of resources in the operational data-acquisition-and-analysis system. The operational entity must plan, fund, and carry out the demonstration of utility before the new technology can be considered for inclusion in the operational system. The European Centre for Medium-Range Weather Forecasts3 (ECMWF) has done an excellent job in aggressively and rapidly demonstrating the utility of measurements developed in the research community. The NASA-NOAA Joint Center for Satellite Data Assimilation (JCSDA)4 promises to be a substantial improvement over the existing approach for demonstration of the use of environmental research data in the United States. TIME SCALES IN THE TRANSITIONING PROCESS Managing multiple and often conflicting time scales for transitions is one of the most difficult pathway challenges. In planning a transition, time scales may include 1-year budget cycles, 2-/4-/8-year political cycles, 3- to 8-year satellite lifetimes (with accompanying uncertainty), lengthy procurement cycles, obsolescence cycles, the doubling of computational power every 18 months, and many other issues. A well-defined transition pathway carefully balances these time scales in establishing schedules and planning for resources. 3   More information on the European Centre for Medium-Range Weather Forecasts is available online at <http://www.ecmwf.int/>. Accessed February 3, 2003. 4   More information on the Joint Center for Satellite Data Assimilation is available online at <http://jcsda.gsfc.nasa.gov>. Accessed February 3, 2003.

OCR for page 12
When the time scale for transition is short (for example, some months to a couple of years), the processes and interfaces are relatively straightforward. The transition process can be managed much as a “product improvement” process, during which research and operations work closely together. The requirements are generally well understood, and the resources necessary to support the improved product can be identified and acquired. For transitions occurring on a medium time scale (for example, 2 to 5 years), the technical and administrative issues become more complicated. The process is analogous to “product development,” during which a research and development (R&D) center is attempting to develop and refine a potential product and bring it to market. The product requirements may not be well understood, and the operational/ production side may be skeptical of the actual costs that may be incurred. Time schedules may be poorly defined, and the project may run late, thus incurring more costs. Continuous evolution in the research requirements may also add to complexity and uncertainty. The transfer of programmatic responsibility for a research satellite mission to an operational agency is an example of such a midrange transition. At long time scales (5 to 15 years), the transition process resembles “market development,” during which needs and capabilities are poorly known but often evolve rapidly as research proceeds and new ways are discovered to use the research data. SPANNING THE VALLEYS OF DEATH AND LOST OPPORTUNITIES Borrowing from the language of technology transfer, there needs to be a “spannable distance” between the research and operational communities, as well as a mechanism that bridges this distance and connects the researchers to the end users of the data and technologies (NRC, 2001b). The gap between research and operations is often called the valley of death (NRC, 2000a). Closely related to the valley of death, the valley of lost opportunities represents the gap between research and unknown or unrecognized applications. The valley of lost opportunities represents the chasm between what might have been possible and what is actually realized. It represents the missing of opportunities that are presented by the push side of the transition process. For example, the operational community may be slow to recognize the potential of new technologies because it cannot foresee uses for them, their impact on operations, or new users created by these technologies (the example of the Global Positioning System is discussed in Chapter 3). The valley of death represents the barrier between research and recognized operational needs (the pull impediment), whereas the valley of lost opportunities represents the gap between research and unknown or unrecognized applications (the push impediment). The combined impediments are referred to here as the

OCR for page 12
valley of death and lost opportunities. There may be more than one such valley in the long transition pathway from research to operations. Figure 2.1 illustrates the valleys of death and lost opportunities—the gap between researchers and operational satellite data providers and the gap between the satellite data providers and end users. Each valley can be measured in several dimensions, including those of organizational structure and culture, mission objective and design, planning and coordination, communication, and financial and human resources, as well as that of the limitations associated with scientific understanding and technical capability. Ideally, the organizational structure of the research and operational agencies would be efficiently aligned to support technology transfer across the transition pathway. Staff would be highly educated, trained, and motivated. The cultures would be open to new ideas from other organizations, as well as being cooperative, team-oriented, and supportive of the technology-transfer process. Planning for technology transfer would occur from the very beginning of a research mission and would be updated continuously as the mission progressed. Communication and coordination would occur between the researchers and the operational personnel throughout the mission. The necessary scientific and technological underpinnings of the mission would be solid, including the scientific understanding required to use the mission results (e.g., observations) effectively to improve operational products and services (e.g., numerical weather forecasts). And finally, adequate financial and human resources would be available, not only to the research and operational sides but also to the transitioning process that bridges the gap. Given the complexity of the transitioning process, there are many ways in which the pathway can be slow and inefficient, or it might break down altogether. The absence of ab initio plans to transfer research results into operations will delay the process when the research results emerge, or it may prevent them from ever being used. Even with good plans from the beginning, poor communication and coordination or cultural issues may prevent the implementation of the plans and their modification as necessary throughout the mission. If the scientific and technological underpinnings are weak, the research/technology part of a mission can fail altogether; or, the operational community may not have the scientific basis to use the research results and observations when they become available. Of course, the success of the entire end-to-end process depends on adequate financial and human resources. In general, financial resources can be invested to obtain the necessary human resources, but this is not guaranteed. It takes time to develop the necessary human resources through education and training, so the human resource needs must be anticipated explicitly throughout the process: “Well trained people are, therefore, one of the most important components of the remote sensing technology transfer process” (NRC, 2001b).

OCR for page 12
SYNERGIES BETWEEN RESEARCH AND OPERATIONS As has been recognized in previous studies (NRC, 2000a, 2001b), the needs and issues that drive the development and use of research satellites are in many ways distinctly different from those that drive their operational counterparts. These differences constrain transition pathways in important but often subtle ways. For example, while systematic research and operational measurements are commonly viewed as being synonymous, in reality, resolving their differences (see Table 2.1) is critical to the success of the research-to-operations transition. Operational measurements are driven by the need for uninterrupted availability; systematic research measurements are driven by the need for long-term stability and are less sensitive to occasional data loss. In spite of their differences, research and operations often experience strong synergies and positive feedbacks. While research is a key to advancing operational capability, operational data in turn provide essential support for research. For more than a hundred years, observations collected primarily for the purpose of operational weather forecasting have been used by scientists to study the structure and behavior of the atmosphere and to monitor climate. Evidence of the real-time access of environmental data by the research community can be seen in the Web sites of many university departments and research laboratories. Improved understanding of the atmosphere-Earth system from this research has been as important to the progress of operational weather forecasting as the data themselves. The quality of research data sets must be based on calibrated and validated data, as their impact on theories, models, and algorithms may last indefinitely. Because the type of data required for research is generally the same type required for operations, the research data sets will often come from the same sensors used to support operational applications, or from experimental prototypes of future operational TABLE 2.1 Key Differences Between Research and Operational Satellites   Approach Issue Research Satellites Operational Satellites Replace on failure? Rarely For high-priority sensors Uninterrupted operation Usually desired Nearly always required Spare satellite on-orbit or rapidly available No Yes Data format and collection May change over mission No change over mission Risk tolerance Moderate Low Data latency Hours to months Hours or less Impact of reduced data quality Reduced science Lower-value products Time value of data High long-term value High immediate value Calibration drivers Long-term stability Pixel-to-pixel stability

OCR for page 12
instruments. Thus, a validation and quality-assurance approach should be adopted that enables the near-real-time generation of research-quality data, which can be archived for future research. At the same time, research data should be made available to operational users to test and evaluate their potential value in operations. The involvement of operational centers in the early testing and use of research data sets has provided invaluable feedback to mission researchers. Early and continual data-management planning is essential if NASA and NOAA are to satisfy the multitudes of research and operational users. Because long-term research data sets are primarily derived from operational systems, NOAA and NASA should jointly develop an approach for generating research-quality data sets from next-generation operational satellite sensors. The relationship involving dynamic, positive feedback between research and operations highlights the importance of a strong cooperative relationship between NASA, NOAA, and DOD. As the rate of technological change increases and society becomes more dependent on the products and information provided by satellite remote sensing, these agencies’ ability to quickly transfer new satellite and sensor technologies, models and algorithms, and scientific knowledge into new applications and user products becomes even more critical (NRC, 2003).