3
Program Conceptualization: Matching Objectives to Constraints

Successful programs start with proper conceptualization, including the identification of well-defined and widely understood objectives and the establishment of resources sufficient to achieve those objectives. This chapter identifies several areas in which it has proved difficult to match objectives and resources for existing PI-led missions and investigates how the issue is addressed in several non-PI-led programs.

STATED OBJECTIVES AND CONSTRAINTS

Table 3.1 summarizes ESE’s stated ESSP objectives and constraints, which are representative of all ESE PI-led missions and are appropriate as guidelines for PI-led missions when considered individually. Nevertheless, the combination of ambitious scientific and programmatic objectives, coupled with tight schedules, capped costs, and (to a lesser extent) management and implementation constraints, makes PI-led missions very challenging, even in comparison to larger, more typical NASA projects.

UNSTATED OBJECTIVES AND CONSTRAINTS

In addition to the stated objectives listed in Table 3.1, there are a number of unstated objectives that PI-led missions are expected to satisfy, including the desire to increase the capacity of university-based research (which is an explicit objective of UnESS). Potential mismatches between unstated objectives and existing constraints are often more difficult to identify than those associated with stated objectives, but they can be equally important in the success of a mission. The following four examples illustrate the impact of mismatches that can arise between unstated objectives and constraints in PI-led Earth science missions.

Example 1: The Limited Capacity of Universities

Objective:

Enhance university-based Earth science research through participation in PI-led missions.

Constraints:

For Earth science, universities generally have limited technical and programmatic experience with spaceborne missions, and relatively few universities have the capacity to lead or support PI-led Earth science missions at acceptable risk levels.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 18
Steps to Facilitate Principal-Investigator-Led Earth Science Missions 3 Program Conceptualization: Matching Objectives to Constraints Successful programs start with proper conceptualization, including the identification of well-defined and widely understood objectives and the establishment of resources sufficient to achieve those objectives. This chapter identifies several areas in which it has proved difficult to match objectives and resources for existing PI-led missions and investigates how the issue is addressed in several non-PI-led programs. STATED OBJECTIVES AND CONSTRAINTS Table 3.1 summarizes ESE’s stated ESSP objectives and constraints, which are representative of all ESE PI-led missions and are appropriate as guidelines for PI-led missions when considered individually. Nevertheless, the combination of ambitious scientific and programmatic objectives, coupled with tight schedules, capped costs, and (to a lesser extent) management and implementation constraints, makes PI-led missions very challenging, even in comparison to larger, more typical NASA projects. UNSTATED OBJECTIVES AND CONSTRAINTS In addition to the stated objectives listed in Table 3.1, there are a number of unstated objectives that PI-led missions are expected to satisfy, including the desire to increase the capacity of university-based research (which is an explicit objective of UnESS). Potential mismatches between unstated objectives and existing constraints are often more difficult to identify than those associated with stated objectives, but they can be equally important in the success of a mission. The following four examples illustrate the impact of mismatches that can arise between unstated objectives and constraints in PI-led Earth science missions. Example 1: The Limited Capacity of Universities Objective: Enhance university-based Earth science research through participation in PI-led missions. Constraints: For Earth science, universities generally have limited technical and programmatic experience with spaceborne missions, and relatively few universities have the capacity to lead or support PI-led Earth science missions at acceptable risk levels.

OCR for page 18
Steps to Facilitate Principal-Investigator-Led Earth Science Missions TABLE 3.1 Stated Objectives and Constraints Within the Earth System Science Pathfinder Program   Text from ESSP-3 AOa Objectives Frequent low-cost missions “launch rate of at least one mission per year” “low-cost, quick turnaround spaceborne missions” “NASA encourages and favors low-cost missions” High-priority, focused, exploratory science “the philosophy behind ESSP embraces small satellite missions addressing high priority Earth System Science objectives” “ESSP … is intended to address exploratory measurements which can yield new scientific breakthroughs … addressing a focused set of scientific questions” Innovative project implementation   “creativity in all aspects of mission development” “innovation in instrumentation and strategies for acquiring and distributing new data products” “the PI’s team will have a large degree of freedom in accomplishing mission objectives within the stated constraints” Constraints Available funding “NASA encourages but does not require contributions from sources other than NASA” “the NASA ESE cost ceiling is $125M” Mission readiness “every aspect of the mission shall reflect a commitment to mission success” “all proposed missions shall be of sufficient technical maturity to achieve launch readiness within a goal of 36 months” “to ensure mission success, there will be appropriate Government oversight and insight” aESSP-3 Announcement of Opportunity (AO), 2001. This and other reference documents for the ESSP program can be found in the ESSP AO library online at <http://centauri.larc.nasa.gov/essp/essplib.html>. The space science community has long required technically advanced remote sensing systems because many critical space science measurements cannot be made with the necessary accuracy or precision from the ground through the intervening atmosphere. Thus, university researchers in the space sciences have of necessity acquired the appropriate management skills and established strategic collaborations with engineering organizations (including university engineering departments) to enable the development, construction, and testing of remote sensing instruments and even entire spacecraft and missions. In contrast, much of the university-based Earth science community has focused on and made significant progress to date using in situ data, given that certain critical subsurface oceanic and land measurements and atmospheric profile measurements cannot be obtained even from space-based platforms. Most Earth science researchers and their academic institutions have thus developed the unique skills and support infrastructures required for the different, and in many cases less demanding, tasks of constructing, testing, and deploying in situ instrumentation. For example, nine academic institutions currently operate the 14 major oceanographic research vessels that make up the nation’s University-National Oceanographic Laboratory System (UNOLS) fleet, but none of these institutions has developed a spaceborne remote sensing instrument, let alone a spacecraft or an entire mission. The development of spaceborne Earth observation measurement techniques, instruments, and remote sensing missions has therefore taken place largely in industry and government (principally at Goddard Space Flight Center (GSFC), the Jet Propulsion Laboratory (JPL), and the Naval Research Laboratory), while the academic focus has been on measurement validation using classic in situ data and the interpretation and utilization of the data once

OCR for page 18
Steps to Facilitate Principal-Investigator-Led Earth Science Missions acquired.1 Consequently, the university-based Earth science community has neither the tradition nor a large number of PIs trained and experienced in managing advanced space-based technical projects. Few universities have sufficiently strong space system and Earth remote sensing instrumentation development programs to complement and support geophysical research efforts sponsored by NASA and other agencies. Furthermore, few senior Earth science faculty can serve as space mission PI role models for younger scientists, and (as noted throughout this report) the stringent cost and schedule constraints of ESSP missions do not make them suitable for initial PI training of junior faculty. Example 2: Risk Tolerance—Theory Versus Practice Objective: Accept relaxed risk tolerance to support innovative approaches to measurement techniques, technical innovation, and novel management approaches. Constraints: NASA is unwilling in practice to accept mission failure, the risk of which might be increased with more innovative approaches. The Earth Explorers Program identifies several objectives—including “pathfinder exploratory measurements,” “technical innovation,” and innovative management approaches—that imply a greater tolerance of risk than would be acceptable for facility-class missions. However, increased risk tolerance in theory does not translate in practice to NASA’s being more willing to accept failures in the program. Congress, the public, and the research community all hold NASA responsible for mission success or failure.2 ESE cannot therefore divest itself of the ultimate programmatic responsibility for the success of any of its space missions. Furthermore, Earth Explorer missions must succeed scientifically if they are to play a vital role in the overall ESE and national Earth science programs as claimed by NASA. Because these missions are used to address unique and high-priority science, they are critically important to the overall ESE science strategy. Another aspect of risk is the introduction of a new and untried technology or methodology in a cost schedule-constrained program. NASA has been explicit about avoiding technology development within the Earth Explorer missions, but there is still some general indication in the AO statement that innovative management techniques should be considered. As is discussed in more detail below, the schedule and cost constraints for Earth Explorer missions, coupled with NASA’s responsibilities for the missions’ scientific success, substantially diminish the likelihood that innovative management approaches could be used successfully. Any attempt to force the introduction of a new technology or methodology into the program makes it far more difficult for inexperienced academic scientists and their institutions to participate fully as mission management leaders. Example 3: NASA Acceptance of Innovative PI Management Approaches Objective: Allow the PI to define and implement the management approach. Constraints: NASA depends on its own management practices to ensure success. The decision to promote management leadership of Earth Explorer missions outside NASA centers was intended partly to provide incentives for the development of innovative management practices. In practice, however, NASA’s stake in ensuring mission success inhibits management practices proposed by a PI that NASA believes could be inconsistent with mission success. The ESSP office at GSFC is tasked with ensuring the effectiveness of a PI’s management approach. The history of successful flight projects at GSFC, and the reliance on proven management approaches to achieve those 1   The Applied Physics Laboratory of Johns Hopkins University, with significant spaceborne instrument and spacecraft development expertise and strong ties to Earth science researchers, represents a notable exception. 2   William B. Gail, Ball Aerospace and Technologies Corporation and a member of the Committee on Earth Studies, “Perspectives on the PI-led AO Mission and the CES PI-Mode Study,” presentation to the Committee on Earth Studies, April 25, 2001, p. 15.

OCR for page 18
Steps to Facilitate Principal-Investigator-Led Earth Science Missions successes, suggest the potential for conflict with management approaches that differ from GSFC processes. The committee recognizes that various organizations, such as GSFC, JPL, the Department of Defense, and industry, all have proven but differing management approaches. In many cases, GSFC may be justified in promoting its processes over those proposed by a PI, but it is not clear that there are viable criteria for distinguishing between effective nontraditional management approaches and approaches that are inherently less effective than proven GSFC techniques. The involvement of academic institutions in mission management introduces other issues as well. The mandated PI-led management mode is consistent with NASA’s aims of building university mission capability and fostering innovative teaming arrangements between government and nongovernmental (e.g., industry and/or university) partners. But the time scale of ESSP Earth Explorer missions, with a maximum of 3 years for prelaunch development and 6 to 7 years total from inception to the end of the baseline mission, is inconsistent with academic career objectives and the tenure of undergraduate or graduate students. Furthermore, a university academic department is not always the best place for hardware development and testing. ESE therefore expects the PI to contract out these tasks to the appropriate development organizations (private industry or NASA laboratories), leaving the PI and the students to concentrate on the research tasks. Thus PI control and leadership do not necessarily mean that the PI is fully the project manager, with the day-to-day concerns for schedule, cost, and work completion. Such responsibilities may be delegated to a professional manager with space mission experience and preferably an understanding of the technology or science, so that the PI is free to be the primary arbiter of the science and the final decision maker when compromises must be made. PI-led management also contributes directly to ensuring that the necessary trade-offs during the mission design and implementation stages preserve the scientific integrity of the mission. During the development phase, the PI-led science team is the only recognized user of the mission’s measurement capabilities. In contrast, the government/private-sector relationships and interactions as practiced by the NASA field centers are conservative and hierarchical. This is appropriate for large observatory- and facility-class missions where NASA has responsibilities to sizable segments of the research community and the overall goal is to meet the requirements of the largest number of users with minimal risk. But it is difficult to introduce innovative changes in such programs. The long time scale of development, greater cost, and significant penalty for failure reduce the incentive for certain kinds of innovation. (Of course, because of the longer time scale, these programs are the ones that could afford to try new management techniques and develop new technology.) In principle, empowered “outsiders” such as selected PIs can be efficient agents of innovation—a principle substantiated in the automotive industry by, for example, the General Motors Saturn enterprise and in portions of the Japanese manufacturing sector.3 But although truly innovative management approaches have been demonstrated in industry (such as the integration of multigroup and multi-institution product teams and end-to-end manufacturing processes), the time, expense, and technical management expertise devoted to these efforts in the private sector have been far larger than can reasonably be expected from a small program like an Earth Explorer mission. Furthermore, there is no basis to expect that a typical inexperienced science-oriented PI at an academic institution, operating under the tight schedule and cost constraints of an Earth Explorer mission, would have the desire or the skills necessary to implement such innovative management approaches. Example 4: Erosion of Cost and Schedule Objectives Objective: Maintain a fixed and regular program launch schedule. Constraints: Unanticipated management rigor and reviews can impose programmatic burdens. The stringent Earth Explorers Program cost and schedule constraints and the mandated PI-mode of management may deprive NASA of many of its standard management tools for ensuring mission success. The classical 3   National Research Council, Space Studies Board, 2000, The Role of Small Satellites in NASA and NOAA Earth Observation Programs, National Academy Press, Washington, D.C., p. 52.

OCR for page 18
Steps to Facilitate Principal-Investigator-Led Earth Science Missions approach to minimizing failures usually involves a combination of active oversight and consistent management of the contractor—an approach that conflicts with the Earth Explorers Program’s PI management approach, increases both NASA and contractor costs, and generally lengthens schedules when unanticipated reviews or additional analysis and review tasks are imposed, thus violating the cost and schedule constraints basic to the original mission proposal solicitation and evaluation and essential to the long-term vitality of a flight program intended to have a fixed, regular launch schedule. Thus, lacking flexibility to change the schedule and cost once a mission is under way, NASA has a greater responsibility to plan, schedule, and obtain agreement on risk monitoring and risk-reduction activities in the mission’s initial phase. These risk management tasks are paramount throughout the lifetime of an Earth Explorer mission, from solicitation and selection through implementation and launch. COMPARISON WITH NON-PI-LED PROGRAMS Comparing PI-led missions with non-PI-led programs can be valuable for understanding how to resolve conflicting objectives and constrained resources. Several small non-PI-led NASA programs that were very successful should be considered in the context of their development by industry or government laboratories. The Clementine program, for example, is often held up as a very successful small satellite program and has been compared to several of the less successful planetary programs. In testimony before the House Science Committee, Pedro Rustan, Clementine’s mission director and later the director of the National Reconnaissance Office’s Small Satellites Program, gave his views on managing these advanced programs according to 10 management practices that he believed were key to the success of the Clementine program.4 Although most of the 10 practices are applicable to all programs, points 1 and 5 can be directly contrasted with the requirements of PI-led missions: 1. Empower a single program manager who is a seasoned leader and make that person responsible and accountable for all aspects of the mission during the entire duration of the program. This approach is in keeping with the stated philosophy of PI-led missions but introduces the challenge of finding strong scientists who are also good program managers. Such a program management practice, which is 4   In his testimony, Dr. Rustan stated, “The ten most important management practices used in the Clementine Program that are relevant to the recent Mars Mission can be summarized as follows: Empower a single PM who is a seasoned manager and a top leader and hold him/her fully responsible and accountable for all aspects (technical, managerial, and financial) of the mission during the entire duration of the program (cradle to grave). Nurture a group of government and industry personnel with a successful legacy for problem solving who are willing to do whatever it takes to guarantee mission success. Select and collocate a team of experienced managers to direct the development and integration of the various systems and subsystems throughout the mission. Remain steadfast without making any significant changes in the requirements (scientific, operational, or technical) of the mission after final design completion. Use the most advanced technologies readily available that can increase mission capabilities and reduce cost. Depend on extensive testing throughout the entire mission and minimize dependence on analysis and simulation. Perform only four major reviews at the program transition points, but be ready to perform an in-depth review whenever there is a problem or discrepancy. Use Integrated Product Teams (IPTs) to maintain online communication with all the stakeholders. Ensure that only one person can make changes in the Interface Control Document (ICD). Complete all new technology developments before the Critical Design Review (CDR). See Testimony of Pedro L. Rustan, PhD, United States House of Representatives Committee on Science hearing, “NASA’s Mars Program After the Young Report, Part II,” May 11, 2000. Available at <http://www.house.gov/science/rustan_062000.htm>. Pedro L. Rustan, adapted from May 11, 2000, testimony to the House Science Committee and published in Space News, August 28, 2000, p. 25.

OCR for page 18
Steps to Facilitate Principal-Investigator-Led Earth Science Missions universally supported but not always evenly applied, is specifically designed for an industry or government laboratory. The academic PI, on the other hand, will probably have to share control with an experienced program manager: 5. Use the most advanced technologies available to increase mission capabilities and reduce cost. Rustan is a strong supporter of the use of advanced technology, which was used in the Clementine program. While all Earth Explorer missions incorporate advanced technologies in the instruments and the science payload, the committee does not recommend trying to develop advanced technology during these missions. Clementine benefited from a dedicated team of spacecraft systems developers, and considerable effort went into the development and testing of new hardware, much of which was being developed for other programs and was made available to the Clementine program. The Earth Explorers Program does not enjoy these advantages.5 As stated in Rustan’s seventh principle, he is also a proponent of a small number of essential reviews at program transition points and has observed that NASA programs have too many reviews that take critical time from the program. FINDINGS AND RECOMMENDATIONS As currently conceptualized, ESE PI-led missions include a wide variety of explicit and implicit objectives while maintaining very tight limitations on resources. Mismatches between objectives and constraints have been the root cause of many difficulties faced by PI-led missions. Finding: The scientific and programmatic objectives of ESE are ambitious compared with the constraints under which PI-led missions are implemented, particularly the capped funding and tight schedule. Recommendation: NASA’s Earth Science Enterprise should focus its programmatic objectives for PI-led missions to better match the available resources and constraints, with achievement of high-quality science measurements being the highest-priority objective. Finding: Universities can derive considerable benefit by participating in an ESE mission; however, using PI-led missions to build the capacity of university-based research is not readily achievable within the structure and resources of current ESE PI-led programs. Recommendation: NASA’s Earth Science Enterprise should not include building the capacity of university-based research as an explicit objective of PI-led missions unless fundamental changes are made to program structure and resources. 5   In his analysis of small satellite programs, Sarsfield also suggests that technology is the key to lower costs and, obviously, better performance, although he recognizes some of the disadvantages of introducing technology too early. See Liam P. Sarsfield, 1998, The Cosmos on a Shoestring: Small Spacecraft for Space and Earth Science, Document Number MR-864-OSTP, RAND Corporation, Santa Monica, Calif.