3
Program Conceptualization: Matching Objectives to Constraints
Successful programs start with proper conceptualization, including the identification of well-defined and widely understood objectives and the establishment of resources sufficient to achieve those objectives. This chapter identifies several areas in which it has proved difficult to match objectives and resources for existing PI-led missions and investigates how the issue is addressed in several non-PI-led programs.
STATED OBJECTIVES AND CONSTRAINTS
Table 3.1 summarizes ESE’s stated ESSP objectives and constraints, which are representative of all ESE PI-led missions and are appropriate as guidelines for PI-led missions when considered individually. Nevertheless, the combination of ambitious scientific and programmatic objectives, coupled with tight schedules, capped costs, and (to a lesser extent) management and implementation constraints, makes PI-led missions very challenging, even in comparison to larger, more typical NASA projects.
UNSTATED OBJECTIVES AND CONSTRAINTS
In addition to the stated objectives listed in Table 3.1, there are a number of unstated objectives that PI-led missions are expected to satisfy, including the desire to increase the capacity of university-based research (which is an explicit objective of UnESS). Potential mismatches between unstated objectives and existing constraints are often more difficult to identify than those associated with stated objectives, but they can be equally important in the success of a mission. The following four examples illustrate the impact of mismatches that can arise between unstated objectives and constraints in PI-led Earth science missions.
Example 1: The Limited Capacity of Universities
Objective: |
Enhance university-based Earth science research through participation in PI-led missions. |
Constraints: |
For Earth science, universities generally have limited technical and programmatic experience with spaceborne missions, and relatively few universities have the capacity to lead or support PI-led Earth science missions at acceptable risk levels. |
TABLE 3.1 Stated Objectives and Constraints Within the Earth System Science Pathfinder Program
|
Text from ESSP-3 AOa |
Objectives |
|
Frequent low-cost missions |
“launch rate of at least one mission per year” “low-cost, quick turnaround spaceborne missions” “NASA encourages and favors low-cost missions” |
High-priority, focused, exploratory science |
“the philosophy behind ESSP embraces small satellite missions addressing high priority Earth System Science objectives” “ESSP … is intended to address exploratory measurements which can yield new scientific breakthroughs … addressing a focused set of scientific questions” |
Innovative project implementation |
“creativity in all aspects of mission development” “innovation in instrumentation and strategies for acquiring and distributing new data products” “the PI’s team will have a large degree of freedom in accomplishing mission objectives within the stated constraints” |
Constraints |
|
Available funding |
“NASA encourages but does not require contributions from sources other than NASA” “the NASA ESE cost ceiling is $125M” |
Mission readiness |
“every aspect of the mission shall reflect a commitment to mission success” “all proposed missions shall be of sufficient technical maturity to achieve launch readiness within a goal of 36 months” “to ensure mission success, there will be appropriate Government oversight and insight” |
aESSP-3 Announcement of Opportunity (AO), 2001. This and other reference documents for the ESSP program can be found in the ESSP AO library online at <http://centauri.larc.nasa.gov/essp/essplib.html>. |
The space science community has long required technically advanced remote sensing systems because many critical space science measurements cannot be made with the necessary accuracy or precision from the ground through the intervening atmosphere. Thus, university researchers in the space sciences have of necessity acquired the appropriate management skills and established strategic collaborations with engineering organizations (including university engineering departments) to enable the development, construction, and testing of remote sensing instruments and even entire spacecraft and missions.
In contrast, much of the university-based Earth science community has focused on and made significant progress to date using in situ data, given that certain critical subsurface oceanic and land measurements and atmospheric profile measurements cannot be obtained even from space-based platforms. Most Earth science researchers and their academic institutions have thus developed the unique skills and support infrastructures required for the different, and in many cases less demanding, tasks of constructing, testing, and deploying in situ instrumentation. For example, nine academic institutions currently operate the 14 major oceanographic research vessels that make up the nation’s University-National Oceanographic Laboratory System (UNOLS) fleet, but none of these institutions has developed a spaceborne remote sensing instrument, let alone a spacecraft or an entire mission.
The development of spaceborne Earth observation measurement techniques, instruments, and remote sensing missions has therefore taken place largely in industry and government (principally at Goddard Space Flight Center (GSFC), the Jet Propulsion Laboratory (JPL), and the Naval Research Laboratory), while the academic focus has been on measurement validation using classic in situ data and the interpretation and utilization of the data once
acquired.1 Consequently, the university-based Earth science community has neither the tradition nor a large number of PIs trained and experienced in managing advanced space-based technical projects.
Few universities have sufficiently strong space system and Earth remote sensing instrumentation development programs to complement and support geophysical research efforts sponsored by NASA and other agencies. Furthermore, few senior Earth science faculty can serve as space mission PI role models for younger scientists, and (as noted throughout this report) the stringent cost and schedule constraints of ESSP missions do not make them suitable for initial PI training of junior faculty.
Example 2: Risk Tolerance—Theory Versus Practice
Objective: |
Accept relaxed risk tolerance to support innovative approaches to measurement techniques, technical innovation, and novel management approaches. |
Constraints: |
NASA is unwilling in practice to accept mission failure, the risk of which might be increased with more innovative approaches. |
The Earth Explorers Program identifies several objectives—including “pathfinder exploratory measurements,” “technical innovation,” and innovative management approaches—that imply a greater tolerance of risk than would be acceptable for facility-class missions. However, increased risk tolerance in theory does not translate in practice to NASA’s being more willing to accept failures in the program. Congress, the public, and the research community all hold NASA responsible for mission success or failure.2 ESE cannot therefore divest itself of the ultimate programmatic responsibility for the success of any of its space missions.
Furthermore, Earth Explorer missions must succeed scientifically if they are to play a vital role in the overall ESE and national Earth science programs as claimed by NASA. Because these missions are used to address unique and high-priority science, they are critically important to the overall ESE science strategy.
Another aspect of risk is the introduction of a new and untried technology or methodology in a cost schedule-constrained program. NASA has been explicit about avoiding technology development within the Earth Explorer missions, but there is still some general indication in the AO statement that innovative management techniques should be considered. As is discussed in more detail below, the schedule and cost constraints for Earth Explorer missions, coupled with NASA’s responsibilities for the missions’ scientific success, substantially diminish the likelihood that innovative management approaches could be used successfully. Any attempt to force the introduction of a new technology or methodology into the program makes it far more difficult for inexperienced academic scientists and their institutions to participate fully as mission management leaders.
Example 3: NASA Acceptance of Innovative PI Management Approaches
Objective: |
Allow the PI to define and implement the management approach. |
Constraints: |
NASA depends on its own management practices to ensure success. |
The decision to promote management leadership of Earth Explorer missions outside NASA centers was intended partly to provide incentives for the development of innovative management practices. In practice, however, NASA’s stake in ensuring mission success inhibits management practices proposed by a PI that NASA believes could be inconsistent with mission success.
The ESSP office at GSFC is tasked with ensuring the effectiveness of a PI’s management approach. The history of successful flight projects at GSFC, and the reliance on proven management approaches to achieve those
successes, suggest the potential for conflict with management approaches that differ from GSFC processes. The committee recognizes that various organizations, such as GSFC, JPL, the Department of Defense, and industry, all have proven but differing management approaches. In many cases, GSFC may be justified in promoting its processes over those proposed by a PI, but it is not clear that there are viable criteria for distinguishing between effective nontraditional management approaches and approaches that are inherently less effective than proven GSFC techniques.
The involvement of academic institutions in mission management introduces other issues as well. The mandated PI-led management mode is consistent with NASA’s aims of building university mission capability and fostering innovative teaming arrangements between government and nongovernmental (e.g., industry and/or university) partners. But the time scale of ESSP Earth Explorer missions, with a maximum of 3 years for prelaunch development and 6 to 7 years total from inception to the end of the baseline mission, is inconsistent with academic career objectives and the tenure of undergraduate or graduate students. Furthermore, a university academic department is not always the best place for hardware development and testing. ESE therefore expects the PI to contract out these tasks to the appropriate development organizations (private industry or NASA laboratories), leaving the PI and the students to concentrate on the research tasks.
Thus PI control and leadership do not necessarily mean that the PI is fully the project manager, with the day-to-day concerns for schedule, cost, and work completion. Such responsibilities may be delegated to a professional manager with space mission experience and preferably an understanding of the technology or science, so that the PI is free to be the primary arbiter of the science and the final decision maker when compromises must be made. PI-led management also contributes directly to ensuring that the necessary trade-offs during the mission design and implementation stages preserve the scientific integrity of the mission. During the development phase, the PI-led science team is the only recognized user of the mission’s measurement capabilities.
In contrast, the government/private-sector relationships and interactions as practiced by the NASA field centers are conservative and hierarchical. This is appropriate for large observatory- and facility-class missions where NASA has responsibilities to sizable segments of the research community and the overall goal is to meet the requirements of the largest number of users with minimal risk. But it is difficult to introduce innovative changes in such programs. The long time scale of development, greater cost, and significant penalty for failure reduce the incentive for certain kinds of innovation. (Of course, because of the longer time scale, these programs are the ones that could afford to try new management techniques and develop new technology.)
In principle, empowered “outsiders” such as selected PIs can be efficient agents of innovation—a principle substantiated in the automotive industry by, for example, the General Motors Saturn enterprise and in portions of the Japanese manufacturing sector.3 But although truly innovative management approaches have been demonstrated in industry (such as the integration of multigroup and multi-institution product teams and end-to-end manufacturing processes), the time, expense, and technical management expertise devoted to these efforts in the private sector have been far larger than can reasonably be expected from a small program like an Earth Explorer mission. Furthermore, there is no basis to expect that a typical inexperienced science-oriented PI at an academic institution, operating under the tight schedule and cost constraints of an Earth Explorer mission, would have the desire or the skills necessary to implement such innovative management approaches.
Example 4: Erosion of Cost and Schedule Objectives
Objective: |
Maintain a fixed and regular program launch schedule. |
Constraints: |
Unanticipated management rigor and reviews can impose programmatic burdens. |
The stringent Earth Explorers Program cost and schedule constraints and the mandated PI-mode of management may deprive NASA of many of its standard management tools for ensuring mission success. The classical
approach to minimizing failures usually involves a combination of active oversight and consistent management of the contractor—an approach that conflicts with the Earth Explorers Program’s PI management approach, increases both NASA and contractor costs, and generally lengthens schedules when unanticipated reviews or additional analysis and review tasks are imposed, thus violating the cost and schedule constraints basic to the original mission proposal solicitation and evaluation and essential to the long-term vitality of a flight program intended to have a fixed, regular launch schedule.
Thus, lacking flexibility to change the schedule and cost once a mission is under way, NASA has a greater responsibility to plan, schedule, and obtain agreement on risk monitoring and risk-reduction activities in the mission’s initial phase. These risk management tasks are paramount throughout the lifetime of an Earth Explorer mission, from solicitation and selection through implementation and launch.
COMPARISON WITH NON-PI-LED PROGRAMS
Comparing PI-led missions with non-PI-led programs can be valuable for understanding how to resolve conflicting objectives and constrained resources. Several small non-PI-led NASA programs that were very successful should be considered in the context of their development by industry or government laboratories. The Clementine program, for example, is often held up as a very successful small satellite program and has been compared to several of the less successful planetary programs. In testimony before the House Science Committee, Pedro Rustan, Clementine’s mission director and later the director of the National Reconnaissance Office’s Small Satellites Program, gave his views on managing these advanced programs according to 10 management practices that he believed were key to the success of the Clementine program.4 Although most of the 10 practices are applicable to all programs, points 1 and 5 can be directly contrasted with the requirements of PI-led missions:
1. Empower a single program manager who is a seasoned leader and make that person responsible and accountable for all aspects of the mission during the entire duration of the program.
This approach is in keeping with the stated philosophy of PI-led missions but introduces the challenge of finding strong scientists who are also good program managers. Such a program management practice, which is
4 |
In his testimony, Dr. Rustan stated, “The ten most important management practices used in the Clementine Program that are relevant to the recent Mars Mission can be summarized as follows:
See Testimony of Pedro L. Rustan, PhD, United States House of Representatives Committee on Science hearing, “NASA’s Mars Program After the Young Report, Part II,” May 11, 2000. Available at <http://www.house.gov/science/rustan_062000.htm>. Pedro L. Rustan, adapted from May 11, 2000, testimony to the House Science Committee and published in Space News, August 28, 2000, p. 25. |
universally supported but not always evenly applied, is specifically designed for an industry or government laboratory. The academic PI, on the other hand, will probably have to share control with an experienced program manager:
5. Use the most advanced technologies available to increase mission capabilities and reduce cost.
Rustan is a strong supporter of the use of advanced technology, which was used in the Clementine program. While all Earth Explorer missions incorporate advanced technologies in the instruments and the science payload, the committee does not recommend trying to develop advanced technology during these missions. Clementine benefited from a dedicated team of spacecraft systems developers, and considerable effort went into the development and testing of new hardware, much of which was being developed for other programs and was made available to the Clementine program. The Earth Explorers Program does not enjoy these advantages.5
As stated in Rustan’s seventh principle, he is also a proponent of a small number of essential reviews at program transition points and has observed that NASA programs have too many reviews that take critical time from the program.
FINDINGS AND RECOMMENDATIONS
As currently conceptualized, ESE PI-led missions include a wide variety of explicit and implicit objectives while maintaining very tight limitations on resources. Mismatches between objectives and constraints have been the root cause of many difficulties faced by PI-led missions.
Finding: The scientific and programmatic objectives of ESE are ambitious compared with the constraints under which PI-led missions are implemented, particularly the capped funding and tight schedule.
Recommendation: NASA’s Earth Science Enterprise should focus its programmatic objectives for PI-led missions to better match the available resources and constraints, with achievement of high-quality science measurements being the highest-priority objective.
Finding: Universities can derive considerable benefit by participating in an ESE mission; however, using PI-led missions to build the capacity of university-based research is not readily achievable within the structure and resources of current ESE PI-led programs.
Recommendation: NASA’s Earth Science Enterprise should not include building the capacity of university-based research as an explicit objective of PI-led missions unless fundamental changes are made to program structure and resources.