Executive Summary

Over the last decade, NASA has increasingly emphasized small, focused principal-investigator (PI)-led science missions as an important element of its space and Earth science programs. NASA has chosen to implement many of these projects by soliciting mission ideas from the scientific community and giving the selected PI responsibility for both the scientific and the programmatic success of the entire project. With the first of these missions now launched and producing valuable scientific results, PI-led missions have established their importance as part of a balanced scientific observing program.

NASA’s Earth Science Enterprise (ESE) initiated the Earth System Science Pathfinder (ESSP), its first program for PI-led missions, in 1996.1 ESSP supports “low-cost, quick turnaround spaceborne missions” intended to provide “exploratory measurements which can yield new scientific breakthroughs and can deliver conclusive scientific results addressing a focused set of scientific questions.”2 Six missions have since been selected in three rounds of ESSP proposals, with one mission successfully launched and four in development for launch in 2004 and beyond (the sixth mission—Vegetation Canopy Lidar—was descoped to a technology development program and has now been canceled).

PI-led missions represent one of several programmatic approaches that ESE takes to obtain scientific data from space, including multi-instrument facility-class missions, data buys, dedicated observatories, and others. The Earth Explorers Program,3 within which PI-led mission projects are executed, has proven to be a valuable and complementary component in this portfolio of mission approaches for obtaining the data required to support the

1  

According to the NPG 7120.5B, a program is “an activity within an Enterprise having defined goals, objectives, requirements, funding, and consisting of one or more projects, reporting to the NASA Program Management Council, unless delegated to a Governing Program Management Council”; a project is “an activity designated by a program and characterized as having defined goals, objectives, requirements, life cycle costs, a beginning, and an end.” See NASA Procedures and Guidelines 7120.5B: NASA Program and Project Management Processes and Requirements. Available online at <http://nodis3.gsfc.nasa.gov/displayDir.cfm?Internal_ID=N_PR_7120_005B_&page_name=main>.

2  

ESSP-3 Announcement of Opportunity (AO), 2001. This and other reference documents for the ESSP program can be found in the ESSP AO library online at <http://centauri.larc.nasa.gov/essp/essplib.html>.

3  

NASA’s Earth Explorers Program is “the component of Earth Science Enterprise that investigates specific, highly focused areas of Earth science research. It is comprised of flight projects that provide pathfinder exploratory and process driven measurements, answering innovative and unique Earth science questions.” Currently, the components of the Earth Explorers Program are the Earth System Science Pathfinder; the Rapid Spacecraft Development Office; the Solar Radiation and Climate Experiment (SORCE), which is in orbit; and Triana, which has been placed in storage indefinitely. See the Earth Explorers Program home page at <http://earthexplorers.gsfc.nasa.gov/index.html>.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
Steps to Facilitate Principal-Investigator-Led Earth Science Missions Executive Summary Over the last decade, NASA has increasingly emphasized small, focused principal-investigator (PI)-led science missions as an important element of its space and Earth science programs. NASA has chosen to implement many of these projects by soliciting mission ideas from the scientific community and giving the selected PI responsibility for both the scientific and the programmatic success of the entire project. With the first of these missions now launched and producing valuable scientific results, PI-led missions have established their importance as part of a balanced scientific observing program. NASA’s Earth Science Enterprise (ESE) initiated the Earth System Science Pathfinder (ESSP), its first program for PI-led missions, in 1996.1 ESSP supports “low-cost, quick turnaround spaceborne missions” intended to provide “exploratory measurements which can yield new scientific breakthroughs and can deliver conclusive scientific results addressing a focused set of scientific questions.”2 Six missions have since been selected in three rounds of ESSP proposals, with one mission successfully launched and four in development for launch in 2004 and beyond (the sixth mission—Vegetation Canopy Lidar—was descoped to a technology development program and has now been canceled). PI-led missions represent one of several programmatic approaches that ESE takes to obtain scientific data from space, including multi-instrument facility-class missions, data buys, dedicated observatories, and others. The Earth Explorers Program,3 within which PI-led mission projects are executed, has proven to be a valuable and complementary component in this portfolio of mission approaches for obtaining the data required to support the 1   According to the NPG 7120.5B, a program is “an activity within an Enterprise having defined goals, objectives, requirements, funding, and consisting of one or more projects, reporting to the NASA Program Management Council, unless delegated to a Governing Program Management Council”; a project is “an activity designated by a program and characterized as having defined goals, objectives, requirements, life cycle costs, a beginning, and an end.” See NASA Procedures and Guidelines 7120.5B: NASA Program and Project Management Processes and Requirements. Available online at <http://nodis3.gsfc.nasa.gov/displayDir.cfm?Internal_ID=N_PR_7120_005B_&page_name=main>. 2   ESSP-3 Announcement of Opportunity (AO), 2001. This and other reference documents for the ESSP program can be found in the ESSP AO library online at <http://centauri.larc.nasa.gov/essp/essplib.html>. 3   NASA’s Earth Explorers Program is “the component of Earth Science Enterprise that investigates specific, highly focused areas of Earth science research. It is comprised of flight projects that provide pathfinder exploratory and process driven measurements, answering innovative and unique Earth science questions.” Currently, the components of the Earth Explorers Program are the Earth System Science Pathfinder; the Rapid Spacecraft Development Office; the Solar Radiation and Climate Experiment (SORCE), which is in orbit; and Triana, which has been placed in storage indefinitely. See the Earth Explorers Program home page at <http://earthexplorers.gsfc.nasa.gov/index.html>.

OCR for page 1
Steps to Facilitate Principal-Investigator-Led Earth Science Missions ESE objective of developing an understanding of the total Earth system and of the effects of natural and human-induced changes on the global environment.4 The explicit objectives of PI-led missions are usually stated clearly in the solicitation,5 but such projects have also historically promoted additional ESE goals. In particular, PI-led missions have been viewed as a means to help develop the capacity of university-based research, building on the potential for leadership by university-based PIs and for the substantial involvement of educational institutions. The experiences with the first six selected ESSP projects underscore the challenges that PI-led missions face. All spaceborne missions are subject to the risks associated with pursuing difficult objectives in the harsh environment of space. PI-led ESSP missions face further challenges that are closely associated with the ambitious objectives of the ESSP program, the limited resources available to satisfy those objectives, the uneven record of the solicitation and selection process in choosing viable missions, and the varying maturity of the processes for executing these missions. The purpose of this study is to identify and evaluate opportunities for enhancing all aspects of PI-led missions and to recommend whether (and if so, how) they should be used to build the capacity of university-based research. The committee concluded that ESE should focus on enhancements for PI-led missions in three areas: the conceptualization of the programs, the institutional investments that support them, and the implementation of the projects themselves. Its findings and recommendations address potential enhancements aligned with these three areas. Finding: The PI-led mission paradigm represents a valuable approach to soliciting and executing missions involving focused science objectives, with demonstrated success in both the Earth and space sciences. PI-led missions provide an important element of the overall ESE observing strategy, complementing other elements such as facility-class missions and data buys. Recommendation: NASA’s Earth Science Enterprise should continue to employ PI-led missions as one element of the ESE observation system. It should ensure regular review and improvement of the programs that employ or are associated with PI-led missions to increase their effectiveness and value to ESE and the science community. PROGRAM CONCEPTUALIZATION: MATCHING OBJECTIVES TO CONSTRAINTS By design, the PI-led missions that are selected by NASA’s ESE are ambitious in their expected science return and frugal in their demands on fiscal and other resources. In the committee’s view, a mismatch between objectives and constraints has been the root cause of many difficulties encountered in the execution of PI-led missions. Finding: The scientific and programmatic objectives of ESE are ambitious compared with the constraints under which PI-led missions are implemented, particularly the capped funding and tight schedule. Recommendation: NASA’s Earth Science Enterprise should focus its programmatic objectives for PI-led missions to better match the available resources and constraints, with achievement of high-quality science measurements being the highest-priority objective. Finding: Universities can derive considerable benefit by participating in an ESE mission; however, using PI-led missions to build the capacity of university-based research is not readily achievable within the structure and resources of current ESE PI-led programs. 4   See the ESE home page at <www.earth.nasa.gov>. 5   The primary stated objectives in the ESSP-3 AO (2001) solicitation included frequent low-cost missions, high-priority focused exploratory science, and innovative project implementation.

OCR for page 1
Steps to Facilitate Principal-Investigator-Led Earth Science Missions Recommendation: NASA’s Earth Science Enterprise should not include building the capacity of university-based research as an explicit objective of PI-led missions unless fundamental changes are made to program structure and resources. INSTITUTIONAL INVESTMENTS: ESTABLISHING THE FOUNDATIONS Success in PI-led missions correlates with direction of the projects by experienced PI-led teams, the use of mature technologies,6 and the existence of adequate project management tools at the time of mission selection. Ensuring an adequate number of potential proposers requires that opportunities exist to develop these tools and capabilities. Finding: The rigorous and ambitious cost and schedule constraints imposed on PI-led missions preclude all but minimal technology development prior to launch. Recommendation: NASA’s Earth Science Enterprise should explicitly nurture and coordinate technology feeder programs—such as the Instrument Incubator Program and the Office of Aerospace Technology’s Mission and Science Measurement Technology Program—that develop technologies with potential application to PI-led missions. A quantitative assessment of the anticipated flow of technology through the technology readiness level chain would help guide this effort. Finding: Proposers of non-selected PI-led missions found to have high scientific priority but known technical risk have limited access to funding for reducing the project’s level of risk prior to the next proposal round. Both ESE and the scientific community would benefit from improved opportunities to reduce the technical risk of recognized high-priority science missions and then re-propose them. Recommendation: NASA’s Earth Science Enterprise should include within the solicitation for PI-led missions a component, following the Solar System Exploration Discovery model, that provides limited technology funding for high-priority non-selected PI-led mission proposals to increase their technology readiness for the next proposal round. Finding: The Earth science community, particularly the university-based community, has historically produced only a small number of scientists with the in-depth space engineering and technical management experience that is required to lead a project in a PI mode of operation. Recommendation: NASA’s Earth Science Enterprise should formally identify and promote activities that develop PIs qualified to propose and lead small, focused science missions. PROJECT IMPLEMENTATION: IMPROVING LIFE-CYCLE PROCESSES The three fundamental elements of the PI-led mission life cycle—solicitation, selection, and execution—are part of a system of checks and balances. The system functions properly when the solicitation process establishes an achievable balance of objectives and resources, the selection process ensures that the chosen missions reflect that balance, and the execution process rigorously maintains the balance throughout mission development. Some of the problems encountered with current ESE PI-led missions could have been avoided if this system had worked more effectively. The committee’s recommendations for enhancing project implementation focus on improving the checks and balances in each of these three life-cycle processes. 6   NASA assesses the maturity of a technology according to technology readiness levels (TRLs). For an explanation of TRLs, see John C. Mankins, “Technology Readiness Levels: A White Paper,” available at <www.ngst.nasa.gov/public/unconfigured/doc_0375/rev_02/technology_readiness.doc>. See also Table 4.1 in Chapter 4.

OCR for page 1
Steps to Facilitate Principal-Investigator-Led Earth Science Missions Finding: Existing NASA guidelines (e.g., NPG 7120.5B) establish a management system relevant to PI-led missions, including an essential checks-and-balances formalism for the three PI-led mission project life-cycle processes of solicitation, selection, and execution. Recommendation: NASA’s Earth Science Enterprise should emphasize formal and regular reviews of the life-cycle system of checks and balances as applied to PI-led missions and should continuously strengthen the processes on which the system is based. Finding: Many of the issues arising throughout a mission’s lifetime are rooted in decisions made by the PI and project team during the formulation phase—early in the project—as the mission concept is developed, team roles and responsibilities (including NASA’s) are defined, and the management approach is established. Ultimate mission success requires that major technical and programmatic issues be identified and jointly addressed by both the PI team and the NASA program office during the formulation phase. While extending competition between PI teams through the entire formulation phase provides NASA with additional insight into the effectiveness of the PI teams and the maturity of the mission designs, it delays the integration of the PI and NASA teams and motivates the PI teams to emphasize strengths and minimize weaknesses. Recommendation: NASA’s Earth Science Enterprise should avoid extensive overlap between competition and execution activities during the formulation phase of PI-led missions, thus providing an adequate schedule for the PI team and NASA to perform critical formulation tasks after the competitive selection is completed. Solicitation The objectives and constraints that drive a PI-led project are determined largely by the first element of the life cycle, solicitation. A carefully constructed solicitation can provide for a more achievable balance between objectives and constraints, thus increasing the probability of receiving viable proposals. Finding: The threat of project cancellation has not proved effective either in motivating the submission of PI-led proposals with adequate reserves or in constraining costs to meet the cost cap. Recommendation: NASA’s Earth Science Enterprise should redefine cost caps from a threshold that triggers an automatic termination review to a threshold for a remedial review that includes an examination of how the division of responsibility and authority between the PI and ESE might be revised to better control costs. Cost caps should be established only when the project has reached a sufficient level of maturity that the proposed cost is credible, such as at mission design review. ESE should also consider the use of a science floor, a PI-proposed minimum scientific achievement needed to justify the mission, in setting and managing within cost caps. Finding: Domestic and international partners have increasingly been included on PI-led mission teams to enhance the quality of science achievable within the available ESE project budget. Despite the many benefits of such collaborations, more complex and diverse teams increase risk and add costs to pay for team interfaces. Recommendation: NASA’s Earth Science Enterprise should recognize not only the benefits but also the risks of having domestic and international partners in a PI-led mission program. The mission solicitation should identify the need for processes by which both the PI team and the relevant NASA office ensure that partnering agreements are completed early in the formulation phase, that definition of an interface is given high priority, and that the management decision chain is clear and is understood by all parties.

OCR for page 1
Steps to Facilitate Principal-Investigator-Led Earth Science Missions Finding: A properly constructed solicitation balances the need for proposals detailed enough to permit thorough evaluation against the time required both to prepare and to evaluate proposals. The two-step proposal process, in particular the use of short Step 1 proposals within ESSP, has provided a workable balance. However, the lack of NASA-funded support for proposals, particularly during Step 2, is increasingly limiting the ability of smaller organizations and universities to participate. Recommendation: NASA’s Earth Science Enterprise should maintain the current two-step proposal process for PI-led missions but should provide funding to proposers for Step 2. Finding: Scientific results are the primary objective in PI-led missions, but postlaunch science funding commitments are not adequately identified in mission solicitations. Recommendation: NASA’s Earth Science Enterprise should clearly specify within the solicitation for a PI-led mission the extent to which scientific investigation and data analysis are expected to be included in the initial mission project budget, as well as the anticipated plans and budget for additional postlaunch science investigations. The science funded for the mission should address a PI-proposed science floor. Finding: Effective communication and the transfer of lessons learned between the Earth Explorers Program Office, current flight projects, and potential PI proposers can both increase the number of qualified proposers and reduce the risk associated with proposed projects. Recommendation: NASA’s Earth Science Enterprise should continue to emphasize and promote communication and the transfer of lessons learned between the Earth Explorers Program Office, current flight projects, and potential PI proposers. Selection Even a well-designed solicitation fails if the second element in the life cycle, the selection process, cannot reliably identify and select PI-led missions that both satisfy the solicitation and can be implemented within cost and schedule constraints. Finding: The quality of the selection process determines whether viable projects proceed to execution and thus greatly influences the overall success of PI-led missions. Selection criteria for PI-led missions, particularly those employed in Step 2, must adequately consider the ability of the project team to successfully implement a project; the ESE associate administrator must be provided sufficient information to determine the likely success of a project; and the selection decision must reflect an objective evaluation of the likelihood of success. Recommendation: NASA’s Earth Science Enterprise should carefully review the selection criteria for PI-led missions to ensure that they adequately identify and promote missions that can succeed. Finding: The number of qualified reviewers for ESE PI-led missions is small, particularly after elimination of scientists with conflicts of interest because of relationships with proposing teams. Recommendation: NASA’s Earth Science Enterprise should consider enlarging the pool of possible reviewers of PI-led missions by adding qualified international scientists (if feasible, given current International Traffic in Arms Regulations constraints) and scientists from the space science community. ESE should also consider requiring as part of the contract for selected PI-led projects that the PI serve subsequently as a reviewer.

OCR for page 1
Steps to Facilitate Principal-Investigator-Led Earth Science Missions Finding: The number of proposals selected for consideration in Step 2 represents a critical compromise between the desire for a large pool of evaluated PI-led mission proposals from which to make the final selection and the need for a pool small enough that available reviewers can perform detailed reviews. Selection for Step 2 of proposals that have a lower probability of final selection results in inefficient use of proposers’ resources. Recommendation: The proposals supported in Step 2 of the selection process for PI-led missions should include only those that have sufficiently high scientific merit and an acceptable initial evaluation of technical, management, and cost risk so as to be fully competitive with all other Step 2 proposals. As an informal guideline, a minimum of two Step 2 proposals should be selected for evaluation for each flight opportunity to be awarded, and the maximum number considered should be one-third of the total proposals submitted in Step 1. Finding: Maintaining and improving the credibility of checks and balances is the highest priority for enhancing the selection process for PI-led missions. An effective and credible proposal review process requires a balanced effort among proposers, reviewers, and the selection official. Proposers are motivated to avoid overly optimistic costing if they respect the cost-review process; reviewers are more diligent when their recommendations are likely to be accepted by the selection official; and the selection official relies more readily on reviewer recommendations when the proposal and review process is effective at identifying the best mission candidates. Recommendation: NASA’s Earth Science Enterprise should strengthen the complementary roles of proposers, reviewers, and the selection official in the selection process for PI-led missions, improving the critical balance between the three roles and focusing on clear traceability of the selection process to independent reviews and established ESE priorities. Finding: The availability of accurate cost estimates is a very important element of the mission selection process, but establishing accurate estimates of project cost has historically provided one of the largest challenges to both proposers and reviewers of PI-led missions. Recommendation: NASA’s Earth Science Enterprise should enhance its cost evaluation capabilities to improve the accuracy of mission selection decisions and to motivate improved fidelity of cost proposals. Execution Finally, selected PI-led missions will not succeed if the execution processes are inadequate. Finding: Although some of the difficulties with recent PI-led missions are unique, many of the problems encountered have root causes in common with non-PI-led missions. In particular, the transition to smaller cost-constrained projects during the 1990s and the contraction and aging of the space industry workforce have affected project success. These problems should not be attributed to flaws in the PI-mode process, but rather applied as general lessons for all small-mission projects. Recommendation: NASA’s Earth Science Enterprise should establish management processes for PI-led missions that emphasize understanding all PI-led and non-PI-led mission issues and the inclusion of appropriate lessons learned from both types of missions. Finding: Mission success is appropriately viewed as the combined responsibility of the PI-led team and NASA. Split as opposed to shared authority is appropriate for achieving mission success and is healthy for the PI community; split authority and the resulting allocation of responsibility should be explicitly recognized in the

OCR for page 1
Steps to Facilitate Principal-Investigator-Led Earth Science Missions project plan and should also reflect the philosophy inherent in PI-led missions that the mission is to be defined and developed by the science community itself. Recommendation: NASA’s Earth Science Enterprise should explicitly recognize that mission success is a combined responsibility of the PI team and NASA and should establish project management plans, organizations, and processes that reflect an appropriate split, not a sharing, of authority, with the PI taking the lead in defining and maintaining overall mission integrity. Finding: While it may be appropriate for PI-led missions to use management processes that differ from NASA standards, NASA-defined minimum management standards are desirable to reduce programmatic risk to acceptable levels. Recommendation: NASA’s Earth Science Enterprise should establish and enforce a comprehensive set of minimum standards for program management to be applied to all PI-led missions, while accepting that such missions may employ management processes that differ from those of NASA. These minimum management standards must invoke the rigor that experience has shown is required for success.