Principal-Investigator-Led Missions in the Space Sciences (2006)

“The most basic lesson from the Clementine mission is that the ability to carry out end-to-end planning and implementation of a (U.S.) planetary mission has evolved beyond NASA’s domain. Thus an underlying assumption of the Discovery program—that a non-NASA principal investigator can be successful when assuming overall responsibility for a deep-space mission—has, in effect, been validated.” (p. 22)

“Recommendation: NASA should (1) place as much responsibility as possible in the hands of the principal investigator, (2) define the mission rules clearly at the beginning, and (3) establish levels of responsibility and mission rules within NASA that are tailored to the particular mission and to its scope and complexity.” (p. 19)

Management, Execution, and Value for Science

“Finding: The PI-led mission paradigm represents a valuable approach to soliciting and executing missions involving focused science objectives, with demonstrated success in both Earth and space sciences….

Recommendation: NASA’s Earth Science Enterprise should continue to employ PI-led missions as one element of the ESE observation system. It should ensure regular review and improvement of the programs that employ or are associated with PI-led missions to increase their effectiveness and value to ESE and the science community.” (p. 2)

Steps to Facilitate Principal-Investigator-Led Earth Science Missions, 2003

“Finding: Many of the issues arising throughout a mission’s lifetime are rooted in decisions made by the PI and project team during the formulation phase—early in the project—as the mission concept is developed, team roles and responsibilities (including NASA’s) are defined, and the management approach is established. Ultimate mission success requires that major technical and programmatic issues be identified and jointly addressed by both the PI team and NASA program office during the formulation phase. While extending competitiveness between PI teams through the entire formulation phase provides NASA with additional insight into the effectiveness of the PI teams and the maturity of the mission designs, it delays the integration of the PI and NASA teams and motivates the PI teams to emphasize strengths and minimize weaknesses.

Recommendation: NASA’s Earth Science Enterprise should avoid extensive overlap between competition and execution activities during the formulation phase of PI-led missions, thus providing an adequate schedule for the PI team and NASA to perform critical formulation tasks after the competitive selection is completed.” (p. 4)

“Finding: Although some of the difficulties with recent PI-led missions are unique, many of the problems encountered have root causes in common with non-PI-led missions. In particular, the transition to smaller cost-constrained projects during the 1990s and the contraction and aging of the space industry workforce have affected project success. These problems should not be attributed to flaws in the PI-mode process, but rather applied as general lessons for all small-mission projects.

Recommendation: NASA’s Earth Science Enterprise should establish management processes for PI-led missions that emphasize understanding all PI-led and non-PI-led mission issues and the inclusion of appropriate lessons learned from both types of missions.” (p. 6)

Steps to Facilitate Principal-Investigator-Led Earth Science Missions, 2003

Management, Execution, and Value for Science,

“Recommendation: NASA’s Earth Science Enterprise should establish and enforce a comprehensive set of minimum standards for program management to be applied to all PI-led missions, while accepting that such missions may employ management processes that differ from those of NASA. These minimum management standards must invoke the rigor that experience has shown is required for success.” (p. 7)

“The Explorer program contributes vital elements that are not covered by the mainline … missions. Explorers fill critical science gaps in areas that are not addressed by strategic missions, they support the rapid implementation of attacks on very focused topics, and they provide for innovation and the use of new approaches that are difficult to incorporate into the long planning cycles needed to get a mission into the strategic mission queues…. The Explorers also provide a particularly substantial means to engage and train science and engineering students in the full life cycle of space research projects. Consequently, a robust … science program requires a robust Explorer program.” (p. 20)

Solar and Space Physics and Its Role in Space Exploration, 2004

“Given Discovery’s highly successful start, the SSE Survey endorses the continuation of this program, which relies on principal-investigator leadership and competition to obtain the greatest science return within a cost cap. A flight rate of no less than one launch every 18 months is recommended.

Particularly critical in this strategy is the initiation of New Frontiers, a line of medium-class, principal-investigator-led missions as proposed in the President’s fiscal year (FY) 2003 budget. The SSE Survey strongly endorses the New Frontiers initiative. These spacecraft should be competitively procured and should have flights every 2 or 3 years, with the total cost capped at approximately twice that on a Discovery mission.” (p. 2)

New Frontiers in the Solar System: An Integrated Exploration Strategy, 2003

“Faster-better-cheaper methods of management, technology infusion, and implementation have produced useful improvements regardless of absolute mission size or cost….

Recommendation 1: Transfer appropriate elements of the faster-better-cheaper management principles to the entire portfolio of space science and Earth science missions sized and cost ranges and tailor the management approach of each project to the size, complexity, scientific value, and cost of its mission.” (p. 3)

Assessment of Mission Size Trade-offs for NASA’s Earth and Space Science Missions, 2000

“Finding 5: The current operation and management styles of the SMEX program—including mutually beneficial cooperation between NASA and non-NASA participants, reduction of documentation, and flexibility in that class—are fostering opportunities for excellent, high-priority science.” (pp. 1-2)

Scientific Assessment of NASA’s SMEX-MIDEX Space Physics Mission Selections, 1997

“Recommendation 2: Adapt some of the management style and procedures associated with the SMEX program, as discussed in Finding No. 5 above, in other science programs. Recent spacecraft-Principal Investigator (PI) mode space physics Explorers (such as Solar and magnetospheric Particle Explorer [SAMPEX] and Fast Auroral Snapshot Explorer [FAST]) successfully demonstrate how high-priority science can be carried out in ‘faster, cheaper, better’ ways.” (p. 2)

“Finding 1. The panel supports use of the “PI mode” by NASA. It brings new vigor to the program at a time when diminishing opportunities could lead to disillusionment amongst the science community. It is an open process that appears to be intrinsically fair. It has exposed a reservoir of ideas for focused science under a cost cap.” (p. 2)

Assessment of Recent Changes in the Explorer Program, 1996

“Finding 3. The panel understands from presentations made to it that the ‘dual mode option’ will be eliminated from future Explorer AOs and that the ‘PI mode’ will be the only management approach allowed. The panel endorses this decision.” (p. 2)

Management, Execution, and Value for Science,

“Recommendation 2. The panel recommends that the Explorer and Discovery programs should continue with separate Headquarters management structures for the next few AOs.” (p. 2)

“Recommendation 3. Each mission must be selected through open competition from proposals presented as an integrated package by a principal investigator. This individual should have full authority to decide the appropriate balance among science performance, mission design, and acceptable risk….” (p. 27)

The Role of Small Missions in Planetary and Lunar Exploration, 1995

“Recommendation 4. NASA should not impose arbitrary constraints (e.g., preselection of launch vehicle, spacecraft bus, payload, data rate, target locale, or management structure) on mission design. Fewer restrictions will permit the most creative and cost-effective solutions for the broadest range of possible mission and target types.” (p. 27-28)

“Recommendation 6. Past NASA practices must change in order to foster the development of a streamlined approach to management of each complete mission….” (p. 28)

Budgetary, Technological, and Schedule Constraints Relating to Mission Selection and Execution

“Finding: The scientific and programmatic objectives of ESE are ambitious compared with the constraints under which PI-led missions are implemented, particularly the capped funding and tight schedule.

Recommendation: NASA’s Earth Science Enterprise should focus its programmatic objectives for PI-led missions to better match the available resources and constraints, with achievement of high-quality science measurements being the highest-priority objective.” (p. 2)

Steps to Facilitate Principal-Investigator-Led Earth Science Missions, 2003

“Finding: The rigorous and ambitious cost and schedule constraints imposed on PI-led missions preclude all but minimal technology development prior to launch.

Recommendation: NASA’s Earth Science Enterprise should explicitly nurture and coordinate technology feeder programs—such as the Instrument Incubator Program and the Office of Aerospace Technology’s Missions and Science Measurement Technology Program—that develop technologies with potential application to PI-led missions. A quantitative assessment of the anticipated flow of technology through the technology readiness level chain would help guide this effort.” (p. 3)

Steps to Facilitate Principal-Investigator-Led Earth Science Missions, 2003

“Recommendation: NASA’s Earth Science Enterprise should include within the solicitation for PI-led missions a component, following the Solar System Exploration Discovery model, that provides limited technology funding for high-priority non-selected PI-led mission proposals to increase their technology readiness for the next proposal round.” (p. 3)

“Finding: The threat of project cancellation has not proved effective either in motivating the submission of PI-led proposals with adequate reserves or in constraining costs to meet the cost cap.

Recommendation: NASA’s Earth Science Enterprise should redefine cost caps from a threshold that triggers an automatic termination review to a threshold for a remedial review that includes an examination of how the division of responsibility and authority between the PI and ESE might be revised to better control costs. Cost caps should be established only when the project has reached a sufficient level of maturity that the proposed cost is credible, such as at mission design review. ESE should also consider the use of a science floor, a PI-proposed minimum scientific achievement needed to justify the mission, in setting and managing within cost caps.” (p. 4)

“Finding: The lack of NASA-funded support for proposals, particularly during Step 2 [of the proposal process], is increasingly limiting the ability of smaller organizations and universities to participate.

Recommendation: NASA’s Earth Science Enterprise should maintain the current two-step proposal process for PI-led missions but should provide funding to proposers for Step 2.” (p. 5)

“Recommendation: NASA’s Earth Science Enterprise should clearly specify within the solicitation for a PI-led mission the extent to which scientific investigation and data analysis are expected to be included in the initial mission project budget, as well as the anticipated plans and budget for additional postlaunch science investigations. The science funded for the mission should address a PI-proposed science floor.” (p. 5)

“Recommendation: NASA’s Earth Science Enterprise should enhance its cost evaluation capabilities to improve the accuracy of mission selection decisions and to motivate improved fidelity of cost proposals.” (p. 6)

“Recommendation 2: Ensure that science objectives—and their relative importance in a given discipline—are the primary determinants of what missions are carried out and their sizes, and ensure that mission planning responds to (1) the link between science priorities and science payload, (2) timelines in meeting science objectives, and (3) risks associated with the mission.” (p. 4)

Assessment of Mission Size Trade-offs for NASA’s Earth and Space Science Missions, 2000

“Recommendation 4: Develop scientific instrumentation enabling a portfolio of mission sizes, ensuring that funding for such development efforts is augmented and appropriately balanced with space mission line budgets.” (p. 4)

“A NASA science mission, by its very nature, will incur cost penalties not applicable to DOD missions such as Clementine. These include:

• The cost of a science team. NASA established and funded Clementine’s science team to validate the data and plan for its archiving.

• Data analysis expenses. In contrast to NASA’s traditional policy, no data analysis was supported by the mission.

• The development of an optimized science payload. Clementine’s instruments were not optimized for scientific observations.

• The proper calibration of the science payload and the data it returns. Clementine’s data calibration is being paid for by NASA’s Office of Space Science.

• Provisions for making the data available through the Planetary Data System.” (pp. 13 and 16)

Lessons Learned from the Clementine Mission, 1997

“Finding 3: To succeed within their severe cost constraints, Explorer missions cannot afford instruments that require lengthy development or space qualification cycles. Therefore, the use of instruments and/or instrument subsystems that have been developed for previous missions is essential. The present funding cap on SMEX and MIDEX could well prove too restrictive for building scientifically first-rate missions without such instrument ‘heritage.’ Lessons learned from the space physics Explorers demonstrate the importance of instrument and spacecraft heritage in meeting science goals while remaining within cost and schedule limits.” (p. 1)

Scientific Assessment of NASA’s SMEX-MIDEX Space Physics Mission Selections, 1997

“General Finding. The panel believes that most of the perceived problems brought to light after the first MIDEX AO were due to the ‘dual mode option’ and the lack of full cost accounting for government contributions. In addition, debriefing of unsuccessful proposal teams was not adequate. While the AO and the selection process both need improvement and while interaction with the science community also needs to be strengthened, the panel believes that the program is now on the right path and that the new Explorer program should be excellent if properly administered. The perception will probably continue that GSFC and its scientists have an advantage, but the panel is satisfied that the Explorer program management is addressing this issue and that elimination of the recognized flaws will bring about a level playing field for both scientists and industry. Given time and continuing effort, it is the belief of the panel that the astronomy and space physics community will strongly support the program.” (p. 1)

Assessment of Recent Changes in the Explorer Program, 1996

“Finding 6. The panel believes that supporting a mix of Explorer mission sizes (MIDEX, SMEX, and UNEX) is an important and valuable feature of the program because it satisfies the needs of multiple constituencies. But this division should not be treated as immutable. Circumstances may change in the future and different cost caps may become preferable.” (p. 2)

Assessment of Recent Changes in the Explorer Program, 1996

“Finding 7. Based on the response to the 1995 MIDEX AO, the panel believes that the flight rate of Explorer missions could probably be substantially increased without any decrease in mission quality, if resources should become available.” (p. 2)

“Recommendation 3. To reduce excess industry investment in detailed costing exercises for large numbers of missions during Step One, these proposals should be submitted on a ‘cost-not-to-exceed’ basis within broad, AO-defined cost ranges. The responsibility for the cost-not-to-exceed estimate rests with the PI, advised by industrial and NASA center partners. The estimate should be accepted in Step One. A selected Step One effort that later failed to meet promised scientific objectives within the accepted cost limitation would be subject to termination and would be replaced.” (p. 3)

“Recommendation 5. The budget, schedule, and risk envelope must be identified in the conceptual and definition phase of mission planning…. It is essential for NASA to adhere to the agreed-upon funding profile….” (p. 28)

The Role of Small Missions in Planetary and Lunar Exploration, 1995

Review and Selection Process

“Finding: The number of qualified reviewers for ESE PI-led missions is small, particularly after elimination of scientists with conflicts of interest because of relationships with proposing teams.

Recommendation: NASA’s Earth Science Enterprise should consider enlarging the pool of possible reviewers of PI-led missions by adding qualified international scientists (if feasible given current International Traffic in Arms Regulations constraints) and scientists from the space science community. ESE should also consider requiring as part of the contract for selected PI-led projects that the PI serve subsequently as a reviewer.” (p. 5)

Steps to Facilitate Principal-Investigator-Led Earth Science Missions, 2003

“Finding: Maintaining and improving the credibility of checks and balances is the highest priority for enhancing the selection process for PI-led missions. An effective and credible proposal review process requires a balanced effort among proposers, reviewers, and the selection official. Proposers are motivated to avoid overly optimistic costing if they respect the cost-review process; reviewers are more diligent when their recommendations are likely to be accepted by the selection official; and the selection official relies more readily on reviewer recommendations when the proposal and review process is effective at identifying the best mission candidates.

Recommendation: NASA’s Earth Science Enterprise should strengthen the complementary roles of proposers, reviewers, and the selection official in the selection process for PI-led missions, improving the critical balance between the three roles and focusing on clear traceability of the selection process to independent reviews and established ESE priorities.” (p. 6)

“Finding 6. The extremely low selection rate (2/50) among the large number of proposed Explorer missions results in much effort spent fruitlessly in proposal preparation. This extra work puts a significant burden on the research community and their industrial partners.” (p. 2)

Scientific Assessment of NASA’s SMEX-MIDEX Space Physics Mission Selections, 1997

Impact on Scientific Community

“Finding: Universities can derive considerable benefit by participating in an ESE mission; however, using PI-led missions to build the capacity of university-based research is not readily achievable within the structure and resources of current ESE PI-led programs.

Steps to Facilitate Principal-Investigator-Led Earth Science Missions, 2003

Impact on Scientific Community,

“Finding 2. The panel believes that the new Explorer program cannot succeed without a high level of support by the science community. In the first (1995) MIDEX solicitation the most readily avoidable errors were failure to consult adequately with the community in the development of the AO and failure to undertake face-to-face debriefings with investigators after the process. These management errors have led to problems with the science community, but they can be resolved through a thoughtful effort in developing future AOs, and again after selection has taken place.” (p. 2)

“Finding 5. The panel believes that the restructured Explorer program can be of outstanding value not only for the science performed, but also for its role in maintaining U.S. scientific capabilities in an important area of space science. This double role for the Explorer program has repercussions with respect to mission sized, foreign participation, and flight rate.” (p. 2)

International Collaboration

“Recommendation: NASA’s Earth Science Enterprise should recognize not only the benefits but also the risks of having domestic and international partners in a PI-led mission program. The mission solicitation should identify the need for processes by which both the PI team and the NASA office ensure that partnering agreements are completed early in the formulation phase, that definition of an interface is given high priority, and that the management decision chain is clear and is understood by all parties.” (p. 4)

Steps to Facilitate Principal-Investigator-Led Earth Science Missions, 2003

“The SSE survey recommends that NASA encourage and continue to pursue cooperative programs with other nations.” (p. 2)

New Frontiers in the Solar System: An Integrated Exploration Strategy, 2003

“Recommendation 6: Encourage international collaboration in all sizes and classes of missions, so that international missions will be able to fill key niches in NASA’s space and Earth science programs. Specifically, restore separate, peer-reviewed announcements of opportunity for enhancements to foreign-led space research missions.” (p. 5)

Assessment of Mission Size Trade-offs for NASA’s Earth and Space Science Missions, 2000

“Recommendation 5. The panel recommends that, at least for the next AO, foreign contributions be included in the Explorer cost cap as was done for the 1995 MIDEX AO. After more experience has been gained with foreign contributors and contributions, NASA and the science community should reassess this issue in workshops to be convened for the consideration of future AOs.” (p. 3)

Assessment of Recent Changes in the Explorer Program, 1996