One of the most significant challenges facing each of the four Earth and space sciences is what balance to strike between the mission size required to achieve science goals and what happens when a high-profile or strategic mission runs over its projected budget and jeopardizes the entire portfolio. Such missions, be they over-budget or simply expensive in relative terms, are colloquially referred to as “flagship” missions. The session on how to plan for high-profile missions, however, gives a more nuanced perspective on what this means, because being a high-profile or strategic mission is not necessarily synonymous with cost.
John Klineberg, Vice Chair, Space Studies Board; President, Space Systems/Loral (retired)
Rick Howard, Program Director, James Webb Space Telescope, NASA (retired)
Ralph McNutt, Senior Space Physicist, Johns Hopkins University Applied Physics Laboratory; Member, 2011 Planetary Science Decadal Survey
Charles Elachi, Director, Jet Propulsion Laboratory
William Gail, Chief Technology Officer, Global Weather Corporation; Member, 2007 Earth Science and Applications from Space Decadal Survey
The panel moderator, John Klineberg, opened the discussion by providing his own definition of a “high-profile” mission. Among the many characteristics of such a mission, Klineberg said, a high-profile program is, in effect, going “all in” as one would in a poker game. He also suggested that NASA reconsider its use of the term “flagship,” either by limiting it to science missions or dropping the term entirely from its lexicon. Rather, he suggested, such missions should be referred to as “strategic” or “all-in” missions.
An all-in mission is characterized by having a long development time that spans administrations and policy changes; incorporates advanced concepts for instruments, spacecraft, and flight control systems; is constructed at more than one NASA center and/or involves international collaboration; could have a large budget that can affect the entire Science Mission Directorate (SMD) or agency as a whole; may be periodically threatened with cancellation; or can be seen as “betting the agency” on the mission, leading to a high degree of visibility and attention by groups external to NASA. Some examples include the Hubble Space Telescope and its first servicing mission, the Mars Science Laboratory, and the James Webb Space Telescope (JWST).
To mitigate such challenges, Klineberg suggests extensive trade-space exploration during pre-Phase A, when the mission is first being designed; selection of a design reference point with large performance margins; and establishment of a maximum threshold of scientific performance expectations to prevent over-scoping and cost growth.
Rick Howard began the panel discussion, noting that JWST is arguably the most ambitious space-based astronomical observatory ever built and that the $8.9 billion mission has earned the flagship moniker. Howard brought his decades of experience in program management, particularly with JWST, to bear on the discussion. According to Howard, the all-in or strategic mission is an important piece of a NASA science division’s overall portfolio. Nevertheless, it is nearly impossible to accurately predict the true cost of such a cutting-edge mission, even in Phase B. Howard emphasized that large, strategic missions need to be treated differently by all stakeholders. For instance, Aerospace Corporation’s cost and technical evaluation (CATE) process is a good approach, but it will always result in very large estimates for flagship missions. He said that creating different “bins” of cost and scope options is a better approach than a single point cost. Similarly, clear decision rules will also help agency decision makers determine the appropriate course of action in designing the mission and what to do if cost growth exceeds tolerances. NASA, he said, will need to maintain good communication with its stakeholders, as well as pay close attention to independent review boards and ensure that funding for large missions is phased appropriately. Once the mission reaches Phase B, Howard recommends vigilant oversight from NASA Headquarters. Howard also said that asking program managers to “do the best you can with the funding you have” leads to delays and cost overruns and is an inefficient management strategy. Finally, a crucial element to the success of these large, strategic missions is adequate and sustained funding from the administration(s) and Congress.
Ralph McNutt commented that the main problem with these large, strategic missions is their cost and technical complexity, which almost always leads to inevitable technical challenges that arise in the development and construction phase. However, he said, these types of spacecraft are also very resilient and once they are flying can last for a very long time (e.g., Hubble Space Telescope). Nevertheless, he said, getting such missions from concept to formal development requires getting all stakeholders involved and coordinating with one another from the very beginning. Trust and respect are crucial elements of stakeholder relationships; and the Space Studies Board needs to play a stakeholder role, as well.
Charles Elachi said that flagship missions are very valuable to a discipline, but it is important to have both flagship and competed missions. Flagships, he said, are an essential part of a successful exploration portfolio, and, indeed, many of the technologies developed for flagship missions are now being used on smaller missions. Another benefit of flagship missions is their high-profile nature, which can draw public interest to the mission and, hopefully, to the rest of NASA’s activities. For instance, he said, 50 million people watched the Mars Science Laboratory (MSL) landing live. When people are engaged, he said, the missions remain supported. As for their management, Elachi—in contrast to previous remarks from McNutt—said that NASA Headquarters’ authority is limited, and ultimate responsibility rests on where the work is actually being done. With Cassini, there was no ambiguity on who was responsible for what; and with MSL, there was no ambiguity on who was at fault for delays and cost growth in that mission. Elachi also touched on another important aspect of spending for flagship missions: the money appropriated for flagship missions can be so large as to be almost an entire account on its own, and as a spacecraft nears completion, many different groups will want that money to be shifted to their work. However, those funds belong to the parent directorate of the division, as will be the case with JWST and the Astrophysics Division in the next few years.
William Gail commented that Earth science is very different from the other space sciences. Moreover, his discipline’s decadal survey was also different from subsequent surveys for numerous reasons. Earth science relies on systems science and interdisciplinary collaboration to a far greater extent than the other space sciences. Furthermore, Earth science has to tackle societal benefits like weather forecasting, environmental security and public safety, agricultural production, and even public policy. Regarding a high-profile or large-scale mission for the Earth sciences, such a spacecraft would necessarily have a high policy impact because of the nature of its observations. Another challenge for the Earth science and applications community is managing operational expectations for spacecraft and also ensuring that high-priority measurements can be continuously gathered with minimal to no gaps in data
collection. If certain measurements are dependent on large-scale, all-in missions—for example, the Joint Polar Satellite System—a cancellation could be very detrimental to both science and national observation capabilities. The challenges of managing large missions—or missions of any size—can be exacerbated by interagency collaboration. Gail concluded by saying that overall program integrity is what matters most.
Workshop participants made comments and posed questions to the panelists, as described below. Topics discussed included the following:
• Addressing high-profile missions specifically in decadal surveys,
• International high-profile missions,
• High-profile mission timescales, and
• Stewardship of high-profile missions.
An audience member began the question and answer session by stating that NASA should tell the decadal survey committees how to factor flagship missions into the decadal planning process, stating that there has been little to no guidance on the matter from the agency. For instance, he explained, no one understood the risk of potential cost growth of JWST, which has come at great cost to both NASA’s Astrophysics Division and other divisions in NASA SMD. He asked, Should the decadal surveys explicitly identify the levels of risk tolerance for recommended missions? and What will happen if a compelling mission profile emerges during the program formulation phase, but the subsequent CATE process reveals that the mission will exhaust its budget at the 30-percent confidence level? The panelists responded by saying that this questions further underlie why decision rules need to be included in the surveys for a wide range of contingencies. In addition, Ralph McNutt noted that stewardship can play an important role, and the Space Studies Board should be one of the stakeholders that is regularly updated on the progress of large missions. William Gail suggested a cleaner break in the decadal survey process between identifying science priorities [first] and then drilling down to select a few missions for more in depth study.
Another audience member who worked on the 2007 Earth science and applications from space decadal survey said that that survey did not include a flagship-type mission, but rather the recommended science program is itself the “flagship” comprised of many different modules that are the missions. Nevertheless, he asked whether the next Earth science decadal survey should consider recommending a more traditional flagship-type mission. William Gail replied that part of the issue here is anticipating what a mission might become, how it will be treated, and the affect that mission will have on the rest of the program. Charles Elachi said that while that modular approach may work in Earth science, it does not work for astronomy, further highlighting the differences between the disciplines. McNutt echoed Elachi’s conclusion and noted that he did not think a modular approach would work for planetary missions either.
An audience member said that while the 2010 astronomy and astrophysics decadal survey was underway, the survey committee was reluctant to tinker with preexisting international missions involving NASA. She then asked what an international partnership for a flagship mission should look like. Charles Elachi said that an international arrangement on such a large mission could be a blessing or a curse. He suggested that the collaboration should be simple enough that if one partner drops out, the first can stay
in, and the mission is not jeopardized. Elachi explained that IXO, or the International X-ray Observatory that would have been a joint U.S.-European venture, was not set up properly because it was too complex. Nevertheless, Rick Howard said, the collaboration should still be able to work even if the international partners are “attached at the hip.”
One audience member said that with respect to flagships in particular, it would be useful for the decadal survey to define the timescale over which the balance between flagships and smaller missions should be maintained. He illustrated his point by explaining that there is a big difference in calling for mission-size balance over a 5-year period versus a 15-year period. The respective differences have major implications for mission-portfolio and funding planning. In response to this, Charles Elachi commented that for large missions, the program should not be constantly adjusted; it is important that the investment in the mission be stable over the necessary period of development and construction time. The questioner then said that it would be useful for the decadal surveys to recognize that large missions could potentially disrupt the balance of mission sizes for an extended period of time. On the other hand, explained Ralph McNutt, the space science community and NASA need to maintain core competencies within the workforce on how to carry out large-scale missions.
Speaking about the different roles between the National Research Council (NRC) and the NASA internal advisory structure, an audience member commented that the NRC is ill suited to keep an eye on the development stages of programs and missions. Instead, he believes the advisory apparatus at NASA needs to be repaired and revitalized and the SSB should be responsible for providing broad, strategic advice. Ultimately, he said, accountability needs to be restored for these large-scale missions.
One audience member suggested including a chapter in future decadal surveys describing the skills and capabilities necessary for managing missions across a range of sizes that would only exist within NASA. Ralph McNutt replied by saying that while in principle it is a good idea, such a chapter might in fact be counterproductive, and the surveys should focus on addressing infrastructure issues instead.
At the conclusion of the panel session, an audience member noted that flagship missions can be both failures or successes and asked the panelists what are the lessons learned from past flagships. The panelists noted the following: good and frequent communication on these large missions across the agency is essential; the community needs to veer away from overly optimistic cost estimates; flagship missions are crucial to the space science community and require a significant amount of investment in technology development; and, the community should not lose sight of conducting science based on the priorities set forth by the community, as opposed to building missions that do not necessarily reflect this question-based science effort.