Space Studies Board Annual Report 2017 (2018)

5.5 Powering Science: NASA’s Large Strategic Science Missions

A Report of the SSB ad hoc Committee on Large Strategic NASA Science Missions: Science Value and Role in a Balanced Portfolio

Summary

NASA’s Science Mission Directorate (SMD) operates dozens of spacecraft performing many different missions. The most high profile of these are the large strategic space science missions often referred to as “flagship” missions. These include missions such as the Hubble Space Telescope (HST) and the Chandra X-Ray Observatory, Curiosity rover, Magnetospheric Multiscale (MMS), and Terra Earth observation satellite. These missions typically are billion-dollar-class missions, the most costly, the most complex, but also the most capable of the fleet of scientific spacecraft developed by NASA. They produce tremendous science returns and are a foundation of the global reputation of NASA and the U.S. space program. Large strategic missions are essential to maintaining the global leadership of the United States in space exploration and in science because only the United States has the budget, technology, and trained personnel in multiple scientific fields to conduct missions that attract a range of international partners. A large strategic mission can be a single spacecraft, or a coordinated constellation, designed to achieve a set of science goals. All large missions are by definition strategic, but not all strategic missions are large. Large strategic missions are critical for the conduct of space science in each of NASA’s four divisions (astrophysics, Earth science, heliophysics, and planetary science) and are required for the pursuit of the most compelling scientific questions.

Large strategic missions are directed by NASA to a specific institution to develop, although their instruments and subsystems are often competed. Large strategic missions tend to

• Focus on reconnaissance and on conducting a broad suite of objectives;
• Have longer lifetimes and sustained attention to details regarding consistency of operations and calibration;
• Operate with an evolving science program that responds to what has been learned as the mission proceeds, as opposed to a more fixed science program;
• Travel to hard-to-reach destinations or challenging environments; and
• Carry a large number of larger and heavier scientific instruments.

In contrast, smaller missions generally

• Focus on a single objective or on a small number of tightly related objectives,
• Travel to easier-to-reach destinations or to more benign environments, and
• Carry fewer and smaller instruments.

These characteristics for large strategic and medium and small missions are not exclusive. For example, medium-size missions such as NASA’s New Horizons spacecraft have traveled to hard-to-reach locations such as the edge of the solar system. In addition, some medium and small spacecraft have had long lifetimes. But capabilities and lifetimes generally scale with the size and cost of a mission.

In 2016 NASA asked the National Academies of Sciences, Engineering, and Medicine to examine the role of large strategic missions within a balanced program across NASA SMD space and Earth sciences programs. (The statement of task is included as Appendix A.)

The Committee on Large Strategic NASA Science Missions: Science Value and Role in a Balanced Portfolio met three times, starting in fall 2016 and concluding in February 2017, and heard from NASA officials, the chairs of several previous Academies studies including the decadal surveys, and the managers of both large and smaller NASA space science missions. The committee also requested data from NASA about the cost and productivity of

___________________

NOTE: “Summary” reprinted from Powering Science: NASA’s Large Strategic Science Missions, The National Academies Press, Washington, D.C., 2017, pp. 1-8.

large and smaller missions across all four NASA SMD divisions. Based on the presentations made to the committee by a wide range of representatives from NASA, the scientific community, congressional staff, and others, the committee reached a number of findings and recommendations that are included in this report.

The committee determined that large strategic missions have multiple benefits. These benefits include the following:

• Capture science data that cannot be obtained in any other way, owing usually to the physics of the data capture driving the scale and complexity of the mission;
• Answer many of the most compelling scientific questions facing the scientific fields supported by NASA’s SMD, and most importantly develop and deepen humanity’s understanding of Earth, our solar system, and the universe;
• Open new windows of scientific inquiry, expanding the discovery space of humanity’s exploration of our own planet and the universe and providing new technology and engineering approaches that can benefit future small, medium-size, and large missions;
• Provide high-quality (precise and with stable absolute calibration) observations sustained over an extended period of time;
• Support the workforce, the industrial base, and technology development;
• Maintain U.S. leadership in space;
• Maintain U.S. scientific leadership;
• Produce scientific results and discoveries that capture the public’s imagination and encourage students to pursue science and technical careers;
• Receive a high degree of external visibility, often symbolically representing NASA’s science program as a whole; and
• Provide greater opportunities for international participation, cooperation, and collaboration as well as opportunities for deeper interdisciplinary investigations across NASA science areas.

NASA’s current SMD portfolio includes several large strategic missions: Hubble, Chandra, MMS, Terra, Aqua, Aura, Cassini, and Curiosity. It also includes dozens of small and medium-size missions.¹ These small and medium-size missions are usually selected via a competitive process as opposed to being called out as priorities by the decadal surveys in each of the space science disciplines.² The large strategic missions are directed to specific institutions for development, with major elements and systems competed via request for proposals and the announcement of opportunities as appropriate. Small and medium-size missions can accomplish some of the goals of large strategic missions, such as continuing data collection for a single or small set of instruments. They also have unique benefits of their own, including

• Higher cadence,
• Greater agility and responsiveness to new scientific discoveries, and
• Different acceptance of risk.³

Smaller missions can often be developed in half a decade (or even less time in the case of the smallest missions), compared to large strategic missions that typically take a decade or more to develop.⁴ This makes smaller

___________________

¹ For this study the committee looked at missions currently in operation or development by the Science Mission Directorate (SMD). The committee did some historic budget analyses of large strategic missions back to the late 1960s that are reflected in Figure 1.1.

² This committee is addressing the space programs conducted by the four SMD divisions: Astrophysics Science Division, Earth Science Division, Heliophysics Science Division, and Planetary Science Division. The committee’s reference to “science disciplines” is based on the general assumption that each of these divisions represents a science discipline. There are other science disciplines, and some overlap, but the categories are relatively distinct.

³ NASA has different risk acceptance levels for payloads. Class A missions are considered high priority, very low (minimized) risk; Class B missions are considered high priority, low risk; Class C missions are considered medium priority, medium risk; and Class D missions are considered low priority, high risk. Risk level calculations include such factors as redundancy for critical functions such as communications and propulsion. Large strategic missions are always Class A missions. Smaller missions are usually Class B, C, or D.

⁴ For example, the initial contract for development of the James Webb Space Telescope was awarded in 2002, and launch is currently scheduled for 2018.

missions better suited for responding to recent discoveries in some cases. The open competition aspect of smaller missions also encourages ingenuity. Large strategic missions have greater agility and flexibility than small missions to respond to discoveries during the mission (e.g., Enceladus’s plume and Titan sea bathymetry for Cassini), while smaller missions have a faster response in terms of development.

The roles of large and small missions are different within each division, however, and are best determined by the decadal survey process. For example, within astrophysics, the “Great Observatories” have offered opportunities for widespread community participation as “guest observers,” especially for missions like Hubble and Chandra that have had long lifetimes, although even smaller missions like Fermi have supported guest observers. Earth science has also been able to use its large strategic missions for broad community engagement. For planetary science, large strategic missions like Voyager, Galileo, Cassini, and Curiosity have supported multiple segments of the larger planetary science community, including atmosphere and magnetosphere studies as well as geochemistry and geophysics. Heliophysics has conducted many strategic missions, including Solar Dynamics Observatory, Van Allen Probes, MMS, Solar Terrestrial Relations Observatory, and Voyager (which became a heliophysics mission after completing its planetary encounters). Not all of these have been “large” strategic missions. The longevity of some missions has helped support the heliophysics community.⁵ Large strategic missions have also contributed to the development of large data archives that are used by many researchers. The HST data archive, as one example, has increasingly been used by astrophysicists, who are able to mine nearly three decades of telescope observations.

Large strategic science missions support scientific investigations by teams of scientists and graduate students that in turn support large fractions, in toto, of the research community. As a result, these missions sustain the development and the health of their respective scientific communities in ways that smaller missions cannot. They can also be vital for training new generations of scientists, instrumentalists, and engineers, not only during initial mission development but even when the missions have been operating for many years.

Large strategic missions are highly scientifically productive. Numbers of smaller missions can be scientifically productive as well, and they provide access to a diverse set of scientific questions that cannot all be accomplished with large missions. Smaller missions are often necessary to provide new insights, respond to recent discoveries, and refine the scientific goals of less frequent large strategic missions.

During the course of this study the committee gathered data that indicate that technology development occurs at many levels: large strategic missions, medium-size missions, and even small missions, as well as separate technology development programs. Large missions can have substantial budgets devoted to maturing technologies. In contrast, smaller missions can have faster turnaround times, introducing and maturing technologies at a faster pace. CubeSats are an example of a rapid technology incubator that helps to infuse new technology into the programs, although the benefits and limitations of CubeSats are still being learned. It is also possible for small missions to benefit from technology developed for large missions.

In terms of workforce development large missions are more advantageous than smaller missions primarily because they have the budgets, the scientific breadth, and the longevity to support more researchers. Many small missions support only small teams of researchers and often cannot provide full support even for their principal investigators (PIs) and co-investigators. In addition, many small missions have relatively short lifetimes compared to larger missions, and they may not operate long enough to support researchers, particularly those early in their careers, before the mission expires.

Large strategic missions are critical for balance and form the backbone of the disciplines encompassed by the respective NASA science divisions. However, each discipline values these missions differently, and their valuation evolves as both the science and technology evolve.

RECOMMENDATION: NASA should continue to plan for large strategic missions as a primary component for all science disciplines as part of a balanced program that also includes smaller missions. (See Chapter 1.)

This committee was tasked with addressing “general principles that SMD could use (e.g., a figure-of-merit approach) to trade off within a limited budget between development and operation of large strategic missions and the cadence and/or cost caps of medium-size and small PI-led mission lines.” After much deliberation, the

___________________

⁵ Magnetospheric Multiscale (MMS) was originally proposed in the 2003 heliophysics decadal survey as a “moderate”-size mission. It later increased in cost and is generally considered to be a “large” mission.

committee concluded that there is no single figure-of-merit approach that could be developed to apply to all four scientific disciplines. The committee also considered whether it was possible to develop different figures-of-merit for different disciplines. Although that might be possible, it would require substantial expertise in each of the disciplines, which a cross-disciplinary committee cannot possess but discipline-focused committees could possibly provide.

The committee also determined that it was not appropriate for this committee to seek to supersede the guidance that is already provided to NASA by the decadal surveys. Balance can be decided only by the decadal surveys themselves. Their definition of balance is likely to change over time, and therefore has to be revisited over subsequent decadal surveys and assessed during the relevant decadal midterm reviews. Furthermore, the definition of balance provided by the decadal surveys is likely the only one that will satisfy a diverse community.

However, the committee concluded that there are many general principles that can be applied to all of the NASA science mission divisions, and this report makes a number of recommendations about them. In particular, the committee was reminded of both the importance and the strength of the decadal survey process for each division, and sought to emphasize that fact and to further buttress the decadal survey process. The committee’s recommendations encompass providing better inputs into the decadal surveys and noting that when a decadal survey is insufficient, NASA has other advisory paths to seek specific input—for instance, for reprioritization or redirection of a program during a midterm review. These advisory methods already have credibility, and NASA benefits by relying on them, particularly in unique circumstances where more general guidance may be insufficient. The committee also reaffirmed the value of the better costing mechanisms that NASA has adopted, and has recommendations concerning them as well.

The committee notes that if SMD seeks “general principles” to trade off within a limited budget between development and operation of large strategic missions and its medium-size and smaller mission lines, any such principles cannot be too general or they will not be very useful. In addition, those principles will be most helpful if they are timely and targeted to the areas most in need of help. NASA’s advisory structure at the National Academies was recently revised to enable discipline committees for each of the space science disciplines to respond to the needs of NASA’s science divisions in a more timely manner. This and other advisory structure changes could greatly assist NASA in making those required decisions.

RECOMMENDATION: When faced with the requirement to trade off between development and operation of large strategic missions and the smaller missions within their portfolios, NASA’s Science Mission Directorate (SMD) divisions should look first to their relevant decadal surveys and their midterm reviews for guidance. If these are insufficient, the SMD divisions should seek the advice of their relevant advisory groups. (See Chapter 2.)

Balance across the entire NASA science program includes an appropriate mix of small, medium-size, and large missions. The detailed meaning of “balance” for the upcoming decade is defined appropriately by each of the decadal surveys based on the required needs of that discipline for the pursuit of the most compelling science identified by the scientific communities by means of the surveys. For example, the most recent planetary science decadal survey defined a balanced program consisting of one or two large strategic missions, two medium-size New Frontiers missions, and at least three smaller Discovery-class missions during the coming decade. Decadal surveys can establish a broader balance that includes scientific capabilities typically provided by other agencies such as the National Science Foundation, the National Oceanic and Atmospheric Administration, and the U.S. Geological Survey. The decadal surveys are closer to their subjects and their communities than this study, and the repeating nature of the decadal surveys (as well as their midterm reviews) enables their recommendations to evolve as their discipline evolves. The committee determined that the decadal surveys can best address how large strategic missions will continue to fit into their programs in the future. But the committee determined as well that the surveys also need help to enable them to better evaluate such missions—for example, mission studies prior to the start of a decadal survey.

RECOMMENDATION: In preparation for the decadal surveys, large strategic mission proposal teams should consider describing ranges of scientific scope for their recommended large strategic missions, such as minimum science goals and maximum budgets, as well as identifying what science goals are most desirable at different budget levels. This approach may allow the scientific community and NASA

to develop less expensive implementation strategies for mission concepts that do not exceed current budget limitations. (See Chapter 2.)

Science is the primary focus of the decadal surveys, and all of the recent decadal surveys have described in detail the highest priority science questions and frontiers. Although the science questions and opportunities change over time, the technology to address that science changes over time as well.

RECOMMENDATION: Budget constraints should be included in the development of a decadal scientific program. Flexibility in the “decision rules” that decadal surveys produce should allow for both the de-scoping of large strategic missions in the face of cost overruns or insurmountable technical barriers as well as the “up-scoping” of missions as new technological or other opportunities arise. (See Chapter 2.)

RECOMMENDATION: The decadal surveys should formulate mission concept variants or other means to assess the boundaries of cost and technical risk and recommend the application of decision rules to provide flexibility to the NASA science divisions and most importantly the scientific community. This will enable further refinement of mission concepts when pursuing the scientific priorities identified by the decadal surveys. (See Chapter 2.)

RECOMMENDATION: Decadal surveys should be informed by, but not narrowly restricted to, future projections of available budgets. Such flexibility may enable new and potentially revolutionary large strategic missions. (See Chapter 2.)

This flexibility may also help avoid inadvertent programmatic cul-de-sacs if future projections are overly optimistic. Past cost overruns of large strategic missions have had substantial impacts on the other programs in NASA’s science portfolio—notably, the James Webb Space Telescope (Figure S.1) and the Curiosity rover.

FINDING: Cost control of large strategic missions remains vital in order to preserve overall programmatic balance. (See Chapter 3.)

It is common for the public, the press, and the scientific community to cite low-fidelity cost estimates made early in the proposal process, thus creating a false impression that a mission’s costs have grown substantially when the reality is that the early “estimates” were not actual estimates, or were made by project advocates rather than an independent authority. It can be difficult for the agency and a program to overcome this false impression. In the last decade NASA has introduced numerous cost control and cost evaluation mechanisms. As discussed in the body of this report, these mechanisms have been effective at limiting unexpected overruns and impacts on programmatic balance.

RECOMMENDATION: NASA should ensure that robust mission studies that allow for trade-offs (including science, risk, cost, performance, and schedule) on potential large strategic missions are conducted prior to the start of a decadal survey. These trade-offs should inform, but not limit, what the decadal surveys can address. (See Chapter 3.)

Although conducting such studies sufficiently in advance of a decadal survey may not always be feasible, the regular cadence of decadal surveys makes planning for them easier. Implementing them will be less troublesome if the decadals have more useful and earlier inputs.

NASA is often conceptualizing new approaches to increasingly ambitious science missions, many of which are first of a kind, or taking advantage of new architectures or technologies. NASA is expected to push the state of the scientific and technological art and adopt new approaches in order to maximize science return. Cost models, whether empirical parametric tools or analogy-based approaches, are dependent on historical data for systems that have previously flown. New technologies and ways of operating can extend beyond the boundaries of the existing cost databases on which the tools and estimates are based, making it important to constantly revise and develop methods of estimating costs. NASA establishes a project’s baseline budget at what the agency designates “Key Decision Point-C” (KDP-C). The formal decision to proceed with a project is made at KDP-C, and the independent estimates made at this point are the only valid ones.

RECOMMENDATION: NASA should continue to use its various cost estimation and cost management tools to assess and control the costs and risks of large strategic missions to ensure that they remain a viable option. As new technologies and new missions arise, new cost estimation tools will be required to enable NASA to determine their likely costs. NASA should support the development of new tools to perform robust cost estimates and risk assessment. These new cost estimation tools will also be helpful in support of the National Academies’ decadal surveys. (See Chapter 3.)

New technologies, like CubeSats—particularly in large constellations—will require new methods of cost estimation, some of which are already being developed by industry. Although NASA has gotten better at estimating costs, the agency will have to adapt its methods as technology evolves. This is something that NASA already does—witness the evolution in cost estimation adopted within the past 10 years such as changing the confidence level for estimating new projects—but the committee’s point is that as technology advances, cost estimation tools will, and will have to, advance as well.

While cost estimation and control are vital, the committee cautions that cost is best appreciated with respect to performance; many Earth and space science missions have operated long beyond their prime missions, providing tremendous value at relatively low operating costs. The agency deserves credit for enabling their long-term productivity.

This study was commissioned in part to examine and discuss the role and scientific productivity of different-size missions. The committee found this task particularly difficult to perform because of the different roles that missions play in different divisions, and the limitations of available data. However, the committee concluded that it is possible for NASA to make this case itself, by publicly presenting the voluminous data that the agency already collects about the missions it operates, particularly as part of its senior review process for extending missions. The committee notes that data collection on missions has improved substantially, particularly since the implementation

of full cost accounting at NASA in the early 2000s. Therefore, much better cost data exist for more recent missions than for earlier missions that may still be operating. This will make it possible for NASA to better present such data in the future.

RECOMMENDATION: In order to demonstrate the role and scientific productivity of large strategic missions in advancing science, technology, and the long-term health of the field, NASA’s Science Mission Directorate (SMD) should develop a publicly accessible database, updated at least annually, that tracks basic data related to all confirmed missions in development as well as operational and past missions from each of the SMD divisions. These data should include development costs; publication numbers and other bibliographic data; outreach data (number of press releases and so on should be tracked); science, engineering, and other full-time equivalents; and other routine data typically sought in senior review proposal submittals once prime missions have been completed. These data should be of sufficient detail and quality to enable basic analyses related to scientific productivity and contributions to the health of the respective fields. (See Chapter 4.)

Although it is difficult to collect and interpret historical data for many of NASA’s missions, including such information in the database, with appropriate caveats and annotations, could be valuable for providing perspective on NASA’s current and future missions. This could include, for example, the cost of servicing the Hubble Space Telescope, because including past data on servicing could be valuable for informing future missions that may include servicing as well. Although the committee acknowledges that establishing such a public database will require effort by NASA, the committee concluded that this would be useful to the agency in communicating the value and output of its missions as a whole, as opposed to via periodic press releases or at scientific conferences. The agency is in a better position today than it was only a decade ago to accurately report on its mission costs and performance across the entire SMD.

The first task in the committee’s charge concerning guiding future prioritization of large strategic space and Earth science missions within a balanced program is addressed in Chapters 1 through 3. The second task, assessing the impact of current and recent SMD missions with a range of life-cycle costs, is addressed in Chapter 4, and Appendixes B through E contain information primarily supplied by NASA in response to the committee’s request in an effort to address this task. Chapter 1 of this report addresses the current and recent history of large strategic missions in NASA’s SMD. Chapter 2 discusses the role that large strategic missions play in achieving balance in each of the four SMD divisions. Chapter 3 discusses the issues of cost estimation and control for space science missions, how the implementation of new procedures and methods has improved these over the past decade, and how cost control is vital to achieving programmatic balance. Chapter 4 addresses issues of comparing large strategic and smaller NASA space science missions in terms of technology development and transfer, workforce training, and scientific productivity.

During its deliberations the committee concluded that whereas some factors such as the cost of a mission are relatively easy to assess (assuming that the data have been collected and are consistent), other factors such as health of a scientific community and scientific productivity of a mission are essentially qualitative, not quantitative assessments. Science that is multidisciplinary, or multiplatform, complicates these measurements even further. In addition, missions with long operational lifetimes create a new dynamic of their own. Consider, for example, the HST. Hubble’s archive of collected data is now so large that it is being used by increasing numbers of scientists to produce new scientific discoveries. Quantifying the scientific return of a wide range of missions with different characteristics and goals and lifetimes is not an easy task and possibly not an achievable one. Nevertheless, the committee concluded that by supporting the creation of the proper tools, NASA can do a better job of communicating the value of large strategic space science missions.

Overall, the committee was impressed with the agency’s extensive portfolio of science missions of all sizes. But it is the large strategic missions that have demonstrated some of the greatest science advances and the capabilities of the United States as a leader in scientific discovery and the exploration of space. Hubble’s Deep Field observation and refinement of the Hubble Constant, Cassini’s observations of Saturn’s rings and Enceladus’s plumes, Voyager’s multiplanet tour and travel through the heliopause, and Curiosity’s explorations of Mars’s past habitability rank among the greatest scientific accomplishments of the past several decades and have become synonymous with American accomplishment.