National Academies Press: OpenBook

Powering Science: NASA's Large Strategic Science Missions (2017)

Chapter: 4 Comparing Large Strategic Missions and Smaller Missions

« Previous: 3 Risks and Realities of Cost Overruns for Large Strategic Missions
Suggested Citation:"4 Comparing Large Strategic Missions and Smaller Missions." National Academies of Sciences, Engineering, and Medicine. 2017. Powering Science: NASA's Large Strategic Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/24857.
×

4

Comparing Large Strategic Missions and Smaller Missions

The statement of task charged the committee to:

Assess the impact of current and recent SMD missions with a range of life-cycle costs. A representative subset of missions within each of SMD’s four science theme areas may be selected for analysis. The committee’s analysis of each representative mission will include a discussion of the relation between mission scientific impact and mission life cycle cost (or cost to date) in order to understand the return on expenditures for various mission classes. In describing the impact of the chosen missions the committee should consider dimensions such as:

  • Scientific productivity,
  • Impact on the current and future health of the relevant scientific community, and
  • Contribution to development and demonstration of technology applicable to future missions.

In 2012 the Jet Propulsion Laboratory (JPL) led a study to try to adequately assess the impact of large strategic missions compared to competed missions for planetary sciences. The study team concluded that there are many rules that need to be followed in order to make an assessment. First, the approach needs to be quantitative and applied in a consistent manner throughout both classes of missions. Second, the information needs to be clearly identifiable and available to the public. Third, the study should be conducted by a group of people with viewpoints that are diverse and across the spectrum of competed and strategic planetary missions. The approach to these evaluations should include (1) the scientific impact, (2) the impact on the number of scientists directly supported, and (3) the impact on the number of mission instruments.

The scientific impact refers to the utilization of existing data, which may include publications, citations, and h-index information taken from the Web of Science on a mission-by-mission basis. The h-index is defined as the number of papers cited at least h times each for a given mission (rather than for a given scientist).1 The JPL study concluded that the citation peak for most publications is reached at the 5- to 6-year mark.

According to the JPL study’s analysis of the h-index and other publication statistics, both large and smaller planetary missions have a significant scientific impact. If larger or smaller missions were eliminated in the future, it would have a negative effect on the program. For instance, eliminating large strategic missions would make

___________________

1 J.E. Hirsch, 2005, An index to quantify an individual’s scientific research output, Proceedings of the National Academy of Sciences 102(46):16569-16572.

Suggested Citation:"4 Comparing Large Strategic Missions and Smaller Missions." National Academies of Sciences, Engineering, and Medicine. 2017. Powering Science: NASA's Large Strategic Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/24857.
×

it difficult to access challenging locations, and eliminating smaller missions would limit the ability to respond quickly to new discoveries. For the scientific community, eliminating either large or smaller missions could lead to major funding gaps that would place strain on the Research and Analysis program. For the engineering community specifically, there could be a loss of opportunity to produce unique classes of instruments required by planetary science, and loss of entry, descent and landing, and deep-space navigation capabilities. The JPL data are retrospective and aggregate over four decades in a variety of political and fiscal environments. The JPL method is not predictive but is informative.

To seek to address the charge regarding comparing the impact of a range of missions, this committee requested data from NASA’s Science Mission Directorate (SMD). The data were sought to inform the committee’s analysis of scientific productivity, impact of missions and mission size on the current and future health of the relevant scientific community, the use of past technology, and the potential feed-forward of technology developed and demonstrated in current missions. Because the committee expected that data on a number of missions might be incomplete or nonexistent due to incomplete historical databases of program costs, it asked for data on more missions than it required for its analysis with the hope of obtaining sufficient information to accomplish the task.

NASA’s SMD expended considerable effort to provide the committee with the requested data. Nevertheless and unfortunately, the data the four science divisions were able to provide are uneven and, in some cases, entirely missing. This affected the committee’s ability to properly and thoroughly address the statement of task. In part, missing data is due to the change in accounting practices at NASA in the early 2000s and the switch to full cost accounting. Missions started before the implementation of full cost accounting, which includes many of the currently operating large strategic missions, have less complete cost data than more recent missions. A greater ability to rapidly provide public data of this kind, however, would have broader implications for the ability of SMD to present their story and successes to Congress, the executive branch (e.g., the Office of Science and Technology Policy and the Office of Management and Budget), the NASA administrator, the scientific community, and the public.

The committee attempted to make some comparisons of the outputs of different-size missions for NASA’s science divisions. The committee cautions, however, that any comparisons of such data are best made within their respective disciplines, not across the disciplines. For example, trying to compare scientific productivity of a heliophysics mission to an astrophysics mission would be highly dubious, because their value cannot be understood without the context of how they fit within their disciplines. Furthermore, publication data alone are not a perfect indicator of importance. The generation of thousands of peer-reviewed scientific papers from a single mission is impressive, but a single observation could significantly change an entire scientific field.

This committee was unable to draw many conclusions from the data on the various space science missions, other than some obvious ones such as larger missions result in funding more full-time equivalent personnel and higher numbers of publications. In general, the data confirmed information and comments that the committee heard from various speakers:

  • Large strategic missions are inherently designed to conduct transformative science with order of magnitude increases in discovery space.
  • Many large strategic missions are managed to engage a broad segment of the community, ranging from guest observer grants in the case of astrophysics to access to large layered databases in the case of Earth science.
  • Many large strategic missions address fundamental questions at the frontiers of the space sciences that are of great interest to the broader scientific community, to policy makers, and to the public.
  • Technology development occurs in large strategic missions, as well as dedicated technology development programs. Technology can flow in multiple directions, from large missions to small and from small missions to large.

What several of the missions discussed in the following appendixes demonstrate is that technology development is different for each division, and there is no inherent path for technology flow. Currently, small spacecraft such as CubeSats are generating technology that is migrating to larger missions. Larger missions have developed their own technology, some of which has been adopted by smaller missions and some of which is unique to those

Suggested Citation:"4 Comparing Large Strategic Missions and Smaller Missions." National Academies of Sciences, Engineering, and Medicine. 2017. Powering Science: NASA's Large Strategic Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/24857.
×

missions that develop it. Finally, separate technology development programs in the divisions, as well as the Space Technology Mission Directorate, have also been important incubators.

FINDING: Large strategic science missions support large teams of scientists and graduate students and therefore support the development and the health of their respective scientific communities in ways that smaller missions cannot.

FINDING: Smaller missions are often necessary to provide new insights, respond to recent discoveries, and refine the scientific goals of less frequent large strategic missions.

FINDING: Technology development occurs at many levels: large strategic missions, medium-size and even small missions, and separate technology development programs.

The limitations of the data across various space science missions are in some cases a result of when these missions were in development. For example, the implementation of full cost accounting at NASA in the early 2000s has made cost comparisons between missions before and after this date—such as Hubble, Chandra, and Cassini—difficult except in broad terms. But now that full cost accounting has become the standard at NASA, it is easier for the agency to acquire and compare these data. In addition, the agency already uses its senior review process to gather data on the scientific productivity of its missions. The senior review data are publicly releasable and NASA has released it, although not consistently across divisions or always easily accessible. The committee concluded that there would be great value to the agency and its programs if the SMD made these data as well as its other programmatic data more publicly available and standardized. If, in the future, the SMD faces questions about the productivity and value of its large strategic missions, then having the data compiled and made public to respond to these questions would be vital.

RECOMMENDATION: In order to demonstrate the role and scientific productivity of large strategic missions in advancing science, technology, and the long-term health of the field, NASA’s Science Mission Directorate (SMD) should develop a publicly accessible database, updated at least annually, that tracks basic data related to all confirmed missions in development as well as operational and past missions from each of the SMD divisions. These data should include development costs; publication numbers and other bibliographic data; outreach data (number of press releases and so on should be tracked); science, engineering, and other full-time equivalents; and other routine data typically sought in senior review proposal submittals once prime missions have been completed. These data should be of sufficient detail and quality to enable basic analyses related to scientific productivity and contributions to the health of the respective fields.

Although the committee acknowledges that establishing such a public database will require effort by NASA, it concluded that this would be useful to the agency in communicating the value and output of its missions as a whole, as opposed to via periodic press releases or at scientific conferences. The agency is in a better position today than it was only a decade ago to accurately report on its mission costs and performance across the entire Science Mission Directorate, and the committee concluded it can and should do so.

This study was initiated in part to evaluate the value and role of large strategic missions and concluded that they are highly valuable, to the scientific endeavor, to NASA, and to the United States. The committee concluded that NASA has substantial information to make that case and can continue to do so provided that it prepares the information in a coherent manner.

Suggested Citation:"4 Comparing Large Strategic Missions and Smaller Missions." National Academies of Sciences, Engineering, and Medicine. 2017. Powering Science: NASA's Large Strategic Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/24857.
×
Page 60
Suggested Citation:"4 Comparing Large Strategic Missions and Smaller Missions." National Academies of Sciences, Engineering, and Medicine. 2017. Powering Science: NASA's Large Strategic Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/24857.
×
Page 61
Suggested Citation:"4 Comparing Large Strategic Missions and Smaller Missions." National Academies of Sciences, Engineering, and Medicine. 2017. Powering Science: NASA's Large Strategic Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/24857.
×
Page 62
Next: Appendixes »
Powering Science: NASA's Large Strategic Science Missions Get This Book
×
Buy Paperback | $75.00 Buy Ebook | $59.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

NASA's Science Mission Directorate (SMD) currently operates over five dozen missions, with approximately two dozen additional missions in development. These missions span the scientific fields associated with SMD's four divisions—Astrophysics, Earth Science, Heliophysics, and Planetary Sciences. Because a single mission can consist of multiple spacecraft, NASA-SMD is responsible for nearly 100 operational spacecraft. The most high profile of these are the large strategic missions, often referred to as "flagships."

Large strategic missions are essential to maintaining the global leadership of the United States in space exploration and in science because only the United States has the budget, technology, and trained personnel in multiple scientific fields to conduct missions that attract a range of international partners. This report examines the role of large, strategic missions within a balanced program across NASA-SMD space and Earth sciences programs. It considers the role and scientific productivity of such missions in advancing science, technology and the long-term health of the field, and provides guidance that NASA can use to help set the priority of larger missions within a properly balanced program containing a range of mission classes.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!