National Academies Press: OpenBook
« Previous: Front Matter
Suggested Citation:"Summary." National Research Council. 2010. Controlling Cost Growth of NASA Earth and Space Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/12946.
×

Summary

STUDY BACKGROUND

Cost growth in Earth and space science missions conducted by the Science Mission Directorate (SMD) of the National Aeronautics and Space Administration (NASA) is a longstanding problem with a wide variety of interrelated causes. To address this concern, the NASA Authorization Act of 2008 (P.L. 110-422) directed the NASA administrator to sponsor an “independent external assessment to identify the primary causes of cost growth in the large-, medium-, and small-sized Earth and space science spacecraft mission classes, and make recommendations as to what changes, if any, should be made to contain costs and ensure frequent mission opportunities in NASA’s science spacecraft mission programs.” NASA subsequently requested that the National Research Council (NRC) conduct a study to:

  • Review the body of existing studies related to NASA space and Earth science missions and identify their key causes of cost growth and strategies for mitigating cost growth;

  • Assess whether those key causes remain applicable in the current environment and identify any new major causes; and

  • Evaluate effectiveness of current and planned NASA cost growth mitigation strategies and, as appropriate, recommend new strategies to ensure frequent mission opportunities.

As part of this effort, NASA also asked the NRC to “note what differences, if any, exist with regard to Earth science compared with space science missions.”

COST GROWTH—MAGNITUDE AND CAUSES

NASA identified 10 cost studies and related analyses that this study uses as its primary references (listed in the References chapter and in Table 1.1). The committee generally concurs with the consensus viewpoints expressed in these studies as a whole, but in some areas, the studies reached different conclusions. For example, the prior studies calculated values for average cost growth ranging from 23 percent to 77 percent. Different studies reach different conclusions because they examine different sets of missions and calculate cost growth based on different criteria. By definition, cost growth is a relative measure reflecting comparison of an initial estimate of mission costs against costs actually incurred at a later time. But studies use initial estimates made at different points in

Suggested Citation:"Summary." National Research Council. 2010. Controlling Cost Growth of NASA Earth and Space Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/12946.
×

mission life cycles (see Figure S.1), as well as cost estimates that cover different phases of mission life cycles. For example, some studies consider only development costs (up to but not including launch), but other studies consider all costs through the end of each mission.

In general, the earlier the initial estimate, the more the cost will grow. In addition, including a larger share of the later phases of a mission (such as launch, operations, and analysis of data collected by a mission) increases the total cost assigned to each mission and the absolute value of the cost growth (in dollars). These differences make it very difficult to derive a single, reliable value for the average cost growth of NASA Earth and space science missions on the basis of previous studies.

The primary references also indicate that most cost growth occurs after critical design review. This implies that the required level of cost reserves remains substantial, even late in the development process. In addition, a relatively small number of missions cause most of the total cost growth. For one large set of 40 missions, 92 percent of the total cost growth (in dollars) was caused by only 14 missions (one-third of the total number). Conversely, the 26 missions with the least cost growth (two-thirds of the total number) accounted for only 8 percent of the total cost growth (see Figure S.2).

The primary references identify a wide range of factors that contribute to cost and schedule growth of NASA Earth and space science missions. The most commonly identified factors are the following:

  • Overly optimistic and unrealistic initial cost estimates,

  • Project instability and funding issues,

  • Problems with development of instruments and other spacecraft technology, and

  • Launch service issues.

Additional factors identified in the primary references include schedule growth that leads to cost growth. Schedule growth and cost growth are well correlated because any problem that causes schedule growth contributes to and magnifies total mission cost growth. Furthermore, cost growth in one mission may induce organizational replanning that delays other missions in earlier stages of implementation, further amplifying overall cost growth. Effective implementation of a comprehensive, integrated cost containment strategy, as recommended herein, is the best way to address this problem.

FIGURE S.1 NASA mission life cycle. SOURCE: Based on NASA Procedural Requirements 7120.5D (NASA, 2007).

FIGURE S.1 NASA mission life cycle. SOURCE: Based on NASA Procedural Requirements 7120.5D (NASA, 2007).

Suggested Citation:"Summary." National Research Council. 2010. Controlling Cost Growth of NASA Earth and Space Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/12946.
×
FIGURE S.2 Ranking of 40 NASA science missions in terms of absolute cost growth in excess of reserves in millions of dollars, excluding launch, mission operations, and data analysis, with initial cost and launch date for each mission also shown. NOTE: Acronyms are defined in Appendix D. SOURCE: Based on data from Tom Coonce, NASA Headquarters, e-mail to committee member Joseph W. Hamaker, December 21, 2009.

FIGURE S.2 Ranking of 40 NASA science missions in terms of absolute cost growth in excess of reserves in millions of dollars, excluding launch, mission operations, and data analysis, with initial cost and launch date for each mission also shown. NOTE: Acronyms are defined in Appendix D. SOURCE: Based on data from Tom Coonce, NASA Headquarters, e-mail to committee member Joseph W. Hamaker, December 21, 2009.

COMPREHENSIVE, INTEGRATED STRATEGY FOR COST AND SCHEDULE CONTROL

NASA sets the strategic direction of its Earth and space science programs using decadal surveys, the SMD science plan, and supporting road maps. A comprehensive, integrated approach to cost and schedule growth is also essential.

The primary references identify dozens of specific causes, make dozens of specific recommendations, and include dozens of additional findings concerning cost growth. The primary references, as a whole, are generally consistent and comprehensive, and so the individual causes of cost growth and the necessary corrective actions are not a mystery. However, rather than simply picking and choosing from among the many suggested causes, findings, and recommendations, development of a comprehensive, integrated strategy offers the best chance that future actions will work in concert to minimize or eliminate cost and schedule growth. An effective strategy would substantially reduce cost growth (beyond reserves) on individual missions and programs so that whatever growth does occur is offset by other missions and programs completed for less than the budgeted amount. This approach would allow NASA to execute the Earth and space science mission portfolio for the appropriated budget. Achieving this goal will require NASA to address both internal and external factors.

Internally, a comprehensive, integrated cost containment strategy would improve the definition of baseline

Suggested Citation:"Summary." National Research Council. 2010. Controlling Cost Growth of NASA Earth and Space Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/12946.
×

costs and enhance the utility of NASA’s independent cost-estimating capabilities. Early development of technologies and more effective program reviews would improve the ability to identify and effectively manage risks and uncertainties. Externally, NASA has the opportunity to collaborate with other federal agencies, the Office of Management and Budget, and Congress to sustain and improve critical capabilities and expertise in the industrial base and the nation’s science and engineering workforce; to address cost and schedule risk associated with launch vehicles; and to improve funding stability.

Successful implementation of a comprehensive, integrated strategy to control cost and schedule growth of NASA Earth and space science missions would benefit both NASA and the nation, while enabling NASA to more efficiently and effectively carry out these critical missions.

Finding. Comprehensive, Integrated Cost Containment Strategy. Recent changes by NASA in the development and management of Earth and space science missions are promising. These changes include budgeting programs to the 70 percent confidence level1 and specifying that decadal surveys include independent cost estimates. However, it is too early to assess the effectiveness of these actions, and NASA has not taken the important step of developing a comprehensive, integrated strategy.


Recommendation. Comprehensive, Integrated Cost Containment Strategy. NASA should develop a comprehensive, integrated strategy to contain cost and schedule growth and enable more frequent science opportunities. This strategy should include recent changes that NASA has already implemented as well as other actions recommended in this report.

KEY PROBLEMS

In addition to developing a comprehensive, integrated cost containment strategy, and as detailed below, NASA should address specific issues related to cost realism and the development process for Earth and space science missions.

Cost Realism

Cost Estimates

NASA project staff generally estimate mission costs using detailed engineering analyses of labor and material requirements, vendor quotes, subcontractor bids, and the like. Non-advocate independent cost estimates in NASA are generally parametric cost estimates using statistical cost-estimating relationships based on historical relationships among cost and technical and programmatic variables (mass, power, complexity, and so on). In both cases, mission cost estimates are created by summing costs at lower levels of a project’s work breakdown structure to obtain total project costs. Parametric cost models rely on observations rather than opinion, are an excellent tool for answering “what-if” questions quickly, and provide statistically sound information about the confidence level of cost estimates. In contrast, the process used within NASA to generate cost estimates on the basis of detailed engineering assessments does not provide a statistical confidence level and, in retrospect, has generally been less accurate than parametric cost models in estimating the cost of NASA Earth and space science missions.2

A project manager or principal investigator who is personally determined to control costs can be of great assistance in avoiding cost growth. People and organizations tend to optimize their behavior based on the environment in which they operate. Unfortunately, instead of motivating and rewarding vigilance in accurately predicting and controlling costs, the current system incentivizes overly optimistic expectations regarding cost and schedule.

1

If programs are budgeted at the 70 percent confidence level, there is a 70 percent probability that all of the missions included in the program can be completed without exceeding mission and program reserves.

2

A detailed list of the strengths and weaknesses of various cost-estimating methods appears in 2008 NASA Cost Estimating Handbook. Washington, D.C.: NASA.

Suggested Citation:"Summary." National Research Council. 2010. Controlling Cost Growth of NASA Earth and Space Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/12946.
×

For example, competitive pressures encourage (overly) optimistic assessments of the cost and schedule impacts of addressing uncertainties and overcoming potential problems. As a result, initial cost estimates generally are quite optimistic, underestimating final costs by a sizable amount, and that optimism sometimes persists well into the development process.

Recommendation. Independent Cost Estimates. NASA should strengthen the role of its independent cost-estimating function by

  • Expanding and improving NASA’s ability to conduct parametric cost estimates, and

  • Obtaining independent parametric cost estimates at critical design review (in addition to system requirements review and preliminary design review), comparing them to other estimates available from the project and reconciling significant differences.

Cost Growth Methodology

The measurement of cost growth has been inconsistent across programs, NASA centers, and Congress. The Government Accountability Office and Congress generally consider the baseline to be the first time a mission appears as a budget line item in an appropriations bill, which is often before preliminary design review. The contents of NASA estimates also differ—some estimates include Phase A and B, some start with Phase C, some (but not all) include launch costs and/or mission operations, and some include NASA oversight and internal project management costs. These differences make it difficult to develop a clear understanding of trends in cost and schedule growth.

Recommendation. Measurement of Cost Growth. NASA, Congress, and the Office of Management and Budget should consistently use the same method to quantify and report cost. In particular, they should use as the baseline a life-cycle cost estimate (that goes through the completion of prime mission operations) produced at preliminary design review.

Development Process

Management of Announcement of Opportunity Missions and Directed Missions

NASA implements two separate and distinct classes of Earth and space science missions—announcement of opportunity (AO) missions and directed missions. NASA headquarters competitively selects AO missions from proposals submitted in response to periodic AOs by teams led by a principal investigator (PI), who is commonly affiliated with a university but may work in industry or for NASA. NASA headquarters determines the scientific goals and requirements for directed missions, which are sometimes referred to as facility class missions or flagship missions. Headquarters then directs a particular NASA center, usually Goddard Space Flight Center or the Jet Propulsion Laboratory, to implement the mission.

The differing nature and goals of directed and AO missions call for different management approaches. AO missions are on average much smaller than directed missions are, and the impact of cost growth in AO missions, which are managed within a mission budget line (e.g., Discovery), is limited to other missions within the line. Flagship missions, however, are typically much larger than AO missions are, and so cost growth in these missions has a much greater potential to diminish NASA’s Earth and space science enterprise as a whole.

Recommendation. Management of Large, Directed Missions. NASA headquarters’ project oversight function should pay particular attention to the cost and schedule of its larger missions (total cost on the order of $500 million or more), especially directed missions (which form a single line item).

Suggested Citation:"Summary." National Research Council. 2010. Controlling Cost Growth of NASA Earth and Space Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/12946.
×

Recommendation. Management of Announcement of Opportunity (AO) Missions. NASA should continue to emphasize science in the AO mission selection process, while revising the AO mission selection process to allocate a larger percentage of project funds for risk reduction and improved cost estimation prior to final selection.


Recommendation. Incentives. NASA should ensure that proposal selection and project management processes include incentives for program managers, project managers, and principal investigators to establish realistic cost estimates and minimize or avoid cost growth at every phase of the mission life cycle, for both directed missions and announcement of opportunity missions.

Technology and Instrument Development

NASA Procedural Requirements (NPR) 7120.5, NASA Space Flight Program and Project Management Requirements, requires that “during formulation, the project establishes performance metrics, explores the full range of implementation options, defines an affordable project concept to meet requirements specified in the Program Plan, develops needed technologies, and develops and documents the project plan” (NASA, 2007, Section 2.3.4). However, despite these requirements, the primary references identify an ongoing need to improve technical and programmatic definition at the beginning of a project. The limited time and resources typically available in phases A and B to mature new technology and solidify system design parameters contribute to cost growth through higher risk and unrealistic cost estimates.

Instrument technology is particularly important because Earth and space science missions generally require special-purpose, one-of-a-kind components. Delays and cost increases for instrument development are pervasive and impact a large number of missions. This problem is exacerbated by shrinkage of the U.S. industrial base that supports space system development.

Recommendation. Technology Development. NASA should increase the emphasis in phases A and B on technology development, risk reduction, and realism of cost estimates.


Recommendation. Instrument Development. NASA should initiate instrument development well in advance of starting other project elements and establish a robust instrument technology development effort relevant to all classes of Earth and space science missions to strengthen and sustain the nation’s instrument development capability.


Recommendation. Decadal Surveys. NASA should ensure that guidance regarding the development of instruments and other technologies is included in decadal surveys and other strategic planning efforts. In particular, future decadal surveys should prioritize science mission areas that could be addressed by future announcements of opportunity and the instruments needed to carry out those missions.

Major Reviews

NASA has increased the size and number of external project reviews to the point that some reviews are counterproductive and disruptive, especially for small missions. Large numbers of reviews diffuse responsibility and accountability, creating an environment where NASA senior managers can become dependent on review teams with many outside members who sometimes do not understand NASA, the field center in question, and/or the mission being reviewed. In addition, major reviews are sometimes conducted as scheduled even though a project may not have progressed as rapidly as expected and, as a result, cannot achieve the intended review criteria, programmatically and/or technologically.3

3

General preliminary design review and critical design review readiness criteria exist within NPR 7120.5D (NASA, 2007). More detailed criteria are provided in center directives such as Criteria for Flight Project Critical Milestone Reviews (NASA/GSFC, 2009).

Suggested Citation:"Summary." National Research Council. 2010. Controlling Cost Growth of NASA Earth and Space Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/12946.
×

Recommendation. External Project Reviews. NASA should reassess its approach to external project reviews to ensure that (1) the value added by each review outweighs the cost (in time and resources) that it places on projects; (2) the number and the size of reviews are appropriate given the size of the project; and (3) major reviews, such as preliminary design review and critical design review, occur only when specified success criteria are likely to be met.

Launch Vehicles

Problems with the procurement of launch vehicles and launch services are a significant source of cost growth. Specific factors include increases in the cost of expendable launch vehicles, vendor issues such as strikes, weather-related issues at the launch site, problems with launch-site-facility capabilities, and delays in the availability of a given launch vehicle. In addition, if a mission is required to change launch vehicles, the costs can be substantial.

Recommendation. Launch Vehicles. Prior to preliminary design review, NASA should minimize mission-unique launch site processing requirements. NASA should also select the launch vehicle with appropriate margins as early as possible and minimize changes in launch vehicles.

DIFFERENCES BETWEEN EARTH AND SPACE SCIENCE MISSIONS

Different classes of missions face different challenges. Earth science missions typically have more complex, more costly, and more massive instruments than do space science missions, because Earth science missions also have more stringent requirements in terms of pointing accuracy, resolution, stability, and so on, although astrophysics missions also have stringent pointing requirements, and planetary spacecraft and instrument technology must be able to survive long cruise phases and radiation environments that are sometimes quite extreme. Space science missions that leave Earth orbit have greater incentives to minimize spacecraft mass and power, and the average cost and average spacecraft mass of these missions are lower than those for Earth science missions. However, the size of the cost growth of Earth and space science missions has been comparable. Both Earth and space science missions have shown good correlation between (1) instrument schedule growth and instrument cost growth, (2) instrument cost/schedule growth and mission cost/schedule growth, and (3) the absolute costs of instruments and instrument complexity.

Suggested Citation:"Summary." National Research Council. 2010. Controlling Cost Growth of NASA Earth and Space Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/12946.
×
Page 1
Suggested Citation:"Summary." National Research Council. 2010. Controlling Cost Growth of NASA Earth and Space Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/12946.
×
Page 2
Suggested Citation:"Summary." National Research Council. 2010. Controlling Cost Growth of NASA Earth and Space Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/12946.
×
Page 3
Suggested Citation:"Summary." National Research Council. 2010. Controlling Cost Growth of NASA Earth and Space Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/12946.
×
Page 4
Suggested Citation:"Summary." National Research Council. 2010. Controlling Cost Growth of NASA Earth and Space Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/12946.
×
Page 5
Suggested Citation:"Summary." National Research Council. 2010. Controlling Cost Growth of NASA Earth and Space Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/12946.
×
Page 6
Suggested Citation:"Summary." National Research Council. 2010. Controlling Cost Growth of NASA Earth and Space Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/12946.
×
Page 7
Next: 1 Size and Historic Causes of Cost Growth »
Controlling Cost Growth of NASA Earth and Space Science Missions Get This Book
×
Buy Paperback | $29.00 Buy Ebook | $23.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Cost and schedule growth is a problem experienced by many types of projects in many fields of endeavor. Based on prior studies of cost growth in NASA and Department of Defense projects, this book identifies specific causes of cost growth associated with NASA Earth and space science missions and provides guidance on how NASA can overcome these specific problems.

The recommendations in this book focus on changes in NASA policies that would directly reduce or eliminate the cost growth of Earth and space science missions. Large cost growth is a concern for Earth and space science missions, and it can be a concern for other missions as well. If the cost growth is large enough, it can create liquidity problems for NASA's Science Mission Directorate that in turn cause cost profile changes and development delays that amplify the overall cost growth for other concurrent and/or pending missions. Addressing cost growth through the allocation of artificially high reserves is an inefficient use of resources because it unnecessarily diminishes the portfolio of planned flights. The most efficient use of resources is to establish realistic budgets and reserves and effective management processes that maximize the likelihood that mission costs will not exceed reserves. NASA is already taking action to reduce cost growth; additional steps, as recommended herein, will help improve NASA's mission planning process and achieve the goal of ensuring frequent mission opportunities for NASA Earth and space science.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!