1
Size and Historic Causes of Cost Growth

STUDY BACKGROUND

The Science Mission Directorate (SMD) of the National Aeronautics and Space Administration (NASA) conducts both Earth and space science missions. The latter type encompasses missions in planetary sciences, heliophysics, and astrophysics.

Cost growth in NASA Earth and space science missions is a longstanding problem with a wide variety of interrelated internal and external causes, both technical and programmatic. Many different organizations, both public and private, have examined the cost growth of NASA Earth and space science missions,1 but the results of these studies have not been assessed as a whole. In response to this situation, the NASA Authorization Act of 2008 (P.L. 110-422) directed the NASA administrator to sponsor an “independent external assessment to identify the primary causes of cost growth in the large-, medium-, and small-sized Earth and space science spacecraft mission classes, and make recommendations as to what changes, if any, should be made to contain costs and ensure frequent mission opportunities in NASA’s science spacecraft mission programs.”

NASA subsequently requested that the National Research Council (NRC) conduct a study to:

  • Review the body of existing studies related to NASA space and Earth science missions and identify their key causes of cost growth and strategies for mitigating cost growth;

  • Assess whether those key causes remain applicable in the current environment and identify any new major causes; and

  • Evaluate effectiveness of current and planned NASA cost growth mitigation strategies and, as appropriate, recommend new strategies to ensure frequent mission opportunities.

The Committee on Cost Growth in NASA Earth and Space Science Missions was established to conduct this study. As part of this effort, NASA also asked the NRC to “note what differences, if any, exist with regard to Earth science compared with space science missions.” NASA identified a list of relevant cost studies and related analyses to use as primary references for the study (see Table 1.1).

1

In general, this report refers to NASA flight projects as missions. NASA programs are generally collections of projects or missions with similar objectives, although in some cases a program has only one flight project/mission.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 8
1 Size and Historic Causes of Cost Growth STuDy BACkGROuND The Science Mission Directorate (SMD) of the National Aeronautics and Space Administration (NASA) conducts both Earth and space science missions. The latter type encompasses missions in planetary sciences, heliophysics, and astrophysics. Cost growth in NASA Earth and space science missions is a longstanding problem with a wide variety of interrelated internal and external causes, both technical and programmatic. Many different organizations, both public and private, have examined the cost growth of NASA Earth and space science missions, 1 but the results of these studies have not been assessed as a whole. In response to this situation, the NASA Authorization Act of 2008 (P.L. 110-422) directed the NASA administrator to sponsor an “independent external assessment to identify the primary causes of cost growth in the large-, medium-, and small-sized Earth and space science spacecraft mis - sion classes, and make recommendations as to what changes, if any, should be made to contain costs and ensure frequent mission opportunities in NASA’s science spacecraft mission programs.” NASA subsequently requested that the National Research Council (NRC) conduct a study to: • Review the body of existing studies related to NASA space and Earth science missions and identify their key causes of cost growth and strategies for mitigating cost growth; • Assess whether those key causes remain applicable in the current environment and identify any new major causes; and • Evaluate effectiveness of current and planned NASA cost growth mitigation strategies and, as appropriate, recommend new strategies to ensure frequent mission opportunities. The Committee on Cost Growth in NASA Earth and Space Science Missions was established to conduct this study. As part of this effort, NASA also asked the NRC to “note what differences, if any, exist with regard to Earth science compared with space science missions.” NASA identified a list of relevant cost studies and related analyses to use as primary references for the study (see Table 1.1). 1In general, this report refers to NASA flight projects as missions. NASA programs are generally collections of projects or missions with similar objectives, although in some cases a program has only one flight project/mission. 8

OCR for page 8
 SIZE AND HISTORIC CAUSES OF COST GROWTH TABLE 1.1 Primary References Provided to the Study Committee NASA Cost Studies 1 R.E. Bitten, D.L. Emmons, and C.W. Freaner. 2006. Using Historical NASA Cost and Schedule Growth to Set Future Program and Project Reserve Guidelines. IEEE Paper #1545. December. 2 NASA Langley Research Center (LaRC). 2007. Cost/Schedule Performance Study for the Science Mission Directorate. Final Report. Prepared by the NASA LaRC Science Support Office and Science Applications International Corporation. NASA LaRC, Hampton, Va. October. 3 NASA. 2007. “Cost and Schedule Growth at NASA.” Presentation provided to the committee by Director of the Cost Analysis Division Tom Coonce, Office of Program Analysis and Evaluation, NASA, Washington, D.C. November. 4 NASA. 2008. “SMD Cost/Schedule Performance StudySummary Overview.” Presentation by B. Perry and C. Bruno, NASA Science Support Office; M. Jacobs, M. Doyle, S. Hayes, M. Stancati, W. Richie, and J. Rogers, Science Applications International Corporation. January. 5 C.W. Freaner, R.E. Bitten, D.A. Bearden, and D.L. Emmons. 2008. “An Assessment of the Inherent Optimism in Early Conceptual Designs and Its Effect on Cost and Schedule Growth.” Paper presented at the Space Systems Cost Analysis Group/Cost Analysis and Forecasting/European Aerospace Cost Engineering Working Group 2008 Joint International Conference, European Space Research and Technology Centre, Noordwijk, The Netherlands, May 15-16. European Space Agency, Paris, France. 6 B. Mlynczak and B. Perry, Science Support Office, NASA. 2009. “SMD Earth and Space Mission Cost Driver Comparison Study. Final Report and Presentation.” March. Related Analyses 7 General Accounting Office. 1992. Space Missions Require Substantially More Funding Than Initially Estimated. GAO/ NSIAD-93-97. Washington, D.C. December. 8 National Research Council. 1997. Reducing the Costs of Space Science Research Missions: Proceedings of a Workshop. National Academy Press, Washington, D.C. 9 J. McCrillis, Office of the Secretary of Defense, Cost Analysis Improvement Group. 2003. “Cost Growth of Major Defense Programs.” Presentation to the Annual Department of Defense Cost Analysis Symposium, January 30, Williamsburg, Va. 10 Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. 2003. Report of the Defense Science Board/Air Force Scientific Advisory Board Joint Task Force on Acquisition of National Security Space Programs. Department of Defense, Washington, D.C. May. The committee generally concurs with the consensus viewpoints expressed in the primary references as a whole. However, as detailed below, in some areas the prior studies reached different conclusions. This chapter describes the size and historic causes of cost growth in NASA Earth and space science missions, based primarily on the above references. As an illustrative example, this chapter also describes the results of a detailed assessment of cost and schedule data for the 40 missions included in Primary Reference 1. 2 Chapter 2 describes the committee’s own assessment of key problems, as they currently exist, and recommended solutions. A full copy of the study statement of task, the tasking letter from NASA, and the legislation that prompted this study appear in Appendix A. The expertise of the study committee is summarized in Appendix B. The findings and recommendations contained in the primary references are summarized in Appendix C. ANNOuNCEMENT OF OPPORTuNITy AND DIRECTED MISSIONS NASA implements two separate and distinct classes of Earth and space science missions: announcement of opportunity (AO) missions, such as the Mars Exploration Rover mission, and directed missions, such as the Hubble Space Telescope mission. NASA headquarters competitively selects AO missions from proposals submitted in response to periodic AOs by teams led by a principal investigator, who is commonly affiliated with a university 2 Primary Reference 1 was selected because it assesses more NASA missions than any of the other primary references. (See Table 1.2.)

OCR for page 8
0 CONTROLLING COST GROWTH OF NASA EARTH AND SPACE SCIENCE MISSIONS but may work in industry or for NASA. These teams seek to address one or more scientific questions within the scope of a particular AO. Teams commonly include a NASA center, such as the Goddard Space Flight Center (GSFC) or the Jet Propulsion Laboratory (JPL), as well as an industrial partner. NASA then uses a moderately complex peer review process to evaluate these “Step 1” proposals. This typically leads to the selection of two or three proposals for further development in what is referred to as Phase A. Phase A concludes with the submission of “Step 2” proposals, from which NASA selects a single mission for implementation. AO missions may carry one or several instruments, and they typically cost from $100 million to several hundred million dollars. Only rarely does the cost exceed $1 billion. NASA headquarters determines the scientific goals and requirements for directed missions, which are some - times referred to as facility class missions or flagship missions. Headquarters then directs a particular NASA center, usually GSFC or JPL, to implement the mission. The spacecraft for directed missions usually constitute large scientific facilities that produce a variety of data using multiple instruments, which may be quite large. Some of the instruments may be selected using an AO process. NASA headquarters establishes a science working group for directed missions to guide the science aspects of the missions as they move through each phase of development and into operations. Directed missions may cost one to several billion dollars. SIzE OF COST GROWTH Finding. Size of Cost Growth. Historic studies of cost growth indicate the following: • Past studies of cost growth in NASA Earth and space science missions calculated values for average cost growth ranging from 23 percent to 77 percent. • Different studies reach different conclusions because they examine different sets of missions and because of differences in how cost growth is calculated. • Relatively little cost growth occurs between preliminary design review (PDR) and critical design review (CDR). A majority of cost growth occurs after CDR, with the rest occurring prior to PDR. • For one large set of 40 missions, 80 percent of the total cost growth (in absolute dollar terms) was caused by only 11 missions. • The size of the cost growth of Earth and space science missions has been comparable. Range of Cost Growth The 10 primary references examined by this study (listed in Table 1.1) address cost growth in both NASA science missions (Primary References 1 to 8) and Department of Defense (DOD) programs (Primary References 9 and 10). Table 1.2 compares the average cost and schedule growth reported by some of these studies. While the lowest reported average cost growth in the 10 primary references was 23 percent, the maximum was 77 percent. The average across all 10 was 51 percent. The primary references confirmed that extensive cost growth exists in many—but not all—NASA Earth and space science missions, but the extent of the cost growth differed significantly from one mission to the next, and from one study of cost growth to the next. However, regardless of the wide range in cost growth reported by the various studies, unwanted cost and schedule growth have certainly made it more difficult to accomplish NASA’s Earth and space science missions than was originally anticipated when these missions were authorized.4 3 Here and in the sections that follow, each section begins with a finding and/or recommendation, which is then explained and justified by the text that follows. 4 Changes in technical or mission requirements increase costs, but this is not always a problem that needs to be fixed. In some cases a deliberate decision is made to increase mission capabilities or otherwise add value to a mission despite the extra cost involved. One way in which successful projects incur extra costs is through extended mission operations. However, unplanned cost and schedule growth, which unexpectedly increase costs without adding value, are always unwelcome.

OCR for page 8
 SIZE AND HISTORIC CAUSES OF COST GROWTH TABLE 1.2 Average Reported Cost and Schedule Growth of Past NASA and Department of Defense Missions Number of Missions Average Cost Growth Primary Reference Missions or Programs (%) Average Schedule Growth 1 NASA 40 27 22 % 2, 4 NASA 15 23 13 months 3 NASA 25 68 56 % 5 NASA 10 76 36 % 77a 7 NASA 29 9 DOD 142 32 NOTE: Primary References 2 and 4 examined the same data. Primary References 6, 8, and 10 did not calculate average cost or schedule growth. Primary Reference 7 concluded that (1) cost increased by more than 50 percent for three-fourths of the missions examined and (2) cost increased by more than 100 percent for one-third of the missions examined. a T he 77 percent cost growth in Primary Reference 7 represents the median cost growth observed. Different Studies—Different Results It is difficult to reconcile fully the different values of cost growth identified in the primary references. The committee identified two primary reasons for these differences: • Differences in how cost growth is calculated, and • Differences in the sets of missions and programs examined. By its very definition, cost growth is a relative measure comparing an initial estimate of mission costs against actually incurred costs at a later time. The times at which the initial and final cost estimates are made differ across studies. Figure 1.1 shows the sequence of program phases, key decision points (KDPs), and key mission reviews for a typical NASA mission. Figure 1.1 also shows how three different references calculate cost growth. Primary Reference 1 only considers growth in phases A through D, except for launch. Primary Reference 2 assesses cost growth from the beginning of Phase B (KDP B) through end of mission. Primary Reference 7, on the other hand, measures cost growth from the first time a mission appears as a line item in the NASA budget submitted to Con - gress until the completion of the mission (end of Phase F). In general, the earlier the initial estimate, the more cost growth will occur. In addition, including more of the later phases (such as launch, operations, and data analysis) in the cost growth assessment increases the total cost assigned to each mission and the absolute value of the cost growth (in dollars). In particular, a key reason that Primary References 1, 2, and 4 show an average cost growth of 23 to 27 percent, whereas Primary Reference 7 shows a 77 percent average cost growth, is that the start and end points of the cost growth measurements made by Primary Reference 7 are very different from those for Primary References 1, 2, and 4. These differences make it very difficult to derive a single, reliable value for the average cost growth of NASA Earth and space science missions based on previous studies. Cost growth estimates can also be distorted by missions with exceptionally long development schedules, by changes in mission scope, and by the cost of extended mission operations. For example, Gravity Probe B was under continuous development for 40 years before it was launched in 2004 (NRC, 1995). The Gravity Probe B cost and schedule growth data used by Primary Reference 1 assume that development began in fiscal year 1994, although NASA had previously expended $128 million developing mission concepts and technology. 5 Costs are reduced when a mission is descoped, but the value of the mission is also reduced. Conversely, the extra value obtained from an increase in mission scope or an extension of mission operations is presumably worth the cost, or else the broader scope or extended operations would not be approved. In both cases, programmatic changes obscure the extent of cost growth that the missions would have experienced if they had been implemented as originally planned. For example, Aqua, an Earth Observing System (EOS) mission formerly known as EOS-PM, 5 Tom Coonce, NASA Headquarters, e-mail to committee member Joseph W. Hamaker, December 21, 2009.

OCR for page 8
 CONTROLLING COST GROWTH OF NASA EARTH AND SPACE SCIENCE MISSIONS Approval Approval NASA Life- Formulation Implementation Cycle Phases Pre-Phase A: Phase A: Phase B: Phase C: Phase D: Phase E: Phase F: Project Life- Concept Concept and Preliminary Final Design System Operations and Closeout Cycle Phases Studies Technology Design and and Fabrication Assembly, Sustainment Development Technology Integration, Completion Test, and Launch Key Decision Points (KDPs) KDP A KDP B KDP C KDP D KDP E KDP F Mission Reviews MCR SRR MDR PDR CDR SIR TRR FRR DR Cost growth only for phases A Initial Ref. 1 through D, excluding launch budget (Sep 2006) submitted Start of Ref. 2 End of Phase B (Oct 2007) Mission Initial estimate Ref. 7 Only missions with development cost > $200M included End of provided in NASA (Dec 1992) Mission Budget to Congress MCR Mission Concept Review SRR System Requirements Review MDR Mission Design Review PDR Preliminary Design Review CDR Critical Design Review SIR System Integration Review TRR Test Readiness Review FRR Flight Readiness Review DR Decommissioning Review FIGURE 1.1 Different studies calculate cost growth over different portions of a mission’s total life cycle. Based in part on NASA Procedural Requirements 7120.5D. was significantly descoped in 1992 when the EOS budget was reduced by NASA. Primary Reference 2 reports that Aqua cost growth is 7 percent (based on changes from CDR through launch). Primary Reference 6, however, reports that Aqua cost growth is 23 percent (based on changes during phases B, C, and D). Similar discrepancies are reported for the Spitzer Space Telescope mission (formerly known as the Space Infrared Telescope Facility, SIRTF), which also was significantly descoped. SIRTF cost growth during phases B though D is reported as 68 percent in Primary Reference 2 and 52 percent in Primary Reference 6. Cost growth for Chandra (formerly known as the Advanced X-ray Astrophysics Facility) is reported as 5 percent in Primary Reference 2 (for phases B through D), as negative 2 percent in Primary Reference 6 (for phases B through D), and as 194 percent in Primary Refer- ence 7 (from project initiation through the current state of development when that report was completed in 1992; Chandra was subsequently descoped). These examples suggest that extreme care must be taken when comparing cost growth as measured by different studies, and they highlight the need for common metrics. Another key complicating factor is that different studies assess different sets of missions and programs. As seen in Table 1.2, the primary references that assessed NASA’s performance examined between 10 and 40 mis - sions. Primary Reference 9 examined 142 DOD projects covering a wide range of applications; the vast majority were not space launch missions. Table 1.3 lists the missions assessed by each of the primary references that examined NASA missions. 6 Note 6 Primary Reference 8 focused on NASA, but it did not assess the cost growth of individual missions.

OCR for page 8
 SIZE AND HISTORIC CAUSES OF COST GROWTH that Primary Reference 7 examined some missions, such as the Advanced Communications Technology Satellite, that are not Earth or space science missions. As shown in Table 1.3, some missions are only discussed in one study while others are included in multiple studies. For example, there is a core group of 12 missions that are included in Primary References 1, 2, 4, and 6. These missions are listed below and offer the best chance for a more in-depth comparison of cost growth calcula - tions across different studies: • Messenger (Mercury Orbiter), • Cloud Satellite (CLOUDSAT), • Galaxy Evolution Explorer (GALEX), • Swift Gamma-Ray Burst Mission, • Spitzer Space Telescope (formerly SIRTF), • Mars Reconnaissance Orbiter (MRO), • Solar Terrestrial Relations Observer (STEREO), • Comet Nucleus Tour (CONTOUR), • Deep Impact, • EOS-Aqua, • Mars Exploration Rover (MER), and • Reuven Ramaty High Energy Solar Spectral Imager (RHESSI). Seven of these 12 are also included in Primary Reference 5. Accordingly, the cost growth experience of these 12 missions is overemphasized in any simple averaging of the results of the primary references. On the other hand, Primary References 1, 3, and 7 assess a large number of missions that are not included in any of the other primary references, and this undoubtedly contributes to the large variance in average cost growth determined by the primary references. Cost Growth versus Schedule Growth To assess the overall cost growth of past missions, it is helpful to consider how missions cluster in terms of their relative cost and schedule growth. Figure 1.2 shows the distribution of the 40 missions examined by Primary Reference 1 in terms of development cost and schedule growth. Missions are grouped as Type I, II, and III based on the cost growth thresholds defined in the NASA Authorization Act of 2005: 7 • Type I, minor or no cost growth (less than 15 percent), • Type II, significant cost growth (15 to 30 percent), and • Type III, excessive cost growth (more than 30 percent). Of the 40 missions in Primary Reference 1, Figure 1.2 shows that 19 missions (roughly half) are Type I, 8 missions (one-fifth) are Type 2, and 13 missions (one-third) are Type III. 8 In other words, half of the Earth and space science missions in this large sample were completed within the 15 percent cost growth target needed to avoid congressional notification. 7 The NASA Authorization Act of 2005 requires congressional notifications for cost growth of 15 percent or more during phases B and C or schedule delays of 6 months or more. Reauthorization is required for cost growth of 30 percent or more or $1 billion or more during phases B and C. These reporting requirements do not apply to all of the missions plotted in Figure 1.2; per statute, they only apply to programs with an estimated life-cycle cost greater than $100 million, and some missions in Figure 1.2 occurred prior to fiscal year 2005. 8 Many of the figures and tables in this report depict data from Primary Reference 1, which were used for these illustrative examples because those data assessed more missions than any of the other studies that examined NASA missions, and because the committee was able to obtain the detailed data used by the authors of Primary Reference 1 in order to conduct the committee’s own supplemental analyses.

OCR for page 8
 CONTROLLING COST GROWTH OF NASA EARTH AND SPACE SCIENCE MISSIONS TABLE 1.3 NASA Missions Examined by the Primary References Primary Reference Launch Mission Name Program/Class year 1 2  4 5 6 7 Messenger Discovery 2004 X X X X X CLOUDSAT Earth System Science Pathfinder 2006 X X X X X GALEX Explorer 2003 X X X X X Swift Gamma-Ray Burst Mission Explorer/Medium-class Explorer 2004 X X X X X Spitzer Space Telescope (formerly Great Observatory 2003 X X X X X SIRTF) MRO Mars 2005 X X X X X STEREO Solar Terrestrial Probe 2006 X X X X X 2002a CONTOUR Discovery X X X X Deep Impact Discovery 2005 X X X X EOS-Aqua Earth Observing System 2002 X X X X MER Mars 2003 X X X X RHESSI Small Explorer 2002 X X X X EO-1 New Millennium 2000 X X X EOS-Aura Earth Observing System 2004 X X CALIPSO Earth System Science Pathfinder 2006 X X GRACE Earth System Science Pathfinder 2002 X X THEMIS Explorer/Medium-class Explorer 2007 X X Landsat-7 Landsat Program 1999 X X FAST Small Explorer 1996 X X TRACE Small Explorer 1998 X X TRMM Mission to Planet Earth 1997 X X Gravity Probe B Astrophysics 2004 X Genesis Discovery 2001 X Lunar Prospector Discovery 1998 X NEAR Discovery 1996 X Stardust Discovery 1999 X Mars Pathfinder Discovery - Mars 1996 X ICESAT Earth Observing System 2003 X SORCE Earth Observing System 2003 X ACE Explorer 1997 X HETE-II Explorer 2000 X IMAGE Explorer 2000 X MAP Explorer 2001 X FUSE Explorer (Origins) 1999 X 1998a MCO Mars X MGS Mars 1998 X 1999a MPL Mars X DS-1 New Millennium 1998 X SWAS Small Explorer 1998 X 1999a WIRE Small Explorer X

OCR for page 8
 SIZE AND HISTORIC CAUSES OF COST GROWTH TABLE 1.3 Continued Primary Reference Launch Mission Name Program/Class year 1 2  4 5 6 7 TIMED Solar Terrestrial Probe 2001 X EOS-Terra Earth Observing System 1999 b b X New Horizons (Pluto) New Frontiers 2006 X X X X Chandra X-ray Observatory Great Observatory 1999 X X X X (formerly AXAF) ACRIMSAT Solar Science 1999 X X Dawn Discovery 2007 X X X Fermi Gamma-ray Space Telescope NASA/DOE/Japan 2008 X X (formerly GLAST) IBEX Small Explorer 2008 X X SOFIA Astrophysics (airborne) 2007 X TWINS A/B Heliophysics 2008 X LCROSS ESMD 2009 X LRO ESMD/SMD 2009 X Kepler Discovery 2009 X 2010c Glory Earth Observing - Directed X 2009a OCO Earth System Science Pathfinder X Planck ESA 2009 X Herschel ESA Horizon 2000 Program 2009 X WISE Explorer/Medium-class Explorer 2009 X 2014c JWST Great Observatory X 2010c Aquarius Instrument X OSTM Joint NASA/CNES/NOAA 2008 X 2011c MSL Mars X CINDI Mission of Opportunity 2008 X Space Technology 7 New Millennium cancelled X Space Technology 8 New Millennium cancelled X M3-Foton Russian-ESA 2007 X 2014c MMS Solar Terrestrial Probe X 2011c NPOESS Preparatory Project NASA/NOAA/USAF X Cassini-Huygens NASA-ESA 1997 X Phoenix Mars Scout 2007 X AIM Small Explorer 2007 X Magellan Venus 1989 X NSCAT NASA/Japan 1996 X ACTS Communications 1993 X GOES I-M Operational weather satellites 1994-2001 X EUVE Explorer 1992 X XTE Explorer/Medium-class Explorer 1995 X Galileo Planetary 1989 X continued

OCR for page 8
 CONTROLLING COST GROWTH OF NASA EARTH AND SPACE SCIENCE MISSIONS TABLE 1.3 Continued Primary Reference Launch Mission Name Program/Class year 1 2  4 5 6 7 CGRO Great Observatory 1991 X HST Great Observatory 1990 X 1992a Mars Observer Mars X TDRS-7 NASA Communications 1995 X TOPEX NASA/CNES 1992 X Ulysses NASA/ESA 1990 X ASRM Space Shuttle cancelled X ATP Space Shuttle 2002 X TSS-1 Space Shuttle 1992 X FTS Space Station cancelled X Freedom (Space Station), now ISS Space Station 1998-2011 X AFE Technology development cancelled X Collaborative Solar Terrestrial International Solar-Terrestrial 1992 X Research Physics Global Geospace Science Program International Solar-Terrestrial 1994/1996 X (Polar and Wind missions) Physics Landsat-D Earth Observing 1982 X OMV Technology development cancelled X NOTE: Acronyms are defined in Appendix D. a M ission failed. b I nitially part of the study, but then dropped from the analysis. c P lanned launch date (has not yet occurred). Figure 1.2 also shows a substantial linear correlation (R2 = 0.64) between cost growth and schedule growth. This correlation would be even stronger if not for the effect of missions with fixed launch windows. For example, launch windows for missions to Mars open every 780 days, and mission staff are highly motivated to be ready to go during the assigned window. If that window is missed, the only alternatives are to cancel the mission or sit on the ground for 2 years waiting for the next window (resulting in a large increase in both cost and schedule). Hence, in Figure 1.2, it is not surprising to see that the Mars Reconnaissance Orbiter (MRO) mission, the Mars Exploration Rover (MER) mission, and the Comet Nucleus Tour (CONTOUR) mission (which had a single, 25- day launch window) had significant cost growth, even though schedule growth was zero or nearly so. In fact, the pressure that such missions experience to meet the launch window may drive cost growth higher than it would otherwise be if the mission had a more flexible launch window. For many missions, there is substantial correlation between cost and schedule growth. However, even when that is the case, schedule delays are not necessarily the primary cause of cost growth. In some cases, schedule delays are a secondary effect of other factors, such as a failure to adequately fund the program or problems with launch vehicle or launch pad availability. In these cases, additional effort is required to replan and manage the mission, and additional costs are incurred due to extension of supporting areas such as program review and cost accounting. In other cases, it may even be necessary to store the spacecraft and supporting systems and keep the launch and support crew trained and capable of supporting a launch. The costs associated with these efforts are sometimes referred to as “sustaining cost” or “capability retention cost,” and they represent real effort that is required for a successful program. In most cases, however, schedule growth and cost growth arise from development problems that require both time and money to solve. Earth and space science programs are very complex, with extensive interactions among

OCR for page 8
 SIZE AND HISTORIC CAUSES OF COST GROWTH tasks, so that schedule delays in one development area impact the schedule and scope of other tasks. Furthermore, delays in one task cause additional effort to replan and manage all of the interdependent project tasks. Thus, cost growth may occur in many areas of a project because of a schedule delay in one area or on one task. When that happens, the total cost growth might be attributed to schedule growth, but the root cause is a development problem in one or more areas. In any case, as discussed below, there are many causes of cost growth in addition to schedule growth. Absolute Cost Growth Additional insight can be gained by considering absolute cost growth (in dollars) in addition to relative cost growth (as a percentage of the initial estimate). Figures 1.3a and 1.3b show in decreasing order the absolute cost growth in excess of reserves for the 40 missions from Primary Reference 1. The two figures are identical, except that the first shows the distribution of Earth science and space science missions, and the second shows the distribu - tion of AO and directed missions. Figure 1.3c also shows the initial cost estimate and launch date for each of these missions.9 The average development cost for each of these 40 missions is $215 million. This does not include the cost of launch, mission operations, data analysis, or inflation. If savings from missions that under run their budgets are allowed to partially offset the cost growth of other missions, the total net absolute cost growth for these 40 missions is $1.3 billion. This corresponds to the total development cost of about 6 missions of average size. As is often the case, about 20 percent of the population causes 80 percent of the problem. In particular, with regard to the 40 missions examined by Primary Reference 1, 11 of them (28 percent of the total) account for about 80 percent of the total cost growth (in absolute terms), and 14 missions (35 percent of the total number of mis - sions) account for about 92 percent of the total cost growth; the other 26 missions (two-thirds of the total number) account for only 8 percent of the total cost growth (see Table 1.4). Based on data from 1970 through 1999, there is not a consistent trend for cost growth of NASA missions over time. As shown in Table 1.5, cost growth for NASA missions increased from the 1970s to the 1980s, but then decreased in the 1990s. More recently, Figure 1.3c and Table 1.4 indicate that, for this set of 40 missions, cost and cost growth for missions launched in 2003 and later are generally higher than for missions launched in 2002 and earlier. The 15 missions launched in 2003 and later had an average initial cost of $244 million, whereas the 25 missions launched in 2002 and earlier had an average initial cost of $146 million, and the 15 missions launched in 2003 and later, as a whole, have a much higher cost growth (29 percent/$72 million) than the 25 mis - sions launched in 2002 and earlier (6 percent/$8 million). In addition, all but 2 of the 14 missions with the most cost growth (in dollars) were launched in 2003 or later, and all but 3 of the 26 missions with the least cost growth were launched in 2002 or earlier. A comparison of Figures 1.2 and 1.4 shows that the slopes of the linear trend lines are nearly identical (y/x = 1.22 in Figure 1.2 and 1.23 in Figure 1.4), indicating that the relationship between schedule growth and cost growth is virtually the same for both small and large missions. However, the impact of even modest cost growth in a large mission can still be substantial. Figure 1.3b shows that both AO and directed missions experience significant cost growth (in absolute terms). Similarly, Figures 1.3a and 1.4 show that both Earth and space science missions experience a similar mix of cost growth in both absolute terms and as a percentage of initial cost. The 10 Earth science missions in this set of 40 projects exhibited a total cost growth of 13 percent (an average of $37 million per mission), while the 30 space sci - ence missions showed a total cost growth of 20 percent (an average of $31 million per mission). Primary Reference 6 concurs that Earth science missions are not significantly different from other Science Mission Directorate (SMD) missions in terms of cost or cost growth (and this is supported by Figure 1.4). Primary Reference 6 concludes that cost growth is more closely associated with increases in spacecraft mass and higher levels of mission complexity rather than with mission type. In any case, as difficult as it is to develop an accurate and consistent view of mis - sion costs and cost growth based on historic studies, it is even more difficult to determine with certainty the cause and effect relationship among the many factors involved. 9 For missions with multiple launches, the date of the first launch is listed in Figure 1.3c.

OCR for page 8
8 CONTROLLING COST GROWTH OF NASA EARTH AND SPACE SCIENCE MISSIONS 150% CALIPSO 140% 130% 120% EO-1 110% 100% y = 1.22x + 0.01 2 R = 0.64 90% 80% CLOUDSAT Percent Cost Growth HETE-II 70% SWAS SWIFT 60% GALEX Messenger 50% ICESAT HESSI 40% Gravity Probe B Type SIRTF STEREO III 30% TIMED See inset Type II 20% Earth Science Missions Space Science Missions Type10% I 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% EOS-Aqua Percent Schedule Growth TRMM -10% MGS Stardust NEAR -20% ACE -30%

OCR for page 8
 SIZE AND HISTORIC CAUSES OF COST GROWTH 30% Deep Impact WIRE 25% CONTOUR y = 1.22x + 0.01 Type II 2 R = 0.64 20% FAST Percent Cost Growth Genesis Landsat-7 MER 15% TRACE MRO DS-1 GRACE FUSE THEMIS 10% SORCE Type I IMAGE MCO/ EOS-Aura MAP MPL 5% Earth Science Missions Space Science Missions Lunar Prospector Mars Pathfinder 0% 0% 5% 10% 15% 20% 25% 30% Percent Schedule Growth Enlarged portion of Figure . showing names of missions classified as Type I and Type II. FIGURE 1.2 (opposite): Clustering of 40 NASA missions from Primary Reference 1 in terms of percent cost and schedule growth in excess of reserves: Type I, less than 15 percent; Type II, 15-30 percent; and Type III, more than 30 percent. The linear trend line shows the best fit to all the missions plotted in the figure. For the names of the missions classified as Type I and Type II, see the enlargement, above, of the lower left portion of the figure NOTE: Acronyms are defined in Appendix D. SOURCE: Cost and schedule data from Tom Coonce, NASA Headquarters, e-mail to committee member Joseph W. Hamaker, December 21, 2009, providing source data for Primary Reference 1.

OCR for page 8
0 CONTROLLING COST GROWTH OF NASA EARTH AND SPACE SCIENCE MISSIONS $200M $150M Earth Science Missions Space Science Missions $100M Absolute Cost Growth $50M EO A R St S- CE TR ust ar NS Aq MM EA d M G ua $0M R NT G TF C nes SI SW UD La IFT SA IC p I -7 MEL M rP C vity M LIP ro EO R ger C ER D ds TI S-A G AS H O/M TH GE M se O B ST -1 EO O SWED ra FU SS UR D SE W GE FA AC SOST IM P I TREM Lu AC M SA pa MC G LEX HO O is ra AP LO EO ee at ET P A -I ar r A S- es S be R R Mu Em En R T ct CE n IR e E A na E R s os 1 Pa pe IS E I th ct fin or T de r –$50M FIGURE 1.3a Ranking of 40 NASA science missions from Primary Reference 1 in terms of absolute cost growth in excess of reserves in millions of dollars (starting at the beginning of Phase B and through the end of Phase D, excluding launch, mis - -$100M sion operations, and data analysis), showing the distribution of Earth science and space science missions. NOTE: Acronyms are defined in Appendix D. SOURCE: Based on data from Tom Coonce, NASA Headquarters, e-mail to committee member Joseph W. Hamaker, December 21, 2009, providing source data for Primary Reference 1. Cost Growth by Phase One of the clear and consistent findings in the primary references is that cost growth does not accumulate uniformly across mission phases; rather, the bulk of cost growth occurs post-CDR (see Figure 1.5). Some post-CDR cost growth is driven by external factors, such as delays in the availability of launch vehicles. However, most post- CDR cost growth is due to internal project development issues, even though (1) CDR is intended to be the final milestone of the design phase and (2) spacecraft configuration should be frozen at CDR. CDR for many missions may be held prematurely—driven by schedule rather than driven by design maturity. CDR approval of an immature design can cause downstream problems during Phase D such as integration difficulties and late changes. Primary Reference 2 found that, excluding external impacts, cumulative average cost growth to CDR is only 7 percent, but this grows to 28 percent by launch. So 75 percent of cost growth occurs after CDR. Primary Reference 5 also analyzed the evolution of cost growth over time and concluded that “it is important to notice that, unlike the mass and power growth time trends, cost growth is typically not recognized until after CDR. This is counter to standard industry guidelines that recommend a decreasing percentage reserve on a reduced cost-to-go basis. The

OCR for page 8
 SIZE AND HISTORIC CAUSES OF COST GROWTH $200M $150M Directed Missions AO Missions $100M Absolute Cost Growth $50M EO A R St S- CE TR ust NS ar MM Aq EA d M G ua $0M G TF R NT La IFT SA C nes Lu AC M rP SI C vity M LIP ro SW UD D ds IC p I -7 G AS ST -1 C ER EO O TI S-A SW ED ra H O/M ME L TH GE M se O B EO R ger FU SS UR D SE W GE FA AC SOST MC IM P I TREM M SA pa G LEX HO ra O is AP ee at ar ro A LO EO ET P es S be S- A -I R R Mu Em En CE n na E e R T ct IR E A R s 1 Pa spe IS E I th cto fin r T de r –$50M –$100M FIGURE 1.3b Ranking of 40 NASA science missions from Primary Reference 1 in terms of absolute cost growth in excess of reserves in millions of dollars (starting at the beginning of Phase B and through the end of Phase D, excluding launch, mission operations, and data analysis), showing the distribution of AO and directed missions. NOTE: Acronyms are defined in Appendix D. SOURCE: Based on data from Tom Coonce, NASA Headquarters, e-mail to committee member Joseph W. Hamaker, December 21, 2009, providing source data for Primary Reference 1. substantial cost growth after CDR implies that a greater percentage reserve on cost to go should be held. Alter- nately, it may mean that cost growth is occurring earlier in the project lifecycle, but isn’t recognized until later” (p. 9).10 Similarly, Primary Reference 6 reported that the highest percentage of schedule growth occurs after the start of integration and testing, i.e., during Phase D, and that this phenomenon is consistent across Earth science, heliophysics, astrophysics, and planetary missions. If risk has not been sufficiently reduced by CDR, then cost and schedule uncertainty remains high, especially because the underlying causes of some post-CDR cost growth may have originated prior to CDR without being recognized. Figure 1.5 suggests that, for one set of 20 missions examined by the committee, there is a slight decrease in cost growth as we go from initial cost estimates at the start of the program to PDR from about 48 percent to about 35 percent cost growth. Thus about 10-15 percent cost growth can be attributed to a lack of design maturity pre-PDR. 10 This conclusion, from Primary Reference 5, implies that the cost growth that occurs post-CDR is disproportionately large compared to the percent of project funds that are expended post-CDR. The committee did not have the data necessary to verify this conclusion.

OCR for page 8
 CONTROLLING COST GROWTH OF NASA EARTH AND SPACE SCIENCE MISSIONS Initial cost—directed missions $950M Initial cost—AO missions Cost growth—14 missions with most cost growth $900M Cost growth—26 missions with least cost growth These 14 missions together account $850M for 92% of the total cost growth for $800M all 40 missions in this figure $750M Initial Cost Estimate / Absolute Cost Growth $700M $650M $600M $550M $500M $450M These 26 missions together account $400M for just 8% of the total cost growth for all 40 missions in this figure $350M $300M $250M $200M $150M $100M $50M $0M -$50M G TF M LIP rob M sen , 2 B, EO R, er, 06 004 ST 1, 03 004 C E0 SW U , 2 M SA pac 99 EO , 2 20 200 TI -A 05 FU SS 20 1 FA AC 999 M RC 996 M E- L, TH GE 01 TR MI 200 M ar 19 7 N S, 99 SI C vity 00 La IFT AT 6 D sa 00 00 IC p I -7, SWED a, G AS 200 04 G EX 99 C es 200 R tou , 20 D E, 20 W 1, 1 999 G E, 98 SO T, 20 H /M 0 IM P, 2 20 98 Lu E 20 St s P osp TR rdu hfin cto M M 19 er, 199 AC R, 98 EO , 1 996 LO O EA 19 7 ra , 2 A P3 es SO e ee t 4 6 Hr0 S- 1 02 E g02 R T, t, on is 3 C E, ET P 0 A II, 19 ar Pr 98 en , 8 R1 M ur AL , 1 1 G , 1 99 19 8 R IR 9 a at e E m 19 nd , 2 , 2 A 0 00 n,0 ER 200 S E, S I, 02 E, AC S, 0 M st, d r, -02 S0 35 E1 O E, O2 S- 997 Aq 2 D0 , 20 1 02 S0 ua ,2 0 3 00 2 96 FIGURE 1.3c Ranking of 40 NASA science missions from Primary Reference 1 in terms of absolute cost growth in excess of reserves in millions of dollars (starting at the beginning of Phase B and through the end of Phase D, excluding launch, mission operations, and data analysis), with initial cost and launch date for each mission also shown. NOTE: Acronyms are defined in Appendix D. SOURCE: Based on data from Tom Coonce, NASA Headquarters, e-mail to committee member Joseph W. Hamaker, December 21, 2009, providing source data for Primary Reference 1. TABLE 1.4 Breakdown of Cost and Cost Growth for the 40 missions from Primary Reference 1 Total Initial Cost Total Cost Growth (billion $) (%) (billion $) (%) 14 missions with the most cost growth 3.9 53 1.2 92 26 missions with the least cost growth 3.4 47 0.1 8 Total for all 40 missions 7.3 100 1.3 100 Average Initial Cost Average Cost Growth (million $) (million $) (%) 15 missions launched in 2003 and later 244 72 29 25 missions launched in 2002 and earlier 146 8 6 SOURCE: Based on data from Tom Coonce, NASA Headquarters, e-mail to committee member Joseph W. Hamaker, December 21, 2009, providing source data for Primary Reference 1.

OCR for page 8
 SIZE AND HISTORIC CAUSES OF COST GROWTH 150% CALIPS O 140% 130% 120% EO-1 110% 100% 90% Percen t Co st Growth CLOUDSAT 80% y = 1.23x + 0.13 2 70% R = 0.63 SW IFT 60% Messenger 50% ICESAT Ty pe Gravity III Probe B 40% SIRTF STEREO Deep Impact 30% Ty pe MER II Earth Scie nce M issions 20% Lands at-7 Space Scie nce M issions MRO EOS- 10% Ty pe Aura I 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Percent Schedule Growth FIGURE 1.4 Clustering of the 14 NASA missions from Primary Reference 1 that accounted for 92 percent of the total cost growth in dollars, in terms of percent cost and schedule growth in excess of reserves. NOTE: Acronyms are defined in Ap - pendix D. SOURCE: Cost and schedule data from Tom Coonce, NASA Headquarters, e-mail to committee member Joseph W. Hamaker, December 21, 2009, providing source data for Primary Reference 1.

OCR for page 8
 CONTROLLING COST GROWTH OF NASA EARTH AND SPACE SCIENCE MISSIONS TABLE 1.5 Decadal Trends in Cost Growth for NASA Missions Cost Growth Average (%) Median (%) 1970s 43 26 1980sa 61, 83 50, 60 1990s 36 26 a T he source cited two different values for cost growth of NASA missions in the 1980s, based on two different prior studies. SOURCE: Based on data from Schaffer, 2004. 200.0% CDR to Launch PDR to CDR Start to PDR 150.0% 100.0% 50.0% 0.0% EO C C ST SW O G M AI nge N D G O Sp Ke FE O IB D M LR G AL LO ew ee AL R ve S AW es C EN R M EX ER itz pl R O -1 AC O IF O p ra se IP EX U M er ES H er N T E Im ll D SO E I/G or SA I pa iz LA r on T ct ST s -50.0% FIGURE 1.5 Cost growth of missions by program phase, showing that most cost growth occurs after critical design review. NOTE: Acronyms are defined in Appendix D. SOURCE: Based on data from Claude Freaner, NASA, e-mail to committee member Joseph W. Hamaker, January 6, 2010, providing source data for Freaner et al. (2010). Surprisingly, the cost growth between PDR and CDR is not very large: only about 5 percent for AO missions and about 12 percent for directed missions. Therefore the majority of cost growth, typically 20 to 30 percent of the initial estimate, occurs post-CDR. This is an important finding, which is supported by many of the primary references. This finding is symptomatic that a number of projects experience “surprises” post-CDR that significantly increase mission cost, even though the design of the instruments and spacecraft is supposed to be “frozen” at CDR. The reasons for the post-CDR cost growth will be discussed in the following sections.

OCR for page 8
 SIZE AND HISTORIC CAUSES OF COST GROWTH CAuSES OF COST GROWTH Finding. Causes of Cost Growth. Past studies identify a wide range of factors that contribute to cost and schedule growth of NASA Earth and space science missions. The most commonly identified factors are as follows: • Overly optimistic and unrealistic initial cost estimates, • Project instability and funding issues, • Problems with development of instruments and other spacecraft technology, and • Launch service issues. In addition, any problem that causes schedule growth contributes to and magnifies total mission cost growth, and cost growth in one mission may induce organizational replanning that delays other missions in earlier stages of implementation, further amplifying overall cost growth. Effective implementation of a comprehen - sive, integrated cost containment strategy, as recommended herein, is the best way to address this problem. The historic causes of cost growth in the primary references are discussed below and summarized in Table 1.6.11 In addition to the four most common causes listed in the above finding, the primary references have identified a wide range of additional factors that contribute to cost growth, such as poor contractor performance and a tendency to over-engineer. Primary Reference 10, which examined DOD projects, also noted the erosion of capabilities to lead and manage the space acquisition process. The committee generally relied on the assessments made in the primary references, rather than begin its assess- ment using raw data collected by the authors of the earlier studies, in part because the raw data were not available to the committee. However, spot checks of data for key missions supported the conclusions reached by the primary references and this study. For example, the Spitzer Space Telescope mission (formerly known as the Space Infrared Telescope Facility, SIRTF) had the largest absolute cost growth of the 40 missions assessed by Primary Reference 1. Cost growth problems encountered by Spitzer included many of the factors cited in the above finding and in Table 1.6: early planning deficiencies; problems with development, integration, and/or testing of the spacecraft as well as all three major instruments; launch vehicle problems; schedule delays associated with all of the above; and cost growth of project-level management functions (Mlynczak and Perry, 2009). The primary references are generally consistent in their conclusions, although, in some cases, different studies disagree about the significance of the impact of some factors on cost growth. For example, Primary Reference 6 uses data-driven correlation analyses to assert that there is little or no correlation between mission cost growth and planned cost reserves or the percent of funds spent in Phase B. Primary References 2 and 4 (which, for the most part, examined the same missions as Primary Reference 6) concur that there is no correlation between cost growth and planned cost reserves. However, several committee members have observed that cost reserves are often “manu - factured” during the early stages of a program by unrealistically reducing baseline budgets (without reserves), so that the total budget can include a cost reserve of the expected magnitude without descoping the mission. This manufactured reserve does not represent a true contingency fund in excess of likely project costs. The committee concludes that the lack of correlation in some studies between cost reserve and cost growth (in excess of the cost reserve) indicates that more care is needed in establishing cost reserves in excess of realistic cost estimates. Primary References 2, 4, and 6 also point out a lack of correlation between funds spent in Phase B and cost growth. Primary Reference 2, however, recommends including a level of cost reserves that is commensurate with mission implementation risk. In addition, during Phase B, many projects are still in a competitive mode, and competitive pressures encourage (overly) optimistic assessments of the cost and schedule impacts of addressing uncertainties and overcoming potential problems. One reason to designate a larger percentage of mission budgets for Phase B technology development is to better 11 The committee’s assessment of historic causes of cost growth is consistent with the conclusions documented in Primary Reference 2, which identifies four key drivers of cost growth: over-optimism early in formulation, unstable or inadequate initial funding profile, instrument development complexity, and launch service issues.

OCR for page 8
 CONTROLLING COST GROWTH OF NASA EARTH AND SPACE SCIENCE MISSIONS TABLE 1.6 Causes of Cost Growth Based on the Primary References Causes of Cost Growth Page Number in Primary Reference Initial Cost Estimates 1 2 3 4 5 6 7 9 10 Inherent optimism in initial design and estimates (in part because 9 38 4 15 1 12 58 2 of the emphasis that is placed on science, not cost; in part because 6 5 mission complexity is underestimated), which leads to unrealistic proposals and estimates Incomplete initial budget estimates (e.g., cost of launch services, 17 58 mission operations, and/or data analysis not included) Cost has replaced mission success as the primary driver in managing iii acquisition processes, resulting in excessive technical and schedule risk. Assumptions about heritage hardware, software, and commercial-off- 6 the-shelf equipment did not materialize. The space acquisition system is strongly biased to produce iii unrealistically low cost estimates throughout the acquisition process. These estimates lead to unrealistic budgets and unexecutable programs. Government budget space acquisition programs should use a most probable (80/20) cost, with a 20-25 percent management reserve for development programs included within this cost. Inflation 17 Project Instability and Funding Issues Program instability/changes (to mission requirements, spacecraft, 25 6 11 58 2 instruments, launch vehicles, upper stage propulsion systems, 12 15 trajectories, and/or operations) Technical and programmatic uncertainty at the beginning of a project 15 36 6 12 (not enough time or resources available in phases A and B; TRL 12 13 lower than claimed; risk identification and mitigation prior to CDR 19 needs attention; uncertainty in spacecraft mass, power, pointing accuracy, data transmission rates, and so on) Budget constraints, lack of stable funding, and/or inadequate initial 25 6 15 11 funding profile 15 Weak independent validation of cost and schedule 6 Inadequate definition of technical and management aspects of 11 projects prior to NASA and OMB approval Inadequate cost and schedule reserves 12 6 Technical complexity 15 Adverse impacts of financial problems experienced by other NASA 6 missions Development of Instruments and Other Spacecraft Technology Instrument development problems caused, for example, by 11 29 6 8 12 117 instrument designs that lack technical details and/or fail to identify 15 13 technical challenges The primary contributor to internal cost growth (i.e., growth caused 4 by factors within the control of a given project) is instrument 11 development problems. Increase in spacecraft development costs, especially integration and 31 114 test costs Increases in spacecraft mass, power requirements, and complexity 4

OCR for page 8
 SIZE AND HISTORIC CAUSES OF COST GROWTH TABLE 1.6 Continued Causes of Cost Growth Page Number in Primary Reference Increases in payload cost (except for planetary missions) 114 117 Launch Service Issues Launch service issues (primarily higher launch vehicle prices or 25 6 15 selection issues) Launch vehicle delays resulting in overall schedule delays 25 6 17 The primary contributor to external cost growth (i.e., growth caused 4 by factors beyond the control of a given project) is problems with the readiness of the launch vehicle. Other Factors Schedule growth that leads to cost growth 11 38 12 19 59 Poor contractor performance 15 Industry has failed to implement proven practices on some programs. 4 Industry guidelines do not in general adequately predict the 13 uncertainty in the initial physical and programmatic parameters claimed in proposals. Tendency to over-engineer 6 Increase in project-level management costs 27 8 Failure to achieve anticipated cost and/or schedule savings from 30 6 mission descopes Increases in mission operations and data analysis costs for science 44 enhancements and/or extended missions Hardware from foreign partners (late delivery or increased costs) 36 6 17 ITAR requirements 44 Department of Defense capabilities to lead and manage the space 3 acquisition process have seriously eroded over time. NOTE: Primary Reference 8 is focused on reducing the absolute costs of NASA space science missions. It does not directly address cost growth, and so its results are not included in this table. Primary References 9 and 10 focus on Department of Defense systems. define system designs, reconcile mission goals and budgets, retire risk prior to PDR, and reduce uncertainties in the project plan. If the baseline cost is established after PDR (that is, after the end of Phase B), increased costs identi - fied in Phase B do not show up as cost growth. Primary References 2, 4, and 6, however, define baseline mission costs at the beginning of Phase B. As a result, technology investments made during Phase B cannot improve the understanding of the technical baseline, which is necessary to improve the accuracy of the initial cost estimate. Initial Cost Estimates The primary references have concluded that mission requirements are typically formulated on the basis of science return on the particular mission much more than cost. In addition, it is easy to underestimate the complex - ity of mission hardware and software and/or make unrealistically optimistic assumptions about how easy it will be to develop and manufacture systems for a new mission based on similarities to systems used in past missions. In a competitive environment, bidders are motivated to argue that science goals can be achieved as inexpensively as possible in order to maximize the science return for a given assumed budget. In addition, initial cost estimates may not include all costs, such as the cost of launch services, mission operations, and/or data analysis. As a result, initial cost estimates and mission proposals tend to be overly optimistic and unrealistic to the point that it becomes very difficult to implement them within the initial budgets. Yet, once projects get under way, cost constraints

OCR for page 8
8 CONTROLLING COST GROWTH OF NASA EARTH AND SPACE SCIENCE MISSIONS may become so important that efforts to minimize cost growth may result in excessive technical and schedule risk. Inflation also becomes a significant factor for missions with a long life cycle, either by design or because of schedule growth. Project Instability and Funding Issues The primary references have concluded that project instability (associated with changes to mission require - ments, spacecraft, instruments, launch vehicles, upper stage propulsion systems, trajectories, and/or operations) are a major cost growth factor. Budget constraints, lack of stable funding, inadequate initial funding profiles, insufficient cost and schedule reserves, and weak independent validation of cost and schedule likewise contribute to cost growth. In many cases, project instability and funding issues are intertwined. At the beginning of a project, actual technology readiness levels may be lower than anticipated, leading to uncertainty in mission requirements in terms of spacecraft mass, power, pointing accuracy, data transmission rates, and overall mission complexity. Technical and programmatic uncertainty tends to persist if the funding in phases A and B is insufficient to resolve risks adequately. As a result, missions are sometimes approved by NASA and OMB before cost and schedule requirements are well understood. Even when a particular mission is well understood and well planned, changes and delays are sometimes imposed when, in a constrained cost environment, cost growth experienced by one NASA mission may require NASA to delay or curtail other missions. Other sources of external changes, which are beyond the control of a given project manager, are changes in overall requirements or processes that may be imposed, for example, by an evolving view of what constitutes acceptable risk. As unanticipated problems and changes arise, either internally or externally, the resulting instability tends to redirect the attention of the project management team from the technical challenges of implementing the mission to work on replanning efforts. These replanning activities themselves generally result in increased cost and schedule. Project instability and funding issues can be especially difficult to avoid for large missions that rely on annual funding appropriations for a mission life cycle that may span a decade or more. Development of Instruments and Other Spacecraft Technology Earth and space science satellites are highly customized, and their design is often driven by the nature of their instruments and their requirements in terms of power, mass, pointing accuracy, thermal control, and so on. The primary references conclude that instrument development problems may be the largest element of mission cost growth that can be attributed to factors within the control of a given project. Common problems are instrument designs that lack technical details and/or fail to identify technical challenges. Cost growth can also arise from problems in the development of other spacecraft technologies and systems, especially in the area of integration and testing and as a result of increases in spacecraft mass, power requirements, and/or complexity. Launch Service Issues Launch service issues include higher launch vehicle prices and/or launch vehicle selection issues. In addition, launch vehicle delays are a common cause of delays in the overall schedule. In fact, Primary Reference 1 concluded that problems with the readiness of the launch vehicle are the primary contributor to external cost growth (i.e., growth caused by factors beyond the control of a given project). DIFFERENCES BETWEEN EARTH AND SPACE SCIENCE MISSIONS A casual look at partial cost data or personal experience may indicate different cost growth potential for mis - sions from different disciplines, because different classes of missions face different challenges. For example, Earth science missions typically have more complex, more costly, and more massive instruments than space science missions do. Earth science missions also have more stringent requirements in terms of pointing accuracy, resolu - tion, stability, and so on, although astrophysics missions also have stringent pointing requirements, and planetary

OCR for page 8
 SIZE AND HISTORIC CAUSES OF COST GROWTH spacecraft and instrument technology must be able to survive long cruise phases and radiation environments that are sometimes quite extreme. Earth science missions also dedicate a higher fraction of mission costs to instru - ments than do other missions, their instruments cost the most, and their spacecraft are typically more complex than most space science spacecraft are. Space science missions that leave Earth orbit have greater incentives to minimize spacecraft mass and power, and the average cost and the average spacecraft mass for space science missions are less than those for Earth science missions. However, as specifically addressed by Primary Reference 6, Earth science missions have not shown a systematic difference in cost or cost growth compared to other SMD missions.12 Both Earth and space science missions have shown good correlation between (1) instrument schedule growth and instrument cost growth, (2) instrument cost/schedule growth and mission cost/schedule growth, and (3) the absolute costs of instruments and instrument complexity. Primary Reference 1 and other studies (Bitten, 2008; Bearden, 2008) confirm that the complexity of Earth science missions results in long development cycles (and associated large development costs as a fraction of total mission costs). Because Earth science missions rarely have highly restricted launch windows, it is easier to post - pone launch (and slip the mission schedule) as necessary to solve technical problems. Restricted launch windows motivate mission teams to avoid schedule delays if at all possible. For some missions, this has involved descoping. Schedule-constrained missions (which are mostly planetary missions) typically respond to late-breaking technical problems not by increasing schedule but by increasing the workforce (and costs). In some cases, despite the best efforts of the mission staff, risk created by late-breaking problems is not fully resolved prior to launch. As a result, schedule-constrained missions tend to have less cost growth—and higher failure rates—than do other missions. 12 For the 40 missions analyzed in Reference 1, the average of the cost growth values for the 10 Earth science missions was 43 percent, almost twice as high as the average cost growth of the 30 space science missions (22 percent). On the other hand, the cost growth of the 10 Earth science missions, as a whole, was just 13 percent, compared to 20 percent for the space science missions. Overall comparisons are po - tentially misleading because one very large Earth science mission, EOS Aqua, had a final cost much less than the initial estimate because of a mission descope. In addition, although the average cost of the Earth science missions was almost twice the average cost of the space science missions ($274 million versus $152 million), the median initial cost of the Earth science missions was actually less than the median cost of the space science missions ($101 million versus $128 million). Regardless, in looking at the real source of the cost growth problem—which is to say, the 14 missions in this set of 40 that accounted for 92 percent of the total cost growth—6 were Earth science missions and 8 were space science missions.