C
Findings and Recommendations from Primary References

In addition to identifying the causes of cost growth, the primary references (see the References chapter) have made dozens of specific findings and recommendations. This appendix summarizes the findings and recommendations contained in these historic studies. In some cases the findings and recommendations listed are quoted from other prior studies.

TABLE C.1 Cost Growth Findings from the Primary References

Finding

Page Number Primary Reference

1

2

3

4

5

6

7

9

10

Cost growth and schedule slips are nearly universal among the projects studied.

 

2

 

 

 

 

 

 

 

The highest percentage schedule growth tends to occur after the start of spacecraft integration and test.

 

 

 

 

 

71

 

 

 

There is no discernable correlation between actual cost performance and planned cost reserve level.

 

33

 

10

 

67

 

 

 

There is no discernable correlation between actual cost performance and percent of funds spent during Phase B formulation.

 

34

 

10

 

63

 

 

 

For the projects in this study, there is no discernable correlation between actual cost performance and percent of funds spent up to CDR.

 

35

 

 

 

 

 

 

 

There is a possible correlation between completing substantial activity prior to CDR and lower cost growth for the total development effort.

 

43

 

 

 

 

 

 

 

There appears to be little-to-no correlation between total flight system dry mass growth and Phase BCD cost growth or between instrument mass growth and instrument Phase BCD cost growth for the 30 SMD missions and 100+ individual instruments included in this study.

 

 

 

 

 

110

 

 

 



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 58
C Findings and Recommendations from Primary References In addition to identifying the causes of cost growth, the primary references (see the References chapter) have made dozens of specific findings and recommendations. This appendix summarizes the findings and recommenda - tions contained in these historic studies. In some cases the findings and recommendations listed are quoted from other prior studies. TABLE C.1 Cost Growth Findings from the Primary References Page Number Primary Reference Finding 1 2 3 4 5 6 7 9 10 Cost growth and schedule slips are nearly universal among the 2 projects studied. The highest percentage schedule growth tends to occur after the 71 start of spacecraft integration and test. There is no discernable correlation between actual cost 33 10 67 performance and planned cost reserve level. There is no discernable correlation between actual cost performance 34 10 63 and percent of funds spent during Phase B formulation. For the projects in this study, there is no discernable correlation 35 between actual cost performance and percent of funds spent up to CDR. There is a possible correlation between completing substantial 43 activity prior to CDR and lower cost growth for the total development effort. There appears to be little-to-no correlation between total flight 110 system dry mass growth and Phase BCD cost growth or between instrument mass growth and instrument Phase BCD cost growth for the 30 SMD missions and 100+ individual instruments included in this study. 8

OCR for page 58
 APPENDIX C TABLE C.1 Continued Page Number Primary Reference Finding 1 2 3 4 5 6 7 9 10 Earth Science missions do not show a systemic difference in cost 5 or cost growth as compared to other SMD missions. Missions from 37 each SMD division experience cost growth; total growth in dollars is greater for missions that have greater baseline costs. There is no correlation between mission cost growth and SMD 5 division, acquisition mode, contractor type, Phase B investment or cost reserve. Directed missions cost more than AO-acquired missions do and are, 47 in general, more complex and more massive. AO-acquired and directed missions have comparable schedule 47 growth in months. In general, cost growth is of a similar percentage for AO-acquired 47 and directed missions, although it may be a larger dollar increase for directed missions since it is against a larger base cost. Mission cost growth correlates strongly with payload cost growth. 5 Payloads from all four SMD divisions experience significant cost 79 growth. Instrument cost shows good correlation to a multivariable 5 instrument Level-of-Difficulty measure. Earth science instruments cost less per unit mass but are more 5 massive, have more stringent requirements and higher levels of difficulty, and therefore are more costly overall than are instruments in the other SMD divisions. There is no systemic difference in spacecraft cost regimes between 5 Earth and space science missions. Department of Defense capabilities to lead and manage the iii acquisition process have seriously eroded. The government should address acquisition staffing, reporting integrity, systems engineering capabilities, and program manager authority. While the space industrial base is adequate to support current iv programs, long-term concerns exist. A continuous flow of new 4 programs—cautiously selected—is required to maintain a robust space industry. Without such a flow, the workforce, as well as critical national capabilities in the payload and sensor areas, are at risk. NOTE: The numbers in each cell of the table indicate the page number(s) in the respective report where the item is discussed. Primary Reference 8 is focused on reducing the absolute costs of NASA space science missions; it does not directly address cost growth, and so its results are not included in this table. Primary References 9 and 10 focus on Department of Defense systems. AO, announcement of opportunity; CDR, critical design review; SMD, Science Mission Directorate.

OCR for page 58
0 CONTROLLING COST GROWTH OF NASA EARTH AND SPACE SCIENCE MISSIONS TABLE C.2 Cost Growth Recommendations from Prior Studies Page Number in Primary Reference Recommendation 1 2 3 4 5 6 7 9 10 NASA should be realistic with itself, Congress, and the public in 12 terms of the goals, capabilities, costs, schedule, and technical risks of a new project. Improve technical and programmatic definition at the beginning of 15 38 7 11 12 11 a project (increase time and funding for Phase A and Phase B and extend them as necessary for complex projects) to allow more time for development of technology, baseline costs, funding profiles, and the overall implementation plan before making significant investments in other mission elements. SMD should work with projects beginning at the start of Phase 20 B, or earlier if possible, to establish a credible baseline plan that fits within the available funding with sufficient margin instead of waiting for projects to present a plan at the end of Phase B. Require more robust initial cost and schedule estimates, including 17 38 7 11 12 project-level management costs. Do a better job of independently validating costs and schedule (this 7 12 5 includes improving cost and schedule estimating tools). SMD should perform independent cost estimates on all decadal 12 planning and similar exercises. Independently validate instrument resources and resulting 12 spacecraft resources needed to meet mission requirements (cost estimators cannot be expected to validate system designs). Give more attention to risk identification and mitigation prior to 12 CDR. Select AO missions with lower risk. 7 Remove funding constraints from AOs for more credible funding 39 7 12 profiles for initial planning. For AO missions, consider funding profiles, mission-specific launch 39 12 date constraints, and program funding availability when making selections. For AO missions, consider alternates to down-selecting to two 7 finalists, delay setting the cost cap until PDR, and require proposes to address cost and schedule feasibility. Direct that source selections evaluate contractor cost credibility and 5 use the estimate as a measure of their technical understanding. Hold basis-of-estimate discussions at the start of Phase A. 7 Spend more money on research and development programs to 7 mature technology readiness levels. Support early instrument development to reduce risk (phased 15 7 development approach). Carefully evaluate design heritage credits. 7 Improve tools for early estimation of science instrument costs. 41 13 12 Increase cost reserves. 15 7 Minimize or eliminate blanket requirements on the level of cost 12 reserves. Instead, match reserves to implementation risk. 14 Hold a budget reserve at the program level at headquarters, in part 43 20 12 to address impacts from changes external to the projects (such as changes in launch costs).

OCR for page 58
 APPENDIX C TABLE C.2 Continued Page Number in Primary Reference Recommendation 1 2 3 4 5 6 7 9 10 Ensure adequate funds (and reserves) to cover cost of spacecraft 42 integration and test. Increase cost reserves for missions relying on foreign hardware and 43 identify backup options. Increase funded schedule reserves. 16 7 Establish and maintain appropriate funding profiles and stable 7 12 funding. Ensure mission are properly scoped. 13 Calculate and report estimates to complete monthly. 17 Manage to schedule. 17 Make more effective use of Earned Value Management, beginning 17 40 12 as early in the development cycle as possible. Identify and disseminate project management best practices. 41 Establish a handbook or memorandum of understanding that details 7 the relationship between PIs and project managers. Program managers should establish early warning metrics and 6 report problems up the management chain for timely corrective action. Avoid changes and redirection, especially after PDR. 39 7 12 Use four-party agreements (among project manager, principal 7 33 investigator, NASA headquarters, and prime contractor) or some other process to control requirements. Include cost and schedule status details at CDR, Assembly 11 Readiness Review, Pre-Environmental Review, and Mission Readiness Review. Assess launch site capabilities before start of Phase B. 39 7 Select the expandable launch vehicle as early as possible and 39 7 minimize changes. 20 Maintain a cost reserve at headquarters to cover unforeseen issues 39 7 affecting launch vehicle price and launch site costs. Develop estimates of cost and schedule savings from descopes 42 7 earlier and with more rigor. Establish a single source of cost data, with routine data collection 44 from missions. Improve NASA’s Cost Analysis Data Requirement (CADRe) system 13 12 by including new missions and expanding the instrument subsystem to identify which instrument types have had the highest historical resource growth. Cancel missions for poor performance. 7 Conduct mission postmortem reviews. 7 NOTE: The numbers in each cell of the table indicate the page number(s) in the respective report where the item is discussed. Primary Reference 8 is focused on reducing the absolute costs of NASA space science missions; it does not directly address cost growth, and so its results are not included in this table. Primary References 9 and 10 focus on Department of Defense systems. AO, announcement of opportunity; CDR, critical design review; PDR, preliminary design review; SMD, Science Mission Directorate.