To address the statement of task, the committee reviewed the literature and heard from many speakers involved in defense system acquisition programs.9 The committee also drew on the expertise and extensive knowledge possessed by its members, who have had many years of personal experience both in defense acquisition programs and in the practice of SE.

The study sponsor did not provide the committee with a list of programs that are considered “successful” or “unsuccessful” by DOD. In addressing statement of task item A, the committee used its own judgment and examined a number of existing case studies of DOD programs in the literature and developed several new ones that it felt illustrate both successful and unsuccessful application of SE, focusing particularly on the pre-Milestone A and early phases. The committee notes, however, that many programs that are judged successful in retrospect were considered to be in trouble during their execution. Thus the perception of “successful” and “unsuccessful” programs can change with time and perspective. For purposes of this report, however, the committee does not believe that a program that requires far more time or money to develop than it would have in the 1960s and 1970s can be judged successful even if it ultimately meets its objectives. In a world in which the threats and technology are both evolving ever more rapidly, one cannot be satisfied with excessive deployment times and costs. The lessons learned from these case studies are summarized in Chapter 2.

The committee quickly determined the near impossibility of quantitatively isolating, testing, and proving direct causal links between pre-Milestone A and early-phase SE and later program cost, schedule, and performance outcomes. Many studies have searched for and proposed actions to address the root causes of the cost, schedule, and performance problems that seemingly have become the norm for current defense acquisition programs. Consistently, such studies have found that the causes and their effects are complex and interrelated.10 The committee believes that high-quality pre-Milestone A and early-phase SE certainly contributes to later positive outcomes; however, available data did not allow the contribution of that SE to be reliably isolated from that of other factors such as requirements maturity and stability, funding stability, and domain knowledge of the development team. In that context, the committee addressed statement of task item A(2) qualitatively.

The committee addressed statement of task items B and C together by developing a checklist of items that constitute good SE practice during both the pre-Milestone A and pre-Milestone B periods. This checklist is presented in Chapter 4 (Box 4-1). As required in statement of task item B, the checklist items are divided into those that should be completed prior to the analysis of alterna-

9

See Appendix B for a list of speakers and the presentations made to the committee.

10

Ronald Kadish, Gerald Abbott, Frank Cappuccio, Richard Hawley, Paul Kern, and Donald Kozlowski, 2006, Defense Acquisition Performance Assessment. Available at http://www.acq.osd.mil/dapaproject/documents/DAPA-Report-web/DAPA-Report-web-feb21.pdf. Last accessed on April 2, 2007.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement