Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
CHAPTER Vl Contracting Issues The proposed SSA computer-based information system would be among the largest ever procured by a federal agency. Similar procurements that have been attempted in the past have been largely by the military ser- vices. These have often been plagued by schedule and funding overruns, at times of significant magnitude. Not surprisingly, therefore, the problems of large computer-information systems have come to public attention. Thus, in March 1977, the General Accounting Office issued a report to the Congress on the problems raised in the acquisition, management, and use of federal data processing systems during the 11 years from 1965 to 1976.* The SSA should take every feasible step early in the planning process, and then on a continuing basis, to minimize the risk of schedule slippage. Consider the analogy of someone who wants to build a new house. The prospective owner either hires a general contractor who then hires, supervises,and manages all subcontractors, or the owner acts as his own general contractor. The tradeoffs are fairly obvious. In the first case, the owner incurs additional cost and has little, if any, impact on the behavior of subcontractors. In the second, he must expend more effort, but is in continuous, close touch with the action. In either case, however, the owner must be strong. He is not likely to achieve his "dream house" if he is poorly or narrowly informed on rele- vant subjects. Most large federal system procurements, including computer-based ones, have so far been in the Department of Defense. Thus, character- istically, in the past, the Air Force has historically awarded a prime contract for a new aircraft or missile system, and the successful bidder has then dealt with second and third tier subcontractors. In such cases, the prime contractor was the sole interface with the Air Force management team, which reviewed the prime contractor's effort largely for schedule, *General Accounting Office, Problems Found with Government Acquisition and Use of Computers From November 1965 to December 1976. FGMSD-77-14, March 15, 1977. 54
55 budget and performance. As electronics, computers, and software subsystems have grown increasingly larger and more complex in a modern aircraft or missile system, the Air Force found that its management visibility via the prime contractor has been inadequate to perceive the problems that often arose, much less to respond with remedial actions. The Air Force has reacted in two ways. On missile programs, it retained an integrating contractor. The integrating contractor has been prohibited from supplying hardware and, in effect, became an extension of the Air Force management structure, but with broader skills and a much higher manpower effort. Alternatively, on more recent aircraft programs, the Air Force has identified certain critical participants and put them in an "associate contractor" status--i.e., they report directly to the Air Force for management rather than through the prime contractor. Computer and avionics subcontractors are frequently so designated. Adequate management visibility by SSA into all facets of its pro- curement is mandatory for a successful program. Success means "on-time, on-budget and meeting functional specifications." The SSA has various means for assuring good management. Whichever one is chosen, there can be no substitute for an in-house cadre skilled in all relevant technol- ogies--computers, software, communications, training--and with experience in managing development programs. A major factor contributing to the failure of some past large computerized systems has been the attempt to implement a modern on-line, third generation system with personnel having experience only in batch computer operations. What was needed was an understanding of develop- ment management. This involves constant attention to detail, frequent and periodic program reviews, constant comparison of actual progress against the schedule (frequently expressed as a PERT chart), and prompt remedial action on problems--with support from specialized information systems collecting the detailed data necessary for full-depth visibility. There are a few elements that have proved particularly significant. The first, system requirements or functional specifications, is especially troublesome for computer systems and software. Historically, the procuring agency issues an REP for a system, often defined very generally and characterized by a few numerical parameters. For well understood technologies that have been used long enough--e."., airframe or propulsion systems--the responses by vendors can be satisfactory. For information systems, such an REP can sometimes create problems because each vendor will respond with his own idea of how to solve the problem. However, federal policy mandates that system procurement define only the objectives that a system must accomplish rather than specify the means to achieve the objectives. The intent is to promote innovation and creativeness among suppliers. Inadequate, incomplete, or poor requirements can only lead to bids that are unrealistic, especially from less experienced or naive vendors --or ones who deliberately bid low to "buy-in" to new business. Even with the most thorough examination by source-selection personnel, proposal deficiences can be missed and the agency can be forced by government regulations to select a bid that is in reality too low, but may not be seen as such.
56 The Air Force regularly uses in-house experts to estimate the cost of RFP's for major systems so that it obtains a good idea of what systems should cost and can be alert to the underpriced proposal. Some federal program overruns have resulted from poorly stated requirements or, what is equally bad, from requirements that are con- stantly changed by the contracting agency as development is in progress. Over the several years required to design and implement a large system, external events may occur that cause changes in requirements--e."., court verdicts, legislation, and political decisions. Therefore, the plan for implementation must accommodate a system of sufficient size and flexibil- ity that such developments do not make it obsolete or inadequate. Software has been a special problem with regard to establishing good requirements and a priori estimates of time and---funding requirements. Fixed price software contracts have almost always proved to be trouble- some, unless the design of the software had already been established or the new product had been a minor variation on a software task that the contractor had already done. Some consideration is being given by the Air Force to include the purchase of a design--or even two--before firm schedules and funding are established for the final product. It is generally accepted that the software industry has not yet evolved good estimating techniques for costs and schedules, except in a few special cases. The relevance of such acquisition and contracting issues to the SSA is simply this: the SSA cannot be inadequately prepared when it seeks approval for its future computer-ir~formation system. The cost and sched- ule must be realistic and soundly based. There must be a comprehensive end-to-end management plan, an end-to-end schedule--perhaps with PERT charts. The time considered necessary for the relevant approvals also needs to be considered in the SSA schedule. Given that SSA realizes that it is embarking for the first time on a huge engineering development program that must be managed as such, and given that a special management apparatus will be created to handle it, the panel offers these suggestions for the SSA's consideration: . Include on the vendor-solicitation list the large system enterprises that have had relevant Department of Defense or NASA experience. · Solicit DoD and/or NASA expertise, because of the large information systems that these agencies have undertaken, as sources of guidance for SSA managers. · Consider sending the SSA managers through the six weeks DoD course for system program managers at Fort Belvoir. Consider hiring recently retired military or NASA officials who have had experience with major acquisition or development programs.
57 · Consider the use of an integrating contractor to extend SSA in-house management capability. Be particularly cautious of bids that deviate widely in cost, especially from an unfamiliar or inexperienced vendor. . Become familiar with the source-selection (i.e., vendor choice) mechanism that the Air Force uses to evaluate bids. (This is a major effort requiring tens of people, often for months, to carefully examine bids.) Carry out independent costing of the end-to-end program-- direct and indirect costs--in order to (1) know what the SSA really has on its hands to do and (2) have a basis for judging proposal costs. Consider hiring individuals from industry with relevant engineer- ing management experience as internal managers. · Realize that successfully managing a major development program is more like running a war than a research effort. The manage- ment must aggressively attend every detail, remain constantly alert for trouble, be quick with remedial actions, and demand that all of its related organization be likewise. Consider the establishment of an external project review board to provide an independent group to be alert for trouble or danger signals. The panel recognizes that the GAS is carrying out or considering a number of these suggestions. The suggestions, therefore, should be viewed as emphasizing the importance that the panel attaches to the various matters.
