WHAT SHOULD THE DEPARTMENT OF DEFENSE DO ABOUT THE ADA PROGRAMMING LANGUAGE?
Ada was developed in the 1970s by the Department of Defense (DOD) and adopted in some DOD programs until its use was required for new software development in 1987. It has been employed as a tool for developing quality software and as a DOD policy lever to encourage DOD organizations and programs to adopt modern software engineering principles. Changes within DOD, the software engineering community, and the commercial software industry have led DOD to reassess its programming language policy.
The Committee on the Past and Present Contexts for the Use of Ada in the Department of Defense was created by the National Research Council's Computer Science and Telecommunications Board to review DOD's programming language policy and the question of Ada's role in it. This study presents findings and recommendations developed by the committee for DOD's consideration in efforts to revise its current policy.
The committee concluded that a vigorous Ada program would enhance the reliability and performance of DOD warfighting systems, and it recommends that DOD continue the use and promotion of Ada in such systems. However, the committee found significant problems with the two primary components of DOD's current strategy for Ada. First, there are problems in the scope, design, and implementation of the current programming language policy, which requires the use of Ada for all software to be maintained by DOD; the committee recommends several modifications. Second, DOD's plan to discontinue investments in Ada technology and user-community support by the end of 1997 will weaken the Ada infrastructure and work against any requirement for DOD systems to use Ada in the future; given the large installed base of Ada code in warfighting systems, targeted investments in Ada are justified.
In the course of this study, the committee also concluded that the currently available data on effects of programming language on project outcomes are insufficient, on their own, to serve as a basis for strong determinations of the impact of programming language choice on the outcome of DOD
programs. Briefings received by the committee also highlighted the difficulty that DOD managers have in gaining access to data that can support informed decisions. The committee did find that trends in the data, anecdotal evidence, and expert judgment provided a basis for its finding that Ada provides benefits in warfighting systems. However, based on its experience with the limitations of currently available data, the committee makes an additional recommendation that DOD institute a corporate data collection effort and develop metrics as a sound basis for evaluating software so as to guide future policy and management decisions.
CONTEXT AND TRENDS
DOD's policy preference for Ada had some merit a decade ago. Most software at the time was entirely custom, and Ada had a good track record in delivering custom software with higher-quality and lower life-cycle costs. However, a custom Ada solution is no longer the best approach in many application areas, due to the following major trends:
- COTS as a source of information infrastructure for applications. Software solutions increasingly depend on commercial off-the-shelf (COTS) software, which provides much of an application's information infrastructure: operating system, database management, networking, user interface, and distributed processing functions. Much of this software is written in programming languages other than Ada that often do not have readily available interfaces to programs written in Ada. Developing these interfaces is not a major technical problem, but, particularly in the area of commercial Internet applications, COTS software is evolving rapidly, making it hard for Ada solutions to keep current.
- Product-line solutions and production factors. Software for many application areas is achieving economies of scale through the development of product-line architectures, enabling software assets to be reused across families of applications. These product-line solutions are driven by strongly coupled "production factors," including software components, processes and methods, human resources, and expertise in particular domains. In warfighting application areas such as weapon control and electronic warfare, there is little commercial development, and DOD has established a strong community of warfighting software developers whose production factors are oriented to Ada. However, for the numerous DOD applications in which the market is dominated by commercial solutions, such as finance and logistics, production factors have been built around programming languages other than Ada, putting Ada solutions at a disadvantage.
Additional conditions that strongly influenced the committee's findings and recommendations include the following:
- DOD emphasis on achieving information dominance. According to Secretary of Defense William Perry, ". .. our warfighting strategy sustains and builds on ... the application of information technology to gain great military leverage to continue to give us [an] unfair competitive advantage" (Perry, 1996a). This assertion highlights the importance of a capability for enhancing military competitive advantage as a criterion for the choice of programming language.
- Large and increasing inventory of DOD Ada software. DOD now has over 50 million lines of operational Ada weapon systems software, with a great deal more under development. Most of this software is in critical warfighting application areas, and there are no quick and cheap ways to translate it into other languages. DOD policies and investment strategies that weaken Ada support for this software are very risky because of the role warfighting software plays in maintaining national security.
- Proliferation of programming languages has decreased, but polylingualism is here to stay. One goal of developing Ada was to reduce the proliferation of programming languages used in DOD systems, estimated to be approximately 450 in the 1970s. The number of languages used throughout DOD has indeed decreased: the use of machine and assembly languages has diminished, and the number of third-generation languages in use has been reduced. However, there has been a rapid increase in development of fourth-generation languages by the commercial sector (there are now more than 100 different such languages), and use of these languages by DOD is increasing. Thus, DOD cannot expect to avoid polylingual software solutions. However, support for multilanguage applications has improved significantly.
- Programming language choice is one of several key software engineering decisions. The requirement to use Ada and the process for obtaining a waiver isolate programming language decisions from other key software engineering decisions (e.g., choices of computer and software architectures, decisions about use of COTS components, and milestone schedules). These decisions are also currently made at the system level, rather than at the component or subsystem level. This arrangement creates an incentive for DOD programs to make decisions that are not optimal for DOD as an organization. Future programming language decisions need to be made as part of an integrated software engineering process.
FINDINGS AND RECOMMENDATIONS
The committee developed the following set of findings and recommendations for future DOD software policy and strategy. The recommendations address the use of Ada in warfighting software, the application in which the committee finds Ada to have demonstrated benefit; the proper scope and implementation of software policy; investment in Ada; and collection of data as a basis for assessing the effectiveness of software and software policy.
Ada Competitive Advantage
Finding. Ada gives DOD a competitive advantage in warfighting software applications, including weapon control, electronic warfare, performance-critical surveillance, and battle management.
Recommendation. Continue vigorous promotion of Ada in warfighting application areas.
Rationale. Available project data and analyses of programming language features indicate that, compared with other programming languages, Ada provides DOD with higher-quality warfighting software at a lower life-cycle cost. DOD can increase its advantage by strengthening its Ada-based production factors (involving software tools, technology, and personnel) for warfighting software (see Chapters 2 and 3).
Applicability of Policy to DOD Domains
Finding. DOD's current requirement for use of Ada is overly broad in its application to all DOD-maintained software.
Recommendation. Focus the Ada requirement on warfighting applications, particularly critical, real-time applications, in which Ada has demonstrated success. For commercially dominated applications, such as office and management support, routine operations support, asset monitoring, logistics, and medicine, the option of using Ada should be analyzed but should not be assumed to be preferable.
Rationale. For warfighting software, supporting Ada-based production factors (involving software tools, technology, and personnel) gives DOD a competitive advantage. In this domain, eliminating the use of Ada would both compromise this advantage and diminish the capabilities for maintaining DOD's existing 50 million lines of Ada. In commercially dominated areas, pushing applications toward Ada would create a disadvantage for DOD (see Chapters 2 and 3).
Scope of Policy
Finding. DOD's current requirement for use of Ada overemphasizes programming language considerations.
Recommendation. Broaden the current policy to integrate the choice of programming language with other key software engineering concerns, such as software requirements, architecture, process, and quality factors.
Rationale. The current policy isolates the Ada requirement and the waiver process from other software engineering decisions, causing programs to make premature or non-optimal decisions (see Chapter 1). DOD has already taken steps to broaden the policy focus in its draft revision of its programming language policy, DOD Directive 3405.1; this report recommends modifications to that draft policy (Appendix A).
Finding. DOD's current Ada requirement and the related waiver process have been weakly implemented. Many programs have simply ignored the waiver process. Other programs make programming language decisions at the system level, but often a mix of Ada and non-Ada subsystems is more appropriate (see Chapter 1).
Recommendation. Integrate the Ada decision process with an overall Software Engineering Plan Review (SEPR) process. Passing such a review should be a requirement for entering the system acquisition Milestone I and II reviews covered by DOD Instruction 5000.2. It should also be required for systems not covered in 5000.2, and recommended by DOD for DOD-directed software development and maintenance of all kinds.
Rationale. The SEPR concept is based on the highly successful commercial architecture review board practice. The SEPR process involves peer reviewing not only the software and system development plans, but also the software and system architecture (building plan) and its ability to satisfy mission requirements, operational concepts, conformance with architectural frameworks, and budget and schedule constraints; the process also involves reviewing other key decisions such as choice of programming language (see Chapter 4).
Investment in Ada
Finding. For Ada to remain the strongest programming language for warfighting software, DOD must provide technology and infrastructure support.
Recommendation. Invest in a significant level of support for Ada, or drop the Ada requirement. The strategy developed by the committee recommends an investment level of approximately $15 million per year.
Rationale. With investment, DOD can create a significant Ada-based complex of production factors (involving software tools, technology, and personnel) for warfighting application domains.
Without such support, Ada will become a second-tier, niche language such as Jovial or CMS-2 (see Chapter 5).
Software Metrics Data
Finding. DOD's incomplete and incommensurable base of software metrics data weakens its ability to make effective software policy, management, and technical decisions.
Recommendation. Establish a sustained commitment to collect and analyze consistent software metrics data.
Rationale. The five sets of findings and recommendations above are based on a mix of incomplete and incommensurable data, anecdotal evidence, and expert judgment. For this study, the patterns of consistency in these sources of evidence provide reasonable support for the results but not as much as could be provided by quantitative analysis based on solid data. A few organizations within DOD have benefited significantly from efforts to provide a sound basis for software metrics; a DOD-wide data collection effort would magnify the net benefits (see Chapter 2).
WHAT THE DEPARTMENT OF DEFENSE SHOULD DO ABOUT ADA
In summary, the committee concluded the following regarding DOD and Ada:
- DOD should continue to require Ada for its warfighting software and should drop the Ada requirement for its other software.
- DOD should provide roughly $15 million per year for Ada infrastructure support, or drop the requirement to use Ada entirely.
- DOD should make programming language decisions in the context of a Software Engineering Plan Review process.
The rationale for the above statements is as follows:
- In commercially dominated areas, although Ada may offer some advantages for custom software development, the preponderance of existing commercial activity and solutions in other languages counters these advantages, thereby shifting the business-case away from mandating Ada in these areas.
- In warfighting applications, Ada's technical capabilities for building real-time, high-assurance custom software are generally superior to those of other programming languages. DOD's investments in Ada to date have provided DOD systems with a competitive advantage in these areas.
- The commercial marketplace alone will not sustain a robust Ada infrastructure.
- A relatively modest ($15 million per year) DOD investment at the margin would be sufficient to sustain a robust Ada infrastructure for warfighting applications.
- DOD's inventory of more than 50 million lines of Ada warfighting software will become a liability without a robust Ada infrastructure.
- DOD's current Ada waiver procedure can be effectively replaced by adoption of the commercially established practice of using architecture review boards, a process that can also strengthen DOD's overall software engineering capability.
ORGANIZATION OF THIS REPORT
Chapter 1 describes the past and present contexts for Ada and DOD software development and programming language policy and points out problems with the current approach. Chapter 2 relates issues in decisions about programming language to other software engineering decision issues and evaluates Ada's support for software engineering processes. It also summarizes the results of comparing the relative cost-effectiveness of Ada and other programming languages, based on analyses of language features and empirical data from software projects.
Chapter 3 presents a business-case analysis for the use of Ada within DOD, gives the committee's findings and recommendations, and analyzes them along with alternatives.
The software policy recommended by the committee for DOD is elaborated in Chapter 4, which also provides additional details on programming language selection criteria and explains the basic elements of the Software Engineering Plan Review process recommended by the committee. Chapter 5 discusses the committee's recommended strategy for investment in infrastructure for Ada.
Appendix A reproduces a draft revision of DOD Directive 3405.1 on programming language policy (DOD, 1987a), a document to which the committee has added further suggested modifications that reflect its recommendations in this report. Appendix B presents detailed descriptions of Ada and other programming languages; a glossary of terms used in the report is contained in Appendix C; Appendix D compares Ada's features with those of other third-generation programming languages; and Appendix E lists briefings and position papers received by the committee during the course of its study.