dependability as a balancing of costs and benefits and a prioritization of risks—is vital.

Unfortunately, it is difficult to assess the dependability of software. The field of software engineering suffers from a pervasive lack of evidence about the incidence and severity of software failures; about the dependability of existing software systems; about the efficacy of existing and proposed development methods; about the benefits of certification schemes; and so on. There are many anecdotal reports, which—although often useful for indicating areas of concern or highlighting promising avenues of research—do little to establish a sound and complete basis for making policy decisions regarding dependability. Moreover, there is sometimes an implicit assumption that adhering to particular process strictures guarantees certain levels of dependability. The committee regards claims of extraordinary dependability that are sometimes made on this basis for the most critical of systems as unsubstantiated, and perhaps irresponsible. This difficulty regarding the lack of evidence for system dependability leads to two conclusions, reflected in the committee’s findings and recommendations below: (1) that better evidence is needed, so that approaches aimed at improving the dependability of software can be objectively assessed, and (2) that, for now, the pursuit of dependability in software systems should focus on the construction and evaluation of evidence.

The committee thus subscribes to the view that software is “guilty until proven innocent,” and that the burden of proof falls on the developer to convince the certifier or regulator that the software is dependable. This approach is not novel and is becoming standard in the world of systems safety, in which an explicit safety case (and not merely adherence to good practice) is usually required. Similarly, a software system should be regarded as dependable only if it has a credible dependability case, the elements of which are described below.

Meeting the burden of proof for dependability will be challenging. The demand for credible evidence will, in practice, make it infeasible to develop highly dependable systems in a cost-effective way without some radical changes in priorities. If very high dependability is to be achieved at reasonable cost, the needs of the dependability case will influence many aspects of the development, including the choice of programming language and the software architecture, and simplicity will be key. For high levels of dependability, the evidence provided by testing alone will rarely suffice and will have to be augmented by analysis. The ability to make independence arguments that allow global properties to be inferred from an analysis of a relatively small part of the system will be essential. Rigorous processes will be needed to ensure that the chain of evidence for dependability claims is preserved.

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement