decline.1 Because their reputation will affect future sales, sellers strive to maintain some minimum level of quality. If consumers rely heavily on such branding, though, it becomes more difficult for new firms to enter the market. In this case, the software industry could lose out on quality or other improvements because new and innovative firms had limited means of proving their quality. Information asymmetries of this type can be mitigated if dependability claims are explicit and backed by evidence, as long as the evidence is available for inspection by potential buyers.

Such transparency, in which those claiming dependability for their software make available the details of their claims, criteria, and evidence, is thus essential for providing the correct market conditions under which informed choices can be made and the more dependable suppliers can prosper.

To assess the credibility of such details effectively, an evaluator should be able to calibrate not only the technical claims and evidence but also the organization that produced them, because the integrity of the evidence chain is vital and cannot easily be assessed without supporting data. This suggests that data of a more general nature should be made available, including the qualifications of the personnel involved in the development; the track record of the organization in providing dependable software, which might include, for example, defect rates on previous projects; and the process by which the software was developed and the dependability argument constructed, which might include process manuals and metrics, internal standards documents, applicable test suites and results, and tools used.

A company is likely to be reluctant to reveal data that might be of benefit to a competitor or that might tarnish the company’s reputation. It is also likely that demands to publish defect rates would result in careful redefinitions of what constitutes a defect. These concerns, however, should not deter users from demanding such information, but the demands should be reasonable, well-defined, and commensurate with the dependability claimed and the consequences of failure. The willingness of a supplier to provide such data, and the clarity and integrity of the data that it provides, will be a strong indication of its attitude toward dependability, since a supplier who truly understands the role of evidence in establishing dependability will be eager to provide such evidence, and a supplier who does not understand the need for evidence is unlikely to understand all the other attributes of dependability.

One would not expect the users of a commodity operating system for standard office purposes to press for such information, although it would


See, for example, George A. Akerlof, 1970, “The market for ‘lemons’: Quality uncertainty and the market mechanism,” Quarterly Journal of Economics 84:488-500.

The National Academies of Sciences, Engineering, and Medicine
500 Fifth St. N.W. | Washington, D.C. 20001

Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement