operational assessments. Our view was that these specific topics were more important and likely to benefit from greater examination.

Finding Failure Modes Earlier It is well known that an effective way to reduce costs and development times is to identify failure modes and design flaws as early as possible during the development of defense systems. What techniques are used in industry to accomplish this? Are there some generally applicable principles and practices that could be learned from the commercial sector and applied to DOD? How useful is it to test under conditions of operational realism early in system development? What aspects of the operational environment can be safely simulated and what can be ignored? What is meant by the envelope of operational performance for a system, and how far beyond that envelope should one test to discover design flaws and system limitations? Related to this, how are accelerated life tests utilized in industry? What are the advantages and disadvantages?

Technological Maturity The inclusion of hardware and software components that are not mature is often the cause of delays in defense system development and reduced performance when fielded. It is insufficient to assess the suitability and effectiveness of individual components of defense systems with respect to component-level requirements and specifications, disregarding how a component functions as part of the whole system. Such an approach represents an assessment of technological maturity in isolation, ignoring the likely environments of operational use, the impact of the employment of typical users, and other potential difficulties involving interoperability with the remaining system. A second related issue is how much of the testing resources should be allocated to just the components and how much should be devoted to testing them as part of the parent system. How do these issues differ for hardware versus software systems?

Use of All Relevant Information for Operational Assessment Data from many different sources are used to design tests and assess operational system performance. These include developmental testing, operational testing, modeling and simulation, and the same types of data from earlier stages of development for both the current system (when evolutionary acquisition is used) and for closely related systems. In evolutionary acquisition, there are also field performance data that are often available from the fielding of earlier versions of the system. As a result, information may be available from the operation of a system in very different contexts and can also involve appreciably different systems, given that the system in question will change during development. It is therefore a challenge to



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement