of breaking a system up into subsystems and subprocesses and having experts examine each carefully, suggesting additional testing when necessary to direct improvements. These tools can also help guide final operational tests.
Hollis and Fries pointed out that the adoption of these methods does require up-front investment, and DoD program managers need to be willing to expend those funds. Support for this investment will come with expected positive experiences, which IRGS has already demonstrated for defense systems.
Finally, the discussants pointed out that the key to the successful use of both of these processes is that there must be an early and constant emphasis on the performance of the system under operational conditions, as opposed to meeting a required reliability level that is based on laboratory performance. It is still the case that operational sources of reliability problems appear very late in system development. These problems are typically ones that could have been identified much earlier. Both approaches can address this problem if necessary change in emphasis is achieved.
The reliability growth management processes outlined above, and reliability growth management more generally, require a variety of sources of information on system reliability as key inputs. Especially important are data from developmental and operational testing and from the field performance of related systems.
Because operational testing is costly (and occurs late in the budget cycle when there is little possibility of a large reallocation of funds for operational test), only a limited amount of information is typically collected in terms of both the number of replications and the number of separate test environments and scenarios that can be examined. Given this limited information, it is typically the case that operational testing data alone cannot confirm, with the usual levels of statistical confidence, that a defense system’s suitability requirements are met. It would be generally useful, therefore, to combine operational test data with appropriate portions of developmental test data on the same system, and data from field use and developmental and operational testing of related systems to provide less variable estimates of system reliability to inform decisions about system promotion. Further, as mentioned previously, early assessment of a system’s