the choice of statistical models used in both the design and the analysis of operational suitability tests.
Speakers suggested the wider use of reliability growth models that are consistent with the maturation process of test, analyze, and fix. Otherwise, since the power law process and related approaches do not explicitly take into account the results of the process of discovering and fixing the faults found in testing, reliability growth models are liable to produce inaccurate predictions. Two models were proposed that assign probabilities of detection to various system faults. These models use the natural assumption that earlier testing is more likely to find errors that have a higher probability of being discovered, whereas subsequent testing is more likely to discover the less probable errors. Such approaches are consistent with the test, analyze, and fix process. They can therefore can be used to examine the maturation that would result from a specific test design given various characteristics of the system under test, and they can provide an estimate of the reliability of the “matured” system.
Speakers described several new approaches based on the use of models for combining information from developmental and operational test when the failure modes in these separate environments of use are well understood (or otherwise satisfy the necessary assumptions underlying the models). In those instances, use of these models can make operational tests much more informative and thereby save test funds in comparison with methods that do not combine information. Use of this approach relates to the following NRC (1998:119) recommendation:
Recommendation 7.7: Methods of combining reliability, availability, and maintainability data from disparate sources should be carefully studied and selectively adopted in the testing processes associated with the Department of Defense acquisition programs. In particular, authorization should be given to operational testers to combine reliability, availability, and maintainability data from developmental and operational testing as appropriate, with the