Reliability Issues for DOD Systems Report of a Workshop (2002) / Chapter Skim
Currently Skimming:
2. The Measurement and Management of Reliability Growth
2. The Measurement and Management of Reliability Growth
Pages 10-34
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 10... ...
It then proceeds to a discussion of six presentations at the workshop specifically dedicated to the subject four addressing tools for measuring reliability growth and two reviewing tools for managing reliability growth. The treatment here alters the order in which these presentations were made to roughly parallel the historical review.
|
From page 11... ...
The Duane model gained substantial popularity through the 1960s and 1970s. It appeared to fit reliability growth processes well enough that attempts were initiated to predict the future reliability of an item based on its fitted Duane curve.
|
From page 12... ...
Gaver studied the performance of a strategy for operational testing in the context of military acquisition in which a prototype is
|
From page 13... ...
increasing reliability of a system as a function of the time spent in development and other relevant variables. Reliability growth modeling has historically played a role in helping to determine whether a system in development is likely to meet reliability requirements in time for graduation to the next development phase, and eventually to operational testing.
|
From page 14... ...
One class of models that has been used for reliability growth modeling is referred to as learning curve models, especially the power law process. The motivation for the latter model is that, plotted on a log-log scale, the empirical cumulative failure rate in practice has appeared to be linear in time.
|
From page 15... ...
retained the power law form but gave it an interpretation that was linked more directly to the TAAF process referred to as exponential reliability growth. Other alternatives to the power law process exist to handle situations such as upward trends in the failure rate, situations in which the time to first failure can be infinite with positive probability, and failure-rate functions that have a bathtub shape (failure rate initially decreasing and subsequently increasing)
|
From page 16... ...
Therefore, the later stages of the system are not tested if an earlier stage fails beforehand. This relative lack of testing of later stages of the process for staged systems is often ignored using current approaches for modeling reliability growth.
|
From page 17... ...
The following are some other questions of interest that this simulation structure can address: · After a specified number of operational test events of the system (and associated fixes) , what is the probability that the system will meet its reliability requirements when fielded?
|
From page 18... ...
are commonly used for modeling reliability growth. As mentioned above, an extremely popular model is the Duane power law model, a particular nonhomogeneous Poisson process in which the failure rate is assumed to be a power function of time.
|
From page 19... ...
While the mathematics underlying inference for this model are complicated given that faults identified previously have an impact on the probability of discovering future faults, Scholz has derived the maximum-likelihood estimates of the residual failure rate at each stage in the fault discovery process using tools from the field of isotonic regression. Scholz has also provided upper bounds for confidence intervals for the residual failure rate.
|
From page 20... ...
in developmental or laboratory tests, versus performance in operational tests, versus performance in the field. Understanding how system performance is related in tests with various degrees of operational realism is extremely valuable for performing reliability growth modeling and for learning how to design laboratory and operational testing with greater operational realism.
|
From page 21... ...
. Field performance data have the following key applications.
|
From page 22... ...
A third use of field performance data is to establish a "transfer function" between developmental and operational tests and between operational tests and field performance. Knowledge of the ways in which developmental and operational tests are unreliable predictors of field performance has great value for reliability growth estimation, and could be useful both in linking developmental and operational test results and in providing information on how to design developmental and operational tests with greater operational realism.
|
From page 23... ...
This approach can be severely biased, producing overly optimistic reliability estimates since the failure modes can be substantially different in the two situations (actually three developmental test, operational test, and field performance)
|
From page 24... ...
In his discussion of the papers by Scholz and Meeker, lames Crouch pointed out that DoD already makes considerable use of operational test and field performance data, at least in the area of reliability testing of jet engines. Performance data are used to manage and control various aspects of turbine engine reliability, specifically (1)
|
From page 25... ...
The Air Force is also using highly accelerated life testing and highly accelerated stress screening to identify failure modes. Analysis of the common and unique failure modes from accelerated developmental testing and operational testing may make it possible to better understand the distinctions between these two types of testing.
|
From page 26... ...
relate the reliability of systems and failure modes in operational test to the reliability of systems and failure modes in developmental test, which could support methods for combining information from developmental and operational testing (as discussed earlier)
|
From page 27... ...
Since operational testing is carried out near the end of the second phase of system development (known as engineering and manufacturing development) , there is little or no opportunity for reliability assessment of a system's operational performance to inform system design during its early stages.
|
From page 28... ...
is a comprehensive framework that facilitates the use of disparate sources of informationsuch as expert opinion, simulations, historical data, test data, and maintenance data for the system in question, or information on similar systems or systems with similar components to produce reliability assessments through use of a combination of information models. PREDICT also provides estimates of the uncertainty of these assessments.
|
From page 29... ...
PREDICT tracks performance as system development proceeds. Once a system has been fielded, PREDICT can be used to track performance in the field; that is, it can continuously update reliability assessments on the basis of new information (e.g., on the aging of the system)
|
From page 30... ...
This is also an expensive way of discovering defects since it is likely that the system will experience difficulties in operational test, and it may have to undergo design modifications and later repeat some operational test events. Today, it is not uncommon for some DoD systems to enter into late-stage developmental test when their reliability is at 30 percent of the ultimate goal, whereas the goal for industrial applications is for a system to be at 75 percent of its eventual reliability before entering into formal testing.
|
From page 31... ...
. Analysis steps include understanding of failure modes and fault tree analysis, analysis of reliability design trade-offs, safety, maintainability, design-stress reliability, material and supplies analysis, and analysis of manufacturing for reliability.
|
From page 32... ...
(In the floor discussion, Dan Willard mentioned that his agency had developed a tool that appears to have some similarities to PREDICT, and they were going to compare the two to see whether there are advantages that could be shared.) One promising idea would be to use IRGS as the process for managing reliability growth, with PREDICT being used for reliability assessment.
|
From page 33... ...
CONCLUDING REMARKS ON THE MEASUREMENT AND MANAGEMENT OF RELIABILITY GROWTH The reliability growth management processes outlined above, and reliability growth management more generally, require a variety of sources of information on system reliability as key inputs. Especially important are data from developmental and operational testing and from the field performance of related systems.
|
From page 34... ...
Certainly such methods cannot be used without some scrutiny, and the benefits of use of these models for defense systems will almost undoubtedly vary with the specific application. The linkage between failure modes and failure frequencies across systems and across environments of testing or field use must be well understood before these models are applied.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.