Reliability Issues for DOD Systems Report of a Workshop (2002) / Chapter Skim
Currently Skimming:
3. Current Research in Reliability Modeling and Inference
3. Current Research in Reliability Modeling and Inference
Pages 35-69
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 35... ...
The idea of exploiting "related information" in the process of interpreting the outcome of a given experiment arises in many different forms. In the DoD acquisition process, data are collected during the various stages of developmental testing, and these data may well be of use in the process of analyzing 35
|
From page 36... ...
These presenters argued that existing methods and others under current investigation constitute promising ways of modeling the data-combination challenges that arise in developmental and operational testing. Some of the earliest work on fatigue modeling occurred in the context of addressing problems that were common in the aircraft industry during and following World War II.
|
From page 37... ...
Even identical systems can have dramatically different reliabilities in developmental and operational testing as a result of the different conditions involved. In developmental testing, the system operators are typically fully acquainted with the system, the test conditions are carefully controlled, and the test is often at the component level (e.g., hardware-inthe-loop testing)
|
From page 38... ...
One session of the workshop focused specifically on the use of models for combining information from developmental and operational tests when the failure modes in these separate environments of use are well understood (or otherwise approximately satisfy the necessary assumptions)
|
From page 39... ...
as displayed in Table 1. For developmental testing, the mean time to failure is 19.53, whereas for operational testing, it is 16.09.
|
From page 40... ...
a specific weighted average of the observed OT individual mean time to failure and the observed DT individual mean time to failure, referred to as the linear Bayes estimator; and (3) an estimator that makes full use of the hierarchical Bayes approach.
|
From page 41... ...
First, one can construct a different linear Bayes estimator that makes use of the mean and variance of the probability distribution. Second, one can make use of a hierarchical Bayes estimator that assumes a joint prior distribution for the means of the distributions of the operational test and developmental test .
|
From page 42... ...
It is generally understood, however, that the loss in efficiency sustained by nonparametric methods when parametric assumptions hold exactly is often quite modest, and is thus a small price to pay for the broad protection these methods offer against model misspecification. Francisco Samaniego offered a brief review of nonparametric methods in reliability, and then suggested some nonparametric approaches to combin· · r · r Tic 1 '' nglntormatlon trom relater ~ experiments.
|
From page 43... ...
, Samaniego instead used nonparametric assumptions about the relationship between the time-to-failure distributions for developmental and operational testing of a system. Three well-known formulations of the notion that a sampled quantity (failure time)
|
From page 44... ...
Should the standard estimates of the empirical cumulative distribution functions (ec~f) for failure times from operational and developmental testing satisfy the property of stochastic precedence, those ec~f's are used, unchanged, to estimate the operational and developmental test time-to-failure distributions.
|
From page 45... ...
To effect this combination, operational evaluation data must be validated by the operational tester independently of the developmental tester. Another caution is that for combining information models to have a good chance of success, it must be determined that the prototypes used in developmental testing are production-representative of the system.
|
From page 46... ...
The environments of developmental and operational testing are very different with very different failure modes. In addition, combining information focuses too much attention on the estimate one obtains instead of on the overall information about the system that one would like to give to the user from the separate test situations.
|
From page 47... ...
Streilein is therefore not sanguine about combining information models until more information is available. TWO MODEL-BASED APPROACHES TO SELECTING INPUTS FOR SOFTWARE TESTING It is well known that software is essentially a ubiquitous component of today's complex defense systems and that software deficiencies are a primary cause of problems in defense system development (see Mosemann, 19941.
|
From page 48... ...
It has been demonstrated empirically that a large majority of the errors can be discovered in such a test set. The second approach uses a graphical model of software usage, along with a Markov chain representation of the probability of selection of inputs, to choose test inputs so that the highprobability inputs are selected for the testing set.
|
From page 49... ...
In this example, eight test cases can be identified that provide coverage of all pairwise field values for each of the seven fields. For instance, the possible input pairs of field 1 and field 2 are (1,1)
|
From page 50... ...
In the test outlined above with seven fields, eight test cases are required. AETG, in contrast, generates the matrix of test inputs shown in Table 3 for the problem of 10 dichotomous fields.
|
From page 52... ...
52 ._ o ._ ._ Cal ._ o ._ o o o ._ to ._ ~7 ._ ._ ._ To ._ GN ._ Do ._ ._ ._ Us ._ ._ ._ ._ ._ cry cry cry cry ~ ~ ~ cry cr.
|
From page 53... ...
are independent of the history of prior movement, conditioned on knowledge of the current state of the system. These transition probabilities can be set to values based on knowledge of specific environments of use, or in the absence of this knowledge, can be set uniformly across all states with which the given state "communicates." (That is, some movements from one state to another may be forbidden given the assumed functioning of the system, and these are the states with which a given state does not communicate.)
|
From page 54... ...
This Markov chain representation of the functioning of a software system supports the following standard analyses (and others) that derive from a test: (1)
|
From page 55... ...
Ferguson remarked that the traditional way of determining software reliability required a great deal of inspection, which is no longer workable. It was stressed that models such as those used by Dalal and Poore need to be developed as early as possible in system development, since, as with hardware systems, it is generally much less costly to fix a problem discovered early in the design phase.
|
From page 56... ...
However, the distribution of fatigue life aboutits expectation was either not considered or ignored. A useful stochastic approach to the problem of fatigue modeling was provided by the development and application of the inverse Gaussian distribution (see, e.g., Folks and Chhikara, 19891.
|
From page 57... ...
where the distribution of Otis substantially non-normal. Saunders described a current application derived from the generalized inverse Gaussian distribution applied to waiting times to failure for polymer coatings.
|
From page 58... ...
For this reason, efforts to validate models and to derive models using relevant physical principles (physics-of-failure models) are of critical importance.
|
From page 59... ...
The damage tolerance methodology assumes that one is sophisticated at predicting how cracks grow. If there is an inspection interval at a given number of cycles, and one can be sure by examination that there are no cracks larger than the inspection limit, one can institute a process whereby any crack will be discovered during the next inspection period before it can grow to a critical size.
|
From page 60... ...
Nicholas then discussed a substantial remaining problem high-cycle fatigue which is due to low-amplitude, high-frequency vibrations. With low-cycle fatigue, a crack typically develops early in an item's life, and gradually propagates until it can be discovered when it grows to be larger than the inspection limit.
|
From page 61... ...
Today there is a widespread perception within DoD that the percentage of the costs of defense systems that is incurred after production is too large, and thus that greater resources should perhaps be expended during the design and development stages to reduce postproduction costs, thereby reducing life-cycle costs. Estimating life-cycle costs and their contributing components can help in evaluating whether this perception is true and what specific actions might be taken to reduce lifecycle costs to the extent possible.
|
From page 62... ...
The earlier life-cycle and associated costs can be estimated, the better it is for the decision maker, though the earlier in development these estimates are attempted, the more difficult they are to produce. Blischke stated his preference for a Bayesian approach in this effort, since that paradigm provides a basis for the use of engineering judgment and information derived from similar systems, as well as a natural method for updating predictions.
|
From page 63... ...
Alternatively, one can carry out testing on prototypes or components of prototypes to obtain information on the distribution of waiting times to failure so they can be modeled. For a simple, real example, Blischke analyzed a free-replacement, nonrenewing warranty (i.e., the replacement item is warrantied to work for the time left in the original warranty period)
|
From page 64... ...
With respect to just the warranty component of lifecycle costs, a Bayesian approach has some real advantages. First, one collects all relevant information, including data on similar systems, similar parts, materials data, and engineering judgment, and aggregates this information into prior distributions for system reliability.
|
From page 65... ...
Third, as systems age the demands for reliability seem to increase, probably because of changes in the way systems are deployed. Fourth, initiating a formal maturation program provides a setting in which to conduct reliability analysis, as well as an element of acquisition planning important to the projection of future system reliability.
|
From page 66... ...
This latter approach makes it possible to learn more about the system and facilitates improvement of the system over time. The underlying question of how much reliability is enough requires a highly complex set of analyses examining a variety of difficult trade-offs between increasing design and production costs and reducing operating costs.
|
From page 67... ...
First, there is a historical lack of emphasis of system reliability relative to system effectiveness. Second, there are very separate environments within DoD for those involved in initial design (acquisition)
|
From page 68... ...
First, in the case of complicated defense systems, it is unlikely that a large number of replications for surveillance testing will be available. Second, there are a multitude of environments and missions with potentially different reliabilities and failure modes for a given system.
|
From page 69... ...
Generalizing the notion of operating characteristics would provide the correct basis for decision rules; for example, a decision rule based on a specific test design would ideally have a high probability of passing a system that met the requirement and a high probability of failing a system that did not meet the requirement. Estimates of the operating characteristics of a test should be communicated to decision makers and recognized in the decision process.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.