formats, be expressed in different forms, and be directed toward answering different (although related) questions. The problem of integrating these diverse sources of information is beyond the scope of the standard statistical methods with which most test analysts are familiar. Several statisticians in the group suggested some less familiar statistical approaches that show promise for this purpose.

BAYESIAN INFERENCE AND HIERARCHICAL MODELING

Almost all statistics courses for nonspecialists adopt the classical frequentist approach to statistical inference. In the frequentist view, statistical methods can be legitimately applied only to data that can be regarded as coming from some random process. The Bayesian approach sees statistics as a set of tools for quantifying how our uncertainty about any phenomenon (random or simply unknown) changes as we acquire more information about it. Problems that are particularly well-suited to Bayesian analysis are those in which (1) there exists substantial, reliable expertise regarding the phenomenon under study and (2) the amount of direct, experimental data is small. If, in such problems, the information contained in expert opinion is not quantified and used, consequent inferences based only on the sparse data will often fail to meet reasonable standards of precision.

The chief motivation for the subsequent discussion of Bayesian inference is that problems of this type arise repeatedly in DoD evaluation and testing. In principle, the Bayesian paradigm can be used to incorporate all sources of information about a weapon system to arrive at an overall assessment of how well the system is likely to perform and the degree of uncertainty surrounding the assessment. In practice, some problems may involve fairly straightforward applications of existing techniques while other problems may involve difficult modeling issues and require the development of new methods.

In the Bayesian approach, one's uncertainty about any phenomenon may be represented by a probability distribution for the quantity of interest under consideration. The field of Bayesian statistics studies how this probability distribution changes as new information about the quantity of interest is obtained. The three basic components of Bayesian inference are:

  • The prior distribution is a probability distribution that represents the state of knowledge about the quantity of interest before the data to be analyzed have been incorporated.

  • The likelihood function is a set of probability distributions, one for each possible value of the quantity of interest, that expresses how likely it is that the data would have arisen given each possible value.

  • The posterior distribution is the result of a Bayesian analysis. Computed



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement