mendous amount of risk for that decision. This is another aspect of the management problem.

Third, the problem with reliability growth modeling is not only the weakness of specific approaches, but also the inability to assess the uncertainty of predictions, an area in which statisticians need to make progress. Fourth, with respect to life-cycle costs, the current inability to forecast a system’s costs of ownership correctly is bankrupting the military. It is crucial that DoD improve its ability to forecast system life-cycle costs.

Fifth, with respect to the development of models for combining information, the linkage required in relating developmental test performance to operational test performance is not the only requirement. It is also important to link test performance to performance in the field.

Finally, Seglie agreed that it is important to get the science underlying reliability assessment right. Therefore, it is important to understand and use models of the physics of failure. A related need is understanding of the impact of “bad actors” on models and estimates. To make progress in this regard, it is important for statisticians and the relevant scientists to work together closely.

The concern was expressed at the workshop that much of the methodological progress described by the participants is not represented in the reliability assessments of defense systems. This is unfortunate since many of these methods offer substantial benefits relative to those used in the 1970s and 1980s. The newer methods are often more efficient, which is important as data collection becomes more expensive. They also make better use of datasets with either subjective elements, censoring, or missing data, again providing more reliable estimates for the same amount of data. Finally, they offer greater flexibility in handling alternative distributional forms, and as a result, the estimates derived are often more trustworthy.

Reinstitution of more active collaboration between the statistical and defense acquisition communities, along with leading to better statistical methods, could also increase the number of academics interested in the most pressing problems faced by the defense acquisition community. In addition, such collaboration could increase the chances of attracting highly trained statisticians to careers in defense testing and acquisition. Many participants in the workshop were strongly in favor of greater interaction between the two communities.