Statistics, Testing, and Defense Acquisition New Approaches and Methodological Improvements (1998) / Chapter Skim
Currently Skimming:
Executive Summary
Executive Summary
Pages 1-8
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 1... ...
Given the importance of the nation's defense, the high cost of many weapon systems, and the substantial cost of testing them, even modest improvements in operational testing by using the most appropriate statistical methods can lead to more efficient use of public funds and considerable improvements in the reliability and effectiveness of the systems deployed. The panel' s examination comes at a time of substantial change in the development and testing process and, more broadly, in military acquisition.
|
From page 2... ...
This set of complicated dynamics affects a variety of aspects of the test and evaluation process budgets, schedules, test requirements, test size, which test events should be excluded because of possible test abnormalities, and even the rules for scoring a test event a "failure." It is critical that the perspectives of the participants are understood and taken into account in decision making concerning test design and test evaluation. Further, for operational tests of most complicated systems, the required sample sizes that would support significance tests at the usual levels of statistical
|
From page 3... ...
This impression stems in part from a lack of communication between the test and acquisition and statistical communities about how statistical theory and methods can be applied to complex problems. For example, effective use of statistical methods is not limited to a determination of the appropriate sample size so that a test yields interval estimates at the required level of statistical confidence.
|
From page 4... ...
An environment of continuous assessment and systematic data collection should enable the use of efficient statistical methods, including decision-theoretic methods, yet would still allow the decision maker to be held accountable for judgments on system procurement.
|
From page 5... ...
The costs of operational testing and the importance of correctly deciding whether to proceed to fullrate production makes it extremely important to base decisions on all of the available, relevant information. · Requiring careful documentation of each test evaluation and inclusion of confidence statements, along with sensitivity analyses and external validation of modeling and simulation used to augment operational tests.
|
From page 6... ...
Recommendation 3.4: All services should explore the adoption of the use of small-scale testing similar to the Army concept of force development test and experimentation. ADVANCES IN STATISTICAL METHODS At a more technical level, the panel also concludes that current practices in defense testing and evaluation do not take full advantage of the benefits available from the use of state-of-the-art statistical methodology.
|
From page 7... ...
Given the importance of the decision on whether to proceed to full-rate production, state-of-the-art statistical methods for combining information should be used, when appropriate, to make tests and their associated evaluations as cost-efficient as possible. Experimental Design Methods State-of-the-art methods for experimental design should be routinely used in designing operational tests.
|
From page 8... ...
of the time to first failure, in reliability design and evaluation. Use of this model, when inappropriate, can often result in unnecessarily large test sizes and inappropriate results for significance tests.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.