Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 302
PREPUBLICATION DRAFT—SUBJECT TO EDITORIAL CORRECTION
Appendix I
Analytical Approaches for Comparing Test Protocols
Existing armor has proven effective on the battlefield. That is, while
statistical rigor was lacking in the Army’s original FAT and LAT protocols, there
is no known evidence that substandard body armor has been sent to the field.
Thus, it is not unreasonable to assume that the manufacturers and their lots of
body armor that passed the Army’s original first article testing (FAT) and lot
acceptance testing (LAT) produced body armor that met the required (or at least
necessary) performance standards.
Of course, it is possible that, given the lack of statistical rigor, substandard
body armor passed both FAT and LAT or, conversely, fully acceptable body
armor failed either FAT or LAT. It is also possible that body armor has failed in
the field and the evidence of such failure has been lost, or that substandard body
armor that inadvertently passed FAT and/or LAT simply has not been put to the
ultimate test. These outcomes are all impossible to determine.
ASSESSING MANUFACTURER RISK
In spite of these unknowns, one way to assess the impact of the new
Office of the Director, Operational Test and Evaluation (DOT&E) protocols for
body armor on manufacturers is to use historical test data for body armor that
passed earlier FAT and LAT tests. The idea is to draw on actual historical test
data to gain some insight into how manufacturers would fare under the new
DOT&E protocol. Such an analysis is based on the following assumption:
All body armor that successfully passed the Army’s original FAT and LAT
protocols was, in fact, fit for use in the field.
Analytical Approach
Given that the foregoing assumption is correct, the effect of the DOT&E
protocol on manufacturers can be assessed as follows:
Simulate a test under the new DOT&E protocol by randomly drawing
(with replacement) from an appropriate pool of historical test data (at a
minimum, by manufacturer and type of plate) using only data from
passed tests.
-302-
OCR for page 303
PREPUBLICATION DRAFT—SUBJECT TO EDITORIAL CORRECTION
Given the pseudo data, determine whether the manufacturer (lot)
would have passed or failed the FAT (LAT).
Repeat steps 1 and 2 as necessary and appropriate to estimate rate of
pass/fail for a given manufacturer/type of plate combination.
Compare and contrast the estimated rates to the historical passing rates
to evaluate whether the new DOT&E protocol is likely to result in
higher failure rates and, if so, under what conditions and by what
magnitude.
This type of simulation cannot determine whether the new DOT&E
protocols will decrease the government’s risk of buying substandard body armor.
What it can do is to assess, from the manufacturers’ perspective, whether the new
DOT&E protocol will result in higher FAT/LAT failure rates for body armor that
probably would have passed under the previous Army protocol.
Discussion
From a manufacturers’ economic risk perspective, this is a very reasonable
comparison since, from that perspective, the issue is not how the armor performs
in the field but whether the new DOT&E protocol increases test failure rates (and
thus costs) for existing products and processes.
Note that this is essentially a nonparametric approach to evaluating
manufacturer risk. That is, by using actual historical data drawn from passed
tests, one does not have to make any parametric assumptions about the
distribution of backface deformation (BFD), nor estimate the probability of
penetration, nor try to model whether and how the two measures are jointly
distributed.
However, this approach does depend on having sufficient historical data
from which to resample. If insufficient data are available, then a parametric
approach may be taken in which distributions are fit to the BFD and penetration
data, and those distributions are used to simulate future data.
COMPARING PROTOCOLS
Another way to assess the impact of the new DOT&E protocols also uses
historical test data for body armor. This approach can be used to compare two
protocols. Instead of considering only manufacturer and design combinations that
passed the historic protocols, consider a representative range of manufacturer and
design results. For illustration, the committee compares the new DOT&E protocol
and a historic Army FAT protocol.
-303-
OCR for page 304
PREPUBLICATION DRAFT—SUBJECT TO EDITORIAL CORRECTION
Analytical Approach
Using historical data, simulate enough test data to satisfy the protocol
with the largest sample size. This can be done either nonparametrically
(by randomly drawing with replacement from an appropriate pool of
historical test data) or parametrically (by using summary statistics
from previous tests).
Note that the simulations may require several summary statistics (or
appropriate data to resample), including probability of first and second
shot complete and partial penetrations by shot order and first and
second shot BFD by shot order. Depending on the criteria for each
protocol, data may be simulated (for example, data on partial
penetrations) that will be used to assess only one protocol.
If the two protocols require a different sample size, randomly select
the smaller sample size from the simulated test data.
Compute whether or not the test would have been passed or failed
under each protocol.
Repeat the simulation as necessary and appropriate to estimate the four
pairs of probabilities: (1) pass under protocol 1/pass under protocol 2,
(2) pass under protocol 1/fail under protocol 2, (3) fail under protocol
1/pass under protocol 2, or (4) fail under protocol 1/fail under protocol
2.
Repeat the simulation for a range of representative manufacturer and
design results.
Discussion
This type of analysis may point to specific design characteristics that are
advantaged or disadvantaged by particular protocols.
-304-
