Testing of Body Armor Materials Phase III (2012) / Chapter Skim
Currently Skimming:
6 Statistical Considerations in Body Armor Testing
6 Statistical Considerations in Body Armor Testing
Pages 107-136
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 107... ...
of the DOT&E, including a discussion of the assumptions underlying the statistical methods and protocol design trade-offs. INTRODUCTION This introduction discusses the concepts of statistically principled testing, how uncertainty and variation drive overdesign, and key test protocol design requirements and considerations.
|
From page 108... ...
to minimize the uncertainty to some acceptable level. Thus, the use of statistically principled test procedures and test methods allow decision makers, test organizations, and manufacturers to all have confidence that the test performance of the sample appropriately characterizes the performance of the population.
|
From page 109... ...
Key Test Protocol Design Requirements The most fundamental requirements for the new protocols are that they are (1) statistically principled and (2)
|
From page 110... ...
As for flexibility, and as previously described, it is critical that the protocol specifies requirements that ensure a scientifically sound, statistically principled test that achieves a minimum standard of body armor performance DoD-wide. However, there are likely benefits to a protocol that is not unnecessarily overly specific.
|
From page 111... ...
. As described in the IG's report, the original Army protocol for body armor testing is statistically based but not statistically principled.
|
From page 112... ...
. The original USSOCOM protocol is statistically principled with sample sizes that can vary from a minimum of 146 plates tested to a maximum of 480 plates tested.
|
From page 113... ...
In addition, the historical Army protocol did not meet the key design requirement as a statistically principled test. DOT&E Protocol for Body Armor FAT In DoD Testing Requirements for Body Armor, the IG recommended that "the Director, Operational Test and Evaluation (DOT&E)
|
From page 114... ...
The new standard establishes a statistically principled protocol that sets minimum requirements for first article tests, including "standard testing references, protocols, procedures, and analytical processes for hard body armor testing." A key component of the protocol is a 60-plate design matrix that specifies the number and sizes of plates to be tested in each of nine environments and under ambient conditions and by shot order (Table 6-1)
|
From page 115... ...
PREPUBLICATION DRAFT -- SUBJECT TO EDITORIAL CORRECTION TABLE 6-1 60-Plate Protocol First Shot Edge/ First Shot Crown/ Environment Second Shot Crown Second Shot Edge Ambient 1 extra small plate 1 small plate (unconditioned) 1 large plate 1 medium plate 1 extra large plate 1 extra large plate Temperature 1 medium plate 1 extra small plate cycling 1 large plate 1 small plate 1 extra large plate 1 medium plate JP-8 soak 1 extra small plate 1 medium plate 1 small plate 1 large plate 1 medium plate 1 extra large plate Oil soak 1 small plate 1 extra small plate 1 medium plate 1 small plate 1 large plate 1 extra large plate Salt water 1 extra small plate 1 extra small plate 1 medium plate 1 small plate 1 extra large plate 1 large plate Weathered 1 small plate 1 extra small plate 1 medium plate 1 large plate 1 extra large plate 1 extra large plate High temperature 1 small plate 1 extra small plate 1 large plate 1 medium plate 1 extra large plate 1 large plate Low temperature 1 extra small plate 1 small plate 1 small plate 1 medium plate 1 extra large plate 1 large plate Altitude 1 extra small plate 1 small plate 1 medium plate 1 large plate 1 large plate 1 extra large plate Total 27 27 2 extra small plates, 1 small plate, 1 medium plate, 1 Impacted a large plate, 1 extra large plate Total plates tested 60 a Shot order is not relevant for impacted plates since the first shot is taken at the most severely damaged part of the plate as identified by X-ray.
|
From page 116... ...
On the other hand, as previously discussed, this is an acceptance test and not an operational test, so if performance in these environments and under these conditions is contractually required, then testing six plates per environment is certainly appropriate. However, it is worth noting that the resulting statistical inference is to a population of plates that experiences environmental conditions in proportion to the fraction of plates tested in each condition in the design matrix.
|
From page 117... ...
The higher the level of confidence the more likely the interval includes the unobserved population parameter. In particular, for the DOT&E protocol, achieving a 90 percent lower confidence bound that is greater than .9 means that the probability of no penetration for the entire population of plates is very likely to be greater than .9.
|
From page 118... ...
During the course of the committee's research and deliberations, the Office of the Director, Operational Test and Evaluation, the Army, and the United States Special Operations Command have endeavored to establish statistically principled test standards that are realistically achieveable with the current body armor designs. The committee found these collaborative efforts to be commendable.
|
From page 119... ...
Finding. The new Office of the Director, Operational Test and Evaluation protocol meets both key protocol design requirements; it is statistically principled and it provides a minimum Department of Defense-wide body armor test standard.
|
From page 120... ...
(NIST, 2010, Section 7.2.6.3) Question 1 leads to a two-sided interval; questions 2 and 3 lead to onesided intervals, called "tolerance bounds." For body armor testing, the relevant question is 3, which requires the calculation of an upper tolerance bound.
|
From page 121... ...
that there will be no failures in the larger population of plates. The DOT&E protocol specifies that the lower confidence limit "is calculated using the Clopper-Pearson method," which is based on the cumulative probabilities of the binomial distribution (DOT&E, 2010a)
|
From page 122... ...
assuming that the data are independent and identically distributed. Given the current test procedures, this assumption is likely not met.
|
From page 123... ...
Lower confidence levels also generally lead to narrower confidence and tolerance intervals, but at the cost of less confidence that the interval contains the quantity of interest. To have a statistically principled protocol, it is critical that a high confidence level is maintained.
|
From page 124... ...
45 Product managers have the option to implement switching procedures, and the requisite sample sizes are listed in Tables 4 thru 6 of the proposed protocol. Other differences include these: Because all plates are tested under ambient conditions, neither Table 6-1 nor any other such design matrix applies to the LAT.
|
From page 125... ...
. Figure 6-3 plots the probability that a lot of body armor passes LAT first shot requirements for the S-3 and S-4 inspection levels for various lot sizes and an AQL of 4 percent.
|
From page 126... ...
PREPUBLICATION DRAFT -- SUBJECT TO EDITORIAL CORRECTION FIGURE 6-3 Probability a lot passes LAT first shot requirements for Pr(nP) for the S-4 and S-3 inspection levels for various lot sizes and an AQL of 4 percent.
|
From page 127... ...
For most lot sizes, and over the higher levels of Pr(nP) , the S-4 inspection level results in a greater probability that lots will pass lot acceptance testing.
|
From page 128... ...
This section illustrates how to assess the trade-offs and Appendix I describes methods for explicitly comparing the performance of various test protocols. The committee would like to illustrate how the risks of the proposed test protocol can be understood and where the testing uncertainties that arise from using clay as a backing material impact the 60-shot protocol.
|
From page 129... ...
The last two lines of Table 6-4 show the sharp increases in required sample size when the requirement is increased beyond .9 and the risks are held roughly constant. Table 6-4 also shows that, for a sample size of 60, a manufacturer must produce hard body armor that has a true probability of no penetration substantially higher than .9 to have a reasonable chance of passing the test.
|
From page 130... ...
. Because of the issues discussed in earlier sections of this report, it is difficult to tell if the observed variation in BFD for hard body armor is attributable mainly to the variation in plates, to the variation in the test process, or to both.
|
From page 131... ...
. The plots show that decreasing variability in BFD by means of more consistent manufacturing processes or more repeatable testing measures lowers the manufacturer's chance of failing testing (given that the manufacturer's plates do meet standards and that all other factors are constant)
|
From page 132... ...
Furthermore, while additional research and coordination may be necessary to finalize the protocol design, and continuing review will likely be required as manufacturing conditions and plate designs change over time, a statistically principled protocol ensures that decision makers have sound information about body armor performance in order to ensure the quality of a critical soldier safety item. RECOMMENDATIONS The committee unequivocally supports the implementation of a statistically principled test protocol that explicitly and scientifically acknowledges and addresses the testing risks described in this report.
|
From page 133... ...
In particular, DOT&E should continue to Collaborate with the Army and the United States Special Operations Command (USSOCOM) to revise the test protocol as necessary, based on the results of Army and USSOCOM "for government reference" first article testing test results and -133
|
From page 134... ...
In particular, the statisticians involved with developing and implementing the statistically principled protocol should be involved with the experimentation recommended in Chapter 4. Over the course of the committee's research and deliberations, the DOT&E, Army, and USSOCOM have endeavored to establish statistically principled test standards that are realistically achievable with the current body armor designs.
|
From page 135... ...
2010a. Standardization of hard body armor testing.
|
From page 136... ...
USSOCOM (U.S. Special Operations Command)
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.