National Academies Press: OpenBook

Testing of Body Armor Materials: Phase III (2012)

Chapter: Appendix I Analytical Approaches for Comparing Test Protocols

« Previous: Appendix H Statistical Tolerance Bounds
Suggested Citation:"Appendix I Analytical Approaches for Comparing Test Protocols." National Research Council. 2012. Testing of Body Armor Materials: Phase III. Washington, DC: The National Academies Press. doi: 10.17226/13390.
×

Appendix I

Analytical Approaches for Comparing Test Protocols

Existing armor has proven effective on the battlefield. That is, while statistical rigor was lacking in the Army’s original FAT and LAT protocols, there is no known evidence that substandard body armor has been sent to the field. Thus, it is not unreasonable to assume that the manufacturers and their lots of body armor that passed the Army’s original first article testing (FAT) and lot acceptance testing (LAT) produced body armor that met the required (or at least necessary) performance standards.

Of course, it is possible that, given the lack of statistical rigor, substandard body armor passed both FAT and LAT or, conversely, fully acceptable body armor failed either FAT or LAT. It is also possible that body armor has failed in the field and the evidence of such failure has been lost, or that substandard body armor that inadvertently passed FAT and/or LAT simply has not been put to the ultimate test. These outcomes are all impossible to determine.

ASSESSING MANUFACTURER RISK

In spite of these unknowns, one way to assess the impact of the new Office of the Director, Operational Test and Evaluation (DOT&E) protocols for body armor on manufacturers is to use historical test data for body armor that passed earlier FAT and LAT tests. The idea is to draw on actual historical test data to gain some insight into how manufacturers would fare under the new DOT&E protocol. Such an analysis is based on the following assumption:

All body armor that successfully passed the Army’s original FAT and LAT protocols was, in fact, fit for use in the field.

Analytical Approach

Given that the foregoing assumption is correct, the effect of the DOT&E protocol on manufacturers can be assessed as follows:

  • Simulate a test under the new DOT&E protocol by randomly drawing (with replacement) from an appropriate pool of historical test data (at a minimum, by manufacturer and type of plate) using only data from passed tests.
Suggested Citation:"Appendix I Analytical Approaches for Comparing Test Protocols." National Research Council. 2012. Testing of Body Armor Materials: Phase III. Washington, DC: The National Academies Press. doi: 10.17226/13390.
×
  • Given the pseudo data, determine whether the manufacturer (lot) would have passed or failed the FAT (LAT).
  • Repeat steps 1 and 2 as necessary and appropriate to estimate rate of pass/fail for a given manufacturer/type of plate combination.
  • Compare and contrast the estimated rates to the historical passing rates to evaluate whether the new DOT&E protocol is likely to result in higher failure rates and, if so, under what conditions and by what magnitude.

This type of simulation cannot determine whether the new DOT&E protocols will decrease the government’s risk of buying substandard body armor. What it can do is to assess, from the manufacturers’ perspective, whether the new DOT&E protocol will result in higher FAT/LAT failure rates for body armor that probably would have passed under the previous Army protocol.

Discussion

From a manufacturers’ economic risk perspective, this is a very reasonable comparison since, from that perspective, the issue is not how the armor performs in the field but whether the new DOT&E protocol increases test failure rates (and thus costs) for existing products and processes.

Note that this is essentially a nonparametric approach to evaluating manufacturer risk. That is, by using actual historical data drawn from passed tests, one does not have to make any parametric assumptions about the distribution of backface deformation (BFD), nor estimate the probability of penetration, nor try to model whether and how the two measures are jointly distributed.

However, this approach does depend on having sufficient historical data from which to resample. If insufficient data are available, then a parametric approach may be taken in which distributions are fit to the BFD and penetration data, and those distributions are used to simulate future data.

COMPARING PROTOCOLS

Another way to assess the impact of the new DOT&E protocols also uses historical test data for body armor. This approach can be used to compare two protocols. Instead of considering only manufacturer and design combinations that passed the historic protocols, consider a representative range of manufacturer and design results. For illustration, the committee compares the new DOT&E protocol and a historic Army FAT protocol.

Suggested Citation:"Appendix I Analytical Approaches for Comparing Test Protocols." National Research Council. 2012. Testing of Body Armor Materials: Phase III. Washington, DC: The National Academies Press. doi: 10.17226/13390.
×

Analytical Approach

  • Using historical data, simulate enough test data to satisfy the protocol with the largest sample size. This can be done either nonparametrically (by randomly drawing with replacement from an appropriate pool of historical test data) or parametrically (by using summary statistics from previous tests).
  • Note that the simulations may require several summary statistics (or appropriate data to resample), including probability of first and second shot complete and partial penetrations by shot order and first and second shot BFD by shot order. Depending on the criteria for each protocol, data may be simulated (for example, data on partial penetrations) that will be used to assess only one protocol.
  • If the two protocols require a different sample size, randomly select the smaller sample size from the simulated test data.
  • Compute whether or not the test would have been passed or failed under each protocol.
  • Repeat the simulation as necessary and appropriate to estimate the four pairs of probabilities: (1) pass under protocol 1/pass under protocol 2, (2) pass under protocol 1/fail under protocol 2, (3) fail under protocol 1/pass under protocol 2, or (4) fail under protocol 1/fail under protocol 2.
  • Repeat the simulation for a range of representative manufacturer and design results.

Discussion

This type of analysis may point to specific design characteristics that are advantaged or disadvantaged by particular protocols.

Suggested Citation:"Appendix I Analytical Approaches for Comparing Test Protocols." National Research Council. 2012. Testing of Body Armor Materials: Phase III. Washington, DC: The National Academies Press. doi: 10.17226/13390.
×
Page 302
Suggested Citation:"Appendix I Analytical Approaches for Comparing Test Protocols." National Research Council. 2012. Testing of Body Armor Materials: Phase III. Washington, DC: The National Academies Press. doi: 10.17226/13390.
×
Page 303
Suggested Citation:"Appendix I Analytical Approaches for Comparing Test Protocols." National Research Council. 2012. Testing of Body Armor Materials: Phase III. Washington, DC: The National Academies Press. doi: 10.17226/13390.
×
Page 304
Next: Appendix J Contemporary Methods for Assessing Behind-Armor Blunt Trauma in Live Animals »
Testing of Body Armor Materials: Phase III Get This Book
×
Buy Paperback | $65.00 Buy Ebook | $54.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

In 2009, the Government Accountability Office (GAO) released the report Warfighter Support: Independent Expert Assessment of Army Body Armor Test Results and Procedures Needed Before Fielding, which commented on the conduct of the test procedures governing acceptance of body armor vest-plate inserts worn by military service members. This GAO report, as well as other observations, led the Department of Defense Director, Operational Test & Evaluation, to request that the National Research Council (NRC) Division on Engineering and Physical Sciences conduct a three-phase study to investigate issues related to the testing of body armor materials for use by the U.S. Army and other military departments. Phase I and II resulted in two NRC letter reports: one in 2009 and one in 2010. This report is Phase III in the study.

Testing of Body Armor Materials: Phase III provides a roadmap to reduce the variability of clay processes and shows how to migrate from clay to future solutions, as well as considers the use of statistics to permit a more scientific determination of sample sizes to be used in body armor testing. This report also develops ideas for revising or replacing the Prather study methodology, as well as reviews comments on methodologies and technical approaches to military helmet testing. Testing of Body Armor Materials: Phase III also considers the possibility of combining various national body armor testing standards.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!