A helmet’s protective capabilities are evaluated on the basis of two primary test measures: resistance to penetration (RTP) and backface deformation (BFD). These are formally defined and their limitations are discussed in this chapter. RTP data available to the committee indicate that the probability of penetration of a helmet shell by a 9-mm bullet, fired under specified conditions, is on the order of 0.005 or less. Available BFD data show that the probability of exceeding the BFD thresholds is around 0.005 or less. The distributions of the BFD data also demonstrate significant differences among helmet sizes and shot locations. Some of the performance differences among helmet sizes may be attributed to the test process, such as headforms and stand-offs. Many others are likely to be due to the differences in the geometry of helmet shells, molds, manufacturing processes, and other factors. In fact, helmets of different sizes are intrinsically different products. Therefore, Recommendation 5-5 proposes changes to DoD’s test protocols so that helmets of different sizes are treated separately. This is one of the major recommendations in the report.
For the purpose of helmet testing, protective capabilities are measured by RTP and BFD. Section 5.2 defines these measures and discusses their limitations. Section 5.3 summarizes results from test data that were made available to the committee. The implications of these results for the Director of Operational Test and Evaluation’s (DOT&E’s) first article testing (FAT) and lot acceptance testing (LAT) protocols are discussed in Section 5.4.
Another measure, called V50,1 is also used in FAT. However, the estimated value of V50 is not used in the decision process. Thus, the committee considers V50 estimation and testing to be an aspect of characterization analyses. This topic is discussed in Chapter 8.
Resistance to Penetration
RTP is measured by shooting a given ballistic projectile at a set of helmets and counting the number of complete penetrations. Most ballistic impacts penetrate the helmet to some degree, so the DOT&E FAT and LAT testing protocols distinguish between complete and partial penetrations. A complete penetration in RTP testing is defined as:
Complete perforation of the shell by the projectile or fragment of the projectile as evidenced by the presence of that projectile, projectile fragment, or spall in the clay, or by a hole which passes through the shell. In the case of the fastener test, any evidence of the projectile, fragment of the projectile, or fastener in the clay shall be considered a complete penetration. Non-metallic material[s] such as paint, fibrous materials, edging, or edging adhesion resin that are emitted from the test specimen and rest on the outer surface of the clay impression are not considered a complete penetration.2
A partial penetration is defined as “any fair impact that is not a complete penetration.”3 In this report, the term penetration is used to refer to complete penetration. In DoD documents, the term “perforation” is used synonymously with “complete penetration.”
According to personnel from the Army Test Center, there is currently no practical way to determine or measure the degree or depth of penetration, and thus helmet penetration testing is currently attribute-based: on a given (fair) shot, the result is recorded as either a complete penetration or a partial penetration. The intuitive notion is that a projectile
1V50 refers to “the velocity at which complete penetration and partial penetration are equally likely to occur” (DoD, 1997).
2The protocols for FAT and LAT testing are given in Appendix B.
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 32
5 Helmet Performance Measures and Trends in Test Data 5.0 SUMMARY testing to be an aspect of characterization analyses. This topic is discussed in Chapter 8. A helmet’s protective capabilities are evaluated on the basis of two primary test measures: resistance to penetration (RTP) and backface deformation (BFD). These are formally 5.2 PERFORMANCE MEASURES defined and their limitations are discussed in this chapter. RTP data available to the committee indicate that the prob- Resistance to Penetration ability of penetration of a helmet shell by a 9-mm bullet, fired RTP is measured by shooting a given ballistic projectile under specified conditions, is on the order of 0.005 or less. at a set of helmets and counting the number of complete Available BFD data show that the probability of exceeding penetrations. Most ballistic impacts penetrate the helmet to the BFD thresholds is around 0.005 or less. The distributions some degree, so the DOT&E FAT and LAT testing protocols of the BFD data also demonstrate significant differences distinguish between complete and partial penetrations. A among helmet sizes and shot locations. Some of the perfor- complete penetration in RTP testing is defined as: mance differences among helmet sizes may be attributed to the test process, such as headforms and stand-offs. Many Complete perforation of the shell by the projectile or frag- others are likely to be due to the differences in the geometry ment of the projectile as evidenced by the presence of that of helmet shells, molds, manufacturing processes, and other projectile, projectile fragment, or spall in the clay, or by a factors. In fact, helmets of different sizes are intrinsically dif- hole which passes through the shell. In the case of the fas- ferent products. Therefore, Recommendation 5-5 proposes tener test, any evidence of the projectile, fragment of the pro- changes to DoD’s test protocols so that helmets of different jectile, or fastener in the clay shall be considered a complete sizes are treated separately. This is one of the major recom- penetration. Non-metallic material[s] such as paint, fibrous mendations in the report. materials, edging, or edging adhesion resin that are emitted from the test specimen and rest on the outer surface of the clay impression are not considered a complete penetration.2 5.1 INTRODUCTION For the purpose of helmet testing, protective capabilities A partial penetration is defined as “any fair impact that is not are measured by RTP and BFD. Section 5.2 defines these a complete penetration.”3 In this report, the term penetration measures and discusses their limitations. Section 5.3 sum- is used to refer to complete penetration. In DoD documents, marizes results from test data that were made available to the the term “perforation” is used synonymously with “complete committee. The implications of these results for the Director penetration.” of Operational Test and Evaluation’s (DOT&E’s) first article According to personnel from the Army Test Center, there testing (FAT) and lot acceptance testing (LAT) protocols are is currently no practical way to determine or measure the discussed in Section 5.4. degree or depth of penetration, and thus helmet penetration Another measure, called V50,1 is also used in FAT. How- testing is currently attribute-based: on a given (fair) shot, ever, the estimated value of V50 is not used in the decision the result is recorded as either a complete penetration or a process. Thus, the committee considers V50 estimation and partial penetration. The intuitive notion is that a projectile 1V refers to “the velocity at which complete penetration and partial 2The protocols for FAT and LAT testing are given in Appendix B. 50 penetration are equally likely to occur” (DoD, 1997). 3Ibid. 32
OCR for page 32
HELMET PERFORMANCE MEASURES AND TRENDS IN TEST DATA 33 that penetrates the shell is apt to cause more serious head injuries than a projectile that does not, but there is no other linkage between what is measured and head injury. Finding 5-1. It is not known whether partial penetrations 11.819 10.161 Shoot Line might be reasonably and usefully measured in order to assess 8.502 the degree to which a non-perforated helmet is penetrated. 6.844 Deepest Point 5.185 3.527 V50 testing refers to estimating the bullet speed at which 1.868 there is a 50 percent chance of penetration. This test uses a –1.868 witness plate mounted inside the headform rather than pack- –3.527 ing the headform with clay as is done with RTP/BFD testing. –5.185 (See Appendix D for details.) Because of this difference, the –6.844 –8.502 DOT&E FAT protocol defines a V50 complete penetration as –10.161 a shot where –11.819 Impacting projectile or any fragment thereof, or any frag- ment of the test specimen perforates the witness plate result- ing in a crack or hole which permits light passage. A break in the witness plate by the helmet deformation is not scored as a complete penetration.4 FIGURE 5-1 Illustrative backface deformation laser scan. SOURCE: Courtesy of the Office of the Director of Operational Finding 5-2. The definition of what constitutes a penetration, Test and Evaluation. and how such penetrations are measured, differs between RTP and V50 tests. V50 specifies a “hole which permits light passage” whereas RTP does not. assumption that the shape of the resultant cavity provides a record of the BFD. Since the relative degree of elastic and Recommendation 5-1. The Office of the Director, Opera- plastic deformation will vary as a function of strain rate, the tional Test and Evaluation, should revise the first article backing material must be characterized under conditions that testing protocol for resistance to penetration and V50 testing are relevant to those under which the tests will be performed. to ensure that the two protocols are consistent. The cavity that results from live-fire ballistic testing is indeed related to the deformation on the back face of the armor, but it is not a true record of maximum deflection. It remains Backface Deformation unknown how the dimensions of the cavity relate to the true Helmet BFD is measured on the non-perforating ballistic BFD and how such a relationship may depend on the rate at which the cavity is formed (NRC, 2012, p. 5). impacts from RTP testing. It is defined as the maximum depth in the post-impact clay surface at the intended impact location as measured from the original clay surface. It is mea- Further, whether the appropriate measure is the depth of sured as follows: After mounting the helmet on the headform the BFD rather than BFD area, BFD volume, or some other and mounting the headform in the test fixture, the helmet is measure such as total or instantaneous force imparted, is not removed from the headform, and the clay surface is scanned known. It is also unclear how well BFD from ballistic impact with a laser. The helmet is then reattached to the headform characterizes the effect of blunt-force trauma, which is one and the shot taken. Finally, the helmet is again removed from of the main types of brain injury that the helmet is intended the headform, inspected for penetration and perforation, and to protect against. the clay is rescanned with the laser to calculate BFD. A typi- cal BFD laser scan is shown in Figure 5-1. Finding 5-3. It is unknown whether the current definition of The definition of BFD as the maximum depth of indenta- BFD is the most appropriate for assessing how well helmets tion left in the clay has a number of issues. First, as discussed protect soldiers and marines from the helmet deformation in the Phase III report (NRC, 2012) report, clay is an imper- due to ballistic impact and other blunt-force trauma. It may fect recording medium. As that report said: be that some other measurement, such as the area or volume of the BFD, or perhaps some measure of force or acceleration The qualitative assertion that RP #1 exhibits little recovery imparted, is more appropriate for assessing the ability of the has been interpreted to mean that the level of elastic recovery helmet to protect against brain injury. If such an alternative is small enough to be safely neglected. This has led to an measurement is found, the protocols and thresholds would have to be changed appropriately. 4Ibid.
OCR for page 32
34 REVIEW OF DEPARTMENT OF DEFENSE TEST PROTOCOLS FOR COMBAT HELMETS Recommendation 5-2. The Department of Defense should Recommendation 5-4. As research progresses, methods, develop a better understanding of the relationship between measures, and thresholds should be continuously reviewed backface deformation and brain damage, including the to determine whether the new knowledge warrants changes examination of alternative metrics to maximum depth. to any of them. The review team should include adequate expertise from a broad range of disciplines, including medi- In addition to the definition of BFD, the DOT&E proto- cal, engineering, and testing professionals. col specifies BFD thresholds at 25.4 mm for front and back shots and 16 mm for side and crown shots. These appear to be based on historical helmet testing precedent and are not 5.3 SUMMARY OF RESULTS FROM AVAILABLE TEST connected to the potential for brain injury. The analysis, how- DATA ever, appears to be based on the presumption that the larger The DOT&E FAT and LAT protocols, as well as any addi- the BFD, the greater the likelihood of serious head injury. tional requirements included in service-specific contractual requirements, specify RTP and BFD pass or fail require- Finding 5-4. The choice of the helmet BFD threshold val- ments. The particular details of these tests are described in ues—25.4 mm for front and back shots and 16 mm for side detail in Chapters 6 and 7. This section summarizes how the and crown shots—does not have a scientific basis. In con- Advanced Combat Helmet (ACH) performs in terms of these trast, the body armor BFD limit was derived from scientific two measures using data made available to the committee. studies. Resistance to Penetration Data As a result, the usefulness of the helmet FAT and LAT test data on BFD is limited. The data can be used for assessing Table 5-1 provides a summary of RTP test data for ACH helmet performance against the requirements in the purchase helmets, provided to the committee, from FAT and LAT. description and the DOT&E helmet testing protocol; the There were two sources of FAT data: the first with 309 shots results can also be used to compare helmet performance and the second one with 816 shots, and there were no pen- within and between manufacturers and over time. But the etrations. So, the estimate of the penetration probability from data cannot be used to determine the level of protection the combined data is 0, and a 90 percent upper confidence provided by a new helmet that is designed and manufactured bound (UCB) is 0.002. The LAT data were from four differ- according to a different set of specifications. This becomes ent vendors (as shown at the bottom of Table 5-1), and there critical when assessing the protection offered by new helmets were only 7 penetrations out of 11,049 shots. This yields an because there are trade-offs between penetration, BFD, and estimated probability of penetration of 7/11,049 = 0.0006. other helmet characteristics, such as weight, form, and fit. The corresponding 90 percent UCB is 0.001. Hence, we see that a Remington 9-mm full-metal-jacket (FMJ) projectile Recommendation 5-3. The Department of Defense should shot at a randomly selected ACH, under test conditions, is examine the basis for backface deformation thresholds and unlikely—with only a 0.1 percent chance—of completely develop appropriate ones based on scientific studies and data. penetrating the helmet. TABLE 5-1 Summary of Resistance to Penetration Test Data Penetration Proportion Test Type Penetrations Number of Shots (90% Upper Confidence Bound) FAT—20-shot, five vendors 0 309 0 FAT—240- or 96-shot, four helmets 0 816 0 FAT—All 0 1,125 0.000 (0.002) LAT—Four vendors (see below) 7 11,049 0.0006 (0.001) Total 7 12,174 0.0006 (0.001) Penetration Proportion (90% Upper Confidence Bound) LAT, Vendor A 5 5,422 0.0009 (0.002) LAT, Vendor B 0 2,872 0.0000 (0.001) LAT, Vendor C 2 1,285 0.0016 (0.004) LAT, Vendor D 0 1,470 0.0000 (0.002) NOTE: FAT, first article testing; LAT, lot acceptance testing. SOURCE: Office of the Director, Operational Test and Evaluation.
OCR for page 32
HELMET PERFORMANCE MEASURES AND TRENDS IN TEST DATA 35 During FAT and LAT, each helmet is subjected to five shots at different locations. So the 11,049 LAT shots corre- 19 Cond. AM spond to roughly about 2,200 helmets. See Appendix D for 18 CO HO additional details. (If a perforation is observed on a helmet, 17 SE that helmet is not tested further, so the seven observed per- 16 Averge BFD forations were all on separate helmets.) One can estimate the 15 probability of helmet failure (rather than penetration at any 14 given location) to be approximately 7/2,200 = 0.003, which 13 is also very low. 12 11 Finding 5-5. Available data indicate that there is very low probability of helmet perforation (less than 0.005) from a 10 22 23 24 25 26 Remington 9-mm FMJ projectile shot under test conditions. standoff This level of penetration probability is considerably FIGURE 5-3 Average backface deformation (BFD) as a function of smaller than the 10 percent “standard” on which the DOT&E stand-off for Data Set 1. Colors represent different environments. protocol is based. The implications of this result are dis- NOTE: AM, ambient; CO, cold; HO, hot; SE, sea water. cussed in Chapter 6. Backface Deformation Data the data across environments as well as across helmet sizes This section summarizes relevant results from BFD data and shot locations in the two location groups. Therefore, that were made available to the committee. the committee has also pooled the data across the environ- ments. The horizontal solid lines in the figure are the BFD Data Set 1 upper limits of 25.4-mm for back and front shot locations and 16-mm for left, right, and crown shot locations. The Data Set 1 is from a test of 48 ACHs (referred to here as BFD measurements are below the thresholds at all loca- Helmet 1). Twelve helmets each are exposed to four differ- tions, and in some cases considerably so. Note also that the ent environments (ambient, cold, and hot temperatures and distributions for the left, right, and crown locations are quite seawater) prior to testing. The test consisted of firing single comparable, while the distribution for the front location is shots at five locations on the helmet: front, back, left side, substantially higher than that of the back. This difference was right side, and crown, leading to a total of 240 shots. The data consistent across the four different environments (figures not are all from a single-sized helmet (size Large), so the effect shown here), and similar effects were seen with other helmet of helmet size cannot be studied from this data set. test data as well. Figure 5-2 shows the BFD measurements by shot loca- The DOT&E protocol based on BFD is formally described tions. DOT&E’s tolerance limit analysis is based on pooling in Chapter 7, and it requires that the upper 90/90 tolerance limit of the BFD distribution not exceed the threshold. Figure 5-2 shows that no BFD values exceeded their limits. Further, for the back/front group of data, the BFD values are consider- 30 ably below their limit. One possible reason for the differences in BFD measure- 25 ments among location is stand-off: the distance between the 20 inside of the helmet shell and the headform (see discussion in Chapter 4). For a large ACH, the stand-offs were as fol- BFD (mm) 15 lows: back, 21.8 mm; front, 22.5 mm; crown, 23.0 mm; and left and right, 25.6 mm.5 Figure 5-3 shows how the average 10 of the BFD measurements differs with stand-off. The colors correspond to different environmental conditions. Note that 5 the data are clearly separated by environment. The average 0 BFDs are clearly different for different values of stand-off, 1-Back 2-Front 3-Crown 4-Left 5-Right Shot Location but the relationship is not monotone, and hence not easy to 5Frank J. Lozano, Product Manager, Soldier Protective Equipment, “Set- FIGURE 5-2 Backface deformation (BFD) measurements by loca- tion for Data Set 1. Specified limits of 25.4 mm and 16.0 mm are ting the Specifications for Ballistic Helmets,” presentation to the committee on April 25, 2013. indicated by solid lines.
OCR for page 32
36 REVIEW OF DEPARTMENT OF DEFENSE TEST PROTOCOLS FOR COMBAT HELMETS interpret. It may be expected that BFD would decrease as stand-off increases, but the average BFD for front and back 30 have the opposite difference. The average BFDs for the 25 crown, left, and right locations are quite close, even though the crown offset is considerably less than the side stand-offs. 20 Perhaps other geometric aspects of the test and the shape of BFD (mm) the helmet contribute to these patterns. 15 10 Data Set 2 5 Data Set 2 was from a test of the Marine helmet (MICH) (Helmet 2). Three helmets each corresponding to four sizes 0 1-Back 2-Front 3-Crown 4-Left 5-Right (small [S], medium [M], large [L], and extra large [XL]) were Shot Location tested at four environmental conditions (ambient, cold, hot, seawater). Again, there were single shots at five locations FIGURE 5-5 Backface deformation (BFD) measurements by loca- (front, back, left side, right side, and crown) for a total of tion for Data Set 3. Specified limits of 25.4 mm and 16.0 mm are 240 shots. This is the suite of shots specified in the DOT&E indicated by solid lines. protocol. Figure 5-4 shows the same sort of location differ- ences for this helmet as for Helmet 1. There is more spread in the Helmet 2 data than for Helmet 1 because the data are pooled over four helmet sizes as well The Figure 5-5 plot shows that there is considerably less as four environments. The BFD distributions for L and XL margin for the BFD data for the crown/left/right shot loca- helmets were different, with the measurements for XL being tions than there was for Helmet 2. Apparently, the design generally smaller than those for L. Perhaps this is due to change increased the magnitude of the dents in the clay. using a single headform for L and XL helmets. There were Eight of the 144 BFDs in this group exceeded the 16.0-mm no appreciable differences among environments. Once again, threshold. The upper 90 percent confidence limit on the prob- the 10 percent standard is easily met by these data. ability of exceeding the limit, based on this outcome, is about 9 percent, so the 10 percent standard is met in this regard. Data Set 3 Figure 5-6 shows that the differences among shot loca- tions for the XL helmet size have a pattern substantially Data Set 3 was from a test of Helmet 3, a repeat of the different from those of the other three sizes. Helmet 2 tests, after a design change to the MICH. Figure 5-5 shows the BFD data by location, pooled over environments and helmet sizes. 30 Back Crown Front Left Right LG MD 25 20 15 20 10 BFD (mm) 15 5 BFD SM XL 20 10 15 5 10 5 0 1-Back 2-Front 3-Crown 4-Left 5-Right Back Crown Front Left Right Shot Location LOCATION FIGURE 5-4 Backface deformation (BFD) measurements by loca- FIGURE 5-6 Backface deformation (BFD) measurements by loca- tion for Data Set 2. Specified limits of 25.4 mm and 16.0 mm are tion and helmet size for Data Set 3. NOTE: MD, medium; LG, large; indicated by solid lines. SM, small; XL, extra large.
OCR for page 32
HELMET PERFORMANCE MEASURES AND TRENDS IN TEST DATA 37 • The effect of environments appears to be small. The 30 same is also true for the effect of shot order. 25 5.4 IMPLICATIONS FOR FIRST ARTICLE TESTING 20 PROTOCOLS BFD (mm) 15 As shown in Table 4.1, the current DOT&E protocols involve testing 48 helmet shells: 12 each corresponding to 10 sizes S, M, L, and XL. Of the 12 shells, 3 are conditioned in 5 each of four different environments. Further, shots are taken at five different locations on the helmet. So, the committee 0 1-Back 2-Front 3-Crown 4-Left 5-Right looked at RTP and BFD data on a total of 240 shots. Chapters Shot Location 6 and 7 describe in detail the pass-fail rules for FAT protocols for RTP and BFD, respectively. Briefly, the RTP protocol states that if there are 17 or fewer penetrations, the test is FIGURE 5-7 Backface deformation (BFD) measurements by loca- tion for Data Set 4. Solid lines are the specified limits of 25.4 mm deemed to be successful. The BFD protocol is applied sepa- and 16.0 mm. rately to the two groups of locations with different thresh- olds: back and front in one group and crown, left, and right in another. The specific approach involves computing 90/90 Data Set 4 upper tolerance limits (UTLs), based on BFD measurements and the assumption that the data are normally distributed, and Data Set 4 was from a FAT for the enhanced combat hel- comparing the UTLs against their respective thresholds. If met (Helmet 4). Three helmets, of each of four sizes, were the UTL is smaller, the test is deemed successful; otherwise tested at four different environments. However, because of it is unsuccessful. excessive helmet damage, the DOT&E protocol was reduced The plots of the BFD distributions in the previous section to only two shots on each helmet. appear to be different across helmets and locations, and this Figure 5-7 shows the BFD data by shot location. There raises the issue of pooling the data to implement the protocol. are 24 shots each in the back and front locations, 16 each in The differences in the two groups of locations (front and back the crown, left, and right locations. Figure 5-7 shows that the versus crown, right, and left) are handled by implementing BFD data for this helmet are well below their limits. the protocols separately for the groups with different thresh- For the data sets analyzed by the committee, 8 of 816 BFD olds: 25.6-mm and 16-mm. Within the groups, differences measurements exceeded their respective thresholds. All of noted at front and back locations indicate that the data should these were for Helmet 3, which suggests something different not be pooled and analyzed as a sample from a single normal about that helmet or the test procedure. distribution. DOT&E has proposed an analysis to check for differences in the mean and variances and pool the data only Finding 5-6. It is clear that manufacturers are capable of if the test is accepted. In addition to the complexity of the producing helmets for which the probability of failing the procedure, the statistical properties of the protocol are not BFD protocol is very small. valid when one applies a pre-test before implementing it. In addition, the committee notes that helmets of differ- Finding 5-7. Based on the available BFD data, one can make ent size are intrinsically different products: different-sized the following observations about heterogeneity: shells are manufactured from different molds and different manufacturing processes or settings (even if some of the • There are substantial differences in BFD data across equipment and process steps are common). Therefore, pool- helmet sizes. ing the BFD data across different-sized helmets and treating • There is also a great deal of heterogeneity across the data as homogeneous does not seem appropriate. It also locations. It was expected that there will be differ- leads to the cumbersome process of pre-testing to see if ences in BFD measurements between two shot-loca- the measurements have the same mean and variance before tion groups: front and back versus crown, left, and combining the data. right. This is reflected in the different BFD thresholds for the two groups. However, the data consistently Recommendation 5-5. The Office of the Director, Opera- indicate that BFD measurements at the front location tional Test and Evaluation, should revise the current proto- are larger than those at the back, which is counter to cols to implement them separately by helmet size. the differences in stand-off at these locations. There is much less variability in the data among the other This recommendation clearly involves a major change three locations: crown, back, and front. in the way helmets are currently tested. It will also require
OCR for page 32
38 REVIEW OF DEPARTMENT OF DEFENSE TEST PROTOCOLS FOR COMBAT HELMETS decisions on how the Department of Defense implements 5.5 REFERENCES procurement decisions. For example, if a particular helmet DoD (Department of Defense). 1997. Department of Defense Test Method size did not pass FAT and others did, DoD will need to decide Standard: V50 Ballistic Test for Armor. MIL-STD-662F. U.S. Army whether the helmet sizes that passed FAT can be procured or Research Laboratory, Aberdeen Proving Ground, Md. not. The committee judges that such decisions should be left NRC (National Research Council). 2012. Testing of Body Armor Materials: to the DoD and should be based on practical considerations Phase III. The National Academies Press, Washington, D.C. rather than statistical properties of the protocol.