LETTER FROM REPRESENTATIVE LOUISE M.
SLAUGHTER TO SECRETARY OF DEFENSE LEON
PANETTA, JUNE 26, 2012
The Honorable Leon Panetta
Secretary of Defense
1000 Defense Pentagon
Washington, DC 20301 -1300
Dear Secretary Panetta:
Since 2006,1 have worked extensively with the Department and the Inspector General to ensure that our troops are being equipped with the best body armor available. Today, 1 write to you to express my concerns regarding new testing standards for Advanced Combat Helmets (ACH).
Our nation&s servicemen and women deserve the highest quality protective armor and equipment when they are in harm&s way for our country. As the Department has learned from past experience, safe armor and equipment in the field starts with comprehensive and effective testing procedures here at home.
However, government employees and contractors have shared concerns with my office that a recent modification to the standard for ballistic testing for ACHs would allow up to 17 penetrations during first article testing. While procurement standards have improved in recent years, the current standard poses an unacceptably high risk for equipment intended to protect the lives of Americans.
In 2010, the Department was forced to recall 44,000 ACHs after it was determined the helmets did not meet service standards due to questionable manufacturing processes. In the field, a penetration to a combat helmet could result in a fatality or extremely serious injury, which is why the Department should continue to take testing and performance concerns seriously.
Our goal should always be the most effective equipment possible and as such, I strongly urge you to modify the ballistics testing procedures currently in place to reduce the risk to military personnel.
Thank you for your continued commitment to the men and woman of our Armed Forces. I am confident that you have our troops& best interests in mind and will continue do your very best to improve the quality standards of all protective body armor and equipment. I look forward to your response.
Cc: Honorable Lynne M. Halbrooks, Acting Inspector General for the U.S. Department of Defense
RESPONSE TO REP. SLAUGHTER FROM J. MICHAEL
GILMORE, DIRECTOR, OPERATIONAL TEST AND
EVALUATION, JULY 13, 2012
The Honorable Louise M. Slaughter
United States House of Representatives
Washington, D.C. 20515
Dear Representative Slaughter:
Thank you for your 26 June letter to Secretary Panetta expressing your concerns regarding the Department&s protocols for testing the Advanced Combat Helmet (ACH). I am responding to you on behalf of the Secretary.
The Department insists—and I insist—on rigorous testing of soldier personal protective equipment, including combat helmets as well as body armor inserts. The protocols for ACH testing were developed to be analogous to the Department’s revised, rigorous body armor test protocols. The revised body armor test protocols were developed in response to the January 2009 Department of Defense Inspector General (IG) report on Department of Defense Testing Requirements for Body Armor. In that report the IG recommended the Director, Operational Test & Evaluation (DOT&E) develop for Department-wide implementation a standard test operations procedure for body armor inserts. The IG further recommended that the protocol include statistical specification of probability of performance and associated confidence in that performance, two concepts discussed further below. As a result of this recommendation, DOT&E published on April 27, 2010 a statistically-based test protocol incorporating a 90/90 standard, as explained below, for hard armor inserts, and then published on December 7,2010 a similar test protocol for combat helmets, including the ACH. The National Research Council (NRC), in its recent independent technical review of the Department&s testing of body armor, indicated this approach to testing is scientifically defensible. The NRC also indicated the previously-used protocols for body armor testing were not scientifically defensible. I also note that you lauded all these actions in your September 7,2011 letter to Secretary Panetta regarding body armor testing in which you stated: “I write to express my sincere appreciation to you and your staff for your work and cooperation in implementing the recommendations of the Inspector General (IG) of the Department of Defense concerning protective body armor equipment.”
The standards in both the body armor and ACH test protocols require that combat helmets and the first shot for body armor inserts demonstrate at least a 90 percent probability of no perforation with 90 percent confidence (a 90/90 standard). The Services and the U.S. Special Operations Command have endorsed the 90/90 standard for no perforation. Probability and confidence are important statistical concepts for understanding the test protocols. A test, such as firing a round at a helmet, must be performed numerous times before the data obtained can be used with confidence to accurately assess performance. (A simple example illustrating this is that three flips of a coin will not accurately predict that the probability of heads (or tails) is 50 percent.) The revised ACH test protocol is better in several ways than the previously used protocol while being designed to demonstrate the same level of protection (probability of
perforation) and also the same level of certainty of our knowledge of the level of protection (which is statistical confidence). While the revised protocol demonstrates the same probability of perforation and confidence, it provides better information in the context of multiple vendors and multiple helmets to be tested from each vendor. The previously used protocol for helmet testing allowed no perforations in 20 shots, equating to 89 percent probability of no perforation at 90 percent confidence. The difference between the old and revised protocol is in the level of knowledge they provide about differences in performance among and within vendors& helmets, not the level of protection helmets are required to provide. (The test protocol using the minimum number of shots—and therefore providing the least information about differences in helmet performance—while satisfying a 90/90 standard would use 22 shots and no perforations.) The revised protocol yields 90 percent probability of no perforation at 90 percent confidence, but allows 17 perforations in 240 shots. Again, this is the same level of protection: if 240 shots were fired at helmets that exactly met the 90/90 standard and had demonstrated the ability to withstand 22 shots with no perforations, about 17 perforations would occur. The revised protocol is not somehow flawed because it allows 17 perforations; rather, it is better because it requires 240 shots. The increase in number of helmets tested and shots taken reduces the risk of accepting a substandard helmet; it also reduces the risk of rejecting a helmet that meets standards. Additionally, because the revised protocol samples more helmets under more conditions, it provides the ability to distinguish if there are differences in performance associated with helmet size or environmental conditions.
Relative to the previously used protocol for helmet testing, the December 2010 protocol has the following specific advantages:
• By testing 48 helmets for perforation—versus four helmets under the previous protocol— to determine whether a contractor&s “first articles” meet standards, the new protocol samples a large enough number of helmets for testing to demonstrate statistically significant differences in performance due to environmental conditions (such as hot and cold temperatures), among helmet sizes, and among different vendors& designs, enabling the Department to rigorously justify purchasing only the best-performing helmets.
• The revised protocol reduces the risk to the government of accepting a substandard helmet. The previous protocol sampled just four helmets, thus increasing the chances of accidently sampling unusually good helmets relative to the vendor&s norms. With 48 helmets sampled (again, to the same 90/90 level of protection), the risk that the government will form an incorrect optimistic assessment of a vendor&s performance is reduced.
• For similar reasons, the new protocol also reduces the risk of rejecting a helmet that meets standards (also called the manufacturer&s risk). This feature is particularly important as the Military Services seek lighter weight helmets (and body armor) that provide needed protection. Soldiers and Marines fighting on the harsh terrain of Afghanistan care greatly about every ounce of weight they must carry. Because the previous protocol tested a relatively small sample of helmets, it incorporated greater risk that a lighter-weight helmet that actually met the 90/90 standard could fail testing. This
is a key reason excessive risk of rejecting a good helmet is a concern—manufacturers’ profits are not the reason it is a concern.
Therefore, the revised protocol does not pose “an unacceptably high risk for equipment intended to protect military personnel,” as stated in your letter.
In conjunction with the revised protocol for first article testing, DOT&E has also published a scientifically defensible protocol for determining whether subsequent production lots of helmets purchased from a vendor whose helmets passed first article testing continue to meet standards. The previously-used lot acceptance protocol tested at most only two helmets from subsequent production lots to assess perforation and was, therefore, not defensible, having a high probability of passing a substandard production lot. This two-shot protocol was in effect when the concerns arose in 2010 regarding the quality of the ACH mentioned in your letter. In my view, that indefensible two-shot protocol did indeed pose “an unacceptably high risk for equipment intended to protect military personnel,” which is why I insisted it be discarded.
I note that you have also communicated your concerns with the revised testing protocol for combat helmets to the Department&s IG. DOT&E and the Services will, of course, cooperate folly with the Inspector General&s office in any investigation it chooses to undertake. Consistent with its December 2009 request that the NRC undertake an independent review of body armor testing (which, as noted above, indicated the revised protocol for that testing is scientifically defensible), DOT&E is requesting that the NRC review the revised, similar protocol for testing combat helmets. This review will complement, and in no way interfere with, whatever action the IG decides to undertake. DOT&E has committed to continual improvement of the Department&s revised test protocols for both body armor and combat helmets. I will use the results of the NRC reviews, whatever findings the IG makes, as well as our own ongoing analytical work and critical review to fulfill that commitment.
In closing, I want to assure you again that the Department remains committed to using rigorous test standards that ensure we provide our Service men and women with safe and effective personal protective equipment.
cc:
DoDIG
