National Academies Press: OpenBook
Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×

THE NATIONAL ACADEMIES

Advisers to the Nation on Science, Engineering, and Medicine

Division on Engineering and Physical Sciences

Board on Army Science and Technology

Mailing Address: 500 Fifth Street, NW Washington, DC 20001 www.nationalacademies.org

December 30, 2009

Dr. J. Michael Gilmore

Director,

Operational Test and Evaluation

Department of Defense

1700 Defense Pentagon Washington, DC 20301-1700

RE: Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army

Dear Dr. Gilmore:

At your request, the National Research Council (NRC) of the National Academies established a study committee to assess the methodologies used by the U.S. Army for the testing of body armor. What follows is the evaluation developed by the Committee to Review the Testing of Body Armor Materials for Use by the U.S. Army, provided in satisfaction of the Phase I component of the committee’s statement of task for this study project (see Attachment A), namely:

  • In Phase I, the committee will comment on the validity of using laser profilometry/laser interferometry techniques to determine the contours of an indent made by a ballistic test in a non-transparent clay material at the level of precision established in the Army’s procedures for testing personal body armor. If laser profilometry/laser interferometry is not a valid method, the committee will consider whether a digital caliper can be used instead to collect valid data.

  • The committee will also provide interim observations regarding the column-drop performance test described by the Army for assessing the part-to-part consistency of a clay body used in testing body armor.

    The committee will prepare a letter report documenting the findings from its Phase I considerations.

This Phase I report is focused primarily on the validity of laser-profiling techniques for body armor test measurements. More comprehensive and detailed evaluations of an array of issues surrounding body armor testing will be presented in the forthcoming Phase II and Phase III reports.

Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×

The findings in this letter report are based on the information that the committee received from the Army and on discussions and observations during a single 4-day meeting that included a site visit to the Aberdeen Test Center (ATC) at the Aberdeen Proving Ground, Maryland. The committee reviewed relevant documentation pertaining to the Army’s body armor testing program in general and to the tasks in the statement of task for Phase I in particular. During the site visit, the committee members engaged in direct observations of how ATC staff performs body armor testing using laser scanning and digital caliper methodologies.1 The committee also witnessed the entire procedure for assessing the part-to-part consistency of the clay used in body armor testing. In addition, the committee made some observations on statistical considerations related to body armor testing measurements.

The committee was impressed with the dedication, qualifications, and openness of the ATC staff. The ATC staff is clearly seeking to achieve the highest possible standards in armor testing and is making major advances in the state of the technology, with the goal of providing the very best armor performance for soldiers.

The committee’s detailed analysis of the Phase I issues is presented in the main body of this report. That analysis follows the committee’s consolidated findings, which are presented below:


Finding 1. The procedure documented in “Internal Operating Procedure No. 001: Measurement of Backface Deformation [BFD] Using Faro® Quantum Laser Scan Arm and Geomagic® Qualify® for Hard and Soft Body Armor” (Aberdeen Proving Ground, Md.: Aberdeen Test Center, September 1, 2009) adequately describes the appropriate use of the laser scanning system.


Finding 2. Surface profilometry by a laser scanning system (including the testing protocols, facilities, and instrumentation) as currently implemented by the Army (or similar equipment), if used in accordance with the Army’s procedures, is a valid approach for determining the contours of an indent in a nontransparent clay material at a level of precision adequate for the Army’s current ballistic testing of body armor.


Finding 3. The digital caliper is adequate for measurements of displacements created in clay by the column-drop performance test: there is a well-defined reference plane, and one can visually see the surface of the clay, given that the depression is relatively shallow (approximately 22 to 28 mm) and fairly smooth.


Finding 4. The column-drop performance test (including the testing protocols, facilities, and instrumentation) is a valid method for assessing the part-to-part consistency of clay boxes used in body armor testing.

1

The system currently in use by the Army is the Faro® system, a profilometry system, not a laser interferometer system. The committee did not assess an interferometer system.

Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×

Sincerely,

MG (ret) Larry G. Lehowicz,

Chair

Committee to Review the Testing of Body Armor Materials for Use by the U.S. Army

Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×

Detailed Analysis with Respect to Phase I Study Tasks and Related Issues

A BRIEF OVERVIEW OF THE BALLISTIC TESTING OF BODY ARMOR BALLISTIC INSERTS/PLATES

All effective body armor systems currently add a significant burden of weight on the soldier. Armor testing therefore has implicit goals of ensuring that body armor meets survivability standards at an acceptable weight while allowing sufficient soldier mobility.

Ceramic materials have been used successfully in personal armor systems to defeat small-arms threats. The committee was informed by the Program Executive Office–Soldier that there have been no known soldier deaths due to small arms that were attributable to a failure of the issued ceramic body armor.2,3,4

Ceramic materials are preferred for personal armor systems because they are relatively light compared to more traditional armor made of metallic alloys. Properties that contribute to the performance of ceramic armor include a high degree of hardness, low density, a high degree of elastic constants, and high compressive strength. However, as standalone items, ceramics would not be particularly good because of their low tensile strength, brittle response, and sensitivity to small mechanical defects such as pores and cracks. Hence, ceramics are used in combination with other materials such as polymers and metals to form laminar composites that provide excellent properties for body protection.

A typical insert (also referred to as a “plate”) of body armor consists of a layer of dense boron carbide or silicon carbide backed by a layer of metal or polymer composite; the entire plate is wrapped in tightly woven ballistics fabric. The ceramic layer breaks up an incoming projectile and dissipates its kinetic energy. The layer of polymer composite provides a measure of ductility and structural integrity and spreads the load over a larger area.

The Army’s procedures for testing body armor include placing the fabric-wrapped armor insert in a fabric envelope with multiple layers of Kevlar™ corresponding to the

2

Question-and-answer session between Debi Dawson, Director, Strategic Communications, PEO Soldier, and the committee, December 1, 2009.

3

Personal communication between LTC Jon Rickey, Product Manager (PM), Personnel Survivability Equipment, PEO–Soldier, and Larry G. Lehowicz, Committee Chair, December 21, 2009.

4

Personal communication between James Zheng, Chief Scientist, Program Executive Office–Soldier, and Larry Lehowicz, Committee Chair, December 29, 2009. According to LTC Rickey and Dr. Zheng, in no case has it been determined that an issued Enhanced Small Arms Protective Insert (ESAPI) or Enhanced Side Ballistic Insert (ESBI) armor plate failed to prevent a penetration of 7.62 × 63 mm armor piercing (AP) and below small-arms projectiles they are designed to defeat. However, in some instances the issued body armor may become separated from the wounded soldier. In these situations it may not be possible to trace the armor back to the original casualty. As a result the word “known” was selected by the Army to qualify the statement “no known deaths.” For a non-military, non-expert audience it is noted that casualties to soldiers wearing body armor may be the result of ceramic armor being defeated by rounds of a caliber larger than 7.62 × 63 mm; projectiles or shrapnel striking a portion of the body not protected by body armor; blast effects from improvised explosive devices (IEDs) or other explosives; and so forth.

Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×

soft armor vest worn by the soldier. The encased plate is placed against an oil-based modeling-clay backing, or “appliqué.” A projectile is then fired against the vest, causing an indentation in the clay. This indentation is referred to as “backface deformation” (BFD). The surface profile of the indentation left in the clay measured relative to the original undisturbed surface enables the determination of the depth and spatial location of the deepest impression. The deepest impression left in the clay is measured and analyzed as an indication of the survivability of a soldier, subjected to a similar shot, protected by a similar plate in a protective vest.5

COMMENTS ON THE VALIDITY OF USING LASER PROFILOMETRY/LASER INTERFEROMETRY FOR TESTING BODY ARMOR

The Historical Measurement Approach Used in Testing Body Armor

Historically, the digital caliper has been the measurement tool of choice for the testing of body armor. The caliper was the state of the art when the body armor test was instituted in 1977 (Prather et al., 1977).

Calipers have several shortcomings that can limit the precision of the measurements of ballistic indentations made in clay. For example, the digital caliper is a point-to-point measuring system. In order to measure a complex curved surface (i.e., the surface of the clay appliqué), a high density of precisely located measurements is needed. This is not easily accomplished with calipers. Also, after the projectile is fired, the deepest location in the clay indent is frequently (approximately 25 percent of the time) displaced from the aim point. The caliper measurement of the initial appliqué surface only at the point of aim requires estimating where one would have expected the original surface to have been, based on the magnitude and direction of the lateral displacement of the point of deepest depression and the nominal curvature of the plate. There can be significant errors in the use of this correction factor. Furthermore, the use of a caliper is subject to operator judgment because one is trying to measure the soft, deformable surface of the clay while “barely” touching but not disturbing the clay with the tip of the caliper. However, in the three decades since 1977, high-precision, laser-based, noncontact measurement systems have become readily available and are commonly used in industry. They are particularly well suited for soft-surface measurements and address many of the limitations of the digital caliper.

5

As shown in the Prather et al. (1977) study, there is a correlation between the depths of penetration as a function of time into various media, including the modeling clay Roma Plastilina #1, and the probability of lethality when the same penetrator enters a human surrogate (goat) (Prather et al.,1977).

Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×

Current Army Profiling Measurement Procedures Used in Testing Body Armor

The Army’s testing procedures for hard body armor using a clay backing for the measurement of deformations in the clay from ballistic impacts are documented in “Test Operations Procedure (TOP) 10-2-210: Ballistic Testing of Hard Body Armor Using Clay Backing,” dated October 1, 2008 (ATC, 2008a) and “Internal Operating Procedure No. 001: Measurement of Backface Deformation [BFD] Using Faro® Quantum Laser Scan Arm and Geomagic® Qualify® for Hard and Soft Body Armor,” dated September 1, 2009 (Huber, 2009). A brief description of the procedure using the Faro® Laser ScanArm® V3 system (Quantum 2.4 m) is given below. The Faro® system is an accepted, high-resolution three-dimensional laser scanning system. It has been and is being used by many manufacturers, including Boeing, General Motors, NASA, and ATI Castings.

A clay box6 and clay chest plate appliqué7 (Figure 1) are assembled, appropriately calibrated for part-to-part consistency, and placed upright in the test holder. A pointing laser aligned with the gun barrel that will fire projectiles for the test projects a red spot of light (about 3 mm in diameter) on the clay to identify the “point of aim.” Independently, a “shoot pack” is prepared. To create a shoot pack, the armor plate is placed in a fabric envelope together with multiple layers of Kevlar™ to replicate the full soft armor vest worn by the soldier. The dimensions of the armor plate depend on the specific size of the vest (i.e., extra small, small, medium, large, or extra large) and can range from 18 × 29 cm to 28 × 36 cm, with a thickness of approximately 2 cm. The vest has a significant nonconstant radius of curvature. Once assembled, the shoot pack is pressed firmly into the surface of the appliqué to ensure a conformal fit. The shoot pack is then removed, and the laser scanning system is used to scan the surface of the appliqué in order to provide a reference surface relative to which subsequent deformations caused by the firing of the projectiles can be compared. Prior to the scanning of the surface of the appliqué, three reference points along the top and side of the aluminum frame around the clay box are used for calibration of the laser scanner’s spatial location on the x-y, x-z, and y-z planes.

The laser scanning system is moved out of the way, and the shoot pack is repositioned onto the surface of the clay, with care taken not to disturb the reference surface, and is secured. (It is worth noting that at this point a relatively large force would be necessary to deform the surface of the clay, as the shoot pack makes a large contact area with the clay.) The selected projectile is then fired into the shoot pack, after which the shoot pack is removed from the clay and inspected for penetration (Army personnel described to the committee the protocols developed for distinguishing more precisely between partial and complete penetrations), exposing the displacement or indent in the clay made by the deformation of the armor plate. Specific deformations in the clay can be correlated with the extent of injury anticipated to be experienced by a soldier, if the same threat were to be experienced in the field.

The laser scanning system is moved back into place, the three alignment reference points are used a second time to establish reference locations (compensating for any

6

A plywood-backed aluminum frame (~61 × 61 × 14 cm) filled with modeling clay is subsequently referred to in this report as a “clay box” or as a “part” when discussing part-to-part variations.

7

The appliqué is an additional layer of clay that has been molded to the shape of the specific armor plate to be tested.

Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×

movements that may have been caused by vibrations or shocks associated with the firing), and the system then is used to measure the three-dimensional profile of the impact area (the clay indent). The data are collected and then used to compute the profile of the indent, allowing its depth distribution to be defined. All of the procedures described above are as prescribed in the two documents previously cited; these procedures are augmented by information in the manuals of the laser system manufacturers.8

FIGURE 1 The clay appliqué applied to the clay box. SOURCE: Richard Sayre, Deputy Director, Office of the Secretary of Defense, Director of Operational Test and Evaluation (OSD DOT&E) Live Fire Test and Evaluation, and Tracy Sheppard, Executive Officer and Staff Specialist, OSD DOT&E Live Fire Test and Evaluation, “DoD in Brief to the National Research Council Study Team,” presentation to the committee, Aberdeen, Maryland, November 30, 2009.

FIGURE 1 The clay appliqué applied to the clay box. SOURCE: Richard Sayre, Deputy Director, Office of the Secretary of Defense, Director of Operational Test and Evaluation (OSD DOT&E) Live Fire Test and Evaluation, and Tracy Sheppard, Executive Officer and Staff Specialist, OSD DOT&E Live Fire Test and Evaluation, “DoD in Brief to the National Research Council Study Team,” presentation to the committee, Aberdeen, Maryland, November 30, 2009.

Finding 1. The procedure documented in “Internal Operating Procedure No. 001: Measurement of Backface Deformation [BFD] using Faro® Quantum Laser Scan Arm and Geomagic® Qualify® for Hard and Soft Body Armor” (Aberdeen Proving Ground, Md.: Aberdeen Test Center, September 1, 2009) adequately describes the appropriate use of the laser scanning system.

Laser Profilometer

Laser scanning is a powerful and effective tool for high-resolution three-dimensional, noncontact, surface profilometry. Over the past several decades, significant progress in the miniaturization of lasers, optical detectors (such as charge-coupled device [CCD] cameras), and optical components, along with enhanced capabilities in high-speed data processing and software for digital data presentation, has made it possible to develop compact laser scanning systems that are fast and have high measurement accuracy and precision. Laser scanning systems enable a high-resolution noncontact surface

8

Additional information can be found at www.faro.com/content.aspx?ct=di&content=pro&item=1. Last accessed December 30, 2009.

Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×

profilometry to be made with a precision exceeding that of conventional mechanical instruments.

The technique of laser profilometry, as used by the Faro® scanning laser instrument, employs a commonly used principle of optical triangulation. A laser generates a collimated beam, which is then focused and projected onto a target surface. A lens re-images the laser spot formed on the surface of the target onto a photodetector array (i.e., a CCD), which generates a signal that is indicative of the spot’s position on the detector. As the height of the target surface changes, the image of the laser spot shifts owing to the parallax.

To generate a three-dimensional image of the part’s surface, the sensor scans in two dimensions, generating a set of data that represent the surface topography of the specimen under inspection. The performed noncontact measurements produce a set of data that are then used to compute the three-dimensional geometrical profile of the surface, with readings essentially continuous over the scanned region. Thus, the laser scanner produces a measurement over the whole surface of the clay, as opposed to just one reading—that is, at one location—obtained with the digital caliper method. In addition, a laser scanning system has the ability to acquire substantial quantities of inspection data in approximately 1 minute.

The laser scanning system collects three-dimensional data by first calibrating the system’s articulated robotic arm movement using a detailed procedure and reference point measurements. The focused laser beam is projected on the surface under study and then scanned in one plane, generating a line of illumination that is imaged by an optical sensor, the CCD camera. In the particular case of the Faro® laser scanning system used by the Army, this line is made up of 640 points that are then projected onto a corresponding number of pixels of the CCD camera in the optical head of the laser scanning system. The sequential scanning of the line over the surface under examination, the imaging of these lines on the CCD camera, and the post-processing of these recorded data enable the generation of a three-dimensional topological image of the surface. The laser scanning system enables the use of raw data or the use of conventional smoothing algorithms or software macros. The latter are used to smoothe the data to resolutions of less than 0.5 mm and to remove single-point spikes. The use of smoothing algorithms is standard practice in the industry.

The laser scanning system enables the three-dimensional measurement of the complete clay indent after the testing of the body armor plate is completed and also of the reference clay surface before the test is conducted. In addition, the ability to calculate and record the entire deformation may allow the extraction of substantially more information about the blast dynamics and performance in the future.

The measurement error of the laser scanning system and the digital caliper has been documented by the Army in several publications, including the following:

  • The U.S. Army Aberdeen Test Center (ATC) report, Quantum Faro®Arm Laser Scanning Body Armor Back Face Deformation (ATC, 2008a);

  • The ATC report entitled Summary Report of Laser Scanning Method Certification Study for Body Armor Backface Deformation Measurements (Walton et al., 2008); and

Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×
  • “Memorandum for Record: Testing of Faro® Laser Scanner” (Rigney, 2008).

Table 1 summarizes the measurement error of the laser scanning system and the digital caliper against an etched metal gage block. Since the gage block is rigid and has known dimensions, it can be used to assess both the accuracy and precision of the measurement systems. Table 2 summarizes the measurement precision of the two systems against a realistic clay target.

TABLE 1 Error Budget Measured for Etched Metal Gage Block Measurement Using Faro® Laser Scanner and Digital Caliper

Type

Component

Components of Uncertainty Per Instrument, (mm)

Faro® Laser Scanner

Digital Caliper

A

Operator and session variability

0.0147

0.0074

B

Inter-instrument variability a,b,d

0.0323

0.0073

Combined standard uncertaintyc

0.0355

0.0104

Expanded uncertaintyd

0.0710

0.0208

Mean lengthe

50.7957

50.8015

a Instrument uncertainty obtained from Faro® manufacturer's documentation was judged by the committee to be sufficiently like that in current use.

b Digital Caliper uncertainty estimate obtained from manufacturer's document was judged by the committee to be sufficiently like that in current use.

cCombined standard uncertainty obtained by taking the root sum of the squares of the component uncertainties.

dTrue gage block length is 50.8002, obtained from calibration certificate dated August 28, 2009.

eExpanded uncertainty obtained by using a coverage factor of k/2 times the combined standard uncertainty. Provides an approximate 95% confidence bound on uncertainty of measurement.

SOURCE: Adapted from Walton et al., 2008.

While it is true that the digital caliper yields better precision on the hard, rigid material, as seen in Table 1, making measurements on hard, rigid material is not the Army’s goal. For measurements on an irregular, soft material, such as clay, the laser scanner is more precise, as shown in Table 2. In particular, for the BFD measurement, the expanded uncertainty (two times the standard deviation) is 0.194 mm for the Faro® laser system, which is much better than the 1.645 mm for the digital caliper.

Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×

TABLE 2 Error Budget Measured for Clay Depression Backface Deformation Measurement Using Faro® Laser Scanner and Digital Caliper

Type

Component

Uncertainty Calculation for Realistic Clay III Corrected, Deepest (mm)

Faro® Laser Scanner

Digital Caliper

A

Operator

0.0410

0.4715

 

random error

0.0817

0.3600

B

Instrument specification

0.0325a

0.0073b

Correction factorc

NA

0.570

Combined standard uncertaintyd

0.0970

0.823

Expanded uncertaintye

0.1939

1.645

a Instrument uncertainty obtained from FARO manufacturer's documentation was judged by the committee to be sufficiently like that in current use.

b Digital Caliper uncertainty estimate obtained from manufacturer's document was judged by the committee to be sufficiently like that in current use.

c Uncertainty in correction factor due to deepest point different from aim point.

d Combined standard uncertainty obtained by taking the root sum of squares of the component uncertainties.

eExpanded uncertainty obtained by using a coverage factor of k/2 times the combined standard uncertainty. Provides an approximate 95 percent confidence bound on uncertainty of measurement.

SOURCE: Adapted from Walton et al., 2008.

In contrast to the situation with the gage block, when measuring a complex surface on a soft material the laser scanner offers better precision. The expanded uncertainty (two times the standard deviation value) is 0.194 mm for the Faro® laser system, which is much better than the 1.645 mm for the digital caliper, when used for a BFD measurement.

Indent Contour Determination with a Laser Profilometer

A representative deformation resulting from the laser test is shown in Figure 2. The nominal design specification is that the maximum depth in the clay relative to the original surface be less than 43 mm. That is, a BFD of less than 43 mm correlates with acceptable performance of body armor in service. Experimental data collected by the Army indicate that under nominally identical conditions the standard deviation for the maximum depth of the BFD (hard armor) is in the range of 2.5 to 4 mm.9

When the laser scanner is used, both the appliqué surface and the complete deformed surface of the clay are measured at approximately 1 million points. The depth of deformation is calculated for all positions for the entire indent by computing the difference between the measurements of the deformed and undeformed surfaces. The measurement is much more difficult and problematic for the case of the digital caliper

9

James Zheng, Chief Scientist, Program Executive Office–Soldier, “Ballistic Protection for Warfighters,” presentation to the committee, Aberdeen, Maryland, November 30, 2009.

Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×

because the original height of the appliqué surface is measured for only one position (at the nominal point of aim). This results in a significant measurement uncertainty when the maximum depth does not occur directly behind the point of aim (also shown in Figure 2). This displacement can be caused by the projectile’s not striking at the point of aim—the blast dynamics causing the greatest displacement to be at a point other than the point of aim—or it can be caused by a shift point in aim due to gun barrel vibrations and recoil.

FIGURE 2 Surface of the backface deformation as measured by the laser scanning system.

FIGURE 2 Surface of the backface deformation as measured by the laser scanning system.

SOURCE: Richard Sayre, Deputy Director, Office of the Secretary of Defense, Director of Operational Test and Evaluation (OSD DOT&E), Live Fire Test and Evaluation, and Tracy Sheppard, Executive Officer and Staff Specialist, OSD DOT&E Live Fire Test and Evaluation, “DoD in Brief to the National Research Council Study Team,” presentation to the committee, Aberdeen, Maryland, November 30, 2009.

As the committee examined its statement of task, it decided that it was necessary to determine first whether the laser scanning system is valid at the level of precision established in the Army’s procedures. It then realized that appropriate documentation to address a level of precision necessary to assess the laser scanning system, as required in the statement of task, does not exist. Unfortunately, the Army’s purchase documents and procedures do not specify a level of precision required. More specifically, the purchase description specifies not a level of precision, but rather a level of accuracy (DOD, 2007, p. 17):

Indentation measurements will utilize measurement devices (±0.1 mm accuracy) incorporating a fixed reference “guide” … that can rest solidly upon two edges of the fixture, establishing the reference plane across the diameter of the indentation.

Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×

Although accuracy and precision in common usage are frequently used interchangeably, they have precise (and quite different) technical definitions. Accuracy is the closeness of agreement between a measured value and the true value, whereas precision is the closeness of agreement between measured values obtained by replicate measurements under specified conditions (ISO, 2007; ASTM, 2008a). These differences are illustrated in Figure 3.

FIGURE 3 Illustration of the distinction between precision and accuracy in measurements.

FIGURE 3 Illustration of the distinction between precision and accuracy in measurements.

SOURCE: Reprinted with permission from Dr. Heather Tavernier Hart, Instructor of Laboratory Science, Chemistry Department, Wellesley College.

As stated above, the committee was asked to comment on the precision of the laser method, and the Army’s purchasing documents do not specifically set standards for precision. Consequently, the committee had to estimate the current level of precision that the Army requires for measuring BFD. The committee adopted two parallel approaches for determining this level of precision. The first approach was to compare the laser’s precision with that of the digital caliper’s when used to make BFD measurements. The second approach was to compare the laser’s precision to the total measurement variation. In particular, a rule of thumb is that the measurement system variance required for a test should be a factor of 10 or better than the total measured variation (McNeese and Klein, 1991). If the measurement system variance is substantially larger than 10 percent of the

Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×

total variation, it becomes difficult to detect potentially significant differences among the products under test. “The factor of 10” heuristic may be reconsidered in the reports to be produced in Phases 2 and 3.

The Faro® laser scanning system is clearly a much more precise tool than the digital caliper for measuring BFD. The Faro® system eliminates operator variability and does not physically contact the soft clay. It has an estimated precision of 0.194 mm for this application—a level of precision well suited for the Army’s purposes. The laser scanning system is eight-fold more precise than the digital caliper, which has been the standard used for the past 30 years, and meets the first approach here.

The scanning system measurement variance clearly meets the second approach above.10 Overall, the use of the laser scanning system represents a significant advance. It is valid for use in the testing process and is clearly a more precise tool for measuring BFD. (See Tables 1 and 2.)

Committee Comments on the Validity of Using Laser Profilometry Techniques

The U.S. Army Aberdeen Test Center demonstrated to the committee that the laser scanning system is housed in a facility with controlled temperature and humidity and without vibration. It has a trained and professional workforce. The ATC has developed thorough operating procedures for using the system.


Finding 2. Surface profilometry by a laser scanning system (including the testing protocols, facilities, and instrumentation) as currently implemented by the Army (or similar equipment), if used in accordance with the Army’s procedures, is a valid approach for determining the contours of an indent in a nontransparent clay material at a level of precision adequate for the Army’s current ballistic testing of body armor.

INTERIM OBSERVATIONS ON COLUMN-DROP PERFORMANCE TESTS TO ENSURE THE CONSISTENCY OF CLAY USED IN BODY ARMOR TESTS

In the testing of body armor, the medium used to record the backface deformation of a ceramic armor system experiencing ballistic impact is a multicomponent modeling-clay body (also referred to as “clay”). With knowledge of the behavior of the individual constituents that make up modeling clay, it is possible to develop an expectation about how the flow behavior will be influenced by the thermal history and shear history experienced by the clay.

10

The committee is comparing the precision of the laser scanning system, as measured by its variance, with the total measurement variance. Since the precision of the laser scanning system is reported as an expanded uncertainty with 95 percent coverage, its standard deviation is 0.194/2 = 0.097 mm, and the variance is (0.097)2 = 0.00941 mm2. The standard deviation for backface deformation (BFD) (hard armor) is in the range of 2.5 to 4 mm, and the total variance is in the range of 6.25 mm 2 to 16 mm2. Thus the measurement system variance of the laser scanning system is clearly adequate for measuring BFD variation.

Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×

Use of Roma Plastilina as a Medium for Recording Backface Deformation

The use of modeling clay as a recording medium dates from at least the 1977 study that established correlations involving the depth of penetration of a 200-g, 80-mm hemispherical missile impacting at approximately 55 meters per second (Prather et al., 1977). The correlations included matching the depth of penetration as a function of time into various media to the probability of lethality when the same penetrator entered a live animal model (in this study goats were used as models). Correlations were observed to be strong between lethality probability and penetration into ballistic gelation and, in turn, penetration into a modeling clay, Roma Plastilina #1. The first conclusion of the Prather et al. (1977) report had a profound effect on testing over the next 30 years. It reads as follows (Prather et al., 1977, p. 11) :

A readily available, easy-to-use backing material, Roma Plastilina 1, has been found which can be correlated to tissue response for use in characterizing both the penetration and deformation effects of ballistic impacts on soft body armor materials.

Roma Plastilina #1 has been adopted as a recording medium used to assess the likelihood of injury or death from ballistics, and its use has been extended from originally assessing soft armor (e.g., Kevlar™ vests) to use in assessing hard armor plates, knife wounds, industrial injuries to a drop-forge operator, and nonlethal projectiles (Lyon, 1997; Karahan, 2008; Vaughan, 2001; O’Callaghan et al., 2001; Chadwick et al., 1999). Roma Plastilina #1 appears to have achieved the status of an industry standard.

The procedures for the use of this modeling clay have evolved with time. In part, this can be explained by the observation that the behavior of the material has changed over time. Army personnel indicated to the committee that the manufacturer confirmed that the formulation sold under the name Roma Plastilina #1 has evolved over time as sources of raw material have changed. Such an evolution is to be expected with commercial manufacturers. Additional performance aspects of the clay material will be addressed in Phase 2 of this study, which is focused totally on clay.

Column-Drop Performance Test Description

As standards have evolved, drop tests have been introduced to ensure that the modeling clay used for each test has well-defined behavior. The drop test consists of dropping a cylindrical steel mass with a hemispherical cap (44.5 mm in diameter) of defined mass (1 kg) from a height of 2 m. The mass is then removed, and penetration is defined by measuring the distance between the original flat clay surface and the deepest point in the indent. As the deepest point is determined by a highly regular hemisphere, it can be readily and reliably located by an operator using a digital caliper. The digital caliper is adequate for this measurement because of the well-defined planar reference, the smooth shallow indentation, and the ease of locating the center of the indention.

Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×

Finding 3. The digital caliper is adequate for measurements of displacements created in clay by the column-drop performance test: there is a well-defined reference plane, and one can visually see the surface of the clay, given that the depression is relatively shallow (approximately 22 to 28 mm) and fairly smooth.


As noted previously, the plywood-backed aluminum frame filled with modeling clay is referred to in this report as a clay box, or as a part when discussing part-to-part variations. Army personnel indicated to the committee that the drop test was specifically introduced in order to account for the complexity of conditions leading to a given behavior of clay.11 That is, whereas heating to 104°F was necessary to achieve a drop-test penetration in the desired range for undisturbed modeling clay, the same result was achieved at a significantly lower temperature when the clay had been recently worked. This indicates an understanding by Army personnel that there are effects on the behavior of the clay from both the thermal history and the shear history of the clay.

Based on reports that have been reviewed by the committee and on what it observed at Aberdeen Proving Ground, the Army employs the following procedures to ensure a common history to the modeling clay used as a backing material in the testing of body armor:

  1. The modeling clay is heavily worked using mallets or tampers when the standard clay box is filled. The procedure described is consistent with standard practice by artists and others to remove behavioral artifacts in the material that are due to the manufacturing process. Thus, this procedure represents good practice.

  2. The clay boxes are thermally equilibrated for at least 3 hours at an elevated temperature, 104°F, prior to use. During drop calibration testing, standard practice is to insert two thermometers into the clay mass. The thermometers are inserted at points approximately along the box face diagonal one-quarter of the way from the opposing corners. The thermometers are inserted to a depth approximately halfway through the clay. The results of these measurements are used to confirm that thermal equilibrium has been established.

  3. All assembly operations (i.e., the attachment of the modeling-clay appliqué used to fill the space between the armor pack and the clay box), drop-test calibration, and armor tests must be completed within 45 minutes of the thermal equilibrium measurement. Empirical data from drop tests were generated by Army personnel to justify that the cooling of the clay box in this period of time results in changes to the drop-test calibration that are less than or equal to 6 mm (that is, within ±3 mm).

  4. The personnel who repair and recondition clay boxes follow procedures that are consistent with standard practice by artists and others for filling space without entrapping air. That is, small additions are made sequentially and each is heavily sheared by hand to express any entrapped air. This procedure represents good practice. Army personnel related that the periodic x-ray of clay boxes only very rarely indicates the presence of entrapped air. The same rebuilding procedure appears to be used both in rebuilding indentations produced during calibration and after an armor test to restore the

11

Scott Walton, ATC, and Shane Esola, ATC, “ATC Perspective on Clay Used for Body Armor Testing,” presentation to the committee, December 1, 2009.

Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×

box.12 However, at the conclusion test clay containing any debris (such as fabric, fragments of body armor, or projectiles) is removed prior to rebuilding the surface.

Given that modeling clay exhibits behavior that is dependent on both thermal and shear history, the committee wanted to assess whether working thermally equilibrated clay to recreate a flat surface would be expected to alter penetration depth. This issue has a bearing on the question of the validity of performing two sets of drop tests prior to using the clay block for a body armor test.

At the committee’s request, the following was done. A drop calibration was made, and the resultant hole was measured, then refilled, releveled, and retested. This was done twice in two separate locations on the surface of a single clay box filled with modeling clay that had been thermally equilibrated. In both cases the two impressions at the same location were within specification. These results support the view that the working of the clay to repair an impression does not significantly modify the behavior of the modeling clay, as shown in the calibration drop tests. One implication of this assessment is that the Army procedure of allowing a second calibration, run after the rebuilding of the surface of the clay box and thermal re-equilibration, does not invalidate the use of that box in an armor test.

Interim Committee Assessment of the Column-Drop Performance Test Method

It is the assessment of the committee that Army personnel demonstrated an awareness of the combined influence of thermal and shear history on the behavior of a modeling-clay body. Further, Army personnel appeared to follow good practice in rebuilding the surface of clay boxes both between drop calibrations (when necessary) and between uses of the clay box for subsequent armor tests. Finally, by following these procedures, as demonstrated to the committee, the clay backing placed behind the plates in subsequent BFD testing was appropriately calibrated.

The committee was provided the results of Army experiments designed to assess the consequence of cooling on the response of the clay. These experiments employed four separate clay boxes, all of which were thermally conditioned to achieve a uniform internal temperature of 104°F. The boxes were then subjected to drop tests over a certain period of time to observe how the clay response changed as cooling was allowed to take place. A randomized 4 × 4 grid was used, and the results of two drops were averaged for each data point. The results that were presented to the committee showed a roughly linear decrease in the penetration depth as a function of cooling for each of the clay boxes. After 45 minutes, the observed drop-test penetration was reduced by 6 mm. Owing to lot-to-lot variation in the clay, different clay boxes resulted in differing penetrations at 104°F. (It is these lot-to-lot variations that necessitate different thermal conditioning to achieve ”in-specification” drop-test results.) The rate of change with time was roughly consistent (i.e., the lines were parallel on the graph of the results). All of the results fell

12

Boxes are used for up to 200 shots or for a year. Appliqués are prepared for each individual armor plate to be tested and are never reused in whole or in part. After boxes have been used for their individual service life, the modeling clay is discarded.

Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×

within a band with a full range at any point in time of approximately 2 mm.


Finding 4. The column-drop performance test (including the testing protocols, facilities, and instrumentation) is a valid method for assessing the part-to-part consistency of clay boxes used in body armor testing.


In its Phase II task, the committee will further analyze the data provided by the Army, conduct an open-literature search for studies and analyses that bear on oil-based modeling clay (which is widely used as a model material in deformation studies in fields as disparate as metalworking and earthquake simulation in addition to ballistic studies), and make a detailed letter report of its findings.

ADDITIONAL COMMITTEE OBSERVATIONS

The committee attended a meeting of subject-matter experts from the Director of Operational Test and Evaluation (DOT&E), the Army Test and Evaluation Command, and the Army Research Laboratory, working together to develop protocols for the testing of body armor that address sample size and statistical confidence. This effort would appear to be consistent with the recommendation in the DOD Inspector General Report DoD Testing Requirements for Body Armor that the DOT&E develop a test operations procedure for body armor ballistic inserts that uses “quantitative methods to develop a sample size for testing” (DOD, 2009). The committee applauds the effort to develop a statistically based protocol, based on appropriate confidence and tolerance intervals, for use in the testing of body armor ballistic inserts.

The committee also observes that it may be helpful, when collecting test data, to consider using ASTM E29-08 (ASTM, 2008a) to establish procedures for rounding and indicating the appropriate number of significant digits as related to measurement precision and the statement of specifications.

Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×

REFERENCES

ASTM (American Society for Testing and Materials). 2008a. ASTM E29-08 Standard Practice for Using Significant Digits in Test Data to Determine Conformance with Specifications. West Conshohocken, Pa.: ASTM International.

ASTM. 2008b. ASTM E177-08 Standard Practice for Use of the Terms Precision and Bias in ASTM Test Methods. West Conshohocken, Pa.: ASTM International.

ATC (U.S. Army Aberdeen Test Center). 2008a. Quantum FARO® Arm Laser Scanning Body Armor Back Face Deformation. Report No. 08-MS-25. December 11, 2008. Aberdeen Proving Ground, Md.: Aberdeen Test Center.

ATC. 2008b. Test Operations Procedure (TOP) 10-2-210 Ballistic Testing of Hard Body Armor Using Clay Backing. Aberdeen Proving Ground, Md.: Aberdeen Test Center.

Chadwick, E., A. Nicol, J. Lane, and T. Gray. 1999. Biomechanics of knife stab attacks. Forensic Science International 105(1): 35-44.

DOD (Department of Defense). 2007. CO/PD 04-19C Purchase Description Personal Armor, Enhanced Small Arms Protective Insert. Arlington, Va.: Department of Defense.

DOD. 2009. DoD Testing Requirements for Body Armor. Report No. D-2009-047. Arlington, Va.: Department of Defense Inspector General.

Huber, J. 2009. Internal Operating Procedure No. 001: Measurement of Backface Deformation [BFD] Using Faro® Quantum Laser Scan Arm and Geomagic® Qualify® for Hard and Soft Body Armor. Aberdeen Proving Ground, Md.: Aberdeen Test Center.

ISO (International Organization for Standardization). 2007. International Vocabulary of Metrology—Basic and General Concepts and Associated Terms (VIM). ISO/IEC Guide 99:2007. Geneva, Switzerland: ISO.

Karahan, M. 2008. Comparison of ballistic performance and energy absorption capabilities of woven and unidirectional aramid fabrics. Textile Research Journal 78(8): 718-730.

Lyon, D. 1997. Development of a 40mm Nonlethal Cartridge. ARL-TR-1465. Defense Technology Federal Laboratories Research Journal.

McNeese, W., and R. Klein. 1991. Measurement systems, sampling, and process capability. Quality Engineering 4(1): 21-39.

O’Callaghan, P., M. Jones, D. James, S. Leadbeatter, S. Evans, and L. Nokes. 2001. A biomechanical reconstruction of a wound caused by a glass shard—A case report. Forensic Science International 117(3): 221-231.

Prather, R., C. Swann, and C. Hawkins. 1977. Backface Signatures of Soft Body Armors and the Associated Trauma Effects. Aberdeen Proving Ground, Md.: Chemical Systems Laboratory.

Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×

Rigney, G. 2008. Memorandum for Record: Testing of Faro® Laser Scanner. September 19, 2008. Aberdeen Proving Ground, Md.: Aberdeen Test Center.

Vaughan, N. 2001. Assessment of Aprons for Protection Against Drop Forging Projectiles. Contract Research Report 395/2001. Sheffield, UK: Health and Safety Laboratory.

Walton, S., A. Fournier, B. Gillich, J. Hosto, W. Boughers, C. Andres, C. Miser, J. Huber, and M. Swearingen. 2008. Summary Report of Laser Scanning Method Certification Study for Body Armor Backface Deformation Measurements. Aberdeen Proving Ground, Md.: Aberdeen Test Center.

Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×
Page 1
Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×
Page 2
Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×
Page 3
Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×
Page 4
Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×
Page 5
Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×
Page 6
Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×
Page 7
Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×
Page 8
Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×
Page 9
Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×
Page 10
Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×
Page 11
Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×
Page 12
Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×
Page 13
Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×
Page 14
Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×
Page 15
Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×
Page 16
Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×
Page 17
Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×
Page 18
Suggested Citation:"Letter Report." National Research Council. 2009. Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12837.
×
Page 19
Next: ATTACHMENT A STATEMENT OF TASK »
Phase I Report on Review of the Testing of Body Armor Materials for Use by the U.S. Army: Letter Report Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The National Research Council of the National Academies established a study to assess the methodologies used by the U.S. Army for the testing of body armor. This Phase I report is focused primarily on the validity of laser-profiling techniques for body armor test measurements. More comprehensive and detailed evaluations of an array of issues surrounding body armor testing will be presented in the forthcoming Phase II and Phase III reports.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!