Testing and Evaluation of Standoff Chemical Agent Detectors (2003)

The detection of chemical warfare agents is a topic that attracts a good deal of attention in the current environment of the war on terrorism. It is well, therefore, to point out at the outset what this report does and does not cover because the broad topic is a critical one with many aspects that need to be addressed urgently.

The Department of Defense (DoD) is responsible for ensuring that standoff CWA detectors will meet user needs under battlefield and homeland defense applications. In fulfilling this responsibility, the Defense Threat Reduction Agency (DTRA) of the DoD sponsored a study by the Battelle Memorial Institute (BMI), The Use of Chemical Agent Simulants in Standoff Detection Testing.⁵ This study issued a final report dated October 2001 which recommended that “using simulants instead of CWAs provides an effective means for conducting outdoor operational testing of standoff detection instruments,” Simulants are chemicals that are less toxic yet approximate the optical and physical characteristics of CWAs used in the detection scheme. Review of the report throughout the DoD establishment raised concerns around the validity of its conclusion based on the proposed test protocol and the supporting information provided in the report. The concern by the DoD that a testing protocol, absent field testing with live CWAs, might not adequately predict the response of these detectors to CWAs in battlefield and homeland defense applications seems intuitively but perhaps not scientifically valid.

DTRA requested that the National Research Council (NRC) undertake an independent assessment of appropriate protocols for testing and evaluating standoff detectors and their expected reliability. The Statement of Work drafted by DTRA that specifies the desired product of this study is given in Appendix A. Testing and evaluating standoff detectors for CWAs and assuring their reliability in actual field environments require the utilization of several technologies. To carry out this study, the NRC assembled

a group of leading experts in the fields of infrared spectroscopy, laser-based measurements such as light detection and ranging (lidar), chemometrics, neural networks, aerosols, atmospheric sciences, and risk assessment (see Appendix B). Of necessity, the study was carried out on a relatively tight schedule, with the first meeting of the committee on November 7-8, 2002, and a second on November 25-26, 2002. The committee had vigorous discussions and debates on the major issues associated with each of the testing protocols and came to unanimous agreement on the recommendations contained in this report.

This report is a narrow and very specific study of the testing and evaluation of infrared-type standoff detectors for CWAs in military situations.⁶ It is not a broad study of various possible methods of detecting CWAs, nor is it a study of the use of such detectors in applications such as homeland security. Rather, it is a study of the best possible way to evaluate such detectors in a realistic way that will ensure that they will detect what they are supposed to, primarily in a wartime scenario. The specifics of field applications, while relevant in the deployment of such detectors, were important in the present study only in the sense that the study had to recognize the breadth of situations in which these devices would be used during and after the military deployment of CWAs.

Although this study is narrowly focused, the committee addressed a number of critical issues. The first issue concerned which types of detectors were to be included in any test protocols. The BMI report focused entirely on passive infrared detection since this technique has been used by the DoD for about 20 years. The committee determined that, while this was current technology, its logical extension was to consider infrared detectors based on lidar technology as well, since this technology was likely to be incorporated in such devices in the not-too-distant future. Hence test protocols were discussed by the committee in the context of these two types of instruments. There was no discussion of alternative methods of detecting CWAs using different technologies as this was considered beyond the charge to this committee.

The second critical issue was the value of field testing of these devices with CWAs. The BMI report tried to make a case for using simulants in field tests to avoid the use of CWAs in such tests. Unfortunately, the protocol recommended in the BMI report did not technically support this conclusion. The present study provides more complex but necessary test protocols that eliminate the need for field-testing with CWAs, a highly desired goal for the testing and evaluation of these detectors.

The third critical issue was the risk assessment of responses from such detectors. These detectors are designed to measure CWAs in the atmosphere. The response of these detectors, like the response of any analytical instrumentation (no matter how simple or sophisticated) has a certain error associated with it. The testing and evaluation of these detectors will provide an assessment of that error and how it is related to the various environments in which the detector is deployed. Nevertheless, the response of the detector, like the result of any analytical measurement, must be interpreted in the context of how that result is to be used. In the case of standoff chemical detectors, the results (an alarm or no alarm) have to be interpreted in the context of actions that should be taken by the troops in a military situation, or a population, in a civilian environment. Thus the report discusses some examples of the risk assessment and indicates in these examples how the information from the detectors must fit into the decision-making process.

The complexities of the protocols recommended in this report are primarily driven by two factors: the lack of predictable backgrounds in which the CWAs are to be detected and the variable chemical nature of the CWAs that results in a wide range of infrared absorption bands. First, the unpredictable

nature of the backgrounds in which these detectors will be used and expected to give reliable alarm information requires that the test protocol is based on data from a very large number of varied backgrounds. The CWAs to be detected must be discriminated against (i.e., detected in the presence of) any type of background. With passive infrared detection, there is no way to simplify this step. Second, because of the wide range of chemical structures of all possible CWAs, the detection scheme must be based on the fact that each potential CWA has a unique infrared absorption spectrum. Thus the CWA must have one or more infrared absorption bands in the spectral window that are being measured that are unique and not found in the background spectra. Detecting and measuring these spectral features in the presence of all the background spectra using sophisticated chemometric techniques requires statistical testing of simulants in both laboratory and field environments as well as laboratory testing of the various CWAs along with complex signal processing. It is these factors that result in the large amount of testing needed to assure detector performance.

If testing and evaluation of this class of detectors for CWAs are to be addressed in a way that provides reliable detection equipment, certain additional information will be necessary. This report recommends research that will be necessary to support the recommended test protocols.

The committee’s protocols do not call for field testing with CWAs, and the committee presents justification to show that this step is not necessary to provide statistically valid protocols.