Testing and Evaluation of Standoff Chemical Agent Detectors (2003)

Testing and evaluation of standoff chemical warfare agent (CWA)⁷ detectors to certify their suitability and reliability for field applications present a number of challenges. Two major challenges are (1) these detectors cannot readily be tested in a field environment with CWAs without extensive precautions and legal approvals by top government officials, and (2) the wide variety of environments in which the detectors might be used may significantly affect instrument performance. Both issues must be addressed in any test protocol evaluating such detectors in order to certify their suitability for use in the field.

This report is confined to two types of standoff detectors, so-called passive and active detectors that are based primarily on the infrared spectral properties of the CWAs, although the protocol for active detectors could also be applied to other wavelength regions. Passive detectors are those that record the infrared spectrum emitted or absorbed by the CWA relative to the surrounding background with the only source of excitation energy being from the ambient background. Active detectors employ transmitted infrared laser radiation that is scattered back to a co-located receiver from aerosols in the atmosphere or from a topographic target. The laser energy is attenuated by the natural atmosphere, the CWA, and other aerosols and gases in the battlefield environment. In both cases the detected radiation is analyzed spectrally and temporally with sophisticated algorithms to extract the signal due to any CWA present in the field. However, there is a fundamental difference regarding the ability to predict the signal source term for the two approaches. For an active detector the signal can be mathematically predicted from the detector system characteristics and from the scattering and absorption cross sections of the aerosols and gases as well as the topographic target characteristics that are present, allowing a complete model to be developed of the measurement process and uncertainties. On the other hand, the source term for passive detection (radiation from a widely varying background) cannot be accurately predicted to develop such

an end-to-end model. This fundamental difference between the analysis of active and passive detection requires significant differences between the two protocols for testing and evaluation of these two types of instruments. The two detector types will have inherently different capabilities. The passive detector will work best when the CWA is in the vapor phase, whereas the active detector has the potential of working better when the CWA is in the form of an aerosol.

The technology incorporated in the existing standoff detectors for CWAs and other devices under near-term development is based on the infrared spectroscopic properties of the CWAs and the differentiation of these properties against the background wherever the detectors are deployed. While such properties of CWAs are unique to the agent, the background in which such detection may be used can be extremely complex from a spectroscopic standpoint. Therefore, the infrared signal that is gathered by the detector must be extensively analyzed in real time utilizing chemometric and/or neural network techniques in order to differentiate a signal resulting from a CWA from the background and other interfering substances (either deliberately released to the environment or occurring accidentally). Adding to the complexity of the spectra is the physical nature of the test environment. CWA release can occur in several ways, including artillery shell bursts, release from aircraft, ground release, and other means. Each of these release mechanisms will likely result in a somewhat different physical form of the CWA and a different localized background signal. In addition, specifically designed mixtures of non-CWA materials could produce spectral properties similar to those of CWAs and might be used to “fool” (spoof) or even “blind” (saturate or completely attenuate the signal of) the detectors.⁸

While logically desirable, testing and evaluation of any test instrumentation under actual operational conditions is not always possible or practical. This is especially true for standoff detectors for CWAs. Using CWAs in field testing presents inherent technical, safety, and environmental concerns as well as significant political, financial, and time costs. The approval process for such testing under existing national laws and international treaties on chemical weapons is extensive. It has been estimated that the financial cost alone for a single field test with a CWA is approximately $136 million whereas a comparable test with a simulant is significantly lower, about $400,000.⁹ Even under the best protocol, a single field test of a standoff detector with a single CWA will not provide sufficient information on the reliability of these test devices for the variety of conditions in which they are expected to operate.

Simulants, on the other hand, are relatively benign materials that can be tested in the open environment at considerably lower cost. Ideally, a test protocol could include field testing of a given detector with simulants coupled with laboratory crossover studies with simulants and CWAs in order to develop confidence in the detector for field detection of CWAs.

This report discusses and addresses the questions raised by the DoD with respect to the value of testing with simulants and CWAs in chamber and field conditions. The committee recommends test protocols for standoff devices and provides the supporting rationale for such protocols. The test protocols recommended in this report are intended to test standoff detectors for the broadest possible field applications. Failure to pass certain steps in the protocol with increasingly complex backgrounds, will define limitations of that particular instrument in its field of application. For example, the instrument may work acceptably in situations where the background is sky or sea, but not a background with buildings, trees etc. It may be the judgment of the government contractor that a particular instrument

while limited in its fields of application can still be useful for specialized deployment. Risk assessment of the proposed test protocols is provided within the context of the scientific test procedures being recommended but is also discussed more broadly in the framework of decisions that must be made in the field. Various tactical considerations of risk that relate to field strategies for the use of these detectors are beyond the scope of this study and are therefore not addressed here. Initially, this report addresses the test protocol and underlying assumptions recommended in the Battelle report. The recommended test protocols resulting from this committee’s discussions and the rationale for them are presented subsequently. There are also identified areas of research and data compilation that are required to support the proposed test protocols. Standoff detectors that can pass the test protocols recommended by the committee will provide high reliability in field performance for the detection of CWAs. The challenge for instrument design and manufacture is to provide standoff detectors that will measure up to these protocols.