The Practicality of Pulsed Fast Neutron Transmission Spectroscopy for Aviation Security (1999)

A legitimate comparison of x-ray CT-based EDSs and PFNTS-based explosives-detection techniques must be based on their baseline performance. Laboratory test results for PFNTS were presented in Chapter 3. In this chapter, both the laboratory performance and the operational performance of the FAA-certified CTX-5000 SP are reviewed. To date, most of the performance data on deployed explosives-detection equipment has been generated from tests conducted at the FAA Technical Center. However, operational (field) test data on false alarm rates are also reviewed in this chapter.

Most of the performance data available for x-ray CT-based EDSs are from certification tests of the InVision CTX-5000 SP in 1996. Data from the certification tests of the recently certified InVision CTX-5500 DS and L3 Communications 3DX-6000 are not available yet but should be included in future analyses. The InVision CTX-5500 DS has been certified for two different inspection modes: SURE98 Mode, which has a lower false alarm rate than the CTX-5000 SP but a lower throughput rate, and CERT98 Mode, which has a similar false alarm rate to the CTX-5000 SP but a much higher throughput rate. The panel's analysis of performance data focuses on the CTX-5000 SP, although performance data for the CTX-5500 DS are also presented for reference. Table 4-1 shows the performance factors for the CTX-5000 SP and CTX-5500 DS, including the P_d, P_fd, and the bag throughput rate. Because the actual P_d and P_fa numbers are classified,¹ P_d is given as a percentage of the overall P_d required for certification (X), and P_fa is given as a percentage of the P_fa required for certification (Y).

In 1995, the FAA initiated the Airport Operational Demonstration Project (FAA, 1995) to determine the operational performance of the InVision CTX-5000 SP in the field as compared to its performance in certification tests. Three test sites were selected for the project: San Francisco International Airport (United Airlines); Atlanta Hartsfield International Airport (Delta Airlines); and Manila International Airport (Northwest Airlines).

Two InVision CTX-5000s were installed in Atlanta and one each in San Francisco and Manila. The demonstration project included four open tests and one blind (red team) test using improvised explosive devices to determine P_d. The P_fa was acquired routinely throughout the project on real passenger bags. Only the data from the San Francisco and Atlanta operational deployments have been documented in final reports (FAA, 1997a, 1997b, 1997c).

Some of the performance data for the CTX-5000 SP installed at San Francisco International Airport are given in Table 4-2. The automated explosives-detection capability of the CTX-5000 SP during open testing was about the same as the capability measured at the FAA Technical Center. However, operator intervention to resolve alarms measurably reduced P_d. This tendency was also observed during blind testing at San Francisco and Atlanta (FAA, 1997a, 1997b). During the operational demonstration project at San Francisco, the automated P_fa was 113 percent to 150 percent higher than the certification standard. Operational data reviewed by the inspector general of the U.S. Department of Transportation suggested that false alarm rates were as much as 169 percent higher than the certification standard. Data from the first of the four open tests show an average of 50 seconds for alarm resolution using the CTX-5000 SP. Although the resolution time was lower during subsequent tests, the combination of a high P_fa and a long alarm resolution time can have a significant impact on throughput rates. In fact, it was determined to be the limiting factor for throughput rate.

Although the CTX-5000 SP exhibited a significantly higher automated (i.e., without operator intervention) P_fa in operation at airports than during certification testing, the

image produced by the system is used by trained operators to assist in resolving the alarms. Thus, the actual P_fa (with operator intervention) is much lower than the automated P_fa measured during certification testing. Nevertheless, the time required for alarm resolution is a factor that must be taken into consideration when evaluating the performance of this EDS.

The FAA's Security Equipment Integrated Product Team (SEIPT) has deployed more than 40 InVision CTX-5000 SP EDSs since 1997. Thus, the FAA now has an outstanding opportunity to collect performance data (e.g., P_d and P_fa) as well as operational performance data (e.g., down time, mean time between failure, and mean time to repair). Unfortunately, this information was not available to the panel, which substantially compromised the panel's ability to assess the performance of deployed equipment and to make comparisons between deployed equipment and alternative technologies, such as PFNTS. Credible future assessments of the performance of deployed explosives-detection equipment will require both performance and operational data.