Engineering Aviation Security Environments—Reduction of False Alarms in Computed Tomography-Based Screening of Checked Baggage (2013)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

3 Alternative Approaches for the Reduction of False Alarms In this chapter the committee encourages the Transportation Security Administration (TSA) to look beyond CT in addition to driving computed tomography (CT) to its best performance and improving the algorithms for detecting threat items in the resulting images. The chapter discusses four potential approaches to reducing false alarms in the field: (1) using multiple CT scans to improve the probability of detection (PD), (2) using mass spectrometry, (3) employing x-ray diffraction technology, and (4) and incorporating data from other sources. The material presented is preliminary, but further consideration and study of these approaches have the potential to positively affect the efforts to reduce false alarms in the screening of checked baggage in U.S. airports. AN ALTERNATIVE APPROACH: MULTIPLE SCANS USING EXISTING TECHNOLOGY One of the ways to improve the performance—that is, to reduce the probability of false alarms (PFAs) and to improve the probability of detection—of CT-based explosive detection system (EDS) scans is to increase the number of cross-sections that the machine takes of an object. More cross-sections usually lead to a better probability of correct discrimination in recognizing whether an object is a threat or a non-threat. The number of cross-sections that a machine takes can be increased either by changing the current hardware or by passing a bag through the CT scanner multiple times in such a manner that the bag is positioned somewhat differently for each scan. When a bag is passed through the CT scanner multiple times, a natural decision-making situation arises—that is, a decision rule is needed with respect to when a bag should be declared a possible threat and sent for manual inspection. The committee offers the following statistical model to present the idea that false alarms can be reduced using the current hardware in a different way. Although according to this model the same bag is being scanned multiple times, the presumption is that neither the machine nor the operator knows that it is the same bag, and in this way the scans remain “independent.” This is reasonable because the bag will most likely be positioned somewhat differently at each pass because of the bumps on the conveyor belt. This may not be possible for unusually large items; however, many of them are manually processed already. This “independence” can also be increased by using multiple machines for the different scans each of which are set to provide cross-sections at slightly different depths. Suppose that a bag is scanned N number of times and out of these N scans, the machine alarmed on it as a potential threat q number of times. The natural questions that arise at this stage are as follows: (1) How many times should a bag be scanned (that is, what is the value of N)? and (2) At what value of q should the bag be declared a potential threat that should be sent for manual inspection? Now suppose that a bag, in fact, contains a threat object that can potentially be detected by a CT scanner. Suppose that the probability of detecting the threat object in a single scan is δ. In this hypothetical, the total number of scans is fixed as N and the decision rule is that the bag is declared a threat (and hence should be sent for manual inspection) if the bag is alarmed q times out of N scans. Then 29 

assuming that the scans are independent of each other, this can be computed by using the binomial distribution 1 as This is the probability of correct detection under the decision rule specified above. Similarly, suppose that the bag does not contain a threat object, but a single CT scan may declare it falsely as a potential threat with probability α. Then, the probability that such a bag will be sent for manual inspection is again obtained by using the binomial distribution as This is the probability of false alarm under the decision rule specified above. Different values of N and q lead to different PDs and PFAs. The detailed description of the solution and computational details are provided in Appendix D in this report. Table 3-1 presents the probability of false alarms and probability of correct detection under different values of N and q in the above decision rule. TABLE 3-1 Probabilities of Correct Detection and of False Alarm for Some Combinations of N and q N q Correct Detection False Alarm 2 1 0.99 0.36 2 0.81 0.04 3 1 0.999 0.488 2 0.972 0.104 3 0.729 0.008 4 1 0.9999 0.5904 2 0.9963 0.1808 3 0.9477 0.0272 4 0.6561 0.0016 5 1 0.99999 0.67232 2 0.99954 0.26272 3 0.99144 0.05792 4 0.91854 0.00672 5 0.59049 0.00032 independent interrogation of a random object. In this scenario, 𝛼 NOTE: These probabilities assume that each scan represents an = 0.2 and 𝛿 = 0.9. 1 G. Casella and R.L. Berger, Statistical Inference (2nd ed.), Duxbury Press, Pacific Grove, Calif., 2002. 30 

A typical entry in the table is read as follows: Suppose that the decision rule is such that a bag is declared a threat if it tests positive at least three times out of the total of five scans. Then such a decision rule will detect the threat correctly 99.14 percent of the times and will give a false-positive alarm 5.79 percent of the times. Presuming the committee’s assumption regarding the independence of the scans is correct, it is clear that multiple scanning can reduce the probability of false alarms, at the same time increasing the probability of correct detection of threats (those that can potentially be detected by a CT scan). If scans are automated it will allow for a greater likelihood of accurate results with no additional personnel costs. However, there will be other costs associated with automating the rescanning, such as increased screening time and additional routing hardware, as well as the costs of tracking the multiple scans of the same bag, and these must be offset by an improvement in false-alarm rates. ANOTHER ALTERNATIVE APPROACH: CHEMICAL ANALYSES One task in the charge to the committee was an examination of the problem of explosives detection by new non-certified methods. This section deals with the use of mass spectrometry to address this charge. Box 3-1 features an extract a 2004 National Research Council report on the subject. Research programs, both internal to TSL and externally supported, have examined the capabilities and potential screening application of mass spectrometers. Nevertheless, there exists the perception that mass spectrometers are too complex, difficult to operate, and insufficiently rugged for deployment. Recent advances in mass spectrometry—mostly made since 2004—have dramatically changed the capabilities of this instrumentation to the extent that accelerated development of baggage and passenger screening methodologies now seem worth revisiting. These advances include (1) the invention of a number of ambient ionization methods that are rapid and do not require any sample preparation and (2) the continued development of small, highly capable mass spectrometers to which these ambient ionization methods can be fitted. These advances are detailed in Appendix C. The recent advances in mass spectrometry ionization methods have yielded processes (such as ambient ionization) that provide mass spectra from materials on solid surfaces in air without sample preparation and in almost real time. These capabilities, some commercially available, could be implemented in trace explosives screening of the external or internal surfaces of baggage using existing commercial mass spectrometers. These new ambient ionization methods have also been implemented on handheld mass spectrometers in research laboratories, a combination that provides high chemical specificity and sensitivity in a small device. Ambient ionization methods can also be used to examine wipes after they have been used to transfer material from baggage in the course of secondary screening. These new methods may offer the sensitivity, speed, and chemical specificity to warrant scrutiny and testing by the Transportation Security Laboratory (TSL) as a possible supplement to or replacement for traditional ion mobility measurements as a secondary baggage-screening device. While additional training may be required, this technology can likely be implemented with limited—if any—additional manpower if it is used to replace the secondary screening that is already in place. If used as a supplement, some additional manpower may be required and this is a trade-off that may need to be considered with regards to the over-all false-alarm rate reduction. X-RAY DIFFRACTION TECHNOLOGY X-ray diffraction technology uses energies in the 30 to 80 kiloelectronvolt range to interact with matter, using diffraction and photoelectric absorption to measure the spacing of crystalline materials within the atomic lattice or the arrangement of atoms in a chemical compound. Because the interaction of 31 

BOX 3-1 Potential Value of Mass Spectrometry in Aviation Security Screening According to Previous National Research Council Reports A 2004 report from the National Research Council, Opportunities to Improve Airport Passenger Screening with Mass Spectrometry,1 has addressed the potential value of mass spectrometry (MS) in aviation security screening. The following recommendations provide an overview of the advantages of this technology: TSA should establish mass spectrometry as a core technology for identifying an expanded list of explosives, as well as chemical and biological agents. Specifically, TSA should • Create a prioritized list of threat materials that are likely to fit a residue scenario and a second list of materials that are not likely to fit the scenario. • Determine appropriate MS [mass spectrometry] sampling procedures, inlet configurations, ionization methods, and analysis strategies for relevant materials on this list.2 If TSA wishes to improve its trace detection capabilities, it should deploy MS-based detectors in a phased fashion, with successive generations of instruments addressing lower quantities of an expanded list of threat materials and more sophisticated security tasks. These tasks range from passenger screening at checkpoints to monitoring of the air handling system.3 1 National Research Council, Opportunities to Improve Airport Passenger Security Screening with Mass Spectrometry, The National Academies Press, Washington, D.C., 2004. 2 Ibid., p. 6. 3 Ibid., p. 7. the energy with the material is chemically specific, some materials contribute more to the false alarm rate than others. X-ray diffraction technology is commercially available and worth consideration as a source of data to help resolve false alarms. INCORPORATING DATA FROM OTHER SOURCES In addition to data from explosive trace detection technology, such as mass spectrometry, data from other sources such as carry-on-baggage and passenger-screening checkpoints, perimeter-surveillance data, and even information about passengers’ behavior or travel habits can be used to inform the screening process (e.g., selected passengers’ bags might be subjected to a more sensitive level of screening. The 2007 report of the National Research Council’s Committee on Assessment of Security Technologies for Transportation, Fusion of Security System Data to Improve Airport Security, 2 provides guidance on how best to make use of data from multiple systems (see Box 3-2). 2 National Research Council, Fusion of Security System Data to Improve Airport Security, The National Academies Press, Washington, D.C., 2007. 32 

BOX 3-2 Systems Approach to Data Fusion The following material is reprinted from the 2007 National Research Council report entitled Fusion of Security System Data to Improve Airport Security:1 For the Transportation Security Administration (TSA) to move from the recognition of data fusion as a key technology for transportation security to having an effective plan for implementing data fusion solutions requires a systems approach. This approach would provide the programmatic basis for integrating security systems for checkpoints, checked-baggage screening, and access control. Key outputs from this systems approach that will enable the successful implementation of data fusion are the following: 1. A set of data standards (e.g., Extensible Markup Language [XML]) for the integration of data from security systems and security personnel; 2. A path for the growth and migration of passenger pre-screening as an input to data fusion; 3. Reference frames for exchanging locational data at all levels from within bags to within airports; 4. Standards for the identification of explosives, hazardous materials, and items that appear as hazardous but are not; 5. Common measures of uncertainty for all data inputs and validated error rates from security systems; 6. Data structures for radio-frequency (RF) tagging and other object identification and marking; 7. Ontologies for potential threat objects, systems, subsystems, and scenarios in baggage screening, checkpoints, and airports that enable the linking of alerts, observations, and historical data and provide for dynamic threat assessment; 8. Data structures for airport and airport perimeter kinematics with a particular focus on trajectories; 9. Visualization methods that enable distributed situational awareness and assessment; 10. Standardized data structures for access control, including biometrics; and 11. Standardized data interfaces for access control with facility security. 1 National Research Council, Fusion of Security System Data to Improve Airport Security, The National Academies Press, Washington, D.C., 2007, p. 44. 33