Engineering Aviation Security Environments—Reduction of False Alarms in Computed Tomography-Based Screening of Checked Baggage (2013) / Chapter Skim
Currently Skimming:
Summary and Recommendations
Summary and Recommendations
Pages 1-11
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 1... ...
Now that CT-based detection systems have been in use for more than 10 years, TSA seeks to improve the performance of its baggage screening systems through such measures as better detection algorithms and more effective EDS equipment -- especially to reduce the number of false alarms and thereby reduce the costs of screening checked baggage. This report, from the National Research Council's Committee on Engineering Aviation Security Environments -- False Positives from Explosive Detection Systems, examines potential technical enhancements, opportunities to foster innovation, and data requirements for reducing the false alarm rate (the committee's full statement of task appears in Appendix E)
|
From page 2... ...
IMPLICATIONS OF A FALSE POSITIVE RATE The TSA estimates that each percentage point of the current false alarm rate costs the government tens of millions of dollars per year. The main element of these costs for resolving false alarms is the total number of personnel required to screen baggage in U.S.
|
From page 3... ...
Reducing the false alarm rate without increasing the rate of false negatives would free up resources to develop and deploy capabilities for identifying explosive materials. 3 TECHNICAL APPROACHES TO REDUCING THE FALSE POSITIVE RATE Although there is no way to completely eliminate all false alarms, there are a number of potential technical approaches that would improve the false positive rate.
|
From page 4... ...
Thus, improvements in the image reconstruction and correction process could lead to a lower false alarm rate. • Slow the bag-processing speed.
|
From page 5... ...
Recommendation: The Transportation Security Administration, through the Transportation Security Laboratory, should support human-factor studies to assess the impact on overall system performance, that is, the EDS plus the screener resolution, when the operating point on the explosive detection system's receiver operating characteristic curve is adjusted so that both the probability of detection and probability of false alarm are lowered. If the results of such studies determine that screener attention is degraded by the expectation that every alarm is a false alarm, the TSA should consider implementing adjustments to the operating point on the receiver operating characteristic curve and allowing vendors to reduce probability of detection in an airport setting to the minimum rate required for certification.
|
From page 6... ...
Under a PBL-based contract, the government and the equipment vendor work together to determine key performance indicators for the equipment, and the government provides incentives for the vendors to invest in improvements with a reasonable expectation that these improvements will be evaluated and implemented if successful. Recommendation: In order to better capitalize on improvements and provide vendors with the necessary incentives to invest in research that will lead to better performance metrics, the TSA should consider adoption of a different contract structure for the procurement and maintenance of the computed tomography-based explosive detection systems used for checked baggage, as well as for other screening technologies.
|
From page 7... ...
Recommendation: The TSA should develop a plan to provide appropriate incentives not only for EDS vendors but also for third parties and researchers in academia in order to improve the overall performance of computed tomography-based EDSs, including their rates of false alarms. Incentives should be provided for both short- and longer-term improvements.
|
From page 8... ...
CERTIFICATION TESTING AT THE TRANSPORTATION SECURITY LABORATORY IS USEFUL BUT DOES NOT REFLECT REAL-WORLD CONDITIONS Certification testing of EDSs and subsequent performance testing in an airport setting is one source of information on EDS performance and causes of false alarms. To be certified, a machine must demonstrate the ability to detect a number of categories of explosives, with each category having a specific detection threshold (i.e., level of detection that must be met)
|
From page 9... ...
USE A DATA-DRIVEN APPROACH TO REDUCE FALSE POSITIVES A detailed, quantitative understanding of the root causes of false positives is important if the TSA is to reduce the costs associated with these false positives without increasing other risks. For instance, the overall false alarm rate includes two distinct "populations" of bags, each of which would require a different approach to reducing false alarm rates: • The first population includes bags for which the EDS cannot make a decision -- so-called "exceptions," such as bags containing solid objects that cannot be penetrated by the EDS x-rays, mistracked bags, and bags that are poorly positioned in the EDS in such a way that the EDS cannot interrogate the entire bag ("cut bags")
|
From page 10... ...
Finding: The low prevalence of the true positives in an airport setting may make it nearly impossible to measure probability of detection with humans-in-the-loop without forcing true positives via red-team testing. Finding: Discussion with TSA officials, airport personnel, and vendors indicates some limitedscale data collection and laboratory studies that have enabled the sources of false alarms to be broadly identified.
|
From page 11... ...
These analyses should also be used to better understand the sources of false positives by determining the dependence of these probabilities on material characteristics of potential explosives threats, the variability in the material characteristics, and the characteristics of non-threat materials typically contained in checked bags. These estimates should then be used as baselines for determining the ability of potential improvements to reduce false alarms.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.