Listed below are some of the main themes arising from each panel session. The themes are not conclusions or findings of the committee; they are ideas extracted from each panel that seem to be the main thrusts of each discussion. Each panel session discussion is more fully elaborated in Chapter 2.
In Session 1, participants from industry, government, and academic research centers discussed the state of the art of biometric systems, the current bottlenecks, and areas where performance could be improved. Among the different types of biometrics, three were highlighted by the panelists—fingerprint, iris, and face—as being those accepted by the International Civil Aviation Organization for use in border-crossing documents. All panelists agreed that biometric systems cannot be made perfect—that is, the focus should be on how to evaluate and reduce, rather than eliminate, error rates. The challenges relevant in varying degrees to all biometric systems were grouped into three categories by the panelists, with primary emphasis during this discussion given to the first category of challenges.
Improving the accuracy of biometric technologies and related performance evaluations through research on sensor resolution and ergonomics, algorithms and techniques for biometric fusion, characteristics of biometric feature spaces, and scientific methods to better quantify biometric systems’ performance under realistic conditions.
Systematically and thoughtfully integrating biometric systems with other security systems.
Promoting interoperability of biometric systems, especially internationally, through a framework of standards, test methodologies, and independent evaluations.
Session 2 explored issues surrounding the measurement, statistics, testing, and evaluation of biometrics and biometric systems. It should be noted that statistical analysis in the context of biometric systems is and can be employed for a range of purposes, including assessments of the underlying technology, analysis of user behavior, data mining, and so on. Indeed, such issues were discussed throughout the workshop in several different contexts. Questions raised for this panel included these: Do biometric systems work? What is meant by “work” in the context of a biometrics system? What is being measured, tested, and evaluated, and how can confidence in the experiments be created? The panelists presented a range of perspectives on these issues, from broad explorations of the nature of experimentation and representative populations to discussions of specific evaluation regimens and real-world deployment at a major international airport. Several overarching themes arose:
Evaluating biometric systems serves three purposes: to guide and support research and development, to assess the readiness of a system for deployment, and to monitor performance of a system in the field.
As in many other domains, appropriate experimental design and solid statistical underpinnings are needed to produce effective testing and evaluation regimes. There is no one-size-fits-all solution, given the many types of systems that are deployed.