Ballistic Imaging (2008)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Executive Summary Since the late 1980s computerized imaging technology has been used to assist forensic firearms examiners in finding potential links between images of ballistics evidence gathered from crime scene investigations, namely, cartridge cases and bullets from fired guns. To support this effort, the Bureau of Alcohol, Tobacco, Firearms, and Explosives (ATF) in 1997 formed the National Integrated Ballistic Information Network (NIBIN). Law enforcement agencies participating in NIBIN contribute to a database of images of bullet and cartridge case evidence recovered from (or test- fired from ­ weapons linked to) crime scenes. This system facilitates rapid comparison with archived evidence and with evidence gathered at other crime sites; when matches look promising, the physical evidence can be retrieved for direct examination and confirmation by an examiner. NIBIN was designed as a tool for search, not for verification, which is always done by an examiner. The rapid development of computerized ballistic imaging technology has led to speculation about its future potential. A particularly interest- ing proposal is to create a national reference ballistic image database (RBID) that would house images from firings of all newly manufactured or imported firearms. Proponents of this proposal argue that such a database could provide a quick investigative lead from evidence recovered at a crime scene to the underlying firearm’s original point of sale. State RBIDs already exist in Maryland and New York, and wide attention was drawn to the issue when California studied the feasibility of creating its own RBID. In 2004 the National Institute of Justice (NIJ) of the U.S. Department of Justice requested that the National Academies appoint a committee of  

 BALLISTIC IMAGING experts to address the issues raised by the computerized ballistic imaging technology. The Committee to Assess the Feasibility, Accuracy, and Tech­ nical Capability of a National Ballistics Database was asked to “assess the feasibility, accuracy and reliability, and technical capability of developing and using a national ballistics database as an aid to criminal investiga- tions.” To accomplish this, the panel’s charge is to: (1) ssess the technical feasibility, through analysis of the uniqueness A of ballistic images, the ability of imaging systems to capture unique char- acteristics and to parameterize them, the algorithmic and computational challenges of an imaging database, the reproducibility of ballistic impres- sions and the ability of imaging systems to extract reproducible informa- tion from ballistic impressions. (2) ssess the statistical probabilities that ballistics evidence presented A would lead to a match with images captured in a database, whether and how the base rate can be estimated for those crimes that present bullet or casing evidence that do in fact come from a gun that produced a database entry, and the probabilities and consequences of false positives and false negatives. (3) ssess the operational utility of ballistics evidence in criminal A investigations—that is the extent to which it is used or can be used to identify crime guns and suspects and to solve specific crimes. (4) ssess the sources of error in ballistics database matching (from A examination, digitization, computer matching, chain of custody and docu- mentation of tests, and expert confirmation), how they may be quantified, and how these errors interact. The charge continues: The committee’s work will provide scientific and technical knowledge to inform the government’s deliberations on three policy options with regard to ballistics databases: (1) aintain the National Integrated Ballistic Information Network M (NIBIN) on ballistics recovered from crime scenes. It is operated by the Bureau of Alcohol, Tobacco, and Firearms. (2) nhance the NIBIN system so that it can be used to match crime E scene evidence with the gun used. (3) stablish a national ballistics database of images from bullets fired E from all, or nearly all, newly manufactured or imported guns for the pur- pose of matching ballistics from a crime scene to a gun and information on its initial owner. Addressing the issues raised by the tasks of the charge permitted the committee to provide guidance to NIJ on the three federal policy options. Specifically, for option 2, enhancing the NIBIN system, we address how 

EXECUTIVE SUMMARY  to increase its effectiveness as a search tool, including changes to the basic imaging standard used by the system, and improving procedures for working with the existing hardware and software. For option 3, establish- ing a national RBID, the committee considers it a counterpart to NIBIN, containing images of ballistic samples from all newly manufactured and imported weapons. The committee also considered the feasibility of alter- native technologies that could achieve the same goal as a national RBID. These alternative technologies include microstamping to imprint a known, unique marker on firearms parts or ammunition: analysis of such marks would complement or perhaps replace the need to examine the currently used toolmarks. Underlying the specific tasks with which the committee was charged is the question of whether firearms-related toolmarks are unique: that is, whether a particular set of toolmarks can be shown to come from one weapon to the exclusion of all others. Very early in its work the committee found that this question cannot now be definitively answered. Finding: The validity of the fundamental assumptions of uniqueness and reproducibility of firearms-related toolmarks has not yet been fully demonstrated. Notwithstanding this finding, we accept a minimal baseline standard regarding ballistics evidence. Although they are subject to numerous sources of variability, firearms-related toolmarks are not completely random and volatile; one can find similar marks on bullets and cartridge cases from the same gun. A significant amount of research would be needed to scientifically determine the degree to which firearms-related toolmarks are unique or even to quantitatively characterize the probability of uniqueness. Assess- ing uniqueness at, say, a submicroscopic level, though probably technically possible, would be extremely difficult and time consuming compared with less definitive but more practical and generally available methods at the macroscopic level. It is an issue of policy and of economics as to whether doing so would be worthwhile. The committee did not and could not undertake such research, nor does it offer any conclusions about undertak- ing such research. Although it appears to the committee that the needs for research are extensive, specifying the nature of that research was not part of the committee’s charge. We also note that the committee does not provide an overall assessment of firearms identification as a discipline nor does it advise on the admissibility of firearms-related toolmark evidence in legal proceedings: these topics are not within its charge. The committee’s charge is to determine the extent to which the tool- marks left on bullets and cartridge casings after firing a weapon can be 

 BALLISTIC IMAGING captured by imaging technology. It is also to assess whether a ballistic image database—particularly a national RBID containing images of exhibits fired from all newly manufactured and imported guns—would be feasible and operationally useful, by which we mean capable of generating leads for f ­ ollow-up and further investigation. Whether or not toolmarks are unique to a given weapon does not preclude the committee from addressing this charge. Indeed, in many situations a sufficient level of toolmark reproduc- ibility can be picked up by imaging or other measurement systems to be use- ful for narrowing a search down to a set of possible weapons, as is currently done. The final determination of a “match” is made by a human examiner. FEASIBILITY OF A NATIONAL REFERENCE BALLISTIC IMAGE DATABASE Independent of the reliability and effectiveness of the technology used in making comparisons of images in a national RBID, there would be sig- nificant limitations in the usefulness of such a database. Most importantly, there is a huge existing supply of weapons and ammunition that would not be entered into the database. In addition, revolvers do not eject cartridge cases at crime scenes as do other handguns. Consequently, even under the best of circumstances, when random variability is kept to a minimum, the database itself would be incomplete. Finally, to implement a national RBID, national protocols would have to be created for the test firing of new and imported guns; ensuring that test-fired cartridge cases or bullets are cor- rectly packaged with their corresponding firearm and maintaining a chain of custody for the exhibits after they are imaged would create a formidable logistical challenge. In our detailed assessment, three additional points regarding the imple- mentation of a national RBID have particular salience. First, the current technology in use for automated toolmark compari- son, based on two-dimensional greyscale images, is useful for gross cat- egorization and sorting of large quantities of evidence. However, it is less reliable for distinguishing extremely fine individual marks that would be necessary to make successful matches in RBIDs in which large numbers of exhibits on file would share gross class and subclass characteristics. Second, basic probability calculations under reasonable assumptions suggest that the process of identifying a subset of possible matches that contains the true match with a specified level of certainty depends critically on as-yet underived measures of similarity between and within gun types. This process is very likely to return too large a subset of candidates to be practically useful for investigative purposes. Third, the large influence of ammunition type and variability introduces a significant source of error in identification. A standard, protocol type of 

EXECUTIVE SUMMARY  ammunition could be specified in an RBID (as it is in NIBIN), but it is likely not to correspond with the ammunition actually used in a crime; the choice of protocol ammunition, or a requirement to use multiple ammunition types, would have significant financial implications for both ammunition and fire- arm manufacturers, as well as on the information systems involved. Conclusion: A national reference ballistic image database of all new and imported guns is not advisable at this time. MAINTAIN OR ENHANCE NIBIN By facilitating access by state and local law enforcement agencies to ballistic imaging technology, the NIBIN program provides a valuable ser- vice in helping to solve gun-related crimes. However, agencies differ in the degree to which they use the NIBIN resources and, consequently, they differ markedly in the benefits they derive in establishing links between crimes and investigative leads. The committee’s principal task includes offering guid- ance on either maintaining NIBIN as it currently operates or enhancing it in various ways to improve its effectiveness. The former is not really a viable option: there are always opportunities for improvement in any program, particularly one as broad as NIBIN. Conclusion: NIBIN can and should be made more effective through operational and technological improvements. To this end, the committee offers 15 specific recommendations to improve NIBIN’s performance and effectiveness. Seven of the recommenda- tions are oriented principally at the operation of the NIBIN program itself and the practices of NIBIN partner agencies, and they address: • priority for NIBIN entry of cartridge casings collected from crime scenes; • ballistic imaging as a part of the criminal investigation process for state and local agencies; • cross-jurisdictional tally of hits using the NIBIN system; • streamlining of the ballistic image acquisition process and reporting requirements; • development of “best practices” in using NIBIN; • a protocol for the entry of more than one exhibit from the same crime scene or test firing; and • allocation of NIBIN system technology. 

 BALLISTIC IMAGING We also offer eight specific recommendations for enhancing the current technical platform for the NIBIN program, the Integrated Ballistics Identifi- cation System (IBIS), and the hardware and software system developed by Forensic Technology WAI, Inc. These eight recommendations address: • research on the distributions of comparison scores; • an “audit trail” in the NIBIN system’s hardware and software systems; • ammunition brand information in NIBIN; • the capacity for national or cross-regional searches against the NIBIN database; • NIBIN’s database partition structure; • enhancements to the NIBIN interface; • side-light imagery of breech face impressions; and • the 20 percent threshold used in the IBIS. In support of this study, the National Institute of Standards and Tech- nology (NIST) was separately contracted by NIJ to perform experimental work at the committee’s request. This experimental work considered the value of one major technical enhancement to the current NIBIN system: a change in imaging standard from two-dimensional, greyscale photography to three-dimensional surface measurement using noncontact microscopy. NIST’s work included analysis of an extract of cartridges from one of the major existing studies of ballistic imaging performance as well as a new dataset of test-fired cartridges designed by the committee. The work highlights the promise of three-dimensional surface measurement, which performs comparably with—and, for some cartridge markings, often ­better than—the current two-dimensional methodology. However, there are major substantive challenges—among them the reduction of data collection time and the refinement of image comparison algorithms that make use of three-dimensional information but are still compatible with existing two- d ­ imensional imagery—that need to be addressed before full consideration can be given to adopting the new standard. ALTERNATIVE TECHNOLOGIES The goal of a national reference ballistic image database is to provide an investigative link from ballistics evidence to the point of sale of the weapon or ammunition used in a crime. The same goal could be achieved through an entirely different approach, microstamping, which is to place a known, unique, and unalterable identifier on gun parts, cartridge cases, or bullets at the time of manufacture. These uniquely microstamped products could then be associated with their purchaser when sold. Microstamping, if 

EXECUTIVE SUMMARY  feasible and practical, would have the advantage of imposing uniqueness as a characteristic of ballistics evidence, substituting known and fixed mark- ings for microscopically fine, individualizing characteristics that result from random processes in manufacture and weapon firing. A distinct advantage of microstamping is that the marks could be examined at a crime scene using equipment no more sophisticated than a magnifying glass, vastly simplifying and speeding up the process of devel- oping investigative leads. The state of California recently passed a law, to take effect in 2010, which requires microstamping on internal parts of new semiautomatic pistols. However, the committee believes that for such a technology to be implemented successfully, in-depth investigations on sev- eral topics are needed. These topics include the reliability and durability of the marks in a variety of firing conditions, their susceptibility to tampering and countermeasures, whether it would be best to place them on guns or ammunition or both, and the cost implications and feasibility of adding a microstamping process to established manufacturing processes. PROCESS FOR IMPROVING COMPUTER-ASSISTED FIREARMS IDENTIFICATION The current technology used in automated examination of images of ballistics evidence is produced and maintained by a single vendor. As a result, it does not benefit from the improvements that could be gained through competition and vetting among the broader research community, and its potential for advancement and innovation is limited. The committee suggests that improvements in matching ballistics evidence be made through government procurement efforts that demonstrate best practices. Two recent examples of government-mandated large-scale imaging system developments based on initially, nonmature technologies include s ­ ystems for fingerprint identification and for facial recognition. Both sys- tems required the creation of dedicated pattern recognition algorithms, similar to the requirements of NIBIN. Instead of relying on a single system produced by a single vendor, both systems were organized as competitions between vendors with the goal of advancing the technology as quickly as possible. Both competitions required that well-vetted datasets from several sources be made available to researchers so that the correct features could be identified for extraction. Finally, the results of both competitions were subjected to independently administered evaluations, using well-defined and published evaluation methodologies that allowed for a direct quan- titative assessment of the relative strengths and weaknesses of different approaches. This approach to procurement—removing strict dependence on a sole-source provider and ensuring government ownership of and access to 

 BALLISTIC IMAGING result data—should be applied for all work related to the improvement in b ­ allistics evidence analysis, including large-scale two-dimensional image search engines, three-dimensional topographical techniques, and micro- stamping processes.