Figure 4-2 shows the basic printed report generated by IBIS, the top 10 ranked pairings by the different cartridge case markings. Reported prominently on the sheet is a sample size of 12,353. In discussing this type of report with other parties—such as investigating detectives, departmental superiors, and legal counsel—the meaning of “sample size” can be explained relatively easily as (roughly) the subset of the database matching the reference exhibit in caliber. But no information is readily provided on the effective sample size that is most relevant to the scores presented on the page—the number of exhibits retained after the coarse pass, for which the full scores were computed. That this effective sample size can be as small as 2,470 would be surprising, and potentially misleading, to observers without a detailed knowledge of all the steps in the IBIS comparison process.
We do not argue that there is anything inherently wrong with a first, coarse cut of the database or the specific method used; however, research should still be done to determine whether 20 percent is an appropriate measure, balancing gains in processing time with the potential to miss hits. We also believe that NIBIN users should have the capacity to easily adjust the threshold level in regenerating comparison score results. Particularly if circumstances lead to court trials where an IBIS-suggested linkage is the primary (or very important) evidence, it would behoove agencies and examiners to be able to demonstrate that the suggested pairing came about in a process where all eligible exhibits were subjected to the same score and rank, rather than roughly 20 percent of them. As with national and cross-regional searches, we also suggest that 100 percent full-comparison requests (that is, waiving the coarse comparison entirely) should be performed by NIBIN management as a matter of routine research and evaluation.
Recommendation 6.15: In light of improvements in computer processing time, the relatively ad hoc choice of 20 percent of potential exhibit pairs from the coarse comparison step should be reexamined. IBIS developers should consider removing the 20 percent threshold restriction or revising the percentage cut if it does not seriously degrade search time over moderate database sizes. In any event, IBIS developers should make it easier for local agencies to adjust the threshold level or to waive the coarse comparison pass altogether if specific investigative cases warrant a full, unfettered regional search of evidence at the expense of some processing speed.