The forensic science disciplines need to develop rigorous protocols for performing subjective interpretations, and they must pursue equally rigorous research and evaluation programs. The development of such research programs can benefit significantly from work in other areas, notably from the large body of research that is available on the evaluation of observer performance in diagnostic medicine and from the findings of cognitive psychology on the potential for bias and error in human observers.
In evaluating the accuracy of a forensic analysis, it is crucial to clarify the type of question the analysis is called upon to address. Thus, although some techniques may be too imprecise to permit the accurate identification of a specific individual, they may still provide useful and accurate information about questions of classification. For example, microscopic hair analysis may provide reliable evidence on the subpopulation of the individual from which the specimen was derived, even if it cannot associate reliably the hair with a specific individual. However, the definition of the appropriate question is only a first step in evaluating the performance of a forensic technique. The research design should address the questions that arise in the specific context of forensics.
A complete research agenda should include studies to establish the strengths and limitations of each procedure, sources of bias and variation, quantification of uncertainties created by these sources, measures of performance, procedural steps in the process of analyzing the forensic evidence, and methods for continual monitoring and improving the steps in that process.
Wide variability is found across forensic science disciplines not only with regard to techniques and methodologies (see Chapter 5), but also with regard to reliability, error rates, reporting, research foundations, general acceptability, and published material. Some of the disciplines are laboratory based (e.g., nuclear and mitochondrial DNA analysis, toxicology and drug analysis, and analyses of fibers and fire debris); others are based on expert interpretation of observed patterns (e.g., of fingerprints, writing samples, toolmarks, bite marks, and hairs). The briefings and materials that informed this report illustrate that the level of scientific development and evaluation varies substantially among the forensic science disciplines.
In most areas of forensic science, no well-defined system exists for determining error rates, and proficiency testing shows that some examiners perform poorly. In some disciplines, such as forensic odontology, the methods of evidence collection are relatively noncontroversial, but disputes arise over the value and reliability of the resulting interpretations.
In most forensic science disciplines, no studies have been conducted