This description of how science creates new theories illustrates key elements of good scientific practice: precision when defining terms, processes, context, results, and limitations; openness to new ideas, including criticism and refutation; and protections against bias and overstatement (going beyond the facts). Although these elements have been discussed here in the context of creating new methods and knowledge, the same principles hold when applying known processes or knowledge. In day-to-day forensic science work, the process of formulating and testing hypotheses is replaced with the careful preparation and analysis of samples and the interpretation of results. But that applied work, if done well, still exhibits the same hallmarks of basic science: the use of validated methods and care in following their protocols; the development of careful and adequate documentation; the avoidance of biases; and interpretation conducted within the constraints of what the science will allow.
One particular task of science is the validation of new methods to determine their reliability under different conditions and their limitations. Such studies begin with a clear hypothesis (e.g., “new method X can reliably associate biological evidence with its source”). An unbiased experiment is designed to provide useful data about the hypothesis. Those data—measurements collected through methodical prescribed observations under well-specified and controlled conditions—are then analyzed to support or refute the hypothesis. The thresholds for supporting or refuting the hypothesis are clearly articulated before the experiment is run. The most important outcomes from such a validation study are (1) information about whether or not the method can discriminate the hypothesis from an alternative, and (2) assessments of the sources of errors and their consequences on the decisions returned by the method. These two outcomes combine to provide precision and clarity about what is meant by “reliably associate.”
For a method that has not been subjected to previous extensive study, a researcher might design a broad experiment to assist in gaining knowledge about its performance under a range of conditions. Those data are then analyzed for any underlying patterns that may be useful in planning or interpreting tests that use the new method. In other situations, a process already has been formulated from existing experimental data, knowledge, and theory (e.g., “biological markers A, B, and C can be used in DNA forensic investigations to pair evidence with suspect”).
To confirm the validity of a method or process for a particular purpose (e.g., for a forensic investigation), validation studies must be performed.
The International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC) developed a joint document,