address all targets and all evidentiary materials that will be encountered. Nor can one validation method or standard address all technologies; each technology must be validated individually.
There are two types of standards: performance standards and material standards.1 Most often, users focus on material standards, but performance is equally important. Traceability, a documentation of measurements to some standard, is needed. In the United States, standards developed by the National Institute of Standards and Technology (NIST) are typically used, but the available standards do not meet all needs. We need to decide who will make the standards to bridge the gaps in what is available. Some of the gaps in standard reference materials appear in Box 6-7. A major concern is how to identify a good standard versus “just a standard.” For example, surrogates may or may not suffice; they are often, but not always, suitable (Anderson et al., 2005). Near neighbors will be required to describe the sensitivity and specificity of an assay. Budowle also posed the question of who will prepare these standard reference materials and who will maintain them? Is it government’s, industry’s, or an individual laboratory’s responsibility? Likely the responsibility will vary based on the need.
Whole-genome sequencing of a metagenomics sample will generate sequences that span the genomes of the microorganisms that reside within the sample. In metagenomic samples, species and/or strains can be represented in widely varying abundance. The limited depth of coverage and amplification-bias stochastic effects on portions of individual genomes might affect representation of critical sequences that define species, causing them to be missed. Part of a genome will likely be represented, but phylogenetically and genetically the parts that are detected may not be able to resolve at the species level (most importantly near neighbors) or even at higher taxonomic levels. An analytical assay may perform perfectly well but may be uninformative on the presence of a target even if it is truly in the sample. So, we have to consider this process of resolving at the near-neighbor level and understanding which sequence reads are informative for taxonomic resolution or classification. One should rightly pose the question, whether only one, two, three, or more reads are sufficient to render an interpretation of identification.
Primer development and the quality of primer synthesis require validation as well. Primer design programs do not validate primers, and the need for such validation is underappreciated. Similarly, bar coding or
1 A performance standard specifies what is to be accomplished but does not dictate the particular method or material to be used as long as the desired end is achieved. A materials standard specifically directs that a certain material or method be used to accomplish the desired end.