BioWatch PCR Assays Building Confidence, Ensuring Reliability: Abbreviated Version (2015) / Chapter Skim
Currently Skimming:
6 Conclusions and Recommendations
6 Conclusions and Recommendations
Pages 137-152
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 137... ...
Responding to a detection that turned out to be false positive drains jurisdictional resources and could undermine confidence in the system when a true positive occurs. On the other hand, a false negative result could be a catastrophic error if a pathogen release was missed.
|
From page 138... ...
Elucidation of performance standards that ensure that the BioWatch PCR assays have performance characteristics that provide sufficient confidence to program stakeholders is the motivating subject of this report. Although the report focuses on fundamental principles for the PCR assay component, it is important to emphasize that interpretation of an operational result (which is essential for user "actionability")
|
From page 139... ...
Newly sequenced strains of target agents eventually will reveal potential false negative situations, and potential false positive situations from newly discovered near-neighbor genomes. Similarly, the program has focused its consideration of the development of multiplex assays primarily on the initial screen, which may reflect earlier days of multiplex PCR development, when successfully multiplexing more than a handful of assays was more difficult.
|
From page 140... ...
into the program as technology and software development makes this increasingly feasible. Recognizing the Work That Has Been Done It is important to recognize the substantial body of work that the BioWatch program and its contractors have undertaken to assemble reference materials, design appropriate testing setups, provide guidance on performance standards, test and evaluate PCR assays, conduct testing to understand current system performance, and establish and operate a quality assurance (QA)
|
From page 141... ...
To be applicable to BioWatch, a PCR standard to characterize assay performance and conduct validation would need to meet the followng i basic goals: • Establish that the assay has sufficient sensitivity to detect the release of a tested biothreat agent at a program-relevant concen tration above the baseline of environmental background; • Establish that the assay has sufficient specificity to detect tested pathogenic strains of concern but not to cross-react with non tested strains and organisms within acceptable program false positive and false negative rates; and • Be sufficiently robust for routine operational use by BioWatch jurisdictional laboratories and be otherwise acceptable to pro gram users. The committee's conclusions and recommendations to address these goals are detailed below.
|
From page 142... ...
To support this goal, the BioWatch program should work with other federal agencies and international partners to ensure that as many strains as possible are sequenced to at least a high- uality draft level and the data and associated reference q m aterials made available to PCR assay and device developers through reference databases. Assay Sensitivity and Specificity Assay sensitivity relates to the ability of the assay to reliably detect the targeted organism at very low concentrations.
|
From page 143... ...
PCR assay performance is critically impacted by the steps that lead into the assay and is influenced by the amplicon detection methodology and feedback from the steps that follow a result. Even a perfectly designed PCR with ideal specificity and sensitivity will fail with either a false positive or false negative, or an inconclusive, result if the sample collection, processing, purification, and reagent input are not optimized and standardized.
|
From page 144... ...
This sampling could build on the existing BioWatch network by using previously tested filters as samples and should also include judicious sampling of water and soil samples. Operational assay performance data in individual jurisdictions based on positive screening results might provide a source of invaluable data, as well.
|
From page 145... ...
Recommendation Federal agencies, including the Department of Homeland Secu rity, the Department of Defense, the Department of Energy, the Department of Health and Human Services, and other partners, should collaborate to produce a reference document or data base identifying which repositories contain strains or strain nucleic acids relevant for testing pathogen detection assays. Information on the requirements or procedures to obtain and use strains or strain materials (e.g., extracted nucleic acids)
|
From page 146... ...
The performance of pathogen identification assays are inherently a process and not an end point: new strains and near neighbors will always be encountered, and the microorganisms themselves are constantly evolving. New sequence information continues to be obtained, which results in a dwindling number of potential unique PCR assays available for species/strain differentiation.
|
From page 147... ...
The committee has attempted to provide a starting point for a performance standards approach for conducting laboratory testing of assays that could provide reasonable statistical confidence for BioWatch users while seeking to keep numbers of tests and numbers of strains reasonable for intended use. Recommendation The following approach can serve as a starting point for a stan dard to provide confidence in PCR performance while seeking to undertake a reasonable amount of laboratory characteriza tion and validation testing: • reasonable approach to determining an assay's analytical A LOD in a laboratory setting is to conduct serial dilution at a range of concentrations bracketing the estimated LOD, using n = 60 replicates with acceptance criteria of at least 58/60 for a given concentration, followed by appropriate curve fit ting.
|
From page 148... ...
. A ROBUST QUALITY ASSURANCE PROGRAM CONTRIBUTES TO CONFIDENCE IN THE SYSTEM BUT COMMUNICATION CHALLENGES REMAIN Process, Transparency, and Communication Various technical factors should be considered when establishing assay performance standards appropriate for BioWatch's mission.
|
From page 149... ...
This effort would assist in estab lishing acceptable performance standards, enhance informed data interpretation and decision making, improve the ability to undertake root cause analysis of assay issues encountered by jurisdictions, and enable the collective identification and dis semination of actions as part of robust quality assurance. STANDARDS APPROACHES FOR MULTIPLEXING REAL-TIME PCR ASSAYS The committee concluded that existing standards approaches should be suitable for characterizing and validating the performance of multiplex PCR assays.
|
From page 150... ...
The committee concluded that existing guidance on singleplex PCR performance for environmental biodetection assays in con cert with FDA information should provide a good starting point for the performance testing and validation of multiplex assays by the BioWatch program. The additional issues that arise when designing, validating, and deploying multiplex assays will need to be considered when making the decision to switch to a mul tiplex format.
|
From page 151... ...
Rather, they are primarily communication and education and training. Building additional, specific mechanisms into the program to share assay and system performance results, discussing what these performance results reveal about the limits of the assay and the translation of assay results into actions, and sharing applicable assay performance information across agencies such as DHS and CDC and with appropriate individuals within the network of jurisdictions appear to the committee to be critical needs.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.