An Evidence Framework for Genetic Testing (2017) / Chapter Skim
Currently Skimming:
3 Genetic Test Assessment
3 Genetic Test Assessment
Pages 35-58
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 35... ...
This chapter reviews and compares available methods that have been proposed for reviewing evidence and making decisions about using tests, including several methods designed specifically for genetic tests, and it offers a synthesis that maps clinically relevant outcomes to a hierarchic evidence structure. EVALUATING GENETIC TESTS Ideally, the clinical use of a genetic test should be preceded by studies to confirm that it is valid and useful.
|
From page 36... ...
METHODS FOR EVALUATING GENETIC TESTS The committee began its review by examining a 2011 report from the Department of Health and Human Services' Agency for Healthcare Research and Quality (AHRQ) 1 that addressed many of the issues surrounding genetic testing, including the feasibility of designing a framework for evaluating genetic tests by modifying existing methods, the strengths and limitations of different methods of literature searching to identify evidence, the feasibility of applying existing rating criteria to analytic-validity studies of genetic tests, and gaps in the evidence on sources and contributors of variability that are common to all genetic tests.
|
From page 37... ...
. The McMaster University evaluation framework was of particular interest to the committee because of the thoroughness of its approach, the richness of its detail, its focus on making coverage decisions for new predictive genetic tests, and its flexibility in applying criteria.
|
From page 38... ...
In evaluating evidence, the USPSTF method considers both the following key questions and the overall certainty of the evidence of net benefit of the preventive service in question: Do the studies have the appropriate research design to answer the key question(s)
|
From page 39... ...
It is a widely used general evaluation structure for medical-test assessment and for clarifying the scope of the assessment and the types of evidence necessary for addressing various aspects of test performance and their consequences, including societal effects (Sun et al., 2011; Morrison and Boudreau, 2012)
|
From page 40... ...
Office of Public Health Genomics established and supported the ACCE Model Project from 2000 to 2004 to develop the first publicly available analytic process for evaluating scientific evidence on emerging genetic tests. ACCE takes its name from the four main criteria or principles used for evaluating a genetic test: analytic validity, clinical validity, clinical utility, and associated ELSI.
|
From page 41... ...
TABLE 3-2 ACCE Model List of 44 Targeted Questions Aimed at a Comprehensive Review of Genetic Testing Element Component Specific Question Disorder/ 1. What is the specific clinical disorder to be studied?
|
From page 42... ...
renders later questions unnecessary, but that has seldom been the case for genetic tests evaluated by EWG. Additional questions outline an indirect-evidence pathway to demonstrate clinical utility and, in more specific terms, address such issues as the following:
|
From page 43... ...
The main goal is to help stakeholders to determine whether the knowledge base is sufficient for assessing the health care benefits of a given molecular-genetic test and identifying specific research subjects that require greater emphasis. Factors considered include epidemiology and genetics of the condition, available diagnostic tools and their analytic and clinical performance, availability of quality-control programs, laboratory and clinical bestpractice guidelines, clinical utility and effects on health and the health care system, the quality of the supporting data, and psychosocial, ethical, and legal implications.
|
From page 44... ...
SOURCE: Rousseau et al., 2010. Reprinted from De Gruyter Clinical Chemistry and Laboratory Medicine, Walter De Gruyter GmbH Berlin Boston, 2010.
|
From page 45... ...
. Evaluation Criteria In their review of the literature, the authors summarized issues germane to genetic tests according to numerous advisory bodies and distilled six evaluation criteria that apply to healthtechnology assessment: the intended purpose of the test, the effectiveness of the test compared with other approaches in accomplishing its purpose, additional effects beyond the intended purpose, the aggregate costs of using the test, the demand for use of the test, and the costeffectiveness of the test relative to that of other covered services that have the same purpose.
|
From page 46... ...
Assembly Questions What is the technology and what is it for? Why address this technology (e.g., "new" genetic tests)
|
From page 47... ...
The authors suggest that analytic validity, clinical validity, and clinical utility "offer too little guidance to structure a rational and predictable interaction between the test developer and the payer or technology assessment body." They identify three axes that describe considerations that might be taken into account during a technology assessment. The first axis represents functional categories of genetic tests, that is, their purpose in a clinical setting.
|
From page 48... ...
Confronting the "gray zon nes" of techno ology assessmen Evaluating genetic testi services for public insu nt: g ing fo urance covera in Canada International age a. Journal of Technology Assessment in Health Car 19(2)
|
From page 49... ...
" To accomplish that, the authors propose a set of six questions to guide assessment of genetic tests: Who should be tested and under what circumstances? What does the test tell us, that we did not know without it?
|
From page 50... ...
are intended to be used by payers and other stakeholders who are considering whether to use or cover the cost of genetic tests. Each of the evaluation methods identifies a "topic" implicitly or explicitly.
|
From page 51... ...
The McMaster University evaluation framework identifies six criteria, one of which (effectiveness) depends on the intended purpose of the test; it also introduces consideration of aggregate costs, use metrics, and cost-effectiveness criteria, which are important from the health care system perspective.
|
From page 52... ...
The EGAPP working group previously outlined a broad set of clinically relevant outcomes that could be considered in the evaluation of genetic tests (Botkin et al., 2010) , so the committee sought to understand how those outcomes could be mapped to the genetic test assessment methods described above.
|
From page 53... ...
GENETIC TEST ASSESSMENT 53 TABLE 3-5 Comparison of Frameworks Method Purpose Approach Strengths Weaknesses USPSTF Preventive Formal analytic Formal grading system Focus on preventive interventions in framework; evidence incorporating services results in a primary care assessment related to evaluation of benefits focus on clinical-utility key questions to and harms and rating outcomes that are not establish a "chain of of evidence relevant to all clinical evidence" applications Fryback– General medical-test Hierarchic Allows an evaluator to Lacks a formal Thornbury assessment representation of determine what evidence-based levels of efficacy for evidence types need to assessment procedure medical tests be assessed for a given test purpose or desired outcome ACCE Analytic process for Standard set of 44 Provides a highly Does not provide evaluating evidence questions that are granular approach to details on evaluating on genetic tests organized according assessing different the strength of to different evidence evidence types evidence; developed types (analytic for single-gene tests validity, clinical and may be difficult to validity, clinical extend to multigene utility, ELSI) panels or genome-scale sequencing tests EGAPP Systematic approach Hybrid approach Flexibility to evaluate Focus on single-gene to evidence-based using analytic different "topics" in tests may be difficult to assessment of framework similar to genetic testing, extend to broader genetic tests USPSTF and including a wide array genomic technologies evaluation of evidence of potential outcomes types articulated by of interest, and ACCE integration of formal evidence-based reviews GETT Structure for List of 72 defined Helps stakeholders to Does not provide systematic items grouped into determine whether the details on evaluating identification and categories knowledge base is the strength of organization of sufficient for genetic- evidence; developed published evidence technology assessment for single-gene tests on genetic testing and may be difficult to extend to multigene panels or genome-scale sequencing tests McMaster Evaluation model to Combines technology Defines criteria for Developed for the University guide public assessment with determining coverage, Canadian health coverage for new coverage decision anticipates the need for system; lacks a formal predictive genetic making from payer's payers to identify evidence-based tests in Ontario, perspective evidentiary thresholds, assessment procedure Canada and considers conditional coverage scenarios continued
|
From page 54... ...
Improving the efficacy of diagnosis is also of interest to payers (Gross et al., 2008) , and new genetic tests that interrogate hundreds or thousands of genes simultaneously could offer comparative advantages in arriving at a diagnosis earlier in the disease course.
|
From page 55... ...
The McMaster University evaluation framework provides a thorough approach and rich detail for making decisions about genetic testing. The three domains of the McMaster evaluation framework -- establishing evaluation criteria, determining acceptable cutoffs for each criterion, and determining conditions of coverage for gray zones -- provide the foundation of the model, whereas the "effectiveness and efficiency," "normative," and "assembly" questions help to fill in the evaluation framework.
|
From page 56... ...
validity diagnostic tests Does the test change a Improved accuracy of prognosis clinician's pretest estimate of the probability of a specific disease? Diagnostic Accurate molecular diagnosis Does the test result distinguish accuracy Clinical sensitivity and specificity patients with and without the target disorder among patients in whom it is clinically reasonable to suspect that the disease is present?
|
From page 57... ...
The committee considered the integration between genetic test assessment methods and relevant outcomes, noting that stakeholders who evaluate genetic tests need to link the purpose of a test with its desired outcome. The committee has added the "efficacy" of personal utility to its modified Fryback–Thornbury hierarchy, that is, information that might not be medically actionable but could have meaning to the individual patient.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.