Summary

Advances in genetics and genomics are transforming medical practice, resulting in a dramatic growth of genetic testing in the health care system. The rapid development of new technologies, however, has also brought challenges, including the need for rigorous evaluation of the validity and utility of genetic tests, questions regarding the best ways to incorporate them into medical practice, and how to weigh their cost against potential short- and long-term benefits. As the availability of genetic tests increases so do concerns about the achievement of meaningful improvements in clinical outcomes, costs of testing, and the potential for accentuating medical care inequality.

STATEMENT OF TASK

Given the rapid pace in the development of genetic tests and new testing technologies, the Department of Defense (DoD) Office of Health Affairs asked the National Academies of Sciences, Engineering, and Medicine to convene a committee to

The DoD Office of Health Affairs assists in the development of strategies and priorities to achieve the health mission of the Military Health System (MHS) and participates in formulating, developing, overseeing, and advocating the policies of the Secretary of Defense. TRICARE is a major part of the MHS that combines the resources of military hospitals and clinics with civilian health care networks. The Office of Health Affairs is responsible for providing a cost-effective, high-quality health benefit to about 9.6 million active-duty uniformed service members, retirees, and their families. The MHS has a $50 billion annual budget and consists of a worldwide network of 59 military hospitals, 360 health clinics, private-sector health-business partners, and the Uniformed Services University of the Health Sciences.

APPROACH TO THE TASK

The 17-member committee met five times over the course of the study to address the clinical application and clinical utility of genetic tests as directed by the statement of task. In support of the committee’s discussions and deliberations, targeted literature searches were conducted, and information from relevant scientific, professional, and federal sources was gathered. The committee quickly found that numerous ad hoc groups, regulatory agencies, organizations, and professional societies were developing approaches to assessment of genetic testing and collecting and using evidence with various methods. None of the approaches was judged to be entirely satisfactory for DoD’s purposes, but generating an entirely new approach was judged both unnecessary and infeasible because of the constraints on time and resources. Thus, the committee reached consensus early in its deliberations to adapt the best of what was already available.

Because a framework aimed only at current technologies might soon be outdated, the committee’s framework focuses on general principles of clinical usefulness that are relevant to any genetic technology. After conversations with DoD and internal discussion, the committee focused its framework on germline DNA-based tests, as a too-broad scope (e.g., to include RNA and other molecular-based tests) would expand the task to an unmanageable level. The committee believes, however, that the recommended framework is broadly applicable to assessment of other molecular technologies. Finally, the committee makes many suggestions about how the framework might be used in practice, but stops short of providing complete illustrative examples because it could not substitute its judgments for those that DoD alone is in a position to make.

USES OF GENETIC TESTING

The committee took as its starting point the definition of genetic testing by the National Institutes of Health’s National Human Genome Research Institute as

The committee organized its analysis around the purpose of the test, using the categories of diagnostic, predictive, and reproductive.

Diagnostic Genetic Testing

Diagnostic genetic testing is used to identify or rule out a specific genetic condition. Genetic testing is often used to confirm a diagnosis when a particular condition is suspected on

the basis of physical signs and symptoms. Family history might also play an important role in identifying appropriate diagnostic tests. For example, genetic testing for inherited cancer syndromes might be performed because a family history suggests an inherited risk of early-onset cancer. The results of a diagnostic test can influence patients’ and clinicians’ choices about clinical management of disorders.

Predictive Genetic Testing

Predictive genetic testing identifies gene variants that increase a person’s risk of developing heritable disorders, such as some types of cancer, before any signs or symptoms appear. In some cases, a diagnostic test of an affected person yields a result that can be used to recommend a predictive test for relatives. For example, if a woman who has breast cancer is found to have a BRCA1 variant, indicating a hereditary breast–ovarian cancer syndrome, her relatives can be offered the option of being tested to determine whether they carry the variant.

Similarly, predictive genetic tests can be used for population screening, notably in the case of newborn screening, which involves testing infants a few days after birth for evidence of treatable diseases that are known to be detrimental to health or development.

Pharmacogenetic testing is another important class of predictive genetic testing. It provides information about individual variation in drug pharmacodynamics (effects on drug receptors) and pharmacokinetics (uptake, distribution, and metabolism), which makes it possible to identify patients who are at increased risk for adverse effects or who are likely to be nonresponders. Pharmacogenetic testing has the potential to help health care providers tailor therapies by adjusting the dose or drug that might work best for an individual patient, to prevent adverse drug reactions, or to select persons who are likely to respond to a given drug.

Reproductive Genetic Testing

Reproductive genetic testing offers the opportunity to identify people who are at increased risk for having a child who has a genetic disease or to identify an affected embryo or fetus. Carrier or heterozygote genetic testing is used to identify people who are at increased risk for having a child who has a genetic disease. Most carrier tests identify people who are heterozygous for or “carry” one variant copy and one normal copy of a gene that is associated with a disorder transmitted as an autosomal recessive trait (that is, a trait that requires both copies of the gene to express the disorder). Carriers typically show no signs of the disease in question, but they have the ability to pass on the variant gene to their children. Carrier testing is offered either because a family history indicates the presence of a specific inherited disease or as a screening test.

Prenatal genetic testing is used to detect abnormalities in the genes or chromosomes of a fetus before birth. Current guidelines recommend that all pregnant women be offered maternal serum screening tests to identify pregnancies that are at increased risk for a trisomy disorder (such as Down syndrome) or for neural-tube defect. If a screening test is positive, a confirmatory test can be performed. Prenatal testing might also be offered if the parents are known to be at risk for having a child who has a specific genetic disorder.

Preimplantation genetic testing, or preimplantation genetic diagnosis, is a specialized technique that can reduce the risk of having a child who has a particular genetic or chromosomal disorder. It is used to detect genetic changes in embryos that were created by the use of assisted reproductive techniques, such as in vitro fertilization.

ETHICAL, LEGAL, AND SOCIAL IMPLICATIONS

Genetic testing has an array of personal and societal implications that require consideration in defining appropriate testing practice. Those implications are related to shared genetic risk among family members, the use of genetic testing in reproductive decision making, and the potential for genetic information to generate stigma or discrimination. In addition, the rapid development of genomic technology and the many remaining uncertainties about the health implications of genetic risk raise ethical considerations related to health care disparities, clinical data sharing, and the scope of result reporting from genome-scale testing.

ASSESSMENT OF GENETIC TESTS

Many of the available methods for assessing genetic tests cover the three common domains of evaluation: analytic validity, clinical validity, and clinical utility. Table S-1 provides a comparison of the different frameworks evaluated by the committee. The ACCE¹ and Fryback–Thornbury² models include an additional domain: societal effect of the test. Some evaluation frameworks (such as the Fryback–Thornbury hierarchy) provide general conceptual guidance, whereas analytic frameworks (such as those of the USPSTF³ and the EGAPP Working Group⁴) provide additional detail regarding the relevant populations, interventions, comparators, outcomes, times, and settings.

The McMaster University Evaluation Framework provides a thorough approach and rich detail for making decisions about genetic testing. The three domains of the McMaster evaluation framework (establishing evaluation criteria, determining acceptable cutoffs for each criterion, and determining conditions of coverage for “gray zones”) provide the foundation of the model recommended by the present committee.

The domains map broadly to the ACCE criteria, with the Fryback–Thornbury hierarchy representing clinical utility in three categories: patient-outcome efficacy, therapeutic efficacy, and diagnostic-thinking efficacy. The USPSTF method evaluates health care interventions in the context of preventive services in the general population and thus emphasizes patient outcomes, such as morbidity and mortality, as high-level end points. The EGAPP method organizes evidence into the ACCE categories and evaluates the chain of evidence by using a framework similar to that of USPSTF. The McMaster University evaluation framework identifies six criteria, one of which (effectiveness) depends on the intended purpose of the test; the framework also considers aggregate costs, use metrics, and cost-effectiveness criteria that are important from the health care system perspective. The Genetic testing Evidence Tracking Tool (GETT), although not explicitly intended as an evaluation method, provides a systematic model for organizing published evidence in 10 main categories. Finally, the evaluation process, the first

___________________

¹ ACCE takes its name from the four main criteria for evaluating a genetic test: analytic validity, clinical validity, clinical utility, and associated ethical, legal, and social implications.

² The Fryback–Thornbury model provides conceptual guidance for evaluating the efficacy of health technologies. It is a widely used general evaluation structure for medical-test assessment and for clarifying the scope of the assessment.

³ USPSTF is the abbreviated form for the US Preventive Services Task Force.

⁴ EGAPP is the abbreviation for the Evaluation of Genomic Applications in Practice and Prevention. The Working Group was established in 2005 to support the development of a systematic process for assessing the available evidence regarding the validity and utility of rapidly emerging genetic tests for clinical practice.

step of which is to determine whether the purpose of genetic testing in a particular clinical scenario is clear and worthwhile, is the subject of two questions posed by Frueh and Quinn.

EVIDENCE

As with other elements of medical care, information about genetic testing is gathered and evaluated through research. The quality and quantity of that information vary, which confers some level of certainty about the resulting findings and conclusions. Additionally, the numerous public and private organizations conducting evidence reviews have differing approaches and criteria for assessing the available evidence regarding the validity and utility of genetic tests. Evidence is generally organized, summarized, and synthesized through systematic reviews or meta-analyses. Evidence summaries then inform clinical practice guidelines and policy decisions.

The evidence base on genetic testing is limited by the frequent lack of direct evidence, for example, from randomized controlled trials (RCTs). The rarity of many genetic diseases and of variants associated with genetic diseases makes it difficult for researchers to gather enough patients who have the disease or variant to conduct a traditional study; thus, the usual sources of evidence often are not available. One response to that challenge has been the establishment of databases that aid in the collection and accumulation of evidence on clinical validity and utility; another potential approach is to provide guidance through decision modeling.

Establishing analytic validity, clinical validity, and clinical utility depends on the characteristics of the genetic test, the clinical scenario, and the prevalence, severity, and nature of the disease. For example, observational studies, such as cohort studies, are appropriate for assessing clinical validity because they can accurately measure the ability to predict or detect disease. In contrast, evidence on the clinical utility of a genetic test is ideally studied by using controlled studies, especially RCTs, inasmuch as this design maximizes internal validity and addresses issues of selection bias and confounding. Although an RCT is the ideal study design for assessing many aspects of clinical utility, the level of evidence sufficient to evaluate clinical utility might depend on the specific clinical indication, the clinical setting, and the perceived value of potential outcomes of the genetic test itself.

Thus, there are challenges to evidence-based decision making around the use of genetic tests, including the paucity of the types of research studies on which evidence-based medical decisions depend, which are exacerbated by the accelerated development of new tests, variants, and technologic discoveries. RCTs to establish clinical utility are expensive and of long duration, so decisions about the clinical value of genetic testing are often based on lower levels of evidence. Policy making is often hampered by the lack of objective methods for setting decision thresholds.

TABLE S-1 Comparison of Frameworks

NOTE: ACCE = analytic validity, clinical validity, clinical utility, and associated ethical legal and social implications; EGAPP = Evaluation of Genomic Application in Practice and Prevention; ELSI = ethical, legal, and social implications; GETT = Genetic testing Evidence Tracking Tool; USPSTF = US Preventive Services Task Force.

RESEARCH RECOMMENDATION

DoD has a number of options for contributing to the evidence base for genetic testing. It could undertake or otherwise support prospective studies of clinical utility when promising tests are identified (such as tests that have potential benefit or effects). DoD could conduct prospective studies, such as RCTs, at its facilities or at affiliated institutions, or sponsor outside research.

As another type of contribution to the evidence base, DoD could implement processes for data collection and analysis of its own experience with genetic testing. Information in TRICARE on medical care and outcomes of a large population of active-duty members, retirees, and their family members could help to answer questions about clinical utility that many researchers and organizations struggle to assess. In addition, access to long-term follow-up data on a particular population and its experience with genetic tests could be a valuable resource to generate evidence of clinical utility.

Full participation of the large DoD TRICARE population in evidence repositories (such as ClinVar) offers the opportunity to advance the evidence base on clinical validity of genomic tests substantially. DoD could also contribute to the clinical validity of tests by supporting discovery efforts regarding the evidence base on next-generation sequencing, in particular the association between variants and phenotypes. That would require a large number of people to undergo testing; DoD is in a position to provide such data on its population.

• supporting high-quality studies of clinical validity and clinical utility (including patient-centered outcomes) of promising tests;
• implementing processes for data collection and analysis of its own experience with genetic testing;
• supporting evidence-based systematic reviews of genetic tests;
• contributing genetic variants and associated clinical data to public evidence repositories; and
• partnering with other organizations to facilitate these recommendations.

OVERVIEW OF THE PROPOSED FRAMEWORK

After reviewing different methods that have been developed for the evaluation of medical tests, some of which were specifically intended for use with genetic tests, the committee developed a hybrid system that incorporates elements of several of the methods described in the genetic test assessment section above. The system that the committee developed—its decision framework—which consists of seven components, is depicted in Figure S-1, and described in the text that follows. The framework is not linear nor is it meant to be a formal algorithm, but rather to provide general guidance to the DoD.

Component 1. Genetic Test Scenario

As a first step, a “genetic test scenario” must be systematically defined to enable relevant information to be identified. The scenario should include the test being performed, the population in which testing is considered appropriate, the purpose of the test in that population (clinical scenario), the outcomes of interest, and comparable alternative methods to accomplish those tasks. Within any given genetic test scenario, the population being tested and the purpose of the test are often considered together as a “clinical scenario.” In many cases, a particular genetic test could be considered for use in one population for one purpose and in a different population for another purpose. The performance of the test and the decision about whether testing is worthwhile might vary dramatically, depending on the clinical scenario.

If a decision has already been made about the use of one genetic test in a particular clinical scenario (considering the population and the purpose of testing), decisions about the use of other genetic tests in the same test scenario, or decisions about the use of the same test in a different clinical scenario, could be streamlined.

Component 2 (A, B, C). Prioritization of Topics to Be Evaluated and Triage Process

The evaluator must prioritize the various genetic test scenarios to determine which ones to address. That determination could be made based on a number of different criteria (e.g., volume of test requests, unit cost of requested tests, stakeholder assessments, test development planning) depending on DoD’s priorities.

The evaluator then carries out an initial triage step to determine if a rapid decision can be made based on existing information. This triage step actually represents three sequential questions (see Box S-1). First, the evaluator determines whether the purpose of the test is “worthwhile” (Component 2A). That determination could be based on whether DoD already offers coverage of other genetic or nongenetic tests used for the same purpose. If the answer is “NO” then a decision can be rendered without further evidence review. If the answer is “YES” (because of similar existing coverage decisions) or “MAYBE” (depending on the characteristics of the test or completeness of the knowledge base), the evaluator performs a streamlined “rapid review” of the available evidence for the genetic test scenario (Component 2B).

If the rapid review reveals sufficient evidence (YES), or areas of missing evidence are deemed non-essential (MAYBE), the evaluator will need to consider contextual issues (Component 2C). Those might include social issues, net benefits and harms, and aggregate costs. The balance of the issues—that is, whether a test is worthwhile, the evidence, and contextual factors—should inform a triage decision. Those considerations, along with the rapid evidence review, will be brought to the decision process (Component 4). If the rapid review does not reveal sufficient evidence (NO), then a formal evidence-based review process (Component 3) would be appropriate to systematically evaluate the evidence to facilitate a decision process (Component 4) as well as identify evidence gaps (Component 7).

Component 3. Evidence Review Process

For topics that require additional evaluation, a formal evidence assessment process is needed. The first stage of the process is to identify the purpose, important outcomes of testing, and any relevant comparators, such as other genetic or medical tests already used in clinical care for the same purpose. An evidence assessment should be conducted using systematic review methodology, for which ample guidance is available.

Component 4. Decision Process

Following the formal evidence assessment process or availability of new information, DoD will need to apply a decision process for evaluating the results of the evidence review and determine whether the available evidence indicates that the genetic test scenario in question is appropriate for coverage. The decision process will incorporate the preceding evidence review and contextual factors such as social issues, potential harms, or benefit/cost considerations and result in either a “YES” or “NO” decision regarding coverage.

In the absence of direct evidence for many genetic tests, decision makers must decide the acceptable level of uncertainty for their purposes. Even when evidence is strong and clear, the judgment of whether benefits outweigh harms is subjective. Decision makers commonly consider clinical experience, expert opinion, and personal judgments regarding potential harms of the test

versus not having the test—with the result that decision makers using identical data can reach different conclusions based on their values.

In addition to questions about whether the available evidence supports the use of the genetic test in a given clinical scenario, this process is likely to also include economic considerations. In that regard, it might be important to consider multiple stakeholder perspectives in the determination and the differences among them. Input from stakeholders might affect the relative weight of net benefits and harms. Ultimately, demand for genetic tests, unit costs, and expert consensus on clinical value will be considered by decision makers.

Thus, because the process of determining what constitutes “sufficient” evidence is subjective and based on value judgments, DoD must set its own standards, preferably in a clear and consistent way, to reflect its values and needs.

Component 5. Decision Repository

A repository of decisions about individual genetic test scenarios will foster the “institutional memory” of the process and facilitate future reviews. Appropriately structured, the repository will allow reviewers to rapidly evaluate new scenarios in light of previous decisions that have been made for similar tests, populations, purposes, outcomes of interest, and methodologies. In addition, the decision repository serves as a record of the value judgments made about whether particular genetic test scenarios are deemed worthwhile, allowing for stakeholders to understand the decisions and so that DoD can ensure consistency across different decisions.

The repository will inform decision makers whether prior decisions have been made for any of the genes on the panel and if coverage has already been decided for one or more of those genes (in the same clinical scenario). For genetic test scenarios that undergo a formal evidence review, the details of the process and the evidentiary thresholds required by DoD will provide transparency to the process and allow identification of evidence gaps that can be addressed by research.

Component 6. Process for Periodic Review and Revision

Given the rapid pace of technologic development and research in genomic medicine, it is critical to develop a mechanism for periodic reassessment of decisions in light of new data. The extent to which this process can be implemented might depend somewhat on the organization of the decision-making process. If designed properly, the decision repository could facilitate a systematic process by including time stamps and automated prompts for re-review of decisions on a certain schedule, or provide a process for stakeholders to request revised decisions.

Component 7. Identification of Evidence Gaps

If the available evidence suggests a potential health benefit associated with a genetic test, especially for tests related to high-priority health conditions, DoD might consider a process for evidence development. Some genetic test scenarios might benefit from traditional research studies to address evidence gaps. Evidence-gap analysis should be conducted on a regular basis as part of a process of continual quality improvement that uses a clinical implementation science process.

FRAMEWORK RECOMMENDATION

The committee’s recommended framework is based on the numerous frameworks it reviewed and adapts the best of those for its intended purpose. The framework is neither linear nor intended to be a formal algorithm; rather, it provides a conceptual approach as general guidance to DoD. It begins with a clear definition of the topic being considered and a triage process to evaluate whether the purpose of a test is worthwhile and an expedited provisional decision can be made. For topics that need to be evaluated in more detail, the committee provides guidance for conducting an evidence review.

1. Define genetic test scenarios on the basis of the clinical setting, the purpose of the test, the population, the outcomes of interest, and comparable alternative methods.
2. For each genetic test scenario, conduct an initial structured assessment to determine whether the test should be covered, denied, or subject to additional evaluation.
3. Conduct or support evidence-based systematic reviews for genetic test scenarios that require additional evaluation.
4. Conduct or support a structured decision process to produce clinical guidance for a genetic test scenario.
5. Publicly share resulting decisions and justification about evaluated genetic test scenarios, and retain decisions in a repository.
6. Implement timely review and revision of decisions on the basis of new data.
7. Identify evidence gaps to be addressed by research.