Important Points Highlighted by the Individual Speakers
- While targeted mutation and gene panel testing are more technically complete than exome sequencing, exome information can be useful for carrier screening, disease diagnosis, and pharmacogenomic testing.
- Rare variant databases and other databases that include both genotype and phenotype information can be valuable shared resources for clinicians and researchers to aid in disease diagnosis, gene–phenotype associations, and drug development.
- A transparent, reproducible, evidence-based method for determining variant actionability is helpful when individual experts have different opinions about what variant information should be returned to patients.
- The actionability of genomic findings depends on the clinical context, such as whether testing is done before conception, prenatally, for newborn screening, during childhood, or for screening, diagnostic, or monitoring reasons.
- “Binning” the genome on the basis of clinical validity and clinical utility and other staged approaches can facilitate pre-test informed consent, analysis, and post-test return of results.
- Given how resource-intensive the process of evaluating variant evidence is, a collective effort using a standardized assessment approach and shared variant databases would be helpful in leading to more efficient variant curation.
- Improving the communication between testing laboratories and clinics would make it possible to update genotype–phenotype information as new data are collected.
- Technical issues—from gene coverage during data collection to bioinformatics interpretation of the data—vary and can impose limits on the information that can be derived from whole-exome or whole-genome sequencing unless they are standardized by the genomics community.
During the workshop a variety of experts in academia and the private sector described current research and clinical perspectives concerning the ways in which genomic data are being generated and linked to human diseases and applied to the practice of medicine. The topics covered during the presentations and discussions included the sources of genomic data, various processes such as “binning” genomic findings into categories with different degrees of actionability, systematic approaches to evaluating gene–phenotype associations, and a collaboration to create a curated resource that can help standardize the interpretation of genetic variation. Other topics addressed were the gathering, assessment, and evaluation of evidence for use in next-generation sequencing in cancer genomics; how new information is reviewed in the context of existing information; and how variant information can be shared more widely.
In recent years, many gene panels have been introduced into the clinical setting, noted Madhuri Hegde, professor of human genetics and executive director of the Emory Genetics Laboratory. The targeted mutation and gene sequencing panels are technically complete in that they cover all the exons of a gene and the entire mutation spectrum of a gene, including point mutations, insertions–deletions, copy number variability, and deep intronic pathogenic changes. By contrast, while exome sequencing covers more overall genes, the majority of the genes covered by exome sequencing are not clinically relevant, and for those genes that are clinically relevant, exome sequencing may not have complete coverage of all exons and may not cover the full spectrum of mutations, Hegde said. Despite this incompleteness, however, exome sequencing can still collect evidence important in assigning genes to a disorder, and it can be useful in yielding information relevant to carrier screening and pharmacogenetic markers.
There is a “critical need in our community to establish what is the [essential] amount of data [for including] a gene in a genetic test,” said Heidi Rehm, director of the Laboratory for Molecular Medicine at the Partners Healthcare Center for Personalized Genetic Medicine and assistant professor of pathology at Harvard Medical School.
Many of the gene panels being offered today have a highly variable number of genes for the same indication, partly because of different evaluations of the evidence for a gene–phenotype association (Rehm, 2013).
Even in the case of a targeted panel where phenotypic information can be gained from the results, complementary assays often need to be included with the gene panel, Hegde noted. For example, with the gene panel for short stature, methylation-based assays are necessary. Whether a gene panel works in a clinical setting therefore “depends on which disorder you are looking at,” Hegde said.
Exome sequencing can be used for either clinical or research purposes, though recently the boundaries between the two have been blurring. In Hegde’s laboratory, exome data are divided according to why the sequencing is being done. For new disease presentations the diagnostic yield, or likelihood that the test will provide enough information to make an appropriate diagnosis, ranges roughly from 30 percent to 40 percent, depending on which laboratory is reporting and what kinds of cases are considered, Hegde said. When writing clinical reports, she said, it is critical to sorting the data into categories of what can be interpreted in the clinic and what is clinically actionable (see Box 2-1).
- Clinical actionability: in the context of incidental findings or in an asymptomatic individual, the degree to which an intervention exists that can mitigate harm before a clinical diagnosis is made.
- Clinical validity: the accuracy and reliability of a variant for identifying or predicting an event with biological or medical significance in an asymptomatic individual.
- Clinical utility: the usefulness of information in clinical decision making and in improving health outcomes.
Genomic Sequencing in Oncology
In the past, oncologists have based treatment largely on traditional immunochemistry, pathology, and, more generally, anatomical staging, said Mark Robson, attending physician of the clinical genetics and breast cancer medicine services in the Department of Medicine at Memorial Sloan–Kettering Cancer Center. Now next-generation sequencing is creating a massive experiment in whether knowing the pattern of genomic aberrations will allow therapies to be targeted more effectively. “Although everybody is very enthusiastic about it,” Robson said, “whether or not we are going to be able to achieve better outcomes on a global scale throughout the cancer population still remains to be seen.”
Most cancer centers are using targeted assays rather than whole-exome or whole-genome sequencing to look at a variable number of genes that have been selected according to an a priori rationale for involvement in the oncogenetic or oncologic process. For example, the Integrated Mutation Profiling of Actionable Cancer Targets (IMPACT) panel probes for biologically or clinically relevant cancer genes (Wagle et al., 2012). Many of the genes are linked to cancer only through somatic mutations, but most of the germline predisposition syndrome genes are included on these panels as well, because many of them are also involved in carcinogenesis in nonhereditary contexts, Robson said.
In the clinical context, mutational profiling is used for variants that are clearly linked to response to a U.S. Food and Drug Administration (FDA)-approved drug, that define clinical trial eligibility, or that are plausibly predictive of response to an approved drug which might not otherwise have been chosen. Variants linked to response to an approved drug have already been defined through the companion diagnostic mechanism, though the companion diagnostic development process can be extremely complicated (IOM, 2014; McCormack et al., 2014). Similarly, variants used to define clinical trial eligibility have generally already been defined.
The more challenging area involves variants that are potentially predictive of response to an already approved drug. “In other words,” Robson said, “you send the test off to [a] commercial entity, get back a series of variations, and now you pull [a drug] off the shelf and use it.” This determination depends on such factors as whether the variant is germline or somatic, whether the link is biologically plausible, the prevalence of the allele for somatic mutations, whether the primary tumor or metastatic
disease has been analyzed, and whether a drug or a combination of drugs is available to use.
The optimal interpretation of a somatic sequence requires the sequencing of normal tissues, Robson said. Sorting out driver and passenger mutations can be very difficult, but finding that something is present in a tumor and not present in the germline is at least an initial piece of evidence that could be relevant to the cancer process. However, if there is a germline alteration, it may not be seen when comparing the two sequences, as many algorithms subtract germline from somatic variants found during sequencing (Bombard et al., 2013). Using next-generation sequencing techniques to generate data and compare germline and somatic mutations has also shown promise for identifying variants that are associated with susceptibility to cancer (Stadler et al., 2014).
Databases for Genomic Case Reports
Databases could be useful repositories for finding information about genes with weak disease associations or with unknown significance. For example, Rehm told of how a patient with the rare disease distal arthrogryposis type 5, a condition related to congenital joint contracture, underwent genome sequencing even though at the time the disorder had no known genetic etiology. Because this patient had unaffected parents, a de novo cause of disease was suspected, Rehm said.
Sequencing the genomes of the patient and the parents revealed two such de novo variants, one of which was quickly ruled out as a common loss-of-function mutation in that population. The remaining variant was a candidate, but there was no evidence to indicate it was causative of the disease, because everyone has de novo variants that are not necessarily related to a phenotype. Rehm and her colleagues contacted a researcher who studied the PIEZO2 protein, the product of the gene in which the variant appeared, and in this way they learned about a second family with a mutation in the same gene who had the same phenotype. The interaction “gave us enough evidence to claim a true causal association with this gene and that phenotype,” Rehm said (Coste et al., 2013).
One cannot expect serendipity to produce such findings too often, so Rehm and her colleagues are working to establish a database to house genomic cases. Various groups have contributed exome and genome data
along with phenotype information to a database that Rehm has developed. The data will be searchable and structured in a way that will allow for the identification of genetic commonalities among phenotypes. This is, she said, “a more robust, international approach to solving these very rare cases in both a clinical testing arena as well as a research context.”
ClinVar and ClinGen
The ClinVar variant database is designed to provide a freely accessible, public archive of reports of the relationships between human genetic variations and phenotypes (Landrum et al., 2014). All of the information being generated in Rehm’s laboratory is also being submitted to ClinVar so that the community can benefit from that information. “By putting a lot of this data that we come across [from] clinical testing and research testing into a common environment,” Rehm said, “that then provides a list of variants that either a researcher or a pharmaceutical company could … study. If they don’t know what variants are out there, there is no project to be done.” The individual efforts of institutions to gather and evaluate evidence can be scaled to benefit the larger genomics community through databases such as ClinVar, Rehm said. Data, including benign variant assessments, are deposited here for sharing it more broadly.
The Clinical Genome Resource, or ClinGen, is a collaboration among research groups dedicated to combining research data with data from clinical tests as well as expert curation to determine which genetic variants are most relevant to patient care (NIH, 2013). As part of this effort, the research groups are examining the standards and processes for evaluating genes and variants and genetic disorders in order to move toward more standardized procedures, said Jonathan Berg, assistant professor in the Department of Genetics at the University of North Carolina at Chapel Hill.
ClinGen starts with the variants, Berg said, so the first step in the effort has been to encourage laboratories to submit data to the project. The next step is to gather phenotypic information about patients in whom the variants are found, along with evidence from the laboratory indicating whether a variant is pathogenic or benign or if there is not enough evidence to be certain. The final step is to understand the clinical validity of gene–phenotype associations, which will provide a standardized framework for curating these associations. “If we can bring that information all together with standardized language and using the same vocabularies to describe what we’re talking about, then we will have a computational
resource that can be mined for clinical validity and the associations of these variants to disease,” he said.
ClinVar is part of the ClinGen collaboration, and together these resources will have a number of valuable uses, Rehm said. For example, they could enable the community to define what the best assays are for assessing a particular gene or disease model. “When you come up with a variant, you can turn toward the appropriate assay … and know where you could get it done,” she said.
Clinical actionability (see Box 2-1) requires both technical accuracy and interpretive accuracy, which together produce high specificity in terms of predictive value. It is important, Berg said, that such an intervention not impose undue hazards to an individual, whether psychosocial, medical, or financial.
Because individual expert opinions vary considerably, there is a need for a transparent, reproducible, evidence-based method for determining whether an identified variant is clinically relevant, Berg said. Thus Berg and his colleagues have divided the concept of actionability into several specific elements that give a semi-quantitative assessment of actionability for every gene–phenotype pair:
- Severity of a disease, which is typically the most severe possible outcome
- Likelihood of a severe outcome
- Effectiveness of an intervention to mitigate the severe outcome
- Acceptability of the intervention, with consideration given to all the hazards of the intervention
- State of the knowledge base, including knowledge about the gene–phenotype association, disease manifestations, and interventions
Each of the 5 elements receives a score from 0 to 3, for a total score of between 0 and 15. Thresholds can be set for dividing variants into bins indicating whether the variants have clinical utility or clinical validity or the clinical implications are unknown (Berg et al., 2011). (More details are provided later in this chapter in the subsection labeled “Binning the Genome.”) Different users could set the thresholds in different places, which provides the system with a measure of flexibility. “It balances the
benefits of the information versus the harms of the information, the paternalism of the physician’s duty to warn versus not doing any harm, and patient preferences for their right to know and not to know,” Berg said.
In addition to being flexible, the advantages of this system are that it is transparent and less subjective than expert opinion, with a clearly defined evidence base, Berg said. Furthermore, some of the workload can be crowd sourced—for example, in the analysis of the consistency or variability of scores. Different end users can use the information in various ways, weighing the parameters depending on the scenario of interest to the particular user (for example, research, diagnostic testing, healthy adults, or newborn screening). Finally, scoring can be revisited as new information becomes available.
This system could be useful in the context of other efforts, such as the return of incidental findings. For example, when Berg and colleagues used Berg’s system to compare 200 genes sorted into bins with a recent list of variants in 56 genes for which the American College of Medical Genetics and Genomics (ACMG) recommends returning information to individuals,1 they found variability in what different groups consider actionable (Green et al., 2013) (see Figure 2-1). The spectrum of actionability raises the question of whether the threshold has been set too low for the ACMG list because, for example, a number of genes on that list score only between 7 and 10 using Berg’s methodology.
As Robert Green, director of the Genomes to People Research Program in Translational Genomics and Health Outcomes in the Division of Genetics at Brigham and Women’s Hospital and Harvard Medical School, observed, thousands of genomes were being sequenced, and physicians were becoming uncomfortable with the idea that potentially lifesaving information discovered in sequencing data was not being reported. The ACMG recommendations were crafted to address this issue. The recommendations propose reporting specific mutations found in those 56 genes to physicians regardless of the indication for which the clinical sequencing was ordered.
With the information in hand, physicians are able to decide what to do with it while taking patient preferences into account. “You can have a
1Following much discussion over the ACMG Genome Sequencing Return of Results guidelines issued in March 2013, ACMG has since updated their recommendations to include an “opt-out” option for patients undergoing whole exome or whole genome sequencing. For more information, see ACMG Updates Recommendation on “Opt Out” for Genome Sequencing Return of Results, https://www.acmg.net.docs/Release_ACMGUpdatesRecommendations_final.pdf (accessed June 11, 2014).
FIGURE 2-1 Application of Berg’s binning metric to genetic variants demonstrates variability in which variants different groups would consider actionable.
NOTE: ACMG = American College of Medical Genetics and Genomics; HFE = hemochromatosis gene
SOURCE: Jonathan Berg, IOM workshop presentation, February 3, 2014.
very clear conversation with a patient about what they do not want to hear about, and you can respect that,” Green said. Berg asked whether some genes not included in the ACMG list, such as those involved in hemochromatosis, for example, should be considered for addition because of their high scores on the metric he developed.
The Medical Exome Project
Hegde’s group has taken an approach to qualifying evidence that is different from Berg’s. The production of a medical exome—the subset of a human genome consisting of the more than 4,000 genes that have been identified as clinically relevant and that can be adequately covered—will require evidence about each gene and a technically complete assay, Hegde said. To do this, Hegde’s laboratory has collaborated with the Children’s Hospital of Philadelphia and Partners HealthCare Laboratory
for Molecular Medicine to create the Medical Exome Project, a “highly curated gene resource and a technically optimized assay to provide a stepping stone for standardizing the interpretation of genetic variation.” The goal of the project is to develop a “medically enhanced exome” capture kit that covers all clinically significant genes so that when physicians are trying to diagnose a patient, they will have confidence that the known clinically relevant genes have complete coverage. Achievements to date have included increasing the coverage of known relevant cardiomyopathy genes from 85 percent to close to 99 percent.
The members of the project have defined the medical exome, Hegde said, by starting with all genes that have possible or proven disease associations, then curating to eliminate false-positive disease association claims, and doing iterative curation to remain current. The Medical Exome Project has worked closely with the ClinGen project to set up a four-tier classification scheme for genes (see Table 2-1). It also went through a pilot curation phase that found many incorrect gene–phenotype associations.
This is a time-consuming process; it takes about 5 hours per gene with at least 2 people researching and curating the gene data. With approximately 4,000 clinically relevant genes, Hegde said, “it is going to take a tremendous amount of [curation] time,” with many of the genes eventually being discarded because of a lack of evidence.
|0||Gene of undetermined (no studies available) or unlikely significance||Undetermined: No reported evidence
Unlikely: Evidence arguing against role in disease
Gene of “uncertain signifcance” (studies available ut insufficient to draw onclusions)
Single or few studies, variants, and families reported AND segregation not established OR no human studies reported but strong animal model data with relevance to human disease
Probably disease associated
Single or few studies, variants, and families reported AND limited segregation observed
Definitely an established isease gene
Multiple studies, variants, and families reported AND significant segregation and or strong functional evidence
SOURCE: Madhuri Hegde, IOM workshop presentation on February 3, 2014.
The Medical Exome Project is working on standardizing assays that will be publicly available for assessing variants, Hegde said. Through the Jain Foundation,2 Hegde and her colleagues have been assessing the biological significance of the variants of unknown significance of dysferlin, a protein involved in muscular dystrophies. By working with the Jain Foundation to acquire clinical data from patients, Hegde’s group is generating information about the variants, which will be submitted to ClinVar, Hegde said.
Actionability depends on the clinical context in which a genetic test is performed, said Katrina Goddard, senior investigator with the Kaiser Permanente Northwest Center for Health Research, in agreement with Berg. For example, actionability can be different depending on whether testing is done for the purposes of prenatal testing or newborn screening versus being performed during adulthood for disease screening (or preconception carrier testing), diagnostic, or monitoring reasons.
In the EGAPP working group with which Goddard has been involved, genes and conditions related to adult screening and predictive testing were proposed for full evidence review and evaluation based on the recommendations of subject matter experts or on the priorities of funding agencies. Topics then were selected for full review and evaluation based on the availability of evidence and other criteria.
Actionability was defined for adult incidental findings on the basis of the following three questions, Goddard said:
- Is there a practice guideline or systematic review for the genetic condition?
- Does the practice guideline or systematic review indicate that the result is actionable in one or more of the following ways?
- o Patient management
- o Surveillance or screening
- o Family management
- o Circumstances to avoid
- Is the result actionable in an undiagnosed adult with the genetic condition?
The group also decided that some areas were not actionable, such as incidental findings that are not related to the indication for testing at the end of a “diagnostic odyssey” where a patient or family has been searching for the explanation of a phenotype, findings that are not relevant for all patients in the EGAPP clinical scenario for reproductive decision making, and findings related to personal utility because they may not be actionable in a clinical context.
While results may not be clinically actionable, Berg said that just finding a molecular explanation for a patient’s previously unexplained symptoms and ending a diagnostic odyssey can have significant personal utility for the patient and his or her family. The information provided in a report to the patient from a test for such a case may be helpful but the report would not necessarily contain information about variants of uncertain significance. For general clinical use, genomic incidental or secondary findings would not be considered to be part of the routine report. While variants that have sufficient clinical actionability should be part of a routine clinical report, consistent with the ACMG recommendations, other classes of conditions that are clinically valid but have insufficient clinical actionability would be subject to more careful consideration on the part of the patient and clinician about whether a patient would prefer to be given such information, Berg said.
Genomic testing at Washington University uses a definition of actionability with components that are very similar to those described by Goddard in that practice guidelines for the genetic condition exist and that professional society practice guidelines recommend action for the purposes of patient management, surveillance or screening, family management, and circumstances to avoid. However, Shashikant Kulkarni, director of cytogenomics and molecular pathology at the Washington University School of Medicine, added that actionability also implies that medical interventions based on new results are effective and that actions are acceptable to the individual in terms of burdens or risks.
Actionability in Oncology
At Memorial Sloan–Kettering Cancer Center, a consensus-based approach is taken for reviewing potential actionability for genomic findings. As Robson explained, a multidisciplinary panel of individuals with expertise in basic science, drug development, clinical trial design, assay development and interpretation, and computational biology and biostatistics does a case-by-case evaluation of the evidence. For a tumor specific
driver mutation, actionability relates to whether a targeted therapeutic is indicated and available while for a deleterious germline mutation, actionability relates to mutation penetrance and the efficacy of available preventive medical interventions.
Incidental germline findings from tumor profiling are reported only in discovery studies with approval from the institutional review board (Yang et al., 2013) and only with proper consent. A multidisciplinary panel within the institutional review board (IRB) reviews the findings and decides whether to initiate the process of contacting the physician and patient. The actionability criteria remain “a bit fluid,” Robson said, and discussions have centered on what level of risk justifies contact. Very high penetrance predispositions, such as those associated with BRCA1 and BRCA2, are relatively straightforward to make decisions about. But variants that confer more modest risks are more problematic. For example, a test result may not be directly relevant for the person who was tested but could be relevant for family members. This remains an issue even with knowledge gained from sequencing specimens from patients who are deceased. Reaching out to family members in such circumstances can be difficult, but this information can be extremely relevant to their health. Today these decisions are being worked out largely on a case-by-case basis, Robson said.
The vast majority of genetic variants have no known clinical relevance. The challenge, Berg said, is therefore to parse through variants to determine which ones can be used to inform clinical decisions. This process requires setting a high bar for which variants from a genome-scale test to report; otherwise, reporting variants with unknown clinical validity (see Box 2-1) or unknown implications for the asymptomatic patient’s health could potentially have negative impacts, such as patient concern about a test result or unnecessary medical costs for testing that may not be clinically useful. Different people may hold different views on the benefits and risks of obtaining genetic information, so individual preferences factor into decisions on whether to return results to patients. Variants reported to physicians and to patients need to be those that can be incorporated into clinical care in an evidence-based fashion, Berg said.
Binning the Genome
Berg and colleagues have developed an a priori structured framework for handling genomic findings that they described as “binning” the genome (Berg et al., 2011). The framework is organized according to the concepts of clinical validity and clinical utility (see Box 2-1), and the binning is intended to facilitate pre-test informed consent, analysis, and post-test return of results. Use of the framework makes it possible to avoid “one-off” decisions that may not be consistent from one patient to the next. “Ideally,” Berg said, “we should know what we’re going to do with different classes of variants up front, so that when we are analyzing the data and we come across something, we know how we will handle it.”
The first step of the binning process is to categorize gene–phenotype pairs into bins according to the clinical actionability elements that Berg described earlier as well as to the risk for psychosocial harm. The second step defines the types of variants that should be reported. For example, known pathogenic variants are reportable, while likely pathogenic variants require further scrutiny before reporting. The third step is to sort an individual’s variants computationally into predetermined bins. Only variants that meet defined bin criteria are reviewed and reported, and new evidence triggers new determinations of how a variant is binned, Berg said.
Using this framework, Berg and his colleagues developed three bins (see Figure 2-2). Bin 1 includes variants that meet clinical utility criteria based on the medical literature and are therefore defined as medically actionable; examples include variants that are known or presumed to be deleterious. Bin 2 includes variants that have clinical validity but not clinical utility. Because of the lack of evidence for clinical action, the return of results to patients for these variants will depend on the individual patient’s interest in receiving the information balanced with any undue stress that may come with learning of the information. The amount of distress that the results could bring to patients is considered by dividing Bin 2 into low-, medium-, and high-risk information. Finally, Bin 3 includes all of the variants that have unknown clinical relevance related to phenotype, outcome, or clinical intervention.
FIGURE 2-2 Genetic variants can be sorted into three bins depending on the level of clinical utility.
NOTE: APOE = apolipoprotein E gene; GWAS = genome-wide association study; Long QT = Long QT syndrome; PGx = pharmacogenomics.
SOURCE: Jonathan Berg, IOM workshop presentation, February 3, 2014.
Systematic Evidence Gathering and Actionability Determination
The recognition of weaknesses in gene–phenotype associations has led those in Rehm’s laboratory to take a more systematic approach to evaluating and scoring the evidence. The approach divides gene–phenotype associations into the following categories: definitive, likely, weak, uncertain or unknown, and no association. Because the numbering systems currently in use vary and can cause confusion, some groups have moved away from labeling these or similar categories with numbers. Although it will take time and significant effort, it is important for those in the field to come to a consensus on a standard system for labeling variants, Rehm said. Genes in Rehm’s first category—“definitive”—are included in predictive tests and can be returned as incidental findings, while genes in the first two categories—“definitive” and “likely”—are included in diagnostic panels where the patient already has a phenotype.
Rehm and her colleagues comprise one of three groups that are working together to define the content for newborn genomic screening. The groups have been using the same categorical gene–phenotype association-based approach described earlier, but they are structuring the
data for making decisions about what should be returned to patients with respect to the age of onset of the disease, the inheritance pattern, penetrance, the phenotype category, and the availability of a clinical test. More than 600 genes have been evaluated, with approximately 3,000 to go, Rehm said. “We hope that by structuring this data, it will allow groups to make cutoffs and decisions about what we think should be returned to individuals.”
As part of the Clinical Sequencing Exploratory Research (CSER) consortium,3 Goddard said, the NextGen project is integrating whole-genome sequencing into preconception carrier status testing and evaluating the downstream costs and use versus those of the current standard of care. Through expert analysis, surveys, and focus groups, the project is gathering information from participants about whether they want to receive results for preconception carrier status screening in various health categories (see Table 2-2). The hope is to gain a better understanding of
|Shortened lifespan||Most children do not live past early childhood, even with medical intervention.|
Most children will have medical problems that require regular medical visits, daily medications, carefully monitored diets, or surgeries; or will have serious problems with learning, vision, hearing, or mobility. Children may have shortened lifespans into early childhood.
Most children will have medical problems that require occasional extra medical visits, occasional medications, a slightly modified diet, or surgery; or will have mild problems with learning, vision, hearing, or mobility.
It is difficult to predict the outcome for many children with these conditions. Some children will have more serious versions but others will have a more mild version or no problems at all.
Few have any symptoms as children, but medical, behavioral, vision, or hearing problems may begin as adults.
SOURCE: Katrina Goddard, IOM workshop presentation, February 3, 2014.
what types of carrier status results patients will be interested in receiving in the future, Goddard said.
Three-Stage Evaluation Process
The evaluation process used to determine which results to return to patients for projects such as NextGen consists of three stages, Goddard said. The first stage is a preliminary assessment to determine whether sufficient information is available to do a full review. In this stage, the actionability concepts described earlier as well as variant penetrance and whether the condition is a significant and important health problem are considered. If the condition does not meet one of these criteria, a full review will not be undertaken. The objective of this stage is to provide a rapid mechanism for determining which conditions do not have sufficient information to warrant further evaluation.
In the second stage, an evidence-based process for each specific gene–phenotype pair is documented in a summary report. Reproducible search methods are used to identify studies and data, which are restricted to systematic reviews, evidence-based practice guidelines, or expert consensus-based practice guidelines. Each gene–phenotype pair is summarized in about two pages, with a goal of keeping the summaries brief, transparent, and reproducible. “This is not a comprehensive method, and we are aware of that, but that was [a] pragmatic choice,” Goddard said. To assess the data, it is sorted into evidence tiers (see Box 2-2) to address expected disagreement among sources and to signal the overall quality of sources. Quality ratings are used as tie-breakers for conflicting evidence at the same tier. In Stage 3 the summary produced in Stage 2 is used by a decision-making group—whether EGAPP or another group—to make recommendations.
During the data assessment stage, Goddard said, information is categorized into tiers of evidence to classify the source of data and its quality. Those tiers are:
- First Tier: Evidence from a systematic review, meta-analysis, or clinical practice guideline based on a systematic reviewa of the objectives, methods, findings, and other criteria.
- Second Tier: Evidence from clinical practice guidelines or broad-based expert consensus with some level of evidence review, but using unclear methods or using sources that were not systematically identified.
- Third Tier: Evidence from another source with non-systematic review of evidence (e.g., GeneTest Reviews, OrphaNet, Clinical Utility Gene Cards, and the opinion of fewer than five experts), with additional primary literature cited.
- Fourth Tier: Evidence from another source with non-systematic review of evidence (e.g., GeneTest Reviews, OrphaNet, Clinical Utility Gene Cards, and the opinion of less than five experts) lacking citation of primary data sources.
Methods for Variant Annotation in Cancer
In 2011 the Washington University School of Medicine began offering next-generation sequencing in addition to the other genomic tests it performs. Because of the school’s particular expertise, it focused on cancer genomics. Curating genomic variants has proven to be a huge task, Kulkarni said. “If the germline is that difficult, consider how difficult cancer variation data curation could be.”
A bioinformatics team is needed to analyze the sequencing data after it is generated, Kulkarni said. Even in the case with a 42-cancer gene panel, there is too much information to process manually, so a software system was designed to perform base calling, alignment, variant calling, and genome annotation in a semi-automated way. In the first phase, custom scripted software programs facilitate an automated step in which the data are compared against publicly available gene information and clinical and mutation terms. Criteria for the searches are set such that relevant papers must contain human data and one or more mutations and must describe a clinical outcome. Where there are commonalities in these three areas, an annotation worthiness score is generated for each variant, and the information is deposited into a searchable spreadsheet for the next phase.
Following this automated process, an external group of six annotators reviews the data over several months. A second evaluation is conducted by the clinical fellows and attending physicians at Washington
University School of Medicine, Kulkarni said. Variants are classified into five levels, which are based on the ACMG guidelines:
- Level 1—Predictive or prognostic in tumor type (includes inherited cancer susceptibility variants).
- Level 2—Predictive or prognostic in another tumor type or types.
- Level 3—Reported in cancer or other disease.
- Level 4—Variant of unknown significance.
- Level 5—Known polymorphism.
The data are then made available on a wiki-based user interface where other clinical fellows and attending physicians could review and modify information about the variants. Presentation of the results sorts the variants by level, with an interpretation of the role of the variants and references to the medical literature. The resulting report provides information about the variant and related data, as well as the ability to examine each step of the variant annotation filtering process. During monthly meetings, new evidence is collectively reviewed. “This is a very comprehensive effort,” Kulkarni said, “and it’s ongoing because there is a lot of new information coming out … on these cancer genes.”
Since March 2012 about 1,500 clinical tests have been ordered, not including those from clinical trials, Kulkarni said. The tumor types tested cover a broad range, including brain, colorectal, lung, pancreatic, and sarcomas and the initial findings suggest that about 45 percent of sequenced cases have specific actionable mutations in targetable genes, including EGFR, KIT, KRAS, and PIK3CA, he said.
Workshop participants described a number of challenges for the future. For example, more than 50 million genetic variants have been found in the human genome, Rehm said, with many of them unique to individuals, and misinterpretation of these variants can affect clinical care and study outcomes.
The vast majority of the variants seen in clinical testing and research studies are rare, which makes it difficult to generate sufficient evidence to make a claim. For example, Rehm said, diagnostic testing of 15,000 probands for a variety of hereditary disorders and for somatic cancer revealed about 1,600 variants reported as either pathogenic or likely patho-
genic; in this case, 68 percent of those variants were seen only once, and 96 percent of the variants were seen fewer than 10 times. Based on testing conducted in Rehm’s laboratory, about one-third of the variants are categorized as having uncertain significance. “Our community will need to develop better approaches to evaluating these variants and their impact,” she said.
The challenge is even greater for the return of incidental findings from exome or genomic sequencing. Rehm cited data from the MedSeq project that indicated that each patient in the study had 20 to 40 variants that had been published as disease causing or as pathogenic (Vassy et al., 2014). However, when these variants were reviewed with strict criteria for pathogenicity, 97 percent were excluded, most of them being uncertain, and many having clear evidence for being benign. Similarly, 30 to 50 variants were linked with loss of function in disease-associated genes in a MedSeq study, but strict review of the evidence determined that 94 percent of these variants did not meet the criteria for pathogenicity. “This is a lot of work with a low yield as we look through patients’ genomes,” Rehm said.
The group is now working to develop better guidance on the interpretation of sequence variants, and a draft of that guidance is expected to be ready for the community in the coming months, Rehm said. A further complication, she noted, is that the evidence constantly changes, so that new information needs to be returned to patients tested in the past. Berg agreed, noting that information will need to be updated over time. A system developed by Partners HealthCare, which features e-mails sent to physicians with new information that can be inserted into patients’ electronic health records (EHRs), has been “very effective,” Rehm said.
Future challenges include continuing to seek the right balance between brevity and comprehensiveness of the assessments and determining whether to relate variants on genes to conditions, Goddard said.
There are certain challenges that come with Kulkarni’s approach to assessing variants for clinical use. It would be possible to scale the system to a large number of genes and variants, Kulkarni said, although it requires a large amount of upfront work. For example, a team of six annotators worked for 6 months on the initial 28-gene-variant curation, sorting out the variants based on given criteria. Additionally, both algorithmic and knowledge-based variant curation methods are necessary for clinical interpretation, and annotating and keeping up with variant management is expensive. As a result, there is an urgent need for implementing universal standards and a variant resource database, he said. Fortunately,
this work does not all have to be done by one organization, but rather can be done collaboratively.
Kulkarni observed that ethnicity is considered in the review of annotations, but more data need to be generated from different racial and ethnic groups. He also noted that data is available through the International Consortium of Cancer Genomics, which has data from different population groups, though in this area, too, more information from different groups is needed. Goddard identified concerns over extrapolating data from high-risk populations to the general population and considering the variation among individuals. For example, X-linked conditions are more relevant for males, and conditions with variable penetrance have different risks depending on the strength of family history.
New data is often produced that can trigger the reinterpretation of a variant–disease link, but a key question is how those data should be communicated broadly so that clinicians and laboratories are working with current information. Perhaps, Berg said, this needs to be an ongoing process such that new evidence would initiate new reviews through a system that could be triggered by inquiries from physicians, researchers, or patients. The reclassification strategy developed at Emory University was designed so that clinicians could generate or point to evidence that a gene is probably not related to a disease, Hegde said. “The labs cannot do it on their own,” she said. “The number of variants of unknown significance has grown so much [that] you can imagine how much time it takes for the lab to go through all those variants and reclassify them. It is a huge help for us if the clinicians actually approach us.”
The ACMG guidelines require that a testing laboratory make an effort to contact physicians who previously tested patients in the event that new information changes the initial clinical interpretation of the sequence variant, Hegde said. To fulfill this guideline, the Emory Genetics Laboratory has set up a Web-based system to release updates on all the variants seen and analyzed by the laboratory (Bean et al., 2013). As variants are reclassified, the system automatically scans the internal database and identifies previously affected patients. It then sends an alert to laboratory directors and issues amended reports. In addition, the system reclassifies variants based on outside requests, with the information then returned to clinical targets.
Asking physicians to provide clinical data can be challenging because they are often too busy to fill out data forms, Rehm said. Nevertheless, these data can be extremely important—for example, when an affected patient in a family tests negative for a variant. “As a community, we need to underscore the importance of the dynamic relationship between the lab and the physician if we hope to improve our understanding of genomic variation,” Rehm said. Berg agreed, noting that the paperwork needed to report on a variant can seem particularly extensive in the context of a pressured clinic. “There needs to be better mechanisms for physicians to be able to supply the phenotypic information to the labs in a structured format,” he said, “because I think that adds to the specificity of the analysis.” There are also challenges of how much clinical information can be shared because of Health Insurance Portability and Accountability Act (HIPAA) issues.
Related to the issue of updating variant information is a concern over the reproducibility of data in the literature. When published studies do not include complete procedures or the primary data are not accessible, it makes evaluating the quality of the data difficult. Rehm noted that the ClinGen project and the ClinVar database are creating a structured mechanism that requires the authors of a paper to make the raw data available so that their results can be verified and extended in a transparent way. But published associations that are inaccurate remain an unresolved problem, because the act of publishing data is often misconstrued as providing a quality piece of evidence in and of itself. Similarly, Hegde observed that important data may not be available in a database, may be out of date, or may be expensive to access. ClinVar will be important for that reason because it will be a free and open database. Other efforts, such as the Global Alliance for Genomics and Health, are also working on ways to responsibly share genomic and clinical information across groups (Callaway, 2014).
Standardized methods for rating quality in the field of genomics do not yet exist, Goddard said. Berg, too, pointed out that “there is really no specific definition of what constitutes a proven gene–phenotype association. There are certainly genes that we know because the evidence is compelling and overwhelming, but then you get to the many gene–phenotype associations based on a couple of case reports or a handful of families, and there are no specific guidelines to say this is where you draw the line.”
There is also the issue of an evidence gap when there is no synthesized evidence in the literature to rely on for evaluating gene–phenotype pairs. When this occurs, we need to prioritize our reviews, Goddard said, so that researchers and clinicians do not spend too much time addressing gene–phenotype associations that have no systematic review or practice guideline that can be referred to.
Kulkarni added that an ideal situation dealing with these evidence gaps would be for researchers to have sufficient tools to model and assess disease progression, clonal evolution, and response to therapies. Maybe in 5 to 10 years, he said, we will have a Clinical Laboratory Improvement Amendments–certified mouse facility for treating primary, secondary, and metastatic tumors from the same patient and then reporting the results back to the clinic.
One collaborative approach to assessing data has been taken by the Evidence-based Network for the Interpretation of Germline Mutant Alleles (ENIGMA) Consortium,4 an international consortium that is taking a multidisciplinary, multi-institutional approach to understanding the involvement of all variants of uncertain significance in BRCA1 and BRCA2 that may be related to breast and ovarian cancers, Robson said. The members of this organization have pooled their clinical expertise, their data, and their laboratory capabilities to resolve issues of evidence involving breast cancer predisposition genes.
There are also technical issues with DNA sequencing that make interpretation of evidence difficult. Rehm said out that no whole-genome sequencing effort today is complete in covering all regions of the genome or detecting all types of variation including substitutions, insertion–deletions, copy number variations, structural variants, and gene fusions. In addition, the bioinformatics techniques applied to raw sequence data in different labs can produce differences in sequences. Finally, the interpretive process can vary from laboratory to laboratory, resulting in different
interpretations of results due to variation in data filtering, alignment, variant calling, quality thresholds, and annotation. “There are so many levels today that are non-standardized and distinct that you will often get different results for different reasons,” Rehm said. “Those are all aspects of this process that we, as a community … need to address.”
On the subject of assessing the comparability of genome sequencing results from different laboratories, Hegde observed that the College of American Pathologists has a next-generation sequencing committee that is working on a database for proficiency testing and cross-validation of variant detection. Select variants will be confirmed by Sanger sequencing, and then the data will be shared more broadly. “This is very important because, as we just heard, these platform differences are significant,” Hegde said. Rehm added that the ClinVar database will help in this regard because it will provide transparency where interpretations differ. When data are submitted to ClinVar, a quality control report is generated that describes where the submitter’s variant interpretations differ from those that are already in the system.
There is also a disadvantage for clinical laboratories in that they generally cannot do functional assays for the biological relevance of variants they observe, Hegde said. Instead, they need to connect with a research laboratory to work on a particular gene or disease. But researchers who have worked on a gene or disease in the past may no longer have the funding to do more research in that area. “The question of how you can do a biological relevance assay is a big one,” Hegde said. Berg agreed, adding that there is an opportunity here for researchers who have robust assays and who can reproducibly separate benign from pathogenic variants in order to overcome these types of technical challenges. However, resources would be necessary to complete the work, he said.
Berg acknowledged the complexity related to genetic variants. Variants are interpreted as being pathogenic or benign with respect to particular disorders, but a given gene can be involved in multiple disorders. “Figuring out how you define pathogenicity means you have to explicitly link the pathogenicity assertion to the gene and the disease that is related to it,” Berg said. Additionally, curation of variants of unknown significance can be more than a technical challenge because the level of understanding of the basic biology of the gene, protein, and pathway involved can influence how variants of unknown significance are classified.
Electronic Health Records
Genomic information needs to be available in some way, but various barriers exist to making it available through an EHR, such as the capacity of the record, Berg said. Rehm noted that the issue has been discussed in the past and that the thinking has been that the information should not be in an EHR. Only a percentage of the variants in genomic data have been rigorously confirmed, and these can be interpreted and put in a report that goes in the EHR. “The consensus right now is you can update [information] that you’ve already put there,” she said, “but we are not ready to expose the 5 million variants in a genome to an environment where a clinician queries that, finds a variant, and goes and treats a patient when in fact that was an incorrect call.” This view may change as the technology advances, she added, “but we are not there yet.”