National Academies Press: OpenBook
« Previous: 4 Overcoming Barriers for Evidence Generation
Suggested Citation:"5 Considerations Moving Forward." Institute of Medicine. 2011. Generating Evidence for Genomic Diagnostic Test Development: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13133.
×

5
Considerations Moving Forward

Key Points Raised by Speakers

  • Developing new models for coverage of tests post-FDA clearance could aid the development of clinical utility evidence.

  • Patient management and outcomes should be the focus of inter­vention development.

  • There needs to be greater accounting of and transparency in experimentation for biomarkers.

  • The public needs to be educated on the need for evidence of utility for new interventions.

In light of the barriers and complexities associated with the various models of evidence generation discussed and of the need for more timely evidence gathering in order to better meet the needs of patients and providers, the workshop participants considered what the next steps should be in generating new evidence for the development of genomic diagnostic tests.

MEDICAL PRACTICE VERSUS EVIDENCE-BASED RECOMMENDATIONS

Roger Klein, a member of EGAPP, asked why medical practice often diverges from the recommendations issued by evidence-based review groups. For example, there are a number of tests for thrombophilia that

Suggested Citation:"5 Considerations Moving Forward." Institute of Medicine. 2011. Generating Evidence for Genomic Diagnostic Test Development: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13133.
×

have been cleared by the FDA. These are very commonly ordered assays, yet there is limited evidence that the results of these tests offer any clinically useful information for most people.

Dan Hayes responded that tests are often ordered because they are available and easy to order and there is little cost to the doctor for using them inappropriately. Physicians order marker tests as part of general data gathering about the patient. “A bad marker is as bad as a bad drug,” he said, adding that providers are making critical treatment decisions based on information that may be wrong. Practitioners could do much better in not overtreating patients who will not benefit and in identifying patients who will benefit. This is where marker development is as important as pharmaceutical development, Hayes said. Becker added that the idea of being able to manage clinical effect (e.g., the hoped-for effect against a tumor or the range of adverse events) is something that lies in the realm of clinical practice. It is a matter of judgment and experience. The question in the mind of a clinician deciding whether to adopt a new test is, What does it add to what is already done in day-to-day practice? Hayes added that clinicians are sometimes of the opinion that practice guidelines are made without considering individual patients and the nuances that come with each. Such contextual issues are difficult to build into the guidelines.

One approach that could affect practice would be for third-party payers who agreed to cover a genomic diagnostic test to then deny coverage of a treatment if the provider treated the patient differently than what the test result had directed (but only if the recommendations from the results are clear cut). Blue Cross and Blue Shield plans, for example, do not pay for trastuzumab treatment for a HER2-negative patient because such a test result indicates it is not a medically appropriate treatment. However, some participants said, this approach does raise concerns about patient autonomy and about patients’ rights to change their minds on treatment course in certain circumstances.

HETEROGENEITY OF EVIDENCE-BASED DECISIONS

Moderator Sharon Terry of Genetic Alliance pointed out the variation in evidence-based decisions. For example, FDA recently cleared the breast cancer recurrence test, MammaPrint, for more broad use in all age groups, while the Blue Cross and Blue Shield Association TEC determined that Oncotype DX met its criteria but MammaPrint did not, and EGAPP has concluded there is insufficient evidence to recommend for or against either test.

Piper responded that the Oncotype DX test was reviewed by TEC several times, and the final vote to cover the test was very close. Similarly,

Suggested Citation:"5 Considerations Moving Forward." Institute of Medicine. 2011. Generating Evidence for Genomic Diagnostic Test Development: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13133.
×

the final EGAPP recommendation was not arrived at easily. (As discussed by Calonge in Chapter 3, the insufficient evidence conclusion for Oncotype DX was classified as “encouraging indirect evidence.”) These nuances do not come across because only the final decision or recommendation is released. TEC assessments summarize the available evidence, Piper noted, but are not a practice guideline, while EGAPP is able to incorporate some of these contextual issues. This difference, she said, may add to some of the disparities between recommendations.

Hayes said that the ASCO committee was widely criticized and accused of being United States–centric for having recommended the 21-gene test, Oncotype DX, which was developed by a company in the United States, and not recommending the 70-gene test, MammaPrint, from an Amsterdam-based company. Hayes explained that ASCO’s Oncotype DX decision was based on the review of studies that asked clinically relevant questions regarding node-negative, estrogen receptor–positive women and tamoxifen that were applicable to practice. Becker said that one of the most important differences between the tests was the fact that Oncotype DX claimed to be able to therapeutically manage patients, while MammaPrint could not specifically make that claim. It came down to clinical utility versus clinical validity.

A test that is cleared or approved by FDA may not necessarily be reimbursed by payers, Becker noted. Clinical validation ties the test to the disease and diagnosis, which may lead to a prognosis, but the payer is focused on the utility. This system, where regulatory approval is disassociated from reimbursement approval, has been in place for a long time in the United States, and, Becker said, “It is not a system which was tailored for the rapidly evolving circumstances that we see now.” A participant warned against linking FDA clearance and reimbursement too closely, as this may lead to other problems. “Either the bar at the FDA would be too high for the commerce aspect or too low for what a rational system should be spending its resources on.”

Currently, a device developer might allot three to five years in its expected development timeline to navigate the FDA clearance process, a participant said. If the target is changed to clinical utility, that could easily push this phase of development out to 7 to 10 years. If payers will reimburse for devices after they are cleared or approved by the FDA, it gives the manufacturer an opportunity both to sell the product and to prepare and conduct clinical utility trials. Developing biospecimen banks will help close the gap between what FDA requires and what third-party payers require. Coverage with evidence development is a valuable tool as well, as it allows for provisional coverage while studies are being conducted.

Suggested Citation:"5 Considerations Moving Forward." Institute of Medicine. 2011. Generating Evidence for Genomic Diagnostic Test Development: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13133.
×

RESEARCH

Piper reiterated that research is too often driven by the desire to secure grants, publications, FDA approval, and reimbursement, when the real focus should be whether the intervention will make a difference in managing the patient. This should be the starting point in choosing tests to evaluate.

It was suggested by several participants that the Institute of Medicine roundtable could help define the top 10 genomics clinical questions that need to be answered. Such a list would not have to be the definitive list, but it could provide well-justified examples and help flesh out other questions that would also be appropriate. It will also be important to compare these questions with the priorities of the medical system at large, one participant commented. The point was made again that the success of Oncotype DX was due to the fact that the clinical question came first, then the developer figured out what kind of dataset was needed to answer the question and found the most appropriate previously conducted clinical trial. As another participant noted, there needs to be a focus on addressing the salient clinical questions.

With regard to building the genomics evidence infrastructure, one participant suggested starting with such a list of ten different, clinically important questions and assembling groups to decide how best to answer those questions and to determine the infrastructure that would need to be developed to facilitate that approach. Some of the answers will come from analysis of biobanked samples, but there may also be ways to answer questions outside of clinical trials. It would be very informative to solve a set of specific, important problems over the next five to ten years and see what evidentiary approaches emerge.

A participant pointed out that research has changed over time as NIH has required grantees to include minorities, children, and women in studies and has required that data-sharing plans be devised. What if NIH required grantees to demonstrate the potential for translation into practice?

Another participant observed that device manufacturers tend to design trials that place the product into a lower category for FDA clearance so that the product can be brought to market more quickly. While it may speed up approval, this approach does not deliver information about the true value of the test for patients and clinicians.

It was noted that one of the themes that has emerged in every roundtable workshop is the idea of having a framework that allows for access to high-quality biological samples connected to clinical data and which facilitates discovery and validation of genetic and genomic biomarkers. “We have to build the sandbox that everybody will play in,” said a participant. It was suggested that GAPPNet (a stakeholders’ group formed by CDC)

Suggested Citation:"5 Considerations Moving Forward." Institute of Medicine. 2011. Generating Evidence for Genomic Diagnostic Test Development: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13133.
×

could offer a forum for taking discussions and ideas from the roundtable workshops and fleshing out how to translate them into practice by pulling together public and private partners to conduct pilot projects to test the ideas in different contexts.

BROAD CONSIDERATIONS

One participant offered several observations relevant to moving forward with an evidence generation model for genomic tests:

  • Individual learning curves vary considerably, and learning is an ongoing process that extends beyond medical/health professional school.

  • Part of the process is learning to be comfortable with being wrong.

  • Variability among both consumers and providers of health care results in different messages being given at different times and in different situations.

  • Familiarity with new tests is variable (e.g., knowing what actions to take in the face of strong test results).

  • There is a general lack of comfort with and understanding of change.

  • There is a lack of comfort with ambiguity.

  • There is no perfect evidence and no absolute truth. We need to set a standard of excellence that providers can be comfortable with.

TRANSPARENCY

A participant said that the evidence base is damaged by selective publication of clinical trials. Although manufacturers are required to register all clinical trial protocols publicly in ClinicalTrials.gov, the participant said she believed that not all trial results are being reported.

Hayes agreed that ClinicalTrials.gov has been a great step forward. The Consolidated Standards of Reporting Trials have helped to ensure transparency in clinical trial reporting, as have the requirements by many journals that authors provide details about basic research studies, such as the specific reagents used. But for some reason, the translational area in between basic and clinical research has been left behind. Hayes suggested that many journal editors, who are generally very careful about not accepting clinical trials that are not registered in ClinicalTrials.gov or that do not provide essential experimental details, have abrogated their responsibility in terms of biomarkers, because markers are not valued in the way that clinical trials are. Hayes mentioned a new effort to initiate a registry for tumor marker

Suggested Citation:"5 Considerations Moving Forward." Institute of Medicine. 2011. Generating Evidence for Genomic Diagnostic Test Development: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13133.
×

studies. This registry would help ensure that studies are being conducted with prospectively written protocols and that negative results come to light.

It was further suggested that the roundtable engage major journal editors and urge them to pay more attention to tumor marker studies with regard to design, quality, and transparency.

PUBLIC EXPECTATIONS

Participants discussed the need for a public education effort regarding genomic interventions. Terry said that many people believe the FDA to be primarily interested in the public’s health and not in commerce and assume that agency clearance implies that the product is beneficial and should be adopted. On the other hand, a participant noted, the public often perceives FDA as being too slow or hindering access to products (such as happened with products targeting the human immunodeficiency virus). Another participant added that the current environment is one of “immediate-communications” and that health literacy and public understanding about evidence are highly variable. While eager for new products to be available on the market, the public in general does not demand evidence to support claims about new interventions. As a participant said, “If we could simply educate the public to ask [for evidence] every time they hear a claim, I think it would help us.” It was also noted that whole-genome analysis will enable providers to give patients their entire genome, but it will not be of much practical use unless the genome can be related to various kinds of clinical evidence.

Suggested Citation:"5 Considerations Moving Forward." Institute of Medicine. 2011. Generating Evidence for Genomic Diagnostic Test Development: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13133.
×
Page 53
Suggested Citation:"5 Considerations Moving Forward." Institute of Medicine. 2011. Generating Evidence for Genomic Diagnostic Test Development: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13133.
×
Page 54
Suggested Citation:"5 Considerations Moving Forward." Institute of Medicine. 2011. Generating Evidence for Genomic Diagnostic Test Development: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13133.
×
Page 55
Suggested Citation:"5 Considerations Moving Forward." Institute of Medicine. 2011. Generating Evidence for Genomic Diagnostic Test Development: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13133.
×
Page 56
Suggested Citation:"5 Considerations Moving Forward." Institute of Medicine. 2011. Generating Evidence for Genomic Diagnostic Test Development: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13133.
×
Page 57
Suggested Citation:"5 Considerations Moving Forward." Institute of Medicine. 2011. Generating Evidence for Genomic Diagnostic Test Development: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13133.
×
Page 58
Next: 6 Final Remarks »
Generating Evidence for Genomic Diagnostic Test Development: Workshop Summary Get This Book
×
Buy Paperback | $29.00 Buy Ebook | $23.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Ten years after the sequencing of the human genome, scientists have developed genetic tests that can predict a person's response to certain drugs, estimate the risk of developing Alzheimer's disease, and make other predictions based on known links between genes and diseases. However, genetic tests have yet to become a routine part of medical care, in part because there is not enough evidence to show they help improve patients' health.

The Institute of Medicine (IOM) held a workshop to explore how researchers can gather better evidence more efficiently on the clinical utility of genetic tests. Generating Evidence for Genomic Diagnostic Test Development compares the evidence that is required for decisions regarding clearance, use, and reimbursement, to the evidence that is currently generated. The report also addresses innovative and efficient ways to generate high-quality evidence, as well as barriers to generating this evidence.

Generating Evidence for Genomic Diagnostic Test Development contains information that will be of great value to regulators and policymakers, payers, health-care providers, researchers, funders, and evidence-based review groups.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!