Important Points Emphasized by Individual Speakers
- Test development may be stifled if creative solutions for demonstrating utility in a timely manner are not developed.
- Sustained dialogue between stakeholders is necessary to understand their views on clinical utility.
- The ability to discover potentially beneficial markers far exceeds the ability to translate them for patient use.
- A critical factor constraining the development and use of molecular diagnostics is a clear link between their use and improved patient outcomes.
- Clear, consistent, and predictable evidentiary expectations are essential to move forward in designing studies of clinical utility.
- Stakeholders need to collectively determine the optimal balance between access to new technologies and the need for certainty about risks and benefits associated with their use.
Bringing diagnostic tests to market previously required developing evidence of technical feasibility, analytic validity, and clinical validity, said Robert McCormack, co-chair of the workshop. Decision makers now need
|In Clinical Use|
|HER2/neu||Trastuzumab, Pertuzumab||Breast Cancer—Recurrence/Survival|
|Oncotype Dx||Use of Adjuvant Chemotherapy||Breast Cancer—Recurrence/Survival|
|EGFR mutation||Erlotinib||Lung Cancer—Progression/Survival|
|K-ras||Cetuximab, Panitumumab||Colorectal Cancer—Progression/Survival|
|EML4-ALK mutation||Crizotinib||Lung Cancer—Progression/Survival|
|BRAF V600E||Vemurafenib||Melanoma Cancer—Progression/Survival|
|BCR-ABL||Imatinib, Dasatinib, Nilotinib||CML—Response|
|TPMT||6-MP, 6-TG||ALL, AML—Toxicity|
|MSI Status||5-FU||Colorectal Cancer—Prognosis/Recurrence/Survival|
|Mammaprint||Treatment Regimen||Breast Cancer—Recurrence/Survival|
|K-ras Mutation||Anti-EGFR Therapy||Lung Cancer—Recurrence/Survival|
|ERCC1||Cisplatin-Based Therapy||Lung Cancer—Recurrence/Survival|
NOTE: ALL, acute lymphoblastic leukemia; AML, acute myeloid leukemia; BCR-ABL, breakpoint cluster region-abelson; BRAF, v-raf murine sarcoma viral oncogene homolog B1; CML, chronic myeloid leukemia; DPD, dihydropyrimidine dehydrogenase; EGFR, epidermal growth factor receptor; EML4-ALK, echinoderm microtubule-associated protein-like 4-anaplastic lymphoma kinase; ERCC1, excision repair cross-complementing rodent repair deficiency, complementation group 1; FU, fluorouracil; GIST, gastrointestinal stromal tumor; HER2, human epidermal growth factor receptor 2; MP, mercaptopurine; MSI, microsatellite instability; TG, thioguanine; TPMT, thiopurine methyltransferase; UGT1A1, UDP glucuronosyltransferase 1 family, polypeptide A1.
SOURCE: Adapted from Andrew Freedman, workshop presentation, May 24, 2012.
the answers to four key questions in considering the use of molecular diagnostics in oncology.1
- Does the genomic application provide correct information? This question addresses the analytic validity of the test by assessing whether an application is measuring what it is supposed to measure.
- Is there a significant association between the results of the genomic application and the clinical phenotype? This question of clinical validity assesses whether a relationship exists between the results of a test and a condition affecting health.
- Does the genomic application provide clinically significant information? This measure of clinical utility determines whether the information from the application leads to a clinical decision that improves outcomes, taking into account the benefits and harms associated with those actions.
- Finally, does the genomic application lead to improved patient outcomes as compared with the alternatives? This is an additional measure of clinical utility that relies on comparisons of utility or added clinical value.
The addition of clinical utility questions has changed the traditional path and altered the processes involved in developing diagnostic tests.
An increasing number of genomic predictive markers are either in clinical use or are undergoing testing to answer at least the first two questions listed above (see Table 2-1). Nevertheless, clinical utility is still largely unknown for most genomic applications. Demonstrating the clinical value of these technologies may potentially reduce the waste of health care resources from inconsistent or unnecessary use of tests and increase the quality of care received. Still, the evidentiary requirements to demonstrate clinical utility for genome-based diagnostics remains unclear, said McCormack. “We are hopeful that we can come to a point where we can understand the level of evidence that is required to get us to the next level of a seamless pathway for introducing these [diagnostic tests] into patient use.”
According to Sean Tunis, director of the Center for Medical Technology Policy, which was a cohost of the workshop, molecular diagnostics
1 Andrew Freedman, chief of the Epidemiology and Genomics Research Program’s Clinical and Translational Epidemiology Branch at the National Cancer Institute, cited these four questions during his workshop presentation, which is summarized in Chapter 4.
could have a transformational impact on medicine. Though molecular diagnostics currently apply to only about 2 percent of the population, that number could eventually rise to 60 percent (Ferreira-Gonzalez et al., 2008). PricewaterhouseCoopers (2009) has estimated that the diagnostic and therapeutic segment of the personalized medicine market will be $42 billion by 2015, with a 10 percent annual growth rate.
The current reality belies this vast potential, however. As Teutsch et al. (2009) have written, “Of most concern, the number and quality of studies [of genetic tests] are limited. Test applications are being proposed and marketed based on descriptive evidence and pathophysiologic reasoning, often lacking well-designed clinical trials or observational studies to establish validity and utility but advocated by industry and patient interest groups” (p. 3). This is a “serious indictment,” said Tunis, and contrasts strongly with the expectations for the field.
The Centers for Medicare & Medicaid Services’ (CMS’s) guidelines for the evaluation of diagnostics tests center on two questions. First, is the evidence adequate to determine whether the test provides accurate diagnostic information? Second, if the test changes accuracy, is the evidence adequate to determine how the changed accuracy affects health outcomes? The clear message is that diagnostic accuracy by itself is not enough, said Tunis. The important factor is whether test results lead to changes in practice that can be linked to improved health outcomes.
Establishing this linkage to improved patient outcomes requires clear, predictable, and consistent standards of evidence by which diagnostic technologies will be judged, stated Tunis. These standards, in turn, will dictate the infrastructure and partnerships that are needed, and they will be essential for investors and entrepreneurs to judge accurately the risk and potential returns on investment.
In the past, coverage decisions have not necessarily been based on clear evidentiary standards, Tunis said. In some cases, coverage decisions have been dictated by legal challenges. Private payers tend to follow the lead of CMS in making their coverage decisions. For the field to move forward, Tunis said, a decision-making process among the payers needs to be more clearly tied to a clear and shared understanding of clinical utility.
Cancer diagnostics have progressed from an era of tumor markers to biomarkers, said McCormack. Tumor markers were helpful in making decisions, but they were also problematic. Many were based on serum tumor markers that were validated using sample banks that were not well pedigreed or well stored and were drawn from readily available patients.
The evidence generated for such markers “met the standard of the day,” said McCormack, “but left a lot to be desired.”
Today, far more biomarkers are available. From 1960 to 1989, fewer than 50,000 publications in the Library of Medicine mentioned biomarkers. In just the first decade of the 21st century, more than 250,000 did. During that period, medicine has moved from classifying cancers based on organ or tissue to classifying cancers based on pathway. Laboratory medicine also has evolved. An explosion of technology, especially at the genomic level, has resulted in procedures and results that require skilled specialists to acquire and analyze data. Yet the answers provided by those data “are not overly obvious,” said McCormack, and the data are being interpreted by a generation of practitioners and researchers who were not necessarily schooled to fully understand that information. “It is apparent to everyone that our ability to discover potentially beneficial markers far exceeds our ability to translate them for patient use,” he added.
Highlighting how the need for clinical utility information has recently altered the development of molecular diagnostics, McCormack drew on his experience in having been involved in several major diagnostic development projects. Of four tests—the development of the prostate specific antigen (PSA) test in 1993, high-throughput hepatitis testing in 1997, testing for tumor cells in the blood in 2004, and a two-gene pathology lab test to detect cancer cells in women undergoing resection for primary breast cancer in 2007—only in the last case did regulatory approval require a demonstration of clinical utility, which was done in a postmarket context. In launching these tests, McCormack was repeatedly asked three questions by providers. First, has the test been validated? Second, is the test covered? And, third, how will this test change the way I practice medicine? “It was this last question that, of course, stumped me the most, because I did not have the clinical utility … they were looking for.”
It was not until this most recent test that the clinical utility question drove a change in the development process, said McCormack. Veridex partnered with the Southwest Oncology Group to study the clinical utility of basing therapy on counts of circulating tumor cells 3 weeks after the start of therapy. The study opened in the fourth quarter of 2006, closed to enrollment in March 2012, and still has 2 years before outcomes can be determined. Nonetheless, noted McCormack, “it shouldn’t take 6 years to [demonstrate the utility of a test]. We need to be creative to deliver the information that people want.”
Tunis emphasized the inherent tension between the level of certainty about risks and benefits and innovation and early access to new technologies. The optimal balance to maximize long-term public health is not easy to determine and varies by stakeholder interest and perspective. Thus, said Tunis, “there is a critical need for stakeholders to come together and develop some common, shared understanding of what constitutes adequate evidence of clinical utility and a process for doing that.”
Clear, consistent, and predictable evidentiary expectations are essential, said Tunis. “Until we translate that dialogue into specific methodological recommendations for designing studies of clinical utility, we can’t really move the field forward in a meaningful and predictable way.”
These methodological recommendations are as much a product of a social consensus as a scientific consensus, Tunis remarked. They reflect a collective social judgment about what is optimal, and they cannot be determined by specialists in a single field working in isolation. In balancing public policy objectives that compete with each other, the stakeholders need to arrive at a consensus that can be translated into scientific and technical terms.