Diffusion and Use of Genomic Innovations in Health and Medicine: Workshop Summary (2008)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

4 Translation of Genomic Technology at the Clinical Level A Primary-Care Provider View of Translating Genomic Innovation Alfred O. Berg, M.D., M.P.H. University of Washington Primary care is of growing importance in the translation of genomic innovations, and genomic innovations will achieve a bigger foothold in this country only if they penetrate into primary care, Berg said. Most of the coming innovations in risk profiling for chronic diseases and many of the pharmacogenomic applications will be very important components of primary care. Primary care is generally thought of as family medicine, general internal medicine, and general pediatrics, but obstetrics and gynecology can also be included. Collectively, these medical specialties account for more than half of all visits to physicians in the United States, who were estimated to number more than 500,000 in 2006. Primary-care physicians serve as the personal medical home for most patients. They are the first medical contact for most patients and are turned to by most patients when a new complaint arises. Primary-care physicians attempt to be comprehensive. Patients can bring any complaint, interest, or concern to them, and the primary-care physician should be able to assist them, either directly by providing services, or indirectly, by making appro- priate referrals. 47 

48 DIFFUSION AND USE OF GENOMIC INNOVATIONS An important component of primary care is that it is continuous, allowing the physician and the patient to develop a relationship over time. This makes it possible for primary-care physicians to accumulate information about per- sonal family history that would not be available to other specialists. Primary-care physicians try to be community- and population-focused. A doctor can practice high-quality primary care only if he or she knows the community, knows what is prevalent in the community, knows the risks in the community, and knows what the community’s health concerns are. An important characteristic of primary-care practice is that the physi- cians see common problems. They specialize in breadth of knowledge and expertise. At the same time, they need to recognize patterns that suggest the unusual. In order to practice in this way, primary-care physicians need information systems and decision support. Because they have a very high volume of practice, their support systems must work on time and all the time. Primary-care physicians cannot wait until the evening or the next day to come up with answers. In primary care, medical tests and interventions must be appropriate for populations in which rare conditions are actually rare. Tests with even small errors can have magnified effects. A test that has a 99.9 percent specificity can still be a catastrophe in primary care if the condition is rare because positive tests will often be false positives, requiring a further cascade of medical testing and intervention. Rare conditions are rare in primary care, as they are in populations. For primary-care physicians to incorporate a new test or innovation, several conditions must be met. First, a new test or innovation must be available, feasible, and acceptable to the patient. It has to do what it says it does. It has to be accurate and reproducible. It has to improve clinical outcomes that patients would notice and care about when compared to current practice. For example, changes in laboratory values are usually not enough in primary care because the patient expects to actually experience improvement with a new test or innovation. A new innovation should not increase adverse effects. Finally, it should be “worth it”—that is, the patient should think it is worth it either with insurance or with out-of-pocket pay- ments. The calculation that goes into determining worth is more complex and nuanced than what typically goes into a cost-effectiveness analysis. Primary-care clinicians need authoritative advice. No one can keep up with the staggering volume of medical information or make sense of all the volumes of literature. Authoritative advice can help clinicians deal with complex decisions by identifying the key factors important to decision making. Furthermore, authoritative advice has the potential to improve the quality of physician decision making. Such advice can provide justification to patients, payers, and the legal system by presenting the criteria used to make decisions. 

TRANSLATION OF GENOMIC TECHNOLOGY AT THE CLINICAL LEVEL 49 Clinical guidelines are one useful form of authoritative advice. They help transmit medical knowledge, and they assist in making decisions. Clinical guidelines are a way to set clinical norms that can be used in ­quality improvement, in privileges and credentialing, and in payment and cost con- trol and that can be useful in medico-legal evaluations as well. Evidence-based guidelines have three major hallmarks. First, they need to be explicit, meaning that they state exactly how to proceed. They need to be transparent, in that all of the information has to be available to users and to consumers. Finally evidence-based guidelines must be publicly accountable. A report of the Institute of Medicine concluded that, in order to be useful, a clinical guideline must specify the clinical condition, the health practice, the target population, the health care setting, the type of clinician, and the purpose of the guideline (IOM, 1992). Therefore, it is not enough to ask, “Is test X a good test?” One must ask, “Is test X a good test in this particular clinical scenario?” The Agency for Healthcare Research and Quality (AHRQ) has further specified process characteristics for developing a clinical guideline. One of the characteristics addresses panel selection. Selection could be based on expertise, which might include individuals with conflict of interest, or selec- tion could identify individuals with a general perspective on evidence so as to avoid conflicts of interest. A second characteristic is that the problem for which the guideline is being developed needs to be specified, as does the literature search strategy. There must also be explicit statements about how the literature is analyzed, the criteria used to judge the quality of the literature, how the evidence is summarized, and how one moves from the evidence to the rationale. Every guideline process has some level of subjectivity. There is no gener- ally agreed-upon algorithm for moving from evidence to recommendation. A linkage between evidence and the recommendation must always be made, however. Furthermore, the decision-making process needs to be specified as explicitly as possible. Another process characteristic is that the guideline needs to focus on clinical outcomes—that is, not simply on intermediate outcomes such as laboratory tests or knowledge, but on actual clinical out- comes that patients or families would notice and care about. Also, clinical guidelines must be sensitive to cost and practicality. AHRQ has developed a list of desirable attributes for guidelines. Guide- lines should be valid, that is, they should be based on criteria that are public and accountable so that validity can be assessed. Guidelines need to be reli- able. This means when guidelines are used in similar circumstances, there should be a similar outcome each time. Guidelines must also be applicable, flexible, clear, multidisciplinary, and documented. In the area of genetics and genomics, Berg described primary-care physicians as being skeptical of genetic exceptionalism, that is, of the 

50 DIFFUSION AND USE OF GENOMIC INNOVATIONS claims that genetic information is qualitatively different from other types of information. And, indeed, many non-genomic tests in current use pro- duce exactly the same kind of information that is promised for genetic tests—they provide information about risk, prognosis, and response to drugs and other therapies. Furthermore, they too have ethical, legal, and social consequences. The one area where genomic tests are unquestionably unique is in their ability to provide information about family members. In almost every other regard, there are many tests used every day by primary-care physicians that provide the same kinds of information that is promised by genetic and genomic testing. There are thousands of genomic tests available, but there is little regula- tion of those tests. There is also direct-to-consumer and direct-to-­physician marketing. The result is that clinicians and consumers are confused and need reliable advice. There are precedents in providing reliable advice, such as the United States Preventive Services Task Force, which evaluates preventive interventions. The Centers for Disease Control and Prevention (CDC) in partnership with AHRQ is sponsoring an initiative called Evalu- ation of Genomic Applications in Practice and Prevention (EGAPP). The EGAPP working group has no regulatory authority and is an independent, non-federal, multidisciplinary panel. Those selected to serve on the panel do not have extensive financial or other ties to various stakeholder groups that would have a stake in the recommendations of the group; they were selected in a manner aimed at minimizing conflicts of interest. As a result, there are a number of generalists on the panel who have a fairly broad view of such things as laboratory testing, primary care, and ethical, legal, and social issues. The first EGAPP guideline, which was released in December, relates to the use of CYP-450 testing in decision making about the kind and dose of selective serotonin reuptake inhibitor (SSRI) that should be used in patients newly diagnosed with major depression. As noted earlier, it is very important to be explicit and clear about the clinical context in which the guideline is going to be used. This guideline is very specific. It is addressed to a primary-care clinician seeing a patient with a new diagnosis of major depression for whom an SSRI is being considered as treatment, and it may not be at all useful in a different clinical scenario. There are a number of other EGAPP reviews underway concerning such things as testing for early detection of ovarian cancer, testing before placing a patient on an antidepressant drug, testing for family-related colon cancer, testing for response to treatment for colon cancer, genetic profiling for cardiac risk, and breast cancer gene-expression profiling. The quantity and quality of evidence that supports testing in typical practice settings has been disappointing. Research designs in the published 

TRANSLATION OF GENOMIC TECHNOLOGY AT THE CLINICAL LEVEL 51 literature are weak. Some potentially important data are proprietary and cannot be examined. Furthermore, there is very little evidence on potential benefits and harms and no head-to-head comparisons with current practice. Comparison with current practice is one of the things that primary-care physicians are looking for. As mentioned earlier, for example, there are 50 years of practice in using warfarin. Could a test actually improve on the clinical outcomes? The tests have not typically been evaluated in real- patient populations but rather only in research centers. There is little infor- mation about cost and cost-effectiveness compared with current practice and essentially no information about the ethical, legal, and social implica- tions, particularly for family members. EGAPP will be publishing a paper that documents the outcomes it considers to be of importance, one of which is family outcomes. Addition- ally, EGAPP has specified a number of things that it believes are potentially important when genomic technologies are evaluated. Genomic innovations that can be used to assess risk or to guide therapy hold great promise, and primary-care physicians are very interested in them. A major issue, however, will be recognizing the importance of appropri- ateness in primary-care settings; many of the most exciting tests that are being discussed do have important implications for whole populations that are typical of primary care. Additionally, new testing technologies must improve on what is in current use. In the next three to five years there are likely to be few examples of genetic tests that will meet standards for com- mon use in the typical primary-care practice. There is an enormous need for more and better-quality research on the effects of testing on clinical outcomes, both good and bad, with publicly available results. Having high-quality information about actual outcomes of testing is critically important, Berg concluded. Introducing a Genomic Innovation to Clinical Practice Steven Shak, M.D. Genomic Health There has been a great deal of discussion about recent genomic inno- vations and the question of whether there is adequate evidence to validate their clinical utility. One challenge for the companies engaged in developing such innovations is to actively collaborate and to fund the studies to obtain that evidence. Because patients urgently need genomics to be translated into 

52 DIFFUSION AND USE OF GENOMIC INNOVATIONS clinical practice, it is important for various stakeholders to work together to conduct the right studies that identify potential breakthroughs that work as well as those that do not work. When Genomic Health was started in 2000, Shak said, little had been done to bring biomarkers into oncology practice. Despite thousands of papers on biomarkers, there were few with any actual diagnostic tests for use in oncology. Genomic Health developed Oncotype DX, the first diagnostic, multi-gene expression test for breast cancer treatment planning which has been commercially available since 2004. There is clinical evi- dence from multiple independent studies demonstrating the test’s ability to predict the likelihood of breast cancer recurrence as well as the magnitude of chemotherapy benefit. Thus the test is useful for individual patients in judging their own likely benefits from chemotherapy and how much those benefits will likely exceed the risks of treatment. Use of Oncotype DX has been growing over the years and more than 39,000 tests have been performed for more than 6,000 physician orders. Furthermore, the test is now reimbursed by Medicare and other major p ­ ayers. And recently an American Society of Clinical Oncology (ASCO) clinical practice guideline recommended the use of Oncotype DX for node- negative, estrogen-receptor-positive breast cancer. That is the group of women for which the test was developed, and it accounts for about half the women diagnosed with breast cancer in the United States. There were a number of challenges to realizing the promise of O ­ ncotype DX. Bringing this test to clinical practice required that multiple, independent clinical studies be conducted that were rigorous in terms of design, performance, and analysis as well as comparing the test to stan- dard measures. Assay precision, standardization, and control, well-known principles in the field of laboratory medicine, were of extreme importance. It was vital to show clinical utility in order to show that Oncotype DX would meet the needs of patients, physicians, payers, regulators, and even investors. Finally, even after meeting those challenges, there is still a need for continuing research. Over the years there had been great innovation in the field of cancer. In the past century cancer treatment was largely one-size-fits-all. Tumors were diagnosed based on their site of origin. Clinicians knew that there were marked individual differences in breast cancers, lung cancers, and other tumors, but they did not have the tools or the insight to analyze those dif- ferences and make practical use of them. During the 1990s, however, new technologies such as gene expression assays were developed that now allow careful measurement and quantification of individual genes in a tumor in order to better understand individual differences. Genomic Health chose to optimize a new technology for quantitative analysis of gene expression for use with tumor blocks. Every time a cancer 

TRANSLATION OF GENOMIC TECHNOLOGY AT THE CLINICAL LEVEL 53 is diagnosed and a tissue sample is taken by either biopsy or surgery, that sample is sent to the pathology laboratory; there the tissue is “pickled” in a fixative and placed into wax or paraffin, where it can be stored indefinitely. It is this fixed paraffin embedded tissue, the tumor block, that is sliced, sectioned, stained, and examined by the pathologist in order to make the diagnosis of cancer. In the past, this material was not considered useful for conducting molecular studies, but scientists at Genomic Health undertook an effort to look at RNA in tumor blocks in a precise and quantitative way and, after two years of effort, developed an assay system that enabled them to do so. This was important for two reasons. The first is that it allowed the test to be practically useful to examine tumors as routinely processed by hospitals and pathology laboratories at the time of diagnosis. The second, and more important, reason was that it enabled Genomic Health to analyze data on women who had been diagnosed with breast cancer in the past and whose outcomes were known, which, in turn, lead to the development and validations of a multi-gene test for breast cancer treatment planning. Genomic Health’s technology uses real-time quantitative reverse transcrip­tion polymerase chain reaction (RT-PCR) to quantify RNA. It is reliable, sensitive and specific, it has a wide dynamic range, and it is highly reproducible. A series of studies was performed to optimize the assay sys- tem so that it would work with the tumor blocks and define and minimize all sources of assay variability. With this innovation, Genomic Health developed a strategy and a plan for developing a breast cancer genomic test that was focused, from the very beginning, on the challenges facing physicians and patients. Physicians and patient advocates said that what was needed for node-negative, estrogen- receptor patients at the time of diagnosis was an increased ability to pick out the truly low-risk patients and, most importantly, to determine who would benefit from the cytotoxic chemotherapy. The critical need for more individual information can be appreciated by reviewing the clinical trial results which examined the benefit of chemo­ therapy. The landmark trial that changed the care of breast cancer was performed by the National Surgical Adjuvant Breast and Bowel Project (NSABP) from 1988 to 1997. This was a controlled trial that randomized patients to either Tamoxifen alone or to Tamoxifen plus chemotherapy. The results of this study demonstrated the benefit of chemotherapy (see Figure 4-1), and, based on these results, chemotherapy became the recom- mended treatment for the vast majority of patients. The study did find, however, that more than 85 percent of women will survive without recurrence with just Tamoxifen and no chemotherapy. By definition, Shak said, the vast majority of women have been overtreated because it was not known which women were going to suffer recurrence or who would benefit by adding chemotherapy. 

54 DIFFUSION AND USE OF GENOMIC INNOVATIONS 1.0 10 yr 0.9 92% Proportion Recurrence-Free 0.8 88% 0.7 0.6 0.5 0.4 0.3 0.2 All Patients N Events 0.1 Tam + Chemo 424 33 Tam 227 31 p = 0.02 0.0 0 2 4 6 8 10 12 Years FIGURE 4-1  NSABP B-20 clinical trial (1988-1997). Tamoxifen vs. Tamoxifen + Chemotherapy—All 651 patients. 4-1.eps SOURCE: Shak, 2007. Another illustration of the problem can be seen in a case presented in 2002 to an audience at an ASCO conference. The case was a 40-year-old woman with an invasive ductal carcinoma. She had a node-negative tumor of 1.1 centimeters. The estrogen receptor/progesterone receptor (ER/PR) was positive, and HER-2 was negative. The presenter asked the physicians in the audience what they would use to treat this patient. Fifty-four percent said they would use hormonal therapy, while 45 percent stated they would use hormonal therapy plus chemotherapy. There was no consensus. The challenge, then, was to determine who would benefit by the addi- tion of chemotherapy. It was to meet this challenge that Genomic Health applied the principles of drug development to the process of developing a diagnostic test. In particular, the company applied the principle of doing multiple studies with a logical sequence and rigor at each step, essentially analogous to the phase I, phase II, and phase III drug development trials. The first series of studies was developmental and was designed to exam- ine a set of genes to identify whether genes matter and, if so, which genes. Two hundred fifty candidate genes were analyzed in a total of 447 patients 

TRANSLATION OF GENOMIC TECHNOLOGY AT THE CLINICAL LEVEL 55 from three separate studies, which eventually led to a 21-gene profile and an algorithm for calculating a Recurrence Score (Cobleigh et al., 2003; Esteban et al., 2003; Paik et al., 2003). After defining a specific assay (in this case an assay of 21 genes), Genomic Health conducted two clinical-validation studies to test that par- ticular assay independently in a prospective way on archival tissue from well-defined cohorts. The first study was performed in collaboration with NSABP to examine the Recurrence Score in the landmark NSABP B-14 clinical trial. The second study was performed in collaboration with the Division of Research at Kaiser Permanente to examine the Recurrence Score in a large, community-based observational study. Finally, treatment-benefit studies of Oncotype DX were undertaken. A study of the NSABP B-20 patients was made to determine the magnitude of the chemotherapy benefit as a function of the 21-gene Recurrence Score assay, with a Recurrence Score provided for each individual tumor. Patients who were randomized in NSABP B-20 to tamoxifen or to tamoxifen plus either CMF or MF chemotherapy were eligible. The primary analysis was prespecified to examine the tamoxifen-treated patients compared with those patients treated with tamoxifen plus either CMF or MF chemotherapy. The hypothesis was that patients with a Recurrence Score of less than 18 would be at lower risk. In fact, the study by Paik and colleagues indi- cated that the risk of recurrence in patients with a Recurrence Score less than 18 was very low, and there was little evidence that chemotherapy was of benefit (Paik et al., 2004). For women with a Recurrence Score that was intermediate—that is, a score of 18 to 30—there was an increased risk of recurrence, although the benefit of chemotherapy was uncertain for this group. For women with a Recurrence Score of greater than or equal to 31 there was a clear, large benefit from chemotherapy. These individual differ- ences make a compelling argument for the use of the Recurrence Score to individualize treatment. The regulations and principles of the Clinical Laboratory Improvement   The first study, the National Surgical Adjuvant Breast and Bowel Project (NSABP) B‑20 study, was a multicenter study in which tissue was analyzed from 233 patients in a h ­ omogeneous patient cohort characterized by having histologically negative nodes, estrogen- receptor-positive tumors, and treatment with tamoxifen and no other systemic therapy. The second study was a single-site study at Rush Presbyterian-St. Luke’s Hospital in which ­tissue was analyzed from 78 patients, all characterized by having more than 10 positive nodes and treatment predominantly with chemotherapy or Tamoxifen, or both. The final study, at Providence St. Joseph’s Hospital, was a single-site study in which tissue was analyzed from 136 patients who were either node positive or negative, ER positive or negative, and treated with tamoxifen or chemotherapy.   “The Recurrence Score is a number between 0 and 100 that corresponds to a specific likelihood of breast cancer recurrence within 10 years of your initial diagnosis” (http://www. genomichealth.com/oncotype/about/patresults.aspx, accessed January 15, 2008). 

56 DIFFUSION AND USE OF GENOMIC INNOVATIONS Amendments (CLIA) require that all assay methods and procedures be defined prior to clinical validation studies. These methods and procedures include specimen eligibility, reagent qualification, instrument validation, controls and calibrators, and linearity, precision, and reproducibility. In the case of Genomic Health’s clinical validation study, it took 6 to 9 months to finalize the assay format and show its analytical performance. The Oncotype DX process is complicated. It involves multiple steps and requires more than 150 standard operating procedures and more than 90 forms to run the test, but the attention to detail assures that the Recurrence Score obtained on an individual tumor a year ago is going to be the same as the score today, and next year, and the year after that. Traditionally, women have been described as having tumors that are simply ER positive or ER negative. With the new molecular, quantitative assay, however, one can see that it is not the case that there are just two types of breast cancer. There is continuous biology and considerable heterogeneity in estrogen- receptor-positive tumors. Prior to the use of Oncotype DX, if one used conventional measures to define patients for treatment, only 7 percent of all women who were node negative and ER positive would be found to be low risk using National Comprehensive Care Network (NCCN) guidelines; the vast majority would be designated high risk, and chemotherapy would be recommended for them. With Oncotype DX, however, many patients are classified differ- ently. For those classified as low risk by conventional measures, about a third may be undertreated because their Recurrence Score is intermediate or high. Conversely, there are many more women (almost half) who are judged by the conventional measures to be at high risk when, in fact, their tumors have a low Recurrence Score (RS < 18) and they might obtain only a minimal or no benefit from chemotherapy. There are currently four separate studies examining the use of O ­ ncotype DX as a guide to treatment planning in clinical practice. One study by Oratz and colleagues showed that 25 percent of treatment recom- mendations changed with the use of Oncotype DX (Oratz et al., 2007). Several important factors facilitated the introduction of Oncotype DX. First, the suite of clinical utility studies was designed to meet the needs of patients, physicians, payers, and regulators. The acceptance of Oncotype DX by physicians and payers is driven by a number of factors, the most prominent being published evidence. Publications do matter and are a method for community education and understanding. Before the publica- tion of clinical validation studies and the presentation of chemotherapy benefit data, there was very little use of the test, but following publication use increased. The investment made in helping physicians, health care pro- viders, and payers understand clinical research and how to interpret data was key to getting them to accept the test. 

TRANSLATION OF GENOMIC TECHNOLOGY AT THE CLINICAL LEVEL 57 Another important factor was federal funding. Genomic Health could turn to the NSABP to collaborate on these studies. The NSABP had the foresight—and the federal government supplied the funding—to collect tumor blocks 15 years ago and save them, not knowing that anyone could develop an assay in the future that might be used on them. One challenge was technology assessment criteria. The various groups involved in assessment do not use uniform criteria or evidence, even if they are using the same data. Each group assesses a new technology from its own perspective and context, which may or may not be the most useful or relevant. In defining the conceptual framework for evaluation of clinical effectiveness, it is important that the focus be on transparency and putting breast cancer patients first. There is a need for studies that establish clinical utility by showing directly, or by inference, that use of the test will improve outcomes and spare toxicity and health care resources. The studies must also rigorously compare the new test with traditional measures for decision making. In the first quarter of 2006, Medicare provided coverage of Oncotype DX. Reimbursement had a dramatic effect on the use of the test, and, as major payers began to reimburse for the test, use increased. Another factor that mattered in the translation of Oncotype DX was the conduct of treatment-decision studies. Although the validation studies were conducted on archival tissue from women treated in the past, it was important to conduct studies that examined the experience of physicians actually using the test in practice. These studies showed that treatment deci- sions changed. It was also important to conduct health economic studies. There are two threats or challenges for Oncotype DX. Traditionally, the reimbursement system has generously reimbursed for drugs and thera- peutics and has reimbursed diagnostics poorly. This has clear consequences and is a clear threat to continued innovation by the diagnostics industry. The second threat is regulatory uncertainty. If the path is clear, one can plan how to proceed. But it is very difficult to make adequate plans in an uncertain regulatory environment. Continued research is needed. Reimbursement for the test is now at $3,650. A significant portion of that money is going into new studies that will examine additional questions in breast cancer and that will begin to look at other tumors such as colon cancer, prostate cancer, as well as treat- ment selection for targeted drugs. The road to realizing the promise of genomics is difficult. It takes innova- tion. It takes multiple, well-designed clinical studies. One must pay incredible attention to assay precision and standardization. One must focus on clinical utility and reimbursement from the beginning all the way through to the end of the translational path. Finally, one must understand the importance of a team working together. None of this would have been possible without an 

58 DIFFUSION AND USE OF GENOMIC INNOVATIONS incredible group of people in industry, academia, the regulatory environment, the National Cancer Institute and the Cooperative Groups, the breast cancer advocates, and patients and their families. The women who participated in the landmark clinical trial 20 years ago, donating their tissue at that time, would likely feel very good that their participation in that trial is now helping women today, perhaps their daughters or granddaughters, to obtain more informed breast cancer treat- ment, Shak concluded. Discussion Wylie Burke, M.D., Ph.D. Moderator One questioner asked Dr. Shak to comment on infrastructure develop- ment within National Institutes of Health (NIH)-funded research, given that Genomic Health was able to take advantage of data that had been collected from randomized controlled trials (RCTs) and that it appears that such opportunities will be limited in the future. Shak responded that fund- ing to collect and save such data is a critical issue. Cooperative groups are still conducting multiple trials. Providing funding to collect data and save tumor blocks would enable others to use those data in the future in order to learn about optimizing treatment. Observational studies might be used to answer some questions, Shak said, but one must be very cautious in using such studies to look at treatment questions because often the treatments instituted are not done without bias. One participant stated that RCTs have been identified as the gold stan- dard for research but are rarely used for the study of genomic innovations. Given this, should there be major effort undertaken to determine how to make the best use of or optimize observational data? Is the knowledge for how to do that already available, or is generating that knowledge a task that also must be done? Berg said that the evidence on the Oncotype DX appears to still be in the indirect category. There are trials underway to answer the question directly about whether use of a test actually changes not only the clinician’s recommendation for treatment but also the patient’s choice and her ulti- mate clinical outcome. What kind of resources would it take to actually answer that question? Will a lot of very good-quality, indirect evidence be enough or, for such an important disease with such an important outcome as breast cancer, should one insist on a properly conducted randomized 

TRANSLATION OF GENOMIC TECHNOLOGY AT THE CLINICAL LEVEL 59 trial or ­trials in order to find out whether the test has the clinical outcomes that are promised? Shak responded that the current evidence is clearly indirect. The most direct and logical approach would have been to do an RCT, but there was no support in the community for that approach, nor were the resources available to conduct such a trial. The TAILORx trial that is being con- ducted assumes that the Oncotype DX test does what it says. The patients who have a low score are all given hormonal therapy. Those with a high score are all given chemotherapy. The critical question that is being asked is, for the group of women for whom there is uncertainty about the ben- efit of chemotherapy (those in the mid range), what is the actual effect of chemotherapy? When this study is completed it will be possible to use the new tech- nologies developed between now and the end of that study to examine the tumor blocks and, it is hoped, have a definitive test. One can return to these cohorts and conduct multiple studies in a rigorous manner in order to build confidence in the test. An audience member asked where biobanking might fit in the discus- sion of evidence and data. Some countries have national biobank systems, and there are smaller biobank systems in the United States. What, she asked, do the presenters think about biobanking, either as a national effort or as a more comprehensive local effort, perhaps a Framingham-for- b ­ iobanks approach? Burke answered that in biobanking large amounts of genomic infor- mation and clinical information are combined. There is a great deal of e ­ mpirical research, currently in the early stages, that is looking at partici- pant attitudes, researchers’ concerns, and IRB personnel concerns around issues of biobanking. The issues are complex, with much concern about harm and participant safety. The example of using stored tumor blocks to ask a question about a therapeutic intervention that was not anticipated illustrates the value of biobanking. For that particular example, the concerns were lessened because the later research addressed precisely the issue that the original samples were collected to address—that is, improved outcomes for breast cancer patients. The concern with biobanks is that samples collected for one purpose might be used for other purposes. This is likely an area around which continued discussion will occur. One questioner asked if prospective trials will be used to examine whether Oncotype DX is having an effect on the clinical outcomes of women with breast cancer, rather than just on the change in the decision about whether to add chemotherapy to the treatment. Shak said that the National Cancer Institute is talking about method- ology for conducting prospective trials on archival tissue. This is a well- 

60 DIFFUSION AND USE OF GENOMIC INNOVATIONS accepted approach in many circles, especially in situations where multiple studies have been conducted and the results have been the same. What is important is rigor. There are good prospective studies and bad ones. In terms of conducting studies on archival tissue, there are good studies and, probably, bad ones. The goal is to become well educated about being able to assess genomic tests, their advantages, and their disadvantages. Another comment was that Oncotype DX is a diagnostic, developed as a business model to be performed as a laboratory test and, as such, is not subject to the same regulatory environment that in vitro diagnostic (IVD) test kits have to deal with. Given this, what will be the effect of the new IVD multivariate index assays (MIA) draft guidance from the FDA? How will that affect Genomic Health’s business model? And how will all the different pathways of bringing diagnostic tests to market compare with one another? Shak responded that there will be a number of gene-expression tests that will be done by different groups in different ways, some by reference laboratories, some by kits. That is where transparency and rigorous assess- ment of tests will be important. One audience member thanked Genomic Health for pioneering a great approach based on archival tissues and for a product development that set a high bar for the rest of the industry to follow. He then asked Shak to describe the magnitude of the effort involved, including the money invested in the development of Oncotype DX before it was launched and the amount of ongoing investment in clinical research. He also asked Shak to comment on whether such an investment was going to be typical of this type of diag- nostic development. Shak responded that the company probably spent somewhere between $50 million and $100 million over about 7 years to arrive at its current situation. Genomic Health is now working on colon cancer and has been able to take advantage of some of the lessons learned with Oncotype DX, so some things are less costly. One of the great things in the field of genomics, Shak stated, is that creative people at the bench are developing better ways of solving problems, so some costs should decrease. It is important to emphasize that much of the cost relates to the ­quality control required to obtain the same result again and again. All of the reagents used are quality controlled. They need to meet particular specifica- tions, as do all the machines. The laboratory personnel are all highly trained and licensed. It is an investment that is often underestimated. An advantage of using archival studies is that the costs of enrollment and long-term follow up of patients in clinical trials are avoided. Incurring such costs would have made it prohibitive for Genomic Health to undertake the studies. In a way, then, Genomic Health has been able to leverage the investment of the community’s resources for the benefit of patients. 

TRANSLATION OF GENOMIC TECHNOLOGY AT THE CLINICAL LEVEL 61 It will still be a challenge to proceed in other areas, such as colon c ­ ancer, particularly if coverage by payers becomes a major issue. One audience member noted that Berg had talked about the importance of clinical guidelines. It is important to think about how to increase the number of clinical guidelines and, more important, how to accelerate the adoption of those guidelines. The data on adoption indicate that it is slow and painful. It is probable that EGAPP will help accelerate the generation of professional guidelines on genetics, but how can adoption of those guide- lines by primary-care and specialty providers be accelerated? Berg responded that one must be careful which guidelines are imple- mented and how they are implemented. Keeping that in mind, however, the way to accelerate adoption of innovation is to tie it to reimbursement. When there is system support (e.g., through development of guidelines) and reimbursement, adoption is often rapid. A member of the audience pointed out that just because something is tied to reimbursement does not mean it is a good thing. Burke responded that it is important to properly align reimbursement incentives. One questioner noted that Shak described paying close attention to what the providers and patients viewed as clinical utility, which in this situation was avoiding the unnecessary use of chemotherapy. From that perspective, Berg was asked to comment on where he believes the gains in genomics might be. For example, will gains be in the area of pharmaco­genetics, or are they more likely to be in some category like disease ­classification? What are the pressure points in primary-care practice where genomics might play a role? Berg responded by pointing out that about 500 women need to be screened to prevent 1 breast-cancer-related death in 5 years in a 50-year- old. That means that for a woman walking into a primary-care practice who says, “I’m 60 and I think I need a mammogram,” the response would be, “Great. You can have your mammogram.” In the example described above of the early trials with adjuvant chemotherapy, the difference between the two groups—those who had the chemotherapy and those who didn’t—was 4 percent. One woman out of 25 benefited from the chemotherapy. If a woman is willing to take a 1-in-500 chance of benefit with a mammogram, is there any feasible sce- nario where she is going to turn down a 1-in-25 chance of benefiting from chemo­therapy with or without the test? This is why it is so important to test these things in practice, to find out what recommendations are made to the patients, what decisions are actually made, and whether the test makes a difference. A strong argument can be made for prospective clinical trials with Oncotype DX, even at this point, because it is not yet known how patients and clinicians will actually use the test, whether they will respond to the 

62 DIFFUSION AND USE OF GENOMIC INNOVATIONS risk information in a novel way or whether they will still opt for treatment with chemotherapy. In one of the trials there was a 1-in-50 chance of ben- efiting. Is that still enough for a woman to say she wants the chemotherapy? At this point, no one knows. Burke said that this illustrates the issues of how many of these kinds of questions must be answered pre-market, how many of them can appro- priately be answered post-market, and how the infrastructure necessary for conducting the studies can be created. One audience member noted that a point had been made earlier about educating physicians. From a primary-care perspective, how can one make physicians more receptive to new innovations? Berg responded that the American Academy of Family Physicians had an annual clinical focus on genetics and genetic testing. It was disappointing in that clinicians said that the kinds of educational offerings were not things that their patients were asking for and they were not things that seemed to be common enough in their practice for them to make the investment in learning. Although acknowledging that he is not an expert in the area, Berg said that he believed that primary-care physicians need to see how the innovation is going to directly benefit their patients and make their practice life better. If one can make those connections, the educational program is much more likely to work. One audience member commented that the compelling question for payers is whether the outcome would be better if Oncotype DX were to be used. There are common tools that are used to make decisions about chemo­ therapy. While there is a substantial benefit from using chemotherapy, there are also risks. What is impressive about the reclassification studies is that one could compare the incremental benefit due to use of Oncotype DX with the benefits from a conventional tool and link that to an outcome known from a randomized controlled trial, albeit one that had been conducted in the past. Although this study approach does not fit easily into the clear conceptual boxes of indirect and direct evidence, the evidence appears more direct, the speaker said. Women in the archival data were women who were on tamoxifen, which is the less prevalent treatment today. Today the more common treat- ment is aromatase inhibitors. To make a further inference to aromatase inhibitors would be an example of indirect evidence, where one needs to link bodies of literature and make assumptions. Berg said that ultimately there will always be some subjectivity in m ­ oving from evidence to a recommendation. One could probably determine a way of objectively characterizing the evidence, as Shak and Aronson have done. But the question is, for a given clinical situation, for whom is the evidence enough? Is it enough for the patient? Is it enough for the clinician? Is it enough for the payer? The EGAPP project is trying to answer those 

TRANSLATION OF GENOMIC TECHNOLOGY AT THE CLINICAL LEVEL 63 questions for primary-care clinicians. The answer that EGAPP arrives at might be quite different than the answer that a group of oncologists would settle on. The context—that is, the clinical setting—matters enormously in making that subjective assessment about evidence. Even though the evidence for Oncotype DX can be characterized as very good, high-quality, almost direct evidence, it is not quite there yet, Berg said. Does it matter? The question is, for whom might it be enough? Maybe it is enough for current investigators. Maybe it is enough for ­payers. Is it enough for all clinicians or for all patients? That question cannot be answered. Aronson responded that the issue is quite complex. But what is per- suasive is that for women making a decision about whether or not to have chemotherapy, Oncotype DX is a better predictive tool. Another audience member drew attention to the fact that it took between $50 million and $100 million to get the test out, which did not include the cost of the original study from which the blocks were obtained. If that cost were included, the figure would be much higher. And this is just the first generation of the test. Will a prospective randomized trial be done for every generation, and, if not, how will the information be obtained? If it has to be done post-marketing, then there is a problem because the payers say, “There must be a decent RCT before this will be paid for.” The data cannot be obtained, however, unless the patients get the test, and the test will not be given unless it is paid for. This is a quandary. Berg responded that this is a common problem. In the case of treat- ments for post-traumatic stress disorder (PTSD), for example, out of the 13 treatments available there was adequate evidence to support only 1 of them. And this is a condition that affects millions of people, not just veterans. PTSD has been known for 30 years. There has been enormous public invest- ment, but the studies are difficult and costly. The question becomes, how much is society willing to invest in answering questions about treatment of PTSD? And how does answering those questions compete with answering other important questions such as have been discussed today? How much money would it take to actually answer the question about Oncotype DX and improved outcomes if we wanted to do an RCT at public expense? What would the trade-offs be with respect to other priorities for research? These are much larger questions than can be answered in this workshop. They are extremely important, Berg said, but it should not be assumed that the area of genomics will be treated any differently than any other area. There are many areas in clinical practice that desperately need better quality research. Shak commented that one thing that might help is if one moves on to other innovations when the early data on a particular innovation do not look compelling. There are two things needed for success: resources must 

64 DIFFUSION AND USE OF GENOMIC INNOVATIONS be focused on solving compelling problems, and, if the solutions developed are not important, if they do not make a large difference, then maybe the effort to continue is not justified. Another participant expressed concern about the discussion of e ­ vidence-based medicine (EBM) because most of what is done in medi- cine today is not completely evidence-based, and it most certainly is not supported at the level of RCTs. Physicians incorporate a great deal of retrospective data when making their judgments. Is it reasonable to say that in the future everything must be prospective, randomized trial data? Presumably the answer is no. When discussing EBM, one must be careful not to equate EBM with prospective, multi-center RCTs as the only way that physicians should make decisions. Not everything will be tested. Whether a drug works bet- ter in a 20-year-old versus a 21-year-old is not something worth testing. Judgment must be used. It is not reasonable for insurers to require multi-center randomized controlled prospective trials if they are not paying for them, the speaker continued. Who will pay for these trials? If a high level of evidence is required, someone must pay to obtain that evidence. Physicians use evi- dence to improve their practice, but if it is incorrectly used or unreasonably required, evidence can be a bar to innovation. Berg agreed but said that in the absence of evidence, no one is suggest- ing that nothing should happen. Clinicians and patients are accustomed to dealing with uncertainty; that is what much of the practice of medicine is about. Nonetheless, it is extremely important that someone draw a scientific line in the sand and say what it is we know and what we do not know, confronting and looking at the evidence objectively in order to determine whether it is worth the investment to fix. There may be many questions that are not worth investing in, but there are many questions where we are currently saying, “Well, is it okay to substitute our judgment?” when in reality an investment should be made. The investment is long overdue, because there are many, many important clinical questions that could be answered but that are not being answered. Finding the answers is a public good that all should support, Berg said.