4
Generating Evidence About Effectiveness and Value

Dr. Steven Cohen of the Agency for Healthcare Research and Quality (AHRQ) introduced a second panel of experts to discuss evidence for effectiveness and value.

THE FDA AND EVIDENCE FOR REGULATORY APPROVAL IN CANCER

The Food and Drug Administration (FDA) operates at the introduction of cancer care drugs and biological therapeutics into the market, Dr. Janet Woodcock of the FDA’s Center for Drug Evaluation and Research said. What role does the FDA review process play for value in cancer care?

Before the FDA efficacy standard was put into place in the 1960s, the requirement that a drug show benefit to patients before reaching the market was very controversial. Since that time, the FDA has required that treatments reaching the market demonstrate evidence of effectiveness—defined as benefit to the patient, not to doctors or to society at large. This requirement that drugs and biologics in the market demonstrate clinical benefit and safety, has been one of the most significant drivers of treatment evidence in medicine.

To determine clinical benefit, the FDA has considered the endpoints of survival extension, improvement in function, and quality of life as a result of a drug, but the FDA has never considered cost-effectiveness. If a drug



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 23
4 Generating Evidence About Effectiveness and Value Dr. Steven Cohen of the Agency for Healthcare Research and Qual- ity (AHRQ) introduced a second panel of experts to discuss evidence for effectiveness and value. THE FDA AND EVIDENCE FOR REGULATORy APPROVAL IN CANCER The Food and Drug Administration (FDA) operates at the introduc- tion of cancer care drugs and biological therapeutics into the market, Dr. Janet Woodcock of the FDA’s Center for Drug Evaluation and Research said. What role does the FDA review process play for value in cancer care? Before the FDA efficacy standard was put into place in the 1960s, the requirement that a drug show benefit to patients before reaching the market was very controversial. Since that time, the FDA has required that treat- ments reaching the market demonstrate evidence of effectiveness—defined as benefit to the patient, not to doctors or to society at large. This require- ment that drugs and biologics in the market demonstrate clinical benefit and safety, has been one of the most significant drivers of treatment evidence in medicine. To determine clinical benefit, the FDA has considered the endpoints of survival extension, improvement in function, and quality of life as a result of a drug, but the FDA has never considered cost-effectiveness. If a drug 

OCR for page 23
 ASSESSINg ANd ImPROVINg VAluE IN CANCER CARE provides longer life, better function, or better quality of life, the FDA would approve it regardless of its cost. In oncology presently, survival and improvement in patient-reported symptoms are considered unequivocally while weighing a drug’s clinical benefits. Other measures may be included as well, such as objective response rate and time to progression. While the FDA is very supportive of using health-related quality of life as an endpoint, generating the evidence has proved difficult, and few drugs have reached the market based only on this measure. Blinding is difficult in trials to determine health-related quality of life; very careful serial assessments are essential, and the clinical significance to patients of changes in quality of life may be unclear or of little utility compared to careful recording of toxicity data. In combination with objec- tive antitumor effects, the quality-of-life outcomes are more credible. Nev- ertheless, it is hoped that advances will be made in methods for generating accurate health-related quality-of-life data, said Dr. Woodcock. Many drugs reaching the market in recent decades have been approved after regulations were implemented allowing approval based on surrogate endpoints (accelerated approval). Put in place during the HIV epidemic, these regulations were set up for serious and life-threatening diseases in which a drug appears to provide benefit over existing therapies based on a surrogate endpoint thought to reasonably predict clinical benefit. Acceler- ated approval has been used extensively in cancer drug approvals in recent years. Accelerated approval is subject to legal requirements that the appli- cant complete longer-term postmarketing studies to verify and describe the clinical benefit of their drug. These postmarketing studies should usually be underway at the time of approval. Therefore the value of therapies approved this way is not fully clear at the time of launch and may remain unclear until confirmatory studies are complete. Recently, time to tumor progression, or progression-free survival, has been suggested as a measure of clinical benefit. Often, time to progression involves measuring radiographic or other evidence of progression. Clearly, if tumor progression occurs, it will eventually lead to negative outcomes, but this must also be weighed against the harm a treatment may cause. This leads to more uncertainty than measuring objective survival (Table 4-1). Ultimately, the importance of time to progression depends on the size of the benefit. If the benefit is large and unequivocal, then our uncertainty is low. If the difference is only apparent statistically with equivocal impact on patient well-being, then the benefit of the drug remains quite uncertain, and the evidence is much less persuasive for approval.

OCR for page 23
 gENERATINg EVIdENCE ABOuT EffECTIVENESS ANd VAluE TABLE 4-1 Comparison of Two Measures of Benefit in FDA Approval: Overall Survival and Time to Tumor Progression Overall Survival Time to Tumor Progression Perfectly accurate event Less accurate event Perfectly accurate time Less accurate time Assessed daily or more frequently Assessed every 2–6 months Unquestioned importance Uncertain importance Measure of safety and efficacy Measure of efficacy only Longer time to reach endpoint Shorter time to reach endpoint May be obscured by secondary Not obscured by secondary interventions interventions SOURCE: Woodcock presentation, February 9, 2009. Response rate as a surrogate measure provides even less certainty of clinical benefit. When considering approval based on response rate, impor- tant questions remain. For instance, what was the number of complete responses compared to partial responses? What was the duration of response and anatomical location? Were they associated with symptom improvement or the extent of metastatic disease? These details matter. Dr. Woodcock reiterated that the FDA has felt pressure to adopt many of these surrogate measures and to consider them measures of full clinical benefit. However, part of achieving value is reducing the uncertainty around treatments that reach the market—people want to know exactly what these drugs can do. The Shape of Cancer Therapeutics to Come Fewer than 5 percent of cancer therapeutics entering phase I trials reach the market, the worst track record of any therapeutic area. While pharma- ceutical discovery and candidate selection in cancer is driven by recent sci- entific discoveries, much more of clinical oncology treatment development is empirical—trial and error—compared to other disease therapeutics areas. This approach limits understanding of cancer drug benefits, since there are no means of assessing drugs’ pharmacodynamic effect. The good news, said Dr. Woodcock, is that cancer is probably the most active area in drug development, with many new cancer drugs coming to market. Between July 2005 and December 2007, the FDA approved 53 new indications in oncology, with 18 new molecular entity approvals and 35 supplemental applications (new drug applications or biologic license

OCR for page 23
 ASSESSINg ANd ImPROVINg VAluE IN CANCER CARE applications). In recent years, the FDA has only approved 18 new molecular entities annually overall, which means that oncology has been taking the lion’s share. FDA has also seen a huge increase in investigational agents studied in cancer, from 925 investigational new drug applications in 2003 to 1,440 in 2008. The question remains: what is their value? Among the 53 new indications approved by the FDA in cancer thera- peutics between July 2005 and December 2007, 38 clearly showed clinical benefit and proceeded through the approval process using regular approval indications, while 10 used accelerated approval, and 5 previous accelerated approvals were converted to regular approvals upon completion of confir- matory trials with a new indication. With respect to the measures used to support these approvals, 10 indications showed benefits in overall survival, 5 indications were approved based on evidence of disease-free survival, 12 indications were approved based on evidence of time-to-progression or progression-free survival, 17 indications were approved based on response rates, and other, novel endpoints, such as reduction in hepatic iron and depletion of asparagines, were used in the remainder of cases. While novel therapies that provide benefits are desired by everyone, concerns exist over health care expenditures and whether or not each novel therapy has needed value, Dr. Woodcock said. But most oncology drugs do not benefit exposed patients uniformly, and many nonresponders only expe- rience the drug toxicities, making the benefit-to-risk ratio quite unfavorable in some cases. Safety problems also decrease value when patient injury or inconvenience reduce quality and raise costs to the health care system. Cur- rently, there is little ability to predict who is going to respond to treatment and who will be harmed. One solution, said Dr. Woodcock, may be the development of predic- tive biomarkers to improve treatment effectiveness, efficacy, or the size of the treatment effect. Genomic markers for tumor susceptibility, imaging technologies for superior assessment of tumor response, proteomic mark- ers for tumor subcategorization, and the use of circulating tumor cells are all promising avenues for predictive biomarker development. There is reason for some skepticism about the value of these technologies because of their added cost, but if they provide the ability to spare many patients from therapy that will not benefit them, the result will be a tremendous savings. Tailoring treatment in this way is also simply the right thing to do for patients. Predictive biomarkers for safety are already being implemented in oncology. Drug-metabolizing enzyme variants and drug-transporter vari-

OCR for page 23
7 gENERATINg EVIdENCE ABOuT EffECTIVENESS ANd VAluE ants are used to predict the total exposure of the patient or their cellular exposure to an agent. Currently, the maximum tolerated dose in groups of patients in phase I trials are examined, but the underlying mechanistic rea- sons why some groups within the population may not tolerate the therapy are not identified. This may lead to underdosing of many patients because a smaller subset with a distinct variant biomarker is very sensitive to the therapy, and standard dosages are set to those outliers. If these variants can be used to avoid serious side effects of treatments in some patients, their use can improve value. Many challenges for oncology will accompany the large number of candidate drugs arriving soon. The FDA does not have a cost-effectiveness standard for putting these drugs on the market, and the community will have to sort out the ones that are truly valuable. In conclusion, Dr. Woodcock reiterated that drug development leading to FDA approval is an important step in evidence development for cancer drugs. For many cancer drugs, it is the only rigorous evidence development process in place. FDA approval is predicated on showing that a treatment’s effectiveness outweighs its harm, harm which may be significant in cancer therapy. Finally, many new methods of targeting therapy could increase value significantly by increasing the size of treatment effects and decreasing the amount of harm. WHAT CONSTITUTES REASONABLE EVIDENCE OF EFFICACy AND EFFECTIVENESS IN CANCER CARE? In the treatment of life-threatening diseases, there is a pressure to adopt new treatments based on less evidence, and treatment risks may be dis- counted when the alternative is death. This can make meeting the standard of “reasonable” and “logical” treatment more difficult in cancer care. Dr. Daniel Sargent of the Mayo Clinic defined a few key terms to begin to address the question he set out to answer through his presenta- tion, “What constitutes reasonable evidence of efficacy and effectiveness in cancer care?” Evidence is “that which tends to prove or disprove some- thing, grounds for belief or proof ” (Evidence, 2009). Regarding criteria for evidence, Dr. Sargent emphasized the difficulty in achieving absolute certainty in cancer research, saying that a P-value less than 0.05 still allows a 1 in 20 chance that a finding is a false positive. In addition, clinical cancer research does not occur in a laboratory. The experiments that are possible in clinical trials may not provide perfect evidence to answer all of the

OCR for page 23
8 ASSESSINg ANd ImPROVINg VAluE IN CANCER CARE important questions. Finally, evidence is often unavailable, inconclusive, or contradictory. Efficacy is defined as “the capacity for producing a desired result or effect” (Efficacy, 2009). What is the desired result? Is it a full clinical benefit endpoint or a surrogate? Can it be measured precisely and reliably? Is the result that is observed in a particular trial transferable to other settings, such as those in the community? These are all challenges for establishing efficacy, Dr. Sargent said. Currently, the FDA determines whether evidence for cancer treatment efficacy is reasonable on scientific grounds with appropriate input from patients and often guided by the Oncologic Drug Advisory Committee. Once the FDA gives its blessing, oncologists in the community decide whether evidence for a treatment is actually sufficient to change practice based on guidelines, relevant literature, marketing, and other resources that may represent a different standard than that required for drug approval. Dr. Sargent asked, is the current practice for evaluating efficacy reasonable? Yes, he said, because there is input from many parties, it is gathered in an organized manner, and there are clear, well-accepted standards by which therapies must demonstrate efficacy. What about effectiveness? Effectiveness is defined as “how well a treat- ment works in practice,” as opposed to efficacy, which measures how well a treatment works in a controlled trial. In most cases, oncology trials do not evaluate how well therapies actually work in the community, and clear standards to judge effectiveness are lacking. While there is input from many parties, collection and reporting of treatment effectiveness data are disorganized, and effectiveness is often unclear because therapies are used in situations beyond those examined in clinical trials. Randomized controlled trials (RCTs) are considered the gold stan- dard of evidence because randomization, intended to balance known and unknown confounding factors between the study groups, allows for causal inference. However, not everything can be randomized, and there are other forms of analysis, such as propensity scores or other statistical modeling adjustments for differences in the treatments received that may facilitate causal inference in nonrandomized settings. For these approaches we can measure as many covariates as we like, but unfortunately we often do not understand many of the covariates that ultimately determine why patients do well or do not, Dr. Sargeant said. Whenever possible, randomized data should be sought.

OCR for page 23
 gENERATINg EVIdENCE ABOuT EffECTIVENESS ANd VAluE Dr. Sargent then listed the critical components of a randomized trial, which include: • Designation of prespecified hypotheses with primary and secondary endpoints; • Prespecified data cutoffs for any continuous measurement to define what constitutes a positive or negative finding; • A defined sample set with eligibility criteria that are as inclusive as possible; • Power calculations to show that there is a reasonable probability of definitively answering the research questions at issue before the experiment is begun; • Unbiased ascertainment of endpoints, including blinding whenever possible and ethical, protocol-specified criteria, and independent review of endpoints; and • Complete information through a standard follow-up schedule and with few patients lost to follow-up. After new therapies are validated as being efficacious in RCTs using these principles, the same treatments are then refined and further studied by the community where the controlled study environment is lost. Doses and schedules are changed. Combinations are made with other agents, or the interventions are used in an off-label manner for other indications. These refinements are rarely studied rigorously, though comparative effectiveness research, meta-analyses, or large-scale observational studies may provide them some level of validation. Because tools for studying treatments in the community are limited in these ways, Dr. Sargent said, this largely prohibits the generation of level I evidence of treatment effectiveness (Table 4-2). How can this gap be bridged? Two solutions are certainly not new but continue to be underused: (1) cluster randomization and (2) large simple (or pragmatic) trials. When it is impossible to randomize individual patients, one can randomize groups by physician, institution, or geographic area. This is cluster randomization. Large, simple trials use streamlined trial designs with no extra investigations and minimal extra workload. One such trial, the QUASAR (QUick And Simple And Reliable) trial (2007), included roughly 7,000 patients and compared approved regimens of 5-fluorouracil and leucovorin for the treat- ment of stage II and stage III colon cancer. The eligibility criteria included a diagnosis of colon cancer, patient consent, and no obvious contraindica-

OCR for page 23
0 ASSESSINg ANd ImPROVINg VAluE IN CANCER CARE TABLE 4-2 Levels of Evidence and Sources of Evidence to Generate Them Level of Evidence Evidence Source Level I Evidence from at least one properly randomized, controlled trial Level II-1 Evidence from well-designed controlled trials without randomization Level II-2 Evidence from well-designed cohort or case-control analytic studies Level II-3 Evidence from multiple time series with or without the intervention Level III Opinions of authorities, clinical experience; descriptive studies and case reports; reports of expert committees SOURCES: Sargent presentation, February 9, 2009; Harris et al., 2001. tions. During the trial, investigators notified the trial office of only serious, unexpected adverse events. Follow-up was yearly to collect data on serious toxicity, recurrence, and death. The study showed benefit of chemotherapy over observation in patients with stage II colon cancer. Economic analyses, compliance, toxicity, and quality-of-life measures were assessed only in a substudy of 600 patients (QUASAR Collaborative Group, 2007). To put large, simple trials into practice requires the use of multicenter trials, mini- mal patient eligibility criteria, intention-to-treat analyses, and minimization of disincentives to data accrual. Dr. Sargent turned to a discussion of how evidence can be evaluated based on a hierarchy of endpoint strength. When evaluating evidence for treatments, true clinical efficacy measures, such as overall survival, are always the gold standard. In some cases a validated surrogate endpoint1 showing effectiveness can predict the true clinical benefit endpoint measure, but this represents a weaker source of evidence than true clinical efficacy measures. The FDA’s accelerated approval process uses surrogate endpoints 1 “Validated” surrogate endpoints are determined as follows: the effect of a given inter- vention on a validated surrogate endpoint reliably predicts the effect of that intervention on the final clinical endpoint of interest. This validation can be accomplished through statistical methods, meta-analyses of RCTs, and the use of clinical information based on the biological disease pathway or the intervention’s mechanism of action. For instance, recommendations from the ACCENT group established a new surrogate endpoint in the adjuvant setting for colon cancer through data from 18 previous randomized trials (Sargent et al., 2007). Similar analyses are now underway for surrogates in other diseases, including advanced colon cancer and advanced breast cancer.

OCR for page 23
 gENERATINg EVIdENCE ABOuT EffECTIVENESS ANd VAluE that are less validated or “reasonably likely” to predict clinical benefit. The lowest-strength evidence is provided by correlate endpoints that solely mea- sure biological activity of the intervention. In the future, biomarkers will likely define patient populations based on both risk and potential benefit, and biomarkers will likely allow early assessment of treatment efficacy, both as trial endpoints and as patient- management tools. This has clear value implications, but very few potential biomarkers have been developed to the point of allowing them to be reli- ably used in clinical practice. Predictive biomarker validation will require randomized clinical trials, either through targeted selection trials (where only patients who express a given biomarker are enrolled) or through unselective enrollment trials with prospectively specified biomarker analysis. While analyses of data from previously conducted trials may also be used to validate biomarkers, the quality of the data may be questionable without standardized protocols and analyses. As a medical community, Dr. Sargent concluded, we do a reasonable job in determining efficacy, though it is costly, data collection is burdensome, and we need further work to develop reliable early endpoints. However, we rarely collect data to reliably determine effectiveness. Carefully generated experiments are critical to generate this evidence, and large, simple trials and cluster randomization can provide some of this data to bridge the gap between efficacy and effectiveness. DISCUSSION Dr. Sean Tunis of the Center for Medical Technology Policy com- mented on trials to better understand effectiveness, saying that there needs to be a model for simpler trial designs that are more pragmatic and effec- tiveness oriented. Frequently companies are hesitant to pursue these types of studies for two reasons: (1) they worry that the FDA will impede the studied drug over safety or toxicity issues that may emerge, and (2) they are concerned that there are many more restrictions on marketing of treat- ments whose effects are determined through studies that are not performed under a traditional regulatory paradigm. How can companies move past such concerns? Dr. Woodcock responded by saying that it is “almost unheard of ” that a cancer drug is delayed because of toxicities in less-controlled trial settings, and some cancer drugs have reached the market based on information gained through access trials with levels of control similar to the large, simple

OCR for page 23
 ASSESSINg ANd ImPROVINg VAluE IN CANCER CARE trials. Regarding the legal restrictions on marketing, there is talk in Congress about removing them. The question is this: what evidentiary standards are required for secondary indications, not drug approval? Dr. Woodcock felt that the FDA would be hard-pressed to accept observational studies. They simply do not provide the level of evidence needed to change the medica- tion label. However, clearly more discussion is required about the amount of data needed and ways to increase the size of large, simple trials to support follow-on indications for a cancer drug. Dr. Cohen said that this calls for a sustained commitment to investment in the methodological infrastructure for such trials. Dr. Martin Murphy of the CEO Roundtable on Cancer asked the panelists about opportunities to retrospectively mine previous clinical trial data to look for signals that might be useful to identify new therapeutics. Dr. Woodcock agreed that the idea was promising and said this would be important for many diseases, especially cancer. REFERENCES Efficacy. 2009. dictionary.com unabridged. Random House, Inc. http://dictionary.reference. com/browse/efficacy (accessed February 9, 2009). Evidence. 2009. dictionary.com unabridged. Random House, Inc. http://dictionary.reference. com/browse/evidence (accessed February 9, 2009). Harris, R. P., M. Helfand, S. H. Woolf, K. N. Lohr, C. D. Mulrow, S. M. Teutsch, D. Atkins, and Methods Work Group of the Third U.S. Preventive Services Task Force. 2001. Current methods of the U.S. Preventive Services Task Force: A review of the process. American Journal of Preventive medicine 20(3 Suppl):21–35. QUASAR Collaborative Group, R. Gray, J. Barnwell, C. McConkey, R. K. Hills, N. S. Williams, and P. J. Kerr. 2007. Adjuvant chemotherapy versus observation in patients with colorectal cancer: A randomised study. lancet 370(9604):2020–2029. Sargent, D., S. Patiyil, G. Yothers, D. G. Haller, R. Gray, J. Benedetti, M. Buyse, R. Labianca, J. F. Seitz, C. J. O’Callaghan, G. Francini, A. Grothey, M. O’Connell, P. J. Catalano, D. Kerr, E. Green, H. S. Wieand, R. M. Goldberg, and A. de Gramont. 2007. End points for colon cancer adjuvant trials: Observations and recommendations based on individual patient data from 20,898 patients enrolled onto 18 randomized trials from the ACCENT group. Journal of Clinical Oncology 25(29):4569–4574.