Large Simple Trials and Knowledge Generation in a Learning Health System: Workshop Summary (2013)

4

Medical Product Regulatory Issues

INTRODUCTION

Clinical trials of all designs, including large simple trials, are often overly complex and costly. This chapter summarizes panel discussions on the current state of trial complexity, various strategies for reducing the complexity of clinical trials and increasing their efficacy, and the U.S. Food and Drug Administration’s (FDA’s) perspective on clinical trial design. Kenneth A. Getz, director of sponsored research programs and research associate professor at the Tufts Center for the Study of Drug Development, discussed the mounting complexity of today’s clinical trials and highlighted possible reasons for the prevalence of such complexity. Christopher B. Granger, director of the Duke University Cardiac Care Unit and professor of medicine at the Duke University Medical Center, continued the panel discussions with a presentation on the promise offered by simplification strategies for reducing the costs of clinical trials and increasing their effectiveness. Rachel E. Sherman, associate director of medical policy for the Center for Drug Evaluation and Research at FDA, brought the panel to a close with a discussion of FDA’s perspective on clinical trial design.

TRIAL COMPLEXITY

In his discussion of complexity in current clinical trial protocols, Kenneth A. Getz underscored the increasing number of elements measured and incorporated into clinical trials today. A typical study has an average of 13 endpoints: 1 primary endpoint, 5 key secondary endpoints, and a number of tertiary, or exploratory, endpoints. Moreover, the average study protocol involves nearly 170 procedures, only half of which support the primary and key secondary endpoints. The typical protocol also has an av-

TABLE 4-1 Rising Protocol Complexity and Burden

NOTE: CRF = case report form.

SOURCE: Reprinted with permission from Kenneth A. Getz.

erage of 35 eligibility criteria, and the case report form for a typical clinical trial has nearly 170 pages, requiring a study volunteer to make 11 visits over an average of 175 days of participation in a trial. Additionally, Getz emphasized, as studies are simultaneously conducted in a growing number of countries, the requisite coordination with multiple health authorities and different regulatory agencies and the logistics of distributing clinical supplies and collecting data in ever more remote regions add large amounts of complexity. Table 4-1 documents these items over the course of three observation periods in the past 10 years. Getz explained that each element has experienced a significant increase, contributing to the rising complexity and burden of clinical trial protocols.

To explain this increase, Getz continued, one could look to the shift in focus from acute illness to chronic illness, for which endpoints are far more difficult to measure. Moreover, many current studies are collecting more genetic material and biomarker data and involve a combination treatment or a diagnostic procedure, both of which add complexity and additional procedures to the clinical trial protocol. More data are being collected at all phases of clinical trials, Getz said, possibly because of pressure from regulatory agencies to collect more safety data or to determine whether a project should be terminated early. Particularly during Phase III trials, more data are collected in anticipation of regulatory requests, and even Phase IV studies have shifted from being more observational in nature to more robust, controlled clinical trials.

This continuously increasing complexity negatively affects the performance of clinical trials in a variety of ways, Getz explained. More complex trials have worse recruitment and retention rates and also have prolonged cycle times. Additionally, more complex trials typically correlate with more amendments to the trial design, which can be incredibly costly and disruptive to implement. Research has shown that the majority of protocols have at least one amendment, 46 percent of which occur before the first

patient receives the first dose. Nearly 40 percent of all amendments are deemed avoidable by those companies sponsoring the studies. Getz cited inappropriate or irrelevant policies, typographical errors, and design flaws as potential reasons for these amendments, underscoring the idea that a protocol amendment is not a problem; it is instead a solution to an underlying design problem. However, one such solution typically adds roughly 2 months to the overall length of a study and costs about half a million dollars in direct study costs.

Getz ended with a discussion of the following question: is all of this complexity meaningful? In a study of companies sponsoring clinical trials and their specific protocols, almost one out every five protocols was classified by the protocol designers to be noncore in nature: the protocol was not tied to a primary or key secondary endpoint, it was not associated with good clinical practice compliance, and it was not evaluating a procedure typically performed in the clinical setting. Possible reasons for such a high proportion of noncore procedures, Getz elaborated, include a desire to collect additional data to gain insight into a particular mechanism of action of a drug or even a new area of development, along with efforts to mitigate the risk that regulatory agencies will request additional data or resistance by purchasers and payers to pay for the drug once it is approved and on the market. Additionally, protocol designs from earlier phases of clinical trials are often reused for later studies, without consideration of whether all procedures from the former studies are necessary for the latter ones.

As a final point of measurement, Getz highlighted that roughly 18 cents of every dollar that is spent on clinical trials goes toward these noncore, or less essential, procedures. This approaches $2 million per Phase III study budget, or $4 billion to $6 billion per year worldwide. Further scrutiny of trial elements and procedures, Getz concluded, would assist with simplification of protocol designs and could not only alleviate some of these misdirected costs but also improve trial performance.

SIMPLIFYING CLINICAL TRIALS

Christopher B. Granger began his remarks by underscoring several key points: trial quality can be defined, large simple trials can be burdened with complexity that does not improve quality, examples of trials that are much simpler and fit for their purpose exist, trials could be either radically or incrementally simplified in most circumstances, and cost reductions resulting from sensible simplification can be quantified. He continued by indicating that trial quality is driven by an important question that will change practice and improve outcomes. With this understanding, Granger elaborated, trial quality is determined by a number of elements, including the availability of an adequate number of events to answer the research

question with confidence, proper randomization, and a plan for ongoing measurement and feedback for improvement of quality measures during the conduct of the trial (see Box 4-1).

Granger delved into the question of how to reduce not only the costs of clinical trials but also their complexity. Using a hypothetical trial of a treatment for a chronic disease with 20,000 patients across 1,000 sites performed over 2 years with 60 case report form pages, 24 site visits, and a study award of $10,000 per patient, researchers prepared models for three different cost estimates: a model of the cost of the full trial, a model of the cost of a streamlined trial, and a model of the cost of a radically streamlined trial. The model of the cost of the full trial provided a cost of roughly $400 million.

To calculate the cost of a streamlined trial, a variety of factors were altered. The trial duration, trial size (in terms of the length of the case report form and the number of sites), and operational issues were all adjusted. Of particular importance was accounting for the large number of unproductive trial sites. Granger described the work of Lisa Berdan and the Duke Clinical Research Institute showing that the top 10 percent of trial sites enroll about 40 percent of all patients. Moreover, the cost of starting up the trial at each site is significant, estimated to be a minimum of $14,000, so the removal of those sites that will not be productive is a critical source

of preventable expense. Reductions in the amount of time for planning and enrollment resulted in relatively modest cost reductions, with only a 1 to 2 percent reduction in the total cost of the trial. However, Granger explained that decreases in the length of the case report form and the number of sites had much more significant impacts, with a nearly 10 percent reduction in cost being obtained after a modest reduction in the total number of sites. Electronic data capture offered the potential for a 10 percent reduction in cost through the elimination of query processing, data entry, and medical coding; and it also decreased the amount of time required for management of queries. Lastly, adjustment of site management factors, including streamlining of trial procedures and reduction of the number of physician visits to the study site, decreased costs by 21 percent. Granger reported that the changes made in the model of a streamlined trial yielded a 35 percent reduction in costs. Additionally, further extension of these changes to include a 50 percent lower per patient study award reduced costs by a dramatic 60 percent.

The model of the cost of a radically streamlined trial, which was limited to 100 high-volume sites, eliminated on-site evaluations and source data verification, used highly focused case report forms, and dramatically reduced payments to the study sites. With this model, the cost of the trial was reduced nearly 10-fold from the cost for the full study.

Granger continued his discussion of approaches to streamlined trial design by highlighting the Thrombus Aspiration in ST-Elevation Myocardial Infarction (TASTE) trial in Sweden as a case study (Fröbert et al., 2010). Built on the Swedish acute myocardial infarction (MI) registry, this trial randomly assigned patients with acute MI to thrombus aspiration with primary percutaneous coronary intervention (PCI) or primary PCI alone. With the registry, researchers were able to identify and register eligible patients, record whether those patients had provided verbal informed consent, and confirm that the inclusion/exclusion criteria had been met. The incremental cost of these procedures has been about $50 per patient, or $350,000 for all 7,000 participants.

To conclude, Granger asked why these cost-reducing strategies have not been adopted, as they have already proven effective. Aversion to the risk of having not collected a particular element requested by auditors is one possible explanation; researchers may believe that it is better to collect 100 unnecessary variables than to miss one important one. Additionally, Granger explained, regulatory departments and contract research organizations have a substantial financial stake in maintaining the status quo, as their business models and margins are created by the complexity inherent to current trial designs. Lastly, the lack of international harmonization among trial designs can force the use of the most complicated common denominator.

Granger underscored the fact that each trial is different, and thus, no

universal solutions for simplification exist. However, substantial reductions in the costs of large-scale clinical trials can be achieved without compromising quality, and these savings can be achieved through both incremental and transformational simplification approaches. Further research on the impact of the simplification of clinical trials will help to transition the current research paradigm to one that is more effective and efficient.

FDA PERSPECTIVE

Rachel E. Sherman concluded the panel’s discussions by sharing the FDA’s perspective and providing guidance on the design of clinical trials. She began her comments with an emphasis on the need for researchers and sponsors to work with FDA more regularly throughout the research process. Moreover, she underscored that unproductive trials, those trials that do not produce useful insights, not only are wasteful but also are unethical for the research enterprise. With the resources available in the United States and around the world, Sherman stressed, it should be possible to address the existing evidence gap in medical research, but continued that wasteful protocols and procedures hinder progress.

Sherman continued by underscoring the idea that the collection of extraneous data points attributed to a fear of regulatory agencies is not supported by the reality of FDA’s regulatory practices; what truly matters is meeting the primary endpoint. Of the utmost importance to FDA is data quality, Sherman explained, and only high-quality data can provide substantial evidence. Substantial evidence, as defined by the Food Drug and Cosmetic (FD&C) Act, is “evidence consisting of adequate and well-controlled investigations, including clinical investigations, by experts qualified by scientific training and experience, to evaluate the effectiveness of the drug involved, on the basis of which it can be fairly and responsibly concluded by such experts that the drug will have the effect it purports or is represented to have under the conditions of use prescribed, recommended, or suggested in the labeling and proposed labeling thereof” (Section 505(d) of the FD&C Act).

The FD&C Act, Sherman emphasized, does not prescribe a particular trial design but instead states that the trial must produce evidence, be appropriately designed, and produce useful information. It is then the responsibility of FDA to communicate that information. As such, large simple trials are not discouraged by FDA but instead only need to adhere to the parameters set forth by FDA’s bylaws. Again, FDA’s emphasis is not on the specific design of a trial but on the quality of the data that it produces.

Moreover, concerns about FDA’s directive to come and inspect well-designed, controlled, and adequate trials are unwarranted, Sherman said. FDA has the authority to put on hold any trial that is not adequate and

well controlled, as well as any Phase II or Phase III study whose design is clearly deficient in its ability to meet the stated objectives of the study. Most importantly, if the trial presents an unreasonable risk to a patient, FDA of course has the regulatory authority to put it on hold; the patient must always come first, Sherman emphasized. If these characteristics do not apply to the trial in question, concerns about FDA inspection or requests for additional data points may be unnecessary.

In terms of harmonization, Sherman continued, FDA has been focused on the streamlining of clinical trials to modernize and harmonize trial design. The goal is to collect only those data needed and to monitor only the necessary procedures. Again, early discussions with FDA can help to ensure that the right questions are being asked and can assuage later fears of FDA monitoring or requests for additional data.

Sherman closed by underscoring that FDA does not emphasize a particular type of trial; the agency instead emphasizes the collection of high-quality data and not a large quantity of data, and once a trial is completed, it will confirm the quality and generalizability of those data. She requested that workshop attendees communicate with the agency if they believe that it is not promoting the use of more efficient designs to ensure that FDA is doing its part to continue the conversation and advance the field.

REFERENCES

Fröbert, O., B. Lagerqvist, T. Gudnason, L. Thuesen, R. Svennson, G. K. Olivecrona, and S. K. James. 2010. Thrombus Aspiration in ST-Elevation Myocardial Infarction in Scandinavia (TASTE trial). American Heart Journal 160(6):1042–1048.

Getz, K. A., R. A. Campo, and K. I. Kaitin. 2011. Variability in protocol design complexity by phase and therapeutic area. Drug Information Journal 45(4):413–420.