Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 55
7
Novel Clinical Trial Designs
“Our clinicians engage in research now. It’s the usual care of
the patient, and we generate research information from it.”
—Louis Fiore, V Boston Healthcare System
A
The workshop session on innovations in trial design began with a series
of challenging questions by the moderator, Michael Krams, Vice President,
Head of Neurology Franchise, Johnson & Johnson. How can we conduct
clinical trials such that actionable results come from them? How can we
translate the creation of knowledge into impact for society? Will innova-
tive trial designs let us emulate the stunning performance improvements
that have been accomplished in computing? And, as was mentioned by
Bram Zuckerman, Director, Division of Cardiovascular Devices, Center for
Devices and Radiological Health (CDRH), FDA, from a practical stand-
point, can we create a library of case studies so that people can see how
these methods work?
ADAPTIVE CLINICAL TRIAL DESIGNS1
While the annual number of new drug approvals in the United States
has remained relatively flat for the past several decades—hovering more
or less between 18 to 25—what has not been constant are the costs of
drug trials, which have been increasing at about triple the inflation rate.
Part of the problem stems from the success in genomic sequencing and
the explosion in the number of new, less well validated targets, with their
resultant high failure rates.
1 Material in this section is based on the presentation by Michael Parides, Professor of
Biostatistics, Department of Health Evidence and Policy, Mount Sinai School of Medicine.
55
OCR for page 56
56 PUBLIC ENGAGEMENT AND CLINICAL TRIALS
These rising costs can be considered not as the price of success, but
the price of failure—an insight credited to Robert Hemmings.2 If clini-
cal trial designs could detect failure sooner, in phase I or even phase II,
then trials would not proceed to the later phases, where costs have been
increasing most dramatically.
There are many ways for a trial to fail, said Michael Parides, Mount
Sinai School of Medicine. Perhaps the compound simply does not work,
or not at the dose being tested, or not in the expected patient population.
Sometimes the study design is not optimal or the entire drug develop -
ment plan is flawed. Many phase III trials that fail have problems that can
be traced to phase I and II trials that did not produce the quality of infor-
mation needed for the confirmatory trial to be designed appropriately.
Improved clinical trial designs hold great promise for making the
clinical trial enterprise more efficient, primarily by earlier detection of
inadequate benefit. At the same time, treatments that do offer benefit need
to be accurately recognized, so that they are not prematurely abandoned,
he said. Reliably discarding compounds that do not work and keeping
those that do increases the overall trial success rate.
A promising approach to improving trial design is “adaptive design.”
Adaptive design is not a new idea, but it is becoming increasingly inter-
esting to researchers. In general, adaptive designs use interim data to
modify an ongoing trial without undermining its validity and integrity
or introducing bias. Modifications might include correcting inaccurate
assumptions or reestimating the sample size. The adaptations are care-
fully planned in advance and are prespecified, such that, while the trial
design is flexible, it is not completely open-ended. There are numerous
variations on the adaptive design theme, some more accepted than others.
Recent developments have made adaptive trial designs more feasible.
Perhaps most important is the increased use of Bayesian statistical meth-
ods, made feasible by desktop computing power. Bayesian approaches
allow continual reassessment of trial findings with respect to, for example,
maximum tolerable dose. Rather than assigning patients to trial doses
according to an algorithm that does not make dose-limiting toxicity
explicit, in the Bayesian approach, the researcher makes an assumption
about the relationship between dose and toxicity; data are collected; the
relationship is reassessed; and the process repeats through some num-
ber of cycles. The key element, Parides said, is the notion of continuous
learning: Each new patient has the benefit of what was learned from each
previous patient. Most such applications require simulations, an approach
2 Dr. Hemmings is Statistics Unit Manager of the U.K. Medicines and Healthcare Products
Regulatory Agency.
OCR for page 57
57
NOVEL CLINICAL TRIAL DESIGNS
that fits the drug development paradigm well, as a series of revisions of
an original idea, updated with additional information. This is a useful
model for exploratory trials.
A second new element is the bolder nature of some types of adap -
tive design, for example, unblinded sample size estimation, changing
the primary end point, patient enrichment (that is, once a trial starts, the
investigator perceives there may be subgroups for which the treatment
works better and stops randomizing people outside of those subgroups),
and seamless phase II/phase III designs. Strategies such as these require
rigorous statistical management.
Information gained from the use of these statistical methods allows
researchers to abandon a trial arm or curtail an entire trial early in devel -
opment if it is not expected to work. For example, “adaptive dose-finding
randomization” helps the researcher decide to drop treatment arms based
on initial responses, whether due to toxicity, efficacy, or both; and even
in a phase III confirmatory trial, where adjustments must be made cau-
tiously, group sequential procedures are an accepted way to gain infor-
mation that can lead to midcourse corrections or even trial termination.
A real-world trial conducted in LVAD recipients—people with end-
stage heart failure surgically implanted with mechanical heart pumps—
provides an example. The investigators wanted to know whether the
trial was generating enough information to warrant continuation. Parides
showed how the trial data would look if there were 20 patients each in
the control group and the active therapy group. The number of treatment
failures in the two groups was 13 (controls) and 10 (treatment). Conven -
tional statistical methods would make it hard to judge whether the trial
should be continued. The p value is 0.52, with a wide confidence interval.
However, the customary reliance on p values in this case would be mis-
guided, Parides said. When the same data are analyzed using Bayesian
approaches, the first step is to assess the success probability for both
groups. Due to the assumptions of clinical equipoise, these probabilities
are the same, albeit unknown. After the data are collected, they are refit -
ted to the model, revealing that the probability that the treatment is better
than control is 75 percent. In this case the Bayesian analysis was a much
more accurate way to present the data and one that made the decision to
move forward with the trial clear.
At the more controversial end of the spectrum of potential trial adap-
tations is changing the study’s primary endpoint. Serious problems can
arise, as occurred a decade ago in the CAPRICORN trial, a multicenter,
multinational, randomized, double-blind, placebo-controlled trial of
whether beta-blockers plus routine medical management performed bet-
ter than routine management alone after a heart attack (Colucci, 2004).
The initial primary endpoint was all-cause mortality. As the trial was
OCR for page 58
58 PUBLIC ENGAGEMENT AND CLINICAL TRIALS
under way, the mortality rate was lower than expected, but, because the
study was blinded, the investigators did not know whether that was
because one of the interventions was working very well, or whether both
were effective. At that point, the researchers elevated a prespecified sec-
ondary endpoint—cardiovascular hospitalization—to be the coprimary
endpoint. Unfortunately, it turned out that the result for this combined
endpoint was not statistically significant, while the original would have
been (a 23 percent reduction in all-cause mortality).
While adaptive designs undeniably have appeal, Parides said, they
are not always better and not necessarily logistically simple or less expen-
sive. On a trial-by-trial basis, adaptive designs may cost more, but they
save money overall, he said, by preventing investment in futile exercises.
Motivating researchers to change the way they work and meth-
ods they use is difficult, Krams said. “As we all know, in the clinical
R&D environment, culture eats strategy for lunch.” When the number
of scientists in academia, industry, and at FDA who are familiar with
adaptive research methods gradually increases, these new methods may
become more acceptable, said Parides. Methodological problems will be
resolved, some approaches will fall by the wayside, and some will eventu-
ally become second nature.
Currently, researchers do what is feasible because it is feasible, rather
than because it will produce the most useful endpoints, Krams said. The
incentives are geared toward obtaining results as quickly as possible.
Krams advised looking beyond an individual trial to an entire research
program and assessing how many times a second trial is needed because
of inconclusive answers to the research question. A more productive way
of framing the incentives, therefore, is to work toward achieving the best
information value per research unit invested.
USING POINT-OF-CARE CLINICAL TRIALS TO
CREATE A LEARNING HEALTH CARE SYSTEM3
Randomized clinical trials remain the gold standard for determining a
treatment’s safety and efficacy, but their high costs and extended timelines
and the delayed integration of their results into clinical care are prob-
lematic. Observational studies are less expensive and produce quicker
results, but their findings are less reliable. Louis Fiore, Assistant Profes -
sor, VA Boston Healthcare System, described an initiative to meld the two
methods, called “point-of-care clinical trials,” which uses randomization
3This section of the report is based primarily on a presentation by Louis Fiore, Assistant
Professor, U.S. Department of Veterans Affairs (VA) Boston Healthcare System.
OCR for page 59
59
NOVEL CLINICAL TRIAL DESIGNS
to remove selection bias in an observational study. The method is being
tested within the VA and led by the Boston VA Healthcare System. The
long-term goal of this new research approach is to create a true learning
health care system,4 he said.
In most medical facilities, research and clinical departments are com-
pletely separate. Poorly funded researchers raise their own money and
not until some years after their project ends do their findings become
adopted by the clinical side of the house. A point-of-care approach would
speed up adoption of treatment improvements and allow researchers to
leverage the resources of the clinical services.
The VA is well positioned to try this approach for several reasons: its
clinicians are interested in it; VA patients continually return to the system
over a period of years; and the VA has a sophisticated EHR system, which
allows patient records to be accessed from any VA facility.
According to Fiore, point-of-care clinical research is a quality improve-
ment strategy, and it has a number of advantages over traditional research
methods:
• he studies compare existing approved drugs with known toxici-
T
ties and therefore do not involve IND applications or require IRB
approval—they ask “which is better?” questions.
• hey are designed so that a substantial portion of their operations
T
can be conducted by clinical staff as part of routine care delivery,
with study data captured passively in the EHR.
• hey do not disrupt regular clinical care, require lengthy and repeated
T
discussions with patients, or demand unusual data collection.
• articipating patients are drawn from the day-to-day caseload of
P
the VA hospital, with minimal exclusion criteria, and thus are a
generalizable population.
• inimal extra outpatient visits are required; when participants
M
are discharged and return as outpatients, the next set of data is
captured.
• he researcher has access to real experiences and outcomes, not
T
surrogate ones, and follow-up can continue as long as desired.
• he research is low cost, even through the follow-up phase, cost-
T
ing only an estimated 10 to 30 percent of the cost of an industry-
sponsored trial.
4 The learning health care system can be defined as the seamless and continuous develop -
ment and application of evidence in the course of patient care. In such a system, each patient
care experience naturally reflects the best available evidence, and, in turn, adds seamlessly
to learning what works best in different circumstances (IOM, 2008).
OCR for page 60
60 PUBLIC ENGAGEMENT AND CLINICAL TRIALS
• inally, since the research is being done in the same health care
F
system that is going to use the results, physicians are more likely
to have confidence in them. In fact, they have generated the results
themselves.
In short, this is a pragmatic approach to study design that produces
very pragmatic results, germane to a hospital’s specific patient population
(see Box 7-1). Generalizing to a different facility requires analyzing dif -
ferences in the health care systems and patients served, not the treatment
being tested, Fiore said.
Administratively, point-of-care research is facilitated by what Fiore
terms a “rational approach” to regulatory oversight and obtaining
informed consent. The question of whether clinicians are participating in
a trial, versus merely going about their regular business, is an important
BOX 7-1a
Point-of-Care Clinical Trial Pilot Study
To test the feasibility of point-of-care research, VA investigators at the eight VA
hospitals in the six-state New England region are comparing weight-based versus
sliding-scale determinations of insulin dose in non-intensive care unit patients
with diabetes. VA physicians and the clinical literature have been divided on which
approach is best. The study’s primary endpoint is hospital length-of-stay, and the
secondary endpoint is glycemic control and readmission within 30 days.
When patients enter the hospital needing insulin, physicians (via EHR) are
presented with three options. This is the “point of care.” Options 2 and 3 provide
an insulin regimen according to usual weight-based or sliding-scale protocols. The
first option is “no preference” and invites clinicians to enroll their patients in a study
comparing the two protocols. If they choose the study, the patient is automatically
randomized to one or the other treatment, a nurse obtains consent, and a progress
note about the study is automatically entered in the record.
From there the computer takes over, writing the orders for the clinician to sign.
The study is fully integrated into the hospital’s informatics system, with the EHRs
tracking which of the two treatments produces the best outcomes. Using adaptive
randomization and Bayesian approaches, randomization may start out 1:1, but as
one arm of the trial becomes statistically superior, randomization will change to
60:40, 70:30, and so on. Eventually, new patients will be randomized 99:1 to the
effective arm, and the study will conclude. In effect, the more successful treat-
ment will become the standard of practice in the facility “directly as the study is
happening,” Fiore said.
a Based on the presentation by Louis Fiore, Assistant Professor, VA Boston Healthcare
System.
OCR for page 61
61
NOVEL CLINICAL TRIAL DESIGNS
one. Declaring them “researchers” might impose regulatory requirements
related to, for example, serious adverse-event reporting. If drugs are well
established (like warfarin), Fiore queried, is it really necessary to report
adverse events (like bleeding)? Similarly, is written informed consent
needed for low-risk comparison studies—for example, one insulin type
versus another—or will oral consent suffice? Ideally, Fiore believes, when
a patient is admitted to a VA facility and provides the usual consent
to care, the form would include consent to participate in this type of
research. This blanket “opt-in” consent would be documented in the EHR.
The electronic record is key to the efficiency and national scalability
of point-of-care research. Researchers work with information technol-
ogy staff to modify the record to incorporate tools and bits of code that
allow it to randomize, extract data, and create notifications. At the end of
the study, the randomization node can be easily changed to a decision-
support node. And, economic analyses are simple because all the costs are
already recorded in the health system.
Because VA patients’ electronic records are available at any VA facility
nationwide, additional opportunities to participate in trials present them-
selves. For example, a veteran at a VA facility remote from any research
center might have prostate cancer (or other tissue) analyzed, with the results
recorded in the EHR. At another VA site engaged in prostate cancer research,
a drug trial might be under way for which the patient would be an appropri-
ate participant. Mining the EHR data allows that patient to be identified and
facilitates the patient’s engagement in the trial. Fiore said this would move
research to the patient, rather than the patient to the research.
Another potential benefit of point-of-care trials would be to create a
culture change in the way clinicians and patients think about treatment
trials. If doctors and patients want treatments based on the best medi-
cal knowledge, with strategies that have been tried and tested—in other
words, if they want to provide and receive evidence-based medicine—
then they need to be part of the evidence-gathering process.
A fundamental challenge to the diffusion of point-of-care research
is the lack of appreciation and reward for collaborative work within
academic medicine. The days of lone investigators owning data and car-
rying out projects in isolation are numbered in the clinical sciences, Fiore
believes. Yet, the academic infrastructure has not even begun to dismantle
these silos. Additionally, Zuckerman mentioned that the training environ-
ment needs to change, so that medical schools produce physicians who
are clinical trialists and clinical research courses become a standard part
of the curriculum.
The point-of-care model requires a reconsideration of the relation-
ship between clinical care and research. Clinical effectiveness research is
“engineering,” and as much as it is needed, there are too many research
OCR for page 62
62 PUBLIC ENGAGEMENT AND CLINICAL TRIALS
questions, too few investigators, and too little funding. Clinical care dol -
lars, being spent in any case, can generate the data. If the health care sys -
tem used point-of-care research methods to learn from its experiences, it
could, Fiore said, “make taking care of patients a whole lot quicker, more
effective, and probably cheaper.”
THE FOOD AND DRUG ADMINISTRATION PERSPECTIVE5
FDA leaders perceive a role for the agency in encouraging innova-
tion and promoting efficient development of new and improved medical
products, said Douglas C. Throckmorton, Deputy Director for Regulatory
Programs, CDER, FDA. FDA staff attempt to make researchers’ jobs easier
by designing clear and thoughtful rules and interpretations, publish -
ing guidances regarding them, and ensuring they are applied equally
to everyone. The goal is to help innovation, he said, not hinder it (FDA,
2010, 2011d). For example, draft FDA guidance on the use of adaptive trial
designs has been released. The document starts with adaptive designs
that are used regularly and would be rather easy to accept and ends with
some of the more complicated and problematic adaptations that might
take a fair degree of discussion.
Promoting an efficient process for medical products development
means more than waiting for researchers to submit their trial applica-
tions, Throckmorton said. It means supporting appropriate collabora-
tions, building opportunities, developing standards that enable efficient
drug development, and building support for academic science. FDA staff
frequently engages in partnerships, collaborations, and consortia, such as
CTTI, a public-private collaboration with Duke University. CTTI mem -
bers, drawn from government, industry, academia, and patient groups,
are examining and prioritizing the major challenges in the conduct of
clinical trials, with the goal of increasing their quality and efficiency.
In addition, FDA is attempting to make its own operations more
efficient, Throckmorton said. The agency is working to focus the clini -
cal trials monitoring program on trial sites where the most problems are
likely, rather than treating all sites equally. In an effort to build quality into
a clinical trial from the beginning, Pfizer and FDA staff are conducting a
pilot test in which they are simultaneously designing a phase III study
and its monitoring program.
A wide range of regulatory approaches is necessary to carry out
FDA’s regulatory authority over devices, which include everything from
5 This section is based on the workshop presentations of Douglas C. Throckmorton, Dep -
uty Director for Regulatory Programs, Center for Drug Evaluation and Research (CDER),
FDA; and of Bram Zuckerman, Director, Division of Cardiovascular Devices, CDRH, FDA.
OCR for page 63
63
NOVEL CLINICAL TRIAL DESIGNS
sterile gloves to LVADs. The most stringent regulations cover the Class
III, high-risk, life-supporting products that require premarket approval
(PMA), which in many ways is similar to the approval pathway for a
new drug.
The agency’s challenge is to ensure the safety and effectiveness of
medical devices even as science is continually evolving, devices are
becoming increasingly complex, and the existing regulatory pathways to
market were established in 1976. In the meantime, Zuckerman said, some
of the distinctions between “drug” and “device” have blurred.
A practical consideration for device developers is the need to engage
with the FDA’s CDRH “early and often” regarding its clinical trial strat-
egy, Zuckerman noted. Adaptive designs may be particularly helpful
to device developers, as many of them are small companies with corre -
spondingly small research budgets. But even large device manufacturers
may want trial results quickly, because a device’s life cycle is often rela -
tively short. An estimated 10 to 15 percent of device applications currently
include some combination of Bayesian or adaptive designs.
OCR for page 64
