National Academies Press: OpenBook

Neuroscience Trials of the Future: Proceedings of a Workshop (2016)

Chapter: 7 Improving the Evidence Base for Real-World Use

« Previous: 6 Ethical Considerations
Suggested Citation:"7 Improving the Evidence Base for Real-World Use." National Academies of Sciences, Engineering, and Medicine. 2016. Neuroscience Trials of the Future: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23502.
×

7

Improving the Evidence Base for Real-World Use

Suggested Citation:"7 Improving the Evidence Base for Real-World Use." National Academies of Sciences, Engineering, and Medicine. 2016. Neuroscience Trials of the Future: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23502.
×

PERSPECTIVES FROM PAYERS

According to several participants, the tension between the rising cost and complexity of clinical trials for innovative treatments for nervous system disorders and the desire to make these treatments accessible for patients reaches its zenith in the reimbursement space. For payers, the dilemma of unsustainability revolves around the expansion and delivery of new technologies and treatments, and how those treatments are paid for, according to Rhonda Robinson Beale, senior vice president and chief medical officer at Blue Cross of Idaho. She noted that in the specialty drug area, new drugs may cost more than $100,000 per year over the lifetime of the patient. As a result, the 3.6 percent of patients who use specialty medications account for 25 percent of health care costs (Milliman, Inc., 2013).

The type of information needed by payers to facilitate adoption and coverage may be very different from that needed by regulators. It may, for example, include the feasibility of implementing a new technology with fidelity, said Robinson Beale. Moreover, different types of payers—health plans, employers, accountable care organizations (ACOs), and individual patients—require different information. For example, ACOs consider not only cost, but also the practicality of implementing a new treatment to achieve the outcomes seen in a clinical trial and whether the treatment is more efficacious compared to existing treatments. Should a new treatment be covered if it only provides incremental improvement over existing treatment? Should off-label use be covered? Ethical choices may also be necessary, said Robinson Beale; for example, if a costly new treatment is to be covered, will something else lose coverage? How does one balance cost with extension of life decisions?

Historically, payers have made decisions based on medical necessity with evidence obtained from peer-reviewed articles, technology reviews, and practice guidelines from credible organizations, said Robinson Beale. She added that payers typically look not only at RCTs, but pragmatic trials as well in order to gain a real-world understanding of the efficacy and practicality of the treatment over a longer period of time than the duration of a clinical trial. Comparative effectiveness studies and a comparative cost‒benefit analysis may also be required.

For very expensive treatments, payers may also ask for definitive evidence that a standard treatment is not effective in a particular patient. For example, said Robinson Beale, transcranial magnetic stimulation has been shown to be effective as a treatment for depression, but is far more

Suggested Citation:"7 Improving the Evidence Base for Real-World Use." National Academies of Sciences, Engineering, and Medicine. 2016. Neuroscience Trials of the Future: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23502.
×

expensive than antidepressant medications; thus, it is typically reserved only for those who receive no benefit from medications. In the precision medicine world, some treatments, including expensive specialty drugs, also work only on subsets of the population. Studies to identify responsive subgroups would thus also be extremely helpful for payers, said Robinson Beale.

In Italy, there is a different approach to reimbursement, as described by Luca Pani in Chapter 5. Outcome-based reimbursement requires companies to refund money if a treatment did not work. This model is particularly challenging in the CNS field, which has a particularly high failure rates. For example, sponsors are developing drugs to treat AD presymptomatically; in addition to the lack of measures for such trials, healthy patients may be exposed to drugs that have some side effects. Studies would need to show some real-life advantages for these patients, which is particularly challenging, said Pani.

PERSPECTIVES FROM THE PHARMACEUTICAL INDUSTRY

Several participants highlighted the importance of payer concerns and how they are factored into pharmaceutical companies’ drug development programs. They noted that this requires generating credible evidence of value in the real world from both observational studies, including registry studies, and randomized trials in real-world settings, also known as pragmatic trials. Steven Romano said that drug developers are now talking with payers and health technology assessment groups even before initiating a Phase II study. This requires investigators to assess the value proposition for a new medicine, including functional outcomes and health economic outcomes.

According to Paul Stang, vice president of global epidemiology for Janssen Research and Development, payers want to understand the performance of products in the specific population they serve. Selecting the appropriate comparator and relevant populations for studies thus may vary, depending on the payer. For example, the population served by the Department of Veterans Affairs may have unique characteristics that influence their use of a treatment and they may also receive care according to a different model than a population served by a commercial insurer. Additionally, patients in the United States frequently change health plans, complicating the value proposition because a payer may question the merit of paying for a treatment that may only provide a benefit in the

Suggested Citation:"7 Improving the Evidence Base for Real-World Use." National Academies of Sciences, Engineering, and Medicine. 2016. Neuroscience Trials of the Future: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23502.
×

future when the patient is no longer in the plan. Different payers also value indirect benefits differently, such as loss of productivity, family impact, or the effect of a treatment on quality of life. Finally, Stang said, both sponsors and payers must grapple with the question of what constitutes a meaningful difference.

METHODOLOGY FOR OBSERVATIONAL STUDIES

While RCTs are considered the “gold standard,” a few participants stated that well-designed observational studies have been shown to approximate the effects of treatment as well as RCTs (Anglemyer et al., 2014; Benson and Hartz, 2000; Concato et al., 2000). Compared to pivotal RCTs, Stang noted that observational trials tend to follow people for a longer period of time, may include much larger and more diverse populations, and may incorporate existing data collection infrastructure such as electronic health records (EHRs). They also involve a more diverse group of investigators with varying levels of experience in conducting trials, increasing variability. Hybrid studies have recently emerged as another alternative, said Stang. These trials randomize patients and use EHRs as the data collection tool.

METHODOLOGY FOR PRAGMATIC TRIALS

Demand is growing from regulators, payers, providers, and patients for pragmatic trials, according to Mark Cziraky, co-founder and vice president of research at HealthCore, Inc. These trials are designed to evaluate the risk and benefits of interventions in real-world, naturalistic community settings. In addition to testing in a real-world population and comparing a treatment to the standard of care rather than a placebo because pragmatic trials are typically conducted by practitioners, they provide a window into the effectiveness and value of a treatment when used as intended. They also provide regulators with real-world evidence around safety, and payers with real-world data about use. Figure 7-1 summarizes some characteristics of pragmatic trials.

Suggested Citation:"7 Improving the Evidence Base for Real-World Use." National Academies of Sciences, Engineering, and Medicine. 2016. Neuroscience Trials of the Future: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23502.
×
Image
FIGURE 7-1 Pragmatic trials versus randomized controlled trials.
SOURCE: Presented by Mark Cziraky at the Workshop on Neuroscience Trials of the Future, March 4, 2016.

In a search of www.ClinicalTrials.gov, Cziraky examined 367 studies identified as pragmatic trials, 11 percent of which were in the neurosciences —primarily in psychiatry (Purgato et al., 2015) and psychopharmacology (Vitiello, 2015). Many of the studies use cluster randomization, where population groups (e.g., in nursing homes, in-patient facilities, or physician practices) rather than individuals were randomized. This approach lessens the burden on practitioners and avoids contamination across interventions, but introduces a loss of statistical efficiency (Meurer and Lewis, 2015). However, in psychiatry, there is the potential for spillover effects in clusters, for example, if many individuals in the cluster received effective antidepressants, suggested Erik Snowberg. While this might be a good thing for the patients involved, it could at the same time make interpretation of results difficult.

One of the challenges with pragmatic trials is selecting the comparator. The standard of care may be well established throughout the field, or may be selected individually by the provider. In the neurosciences the standard of care is often not obvious, said Stang, making these pragmatic

Suggested Citation:"7 Improving the Evidence Base for Real-World Use." National Academies of Sciences, Engineering, and Medicine. 2016. Neuroscience Trials of the Future: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23502.
×

trials noisier than those done on other fields such as cardiology, where standards of care are more widely recognized. Even in cardiology, however, standard treatments such as statins for lowering cholesterol are widely underused, said Cziraky, adding even more complexity to the trial. An additional question related to the standard of care is whether the experimental treatment is given in addition to, or instead of, the standard treatment, noted Frank Rockhold. In neuropsychiatry there are even more complications, he said, because the comparator may be psychotherapy, which may be problematic or unethical to withhold. On the device side, additional variability in the standard of care arises because of variability in the skill of the operators (i.e., surgeons), said Stang.

According to a few participants, real-world trials add other complexities, including variability around diagnoses, whether patients are taking medications as directed, and what other treatments are being taken concurrently. Daniel Burch and Michael Pollock, vice president of real-world outcomes at Pharmaceutical Product Development (PPD), cited other differences between explanatory (efficacy) and pragmatic (effectiveness) trials. Explanatory trials try to set ideal conditions to achieve a well-controlled study, yet result in questionable external validity and generalizability. They also may be affected by non-adherence. Burch added that while placebo responses and variability complicate explanatory trials, they are essential parts of the real world that patients, clinicians, and payers live in.

New technologies are increasingly being employed in real-world trials, including automated reminder systems to maximize compliance and biomeasures (such as those using smartphones) to help further identify patients who are likely to benefit or be harmed by a treatment, said Stang.

DATA SOURCES FOR REAL-WORLD TRIALS

The databases maintained by health plans and large provider networks offer a rich source of minable data for pragmatic trials, according to several participants. For example, HealthCore maintains an integrated research database1 that captures 10 years of data from about 65 million individuals, including data from administrative claims, physician and facility claims, prescriptions, and laboratory tests, said Cziraky. Togeth-

___________________

1 For more information, go to https://www.healthcore.com/database (accessed June 3, 2016).

Suggested Citation:"7 Improving the Evidence Base for Real-World Use." National Academies of Sciences, Engineering, and Medicine. 2016. Neuroscience Trials of the Future: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23502.
×

er, these data can provide a picture of a patient’s exposure to the health care system, including procedures and tests performed, treatments, hospitalizations, and costs, he added. For about 30 percent of HealthCore’s population, results from laboratory tests are also included, providing a proxy for diagnosis. Moreover, Cziraky stated that working through providers, these data can be connected with clinical data in the trial setting to create an integrated dataset for analysis.

Burch commented that the workshop discussions suggest that distinctions between regulatory and pragmatic trials may become blurred, which raises concerns because of the potential for real-world trials to impede assay sensitivity. However, Robert Califf used the example of opioid use and abuse to illustrate why pragmatic trials are needed. Although there is good evidence that opioids provide much-needed pain relief for 2 months, there has never been a study to show that there is a benefit beyond 3 months. Yet patients often take these drugs for extended periods of time, resulting in many overdose deaths. Part of the FDA’s postmarketing requirement now for opioids is a randomized withdrawal study out to a year to provide doctors with prescribing guidance.

Similarly, said Pani, there is virtually no evidence of what will happen if disease-modifying drugs for a disease like AD are given for an extended period of time and outcomes needed for registration may different from health-outcomes needed for price negotiation pointing again to the need of early scientific and HTA advice. Califf commented that tools like Sentinel should provide an inexpensive way to collect long-term data for these types of follow-up studies. Moreover, he thinks these very large databases will provide a self-correcting system to prevent the reporting of false-positive results of a problem when, in fact, a drug is quite safe. Thomas Laughren added that most of these databases have committees that review analysis plans before sharing data to ensure that the study being proposed is legitimate and rigorous. A workshop participant said this problem is being addressed by the National Center for Health Statistics and other federal agencies by a process called a research data center, where people can submit a research design, which, if deemed worthwhile by a group of experts, is turned over to a team of technologists and statisticians who will execute the research plan for a fee.

Story Landis relayed an experience with PatientsLikeMe, that while not a pragmatic trial per se, further illustrates the value of the patient experience. After a paper published in a reputable journal suggested that lithium delayed the progression of amyotrophic lateral sclerosis (ALS) (Fornai et al., 2008), many ALS patients obtained off-label prescriptions

Suggested Citation:"7 Improving the Evidence Base for Real-World Use." National Academies of Sciences, Engineering, and Medicine. 2016. Neuroscience Trials of the Future: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23502.
×

and began taking the drug. As NINDS and partners were conducting a multicenter, double-blind, placebo controlled trial, PatientsLikeMe and its ALS patient community using lithium off-label found that there was no effect on disease progression (Wicks et al., 2011). Consistent with its conclusion, NINDS halted its trial early based on futility (Aggarwal et al., 2010).

FINAL REMARKS

At the start of the workshop, Romano argued that the field was reaching an inflection point. According to Romano, “Neuroscience basic research is exploding. The knowledge is exploding for clinical biology and genetics. Human experimental biology platforms are being refined and incorporated into development paradigms, which will help us going forward. Clarification of functional domains and relevant neurocircuitry should allow for more effective de-risking in the early phase of development. Enhancements in clinical trial methodology and trial execution should increase the probability of success in late-phase development.” Throughout the workshop, several participants highlight potential near- and long-term opportunities for clinical trial improvement for nervous system disorders. According to Thomas Laughren and a few other participants, advances in methodology continues to progress offering novel designs, assessments, recruitment tools, among others, to be used in the near term. Stephen Brannan noted that collaborative efforts are increasing and the field may begin to see more ADNI-like initiatives and public-private partnerships to address complex problems (e.g., biomarker identification and validation). Petra Kaufmann added that part of these collaborations include increasing data sharing which might improve the efficiency of trials and decrease duplicative efforts.

Many workshop participants highlighted the increased focus on patients and real-life outcomes in neuroscience trials of the future. Enabling technologies discussed include mobile health, remote monitoring, and wearable devices as well as other technologies that have yet to be developed, said Atul Pande. A few participants noted the need for increased interaction and communication between pharmaceutical and technology companies to better understand the application and use of these technologies in clinical trials. Given the fact that disruptive innovation requires a process of change, several participants recognized that this will be an ongoing process and will not happen overnight. Opportunities that are

Suggested Citation:"7 Improving the Evidence Base for Real-World Use." National Academies of Sciences, Engineering, and Medicine. 2016. Neuroscience Trials of the Future: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23502.
×

further down in the pipeline, according to some participants, include understanding how to capitalize on biomarkers as a way of teasing out subgroups within targeted populations. In addition, understanding how new technologies might be used for trialists to recruit participants more globally.

Highlighting the collective effort that is needed to address the challenges in the field, Pande stated that in order to see the ideas discussed at the workshop come to fruition, someone is going to have to take the first step. Quoting Kurt Vonnegut (2013), Pande ended by saying, “Sometimes we just have to jump off a cliff and grow our wings on the way down.”

Suggested Citation:"7 Improving the Evidence Base for Real-World Use." National Academies of Sciences, Engineering, and Medicine. 2016. Neuroscience Trials of the Future: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23502.
×

This page intentionally left blank.

Suggested Citation:"7 Improving the Evidence Base for Real-World Use." National Academies of Sciences, Engineering, and Medicine. 2016. Neuroscience Trials of the Future: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23502.
×
Page 59
Suggested Citation:"7 Improving the Evidence Base for Real-World Use." National Academies of Sciences, Engineering, and Medicine. 2016. Neuroscience Trials of the Future: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23502.
×
Page 60
Suggested Citation:"7 Improving the Evidence Base for Real-World Use." National Academies of Sciences, Engineering, and Medicine. 2016. Neuroscience Trials of the Future: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23502.
×
Page 61
Suggested Citation:"7 Improving the Evidence Base for Real-World Use." National Academies of Sciences, Engineering, and Medicine. 2016. Neuroscience Trials of the Future: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23502.
×
Page 62
Suggested Citation:"7 Improving the Evidence Base for Real-World Use." National Academies of Sciences, Engineering, and Medicine. 2016. Neuroscience Trials of the Future: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23502.
×
Page 63
Suggested Citation:"7 Improving the Evidence Base for Real-World Use." National Academies of Sciences, Engineering, and Medicine. 2016. Neuroscience Trials of the Future: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23502.
×
Page 64
Suggested Citation:"7 Improving the Evidence Base for Real-World Use." National Academies of Sciences, Engineering, and Medicine. 2016. Neuroscience Trials of the Future: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23502.
×
Page 65
Suggested Citation:"7 Improving the Evidence Base for Real-World Use." National Academies of Sciences, Engineering, and Medicine. 2016. Neuroscience Trials of the Future: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23502.
×
Page 66
Suggested Citation:"7 Improving the Evidence Base for Real-World Use." National Academies of Sciences, Engineering, and Medicine. 2016. Neuroscience Trials of the Future: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23502.
×
Page 67
Suggested Citation:"7 Improving the Evidence Base for Real-World Use." National Academies of Sciences, Engineering, and Medicine. 2016. Neuroscience Trials of the Future: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23502.
×
Page 68
Next: Appendix A: References »
Neuroscience Trials of the Future: Proceedings of a Workshop Get This Book
×
Buy Paperback | $46.00 Buy Ebook | $36.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

On March 3-4, 2016, the National Academies of Sciences, Engineering, and Medicine’s Forum on Neuroscience and Nervous System Disorders held a workshop in Washington, DC, bringing together key stakeholders to discuss opportunities for improving the integrity, efficiency, and validity of clinical trials for nervous system disorders. Participants in the workshop represented a range of diverse perspectives, including individuals not normally associated with traditional clinical trials. The purpose of this workshop was to generate discussion about not only what is feasible now, but what may be possible with the implementation of cutting-edge technologies in the future.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!