Examining the Impact of Real-World Evidence on Medical Product Development Proceedings of a Workshop Series (2019) / Chapter Skim
Currently Skimming:
6 When Is a Real-World Data Element Fit for Assessment of Eligibility, Treatment Exposure, or Outcomes?
6 When Is a Real-World Data Element Fit for Assessment of Eligibility, Treatment Exposure, or Outcomes?
Pages 71-98
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 71... ...
(Altan, Ball, Yaist) • Patient-generated health data can be collected nearly continu ously, come from many sources, answer research questions that were not previously answerable, and potentially facilitate access and participation from otherwise unrepresented patient populations.
|
From page 72... ...
During the second workshop, individual participants suggested sets of questions organized by topic based on the workshop's three sessions. These questions, the discussions at the second workshop, and additional work by several individual workshop participants between the second and third workshops informed further refinement of the questions into several "decision aids." The decision aids were discussed at the third workshop and were intended to prompt discussion among the participants and inform them and potentially other stakeholders about topics in study design.
|
From page 73... ...
Primary data collection observational study Hybrid data (secondary use Patient/disease state & primary data Real registries collection) World Data Public health surveys Patient generated health data Patient & provider surveys Social media Pragmatic trials FIGURE 6-1 Possible sources of real-world data.
|
From page 74... ...
A 2014 metaanalysis showed that all four trials favored NOACs over warfarin for the risk of stroke and systemic embolic events, as well as secondary outcomes, such as ischemic stroke, hemorrhagic stroke, myocardial infarction (MI) , and all-cause mortality (Ruff et al., 2014)
|
From page 75... ...
. Friends of Cancer Research Pilot Project At the third workshop, Jeff Allen, president and chief executive officer of Friends of Cancer Research, talked about a pilot project that investigated the performance of real-world endpoints among patients with advanced non-small cell lung cancer (aNSCLC)
|
From page 76... ...
Using data for patients on any of three different immune checkpoint inhibitors, the investigators compared the real-world overall survival rates with the ranges that had been observed in pivotal clinical trials for each drug. The overall survival rates from the databases were generally in line with the rates from the clinical trials, said Allen (see Figure 6-2)
|
From page 77... ...
SOURCE: Allen presentation, July 17, 2018. TABLE 6-2 Correlation Between Real-World Overall Survival and RealWorld Extracted Endpoints rwOS Versus rwTTNT rwOS Versus rwTTD Correlation Correlation Data Set N [95% CI]
|
From page 78... ...
However, more research is needed to determine whether the endpoints could be reliable surrogates for OS, and whether these endpoints could support decision making by regulators and payers. Finally, the overall survival rates assessed from EHR and claims data were quite consistent with the rates observed in clinical trials, he said, suggesting a need for additional research on the association between data from real-world sources and data from clinical trials.
|
From page 79... ...
A probabilistic approach would help to better capture the fuzziness of medical conditions, but could also be challenging for researchers and regulators to understand. Exposure, said Berlin, can also be difficult to determine using RWD.
|
From page 80... ...
embedded prompts in the EHR to improve data collection, said Simon. While much of the attention around data is often on assessing data quality once they are collected, said one workshop participant, the start
|
From page 81... ...
Joanne Waldstreicher noted that there is empirical literature about expert adjudication and when it makes a difference. Safety Issues In clinical trials, said Simon, adverse safety events can be detected because trial participants' baseline health status is measured before the trial begins, so any adverse events that occur after the exposure may be attributable to the intervention.
|
From page 82... ...
However, random misclassification may also result in missing safety events, which could "lead to conclusions that would damage the public's health or be unsafe." DECISION AID The general issues discussed by individual workshop participants in the first and second workshops were used to identify topics that could benefit from further exploration in the third workshop. Draft "decision aids" were developed by some individual workshop series participants on discrete aspects of study design to organize the topics that could benefit from further exploration and to facilitate deeper discussions at the workshop.
|
From page 83... ...
Yaist said the first step is to see if there are already validated ways to get information; for example, major adverse cardiac events in claims data have been extensively studied. Next, the researcher would look to see where the needed data elements could be found -- are there existing data sources, either from clinical care or from patients?
|
From page 84... ...
OF E LIGIBILITY , T REATMENT E XPOSURE , OR O UTCOMES ? When is an RWD element generated in real-world When is an RWD element generated outside of a clinical When is an for assessmentgenerated in real-world practice fit RWD element of eligibility, treatment When is an fit for element generated outside of a clinical setting RWD assessment of eligibility, treatment practice fit for assessmentoutcomes?
|
From page 85... ...
element transparent and traceable? FIGURE 6-3 Continued BOX 6-1 Feedback on the Decision Aid as Discussed by Individual Workshop Participants Several workshop participants offered specific feedback on the decision aid "When Is a Real-World Data Element Fit for Assessment of Eligibility, Treatment Exposure, or Outcomes?
|
From page 86... ...
data PATIENT GENERATED "TRADITIONAL" HYBRID collection HEALTH DATA RWD FIGURE 6-4 Decision tree to identify possible data sources. NOTE: EHR = electronic health record; ER = emergency room; RWD = real-world data.
|
From page 87... ...
FIGURE 6-5 Linked, de-identified data sources at OptumLabs. NOTE: CDHP = consumer directed health plan; EHR = electronic health record; SSA = Social Security Administration.
|
From page 88... ...
Common sources of bias may include missing data, a lack of representativeness of the sample, and the "present patient bias," where the data only reflect patients who presented for care, but not those who did not seek care. In addition, there may be other biases such as differing policies or care practices.
|
From page 89... ...
Regardless of the source of the difference, a discrepancy like this indicates there are missing data in the dataset, and that the calculated average for aspirin use may not be accurate. Robert Ball, deputy director of the Office of Surveillance and Epidemiology at FDA's CDER, noted that even as RWD sources and the tools for using them improve, it is likely that there will continue to be issues with missing data.
|
From page 90... ...
TABLE 6-3 Patient-Generated Health Data for a Study of Multiple Sclerosis (MS) Activity Trackers Only MS Trackers Matched Control Trackers N 498 1,400 Percent of Days with Tracked Steps*
|
From page 91... ...
In fact, he worked with the Clinical Trials Transformation Initiative (CTTI) on its recently released guidance about how to use mobile devices for data FIGURE 6-7 Effect of medical procedure or surgery on resting heart rate.
|
From page 92... ...
capture in clinical trials.1 Foschini said the findings and recommendations of the CTTI report, as well as from the Duke-Margolis Center for Health Policy work on characterizing RWD quality and relevancy,2 will be relevant for many sources and uses of PGHD. Other workshop participants discussed the potential of PGHD when linked with additional data sources (see Box 6-2)
|
From page 93... ...
DISCUSSION: REAL-WORLD DATA CONCERNS FOR FUTURE RESEARCH During the second and third workshops, participants engaged in discussions that reflected on specific questions around the use of RWD (some of which were listed in the draft decision aid) , as well as exploring related or new topics that emerged.
|
From page 94... ...
. BOX 6-3 Potential Sources of Bias in Real-World Data as Discussed by Individual Workshop Participants Individual workshop participants identified a number of potential biases in real-world data during the discussion, including • Present patient bias.
|
From page 95... ...
Another way in which RWD can change over time, said Altan, is through changes to practice patterns for particular diseases. In a longitudinal study that may last multiple decades, the data about clinical care and clinical outcomes can change substantially as there are regulatory changes, formulary changes, or changes to standard of care.
|
From page 96... ...
Transparency about the transformation from the source data to the analytic dataset is essential, he said, particularly in cases where the diagnosis is less straightforward (e.g., different types of depression versus myocardial infarction)
|
From page 97... ...
are useful and cover many of the key topics, FDA would need the questions to be systematized into an industrialized process. Ball noted that the current system for assessing and using RWD is quite resource intensive.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.