National Academies Press: OpenBook
« Previous: 9 Neuroimaging Data
Suggested Citation:"10 Real-World Data." National Academies of Sciences, Engineering, and Medicine. 2020. Neuroscience Data in the Cloud: Opportunities and Challenges: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25653.
×
Page 59
Suggested Citation:"10 Real-World Data." National Academies of Sciences, Engineering, and Medicine. 2020. Neuroscience Data in the Cloud: Opportunities and Challenges: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25653.
×
Page 60
Suggested Citation:"10 Real-World Data." National Academies of Sciences, Engineering, and Medicine. 2020. Neuroscience Data in the Cloud: Opportunities and Challenges: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25653.
×
Page 61
Suggested Citation:"10 Real-World Data." National Academies of Sciences, Engineering, and Medicine. 2020. Neuroscience Data in the Cloud: Opportunities and Challenges: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25653.
×
Page 62
Suggested Citation:"10 Real-World Data." National Academies of Sciences, Engineering, and Medicine. 2020. Neuroscience Data in the Cloud: Opportunities and Challenges: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25653.
×
Page 63
Suggested Citation:"10 Real-World Data." National Academies of Sciences, Engineering, and Medicine. 2020. Neuroscience Data in the Cloud: Opportunities and Challenges: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25653.
×
Page 64

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

10 Real-World Data Highlightsa • Real-world data (RWD) from electronic health records, claims and billing activities, registries, digital devices, and patients are playing an increasingly important role in clinical care, clini- cal trials, and regulatory and reimbursement decision making (Sutherland). • Neurodegenerative diseases like Parkinson’s disease could potentially be understood by leveraging RWD to learn more about the patient experience, natural history, and disease pro- gression (Sutherland, Wahl). • Assigning a global unique identifier to each participant in a study allows de-identified data to be integrated across multiple databases and studies (Sutherland). • The quality and validity of RWD is variable; to collect higher quality RWD, incentives will be needed for clinicians (Hill, Marinshaw, Sutherland). • Capturing the context and provenance of how RWD were g ­enerated is necessary to interpret outcomes (Hill, Horgan, Wahl). • Validated diagnostic standards are needed for mental health conditions where objective standards are not currently estab- lished to make RWD maximally useful (Di Martino). 59 PREPUBLICATION COPY—Uncorrected Proofs

60 NEUROSCIENCE DATA IN THE CLOUD • Large cloud-based databases of neuroscience data can be use- ful for testing hypotheses, but the size of the datasets makes accounting for confounding variables difficult (Di Martino). a These points were made by the individual workshop participants identified above. They are not intended to reflect a consensus among workshop participants. Real-world data (RWD), as defined by FDA, are any data related to patient health status and/or the delivery of health care routinely collected from a variety of sources, [including] electronic medical records (EMRs), claims and billing activities, product and disease registries, patient-generated ­ data including in home-use settings, or data gathered from other sources that can inform on health status, such as mobile devices.1 RWD and real-world evidence (RWE) are playing an increasingly important role in clinical practice, clinical trials, and regulatory and reim- bursement decision making, said Margaret Sutherland, program manager of science at the Chan Zuckerberg Initiative and former program direc- tor at NINDS. As digital devices and other new modes of data collection are implemented, the amounts of data are going to be very large and will require either a huge server bank or cloud support systems to handle the volume, said Sutherland. RWD may be useful for hypothesis testing and identifying questions that need to be answered, said Adriana Di Martino. Stuart Hoffman noted that smaller studies can help investigators generate hypotheses, but test- ing them may require large populations, large cloud-based databases, and cloud-based analytics. CURRENT PROMISING PRACTICES FOR MANAGING REAL-WORLD DATA IN THE CLOUD Sutherland described how RWD were collected and stored for a study called Assessing Tele-Health Outcomes in Multiyear Extensions of PD Trials (AT-HOME PD).2 AT-HOME PD is an ancillary study to two NINDS-funded Phase 3 clinical trials in PD patients—STEADY-PD III, a trial of Isradipine in recently diagnosed PD patients, and SURE PD3, a trial of Inosine. Participants in those trials had consented to be recontacted 1  To learn more about FDA’s definitions of real-world data, see https://www.fda.gov/science- research/science-and-research-special-topics/real-world-evidence (accessed November 7, 2019). 2  For more information about the AT-HOME PD study, see https://clinicaltrials.gov/ct2/ show/NCT03538262 (accessed November 7, 2019). PREPUBLICATION COPY—Uncorrected Proofs

REAL-WORLD DATA 61 by study coordinators for ancillary studies. Those who agreed to take part in AT-HOME PD were then reconsented for raw data sharing using cloud infrastructure. Data they agreed to share included that based on clinical telemedicine visits; data collected on the mPower platform created by Sage Bionetworks3; and participant-entered data collected through The Michael J. Fox Foundation’s Fox Insight.4 The goal of AT-HOME PD, according to Sutherland, is to link i ­ndividual-level data from different data streams to create a better picture of the patient experience of living with PD across multiple data modalities. To make these data work together through cloud computing, they established a global unique identifier (GUID) for each participant, said Sutherland. The GUID has been adopted by several NIH institutes for use in clinical studies, she said. It is a computer-generated code based on personally identifiable information such as name, date of birth, etc., that allows investigators to strip data of personal information before submitting the de-identified infor- mation to the database. The GUID allows de-identified data to be integrated across multiple databases and studies. ISSUES TO BE RESOLVED TO INCORPORATE REAL-WORLD DATA INTO CLINICAL STUDIES The quality of RWD, its reproducibility, robustness, and usefulness must be established, which may require validating the various data sources to confirm what they contribute to the overall picture of the patient going forward, said Sutherland. Obtaining quality EMR data, for example, is challenging in part because clinicians may see EMR as interfering with their interactions with patients and do not find EMR data useful, said Sean Hill. Incentives are needed to encourage clinicians to collect higher quality data, he said. He and his colleagues have developed a visualization chart intended to build quality into the medical health record by integrating measurements that are valuable for clinicians and giving them a quick way to see where data are missing, inaccurate, or irrelevant. EMR data collected using this approach, deidentified using a GUID model, can also be entered into a research database. Hill said they are also planning to integrate decision support tools into this model. Datasets may also have substantial differences in terms of how much data were recorded, said Peter Wahl, senior director of clinical research at Optum Analytics. In a study that tried to leverage EMR data in a health 3  For more information about mPower, see https://sagebionetworks.org/research-projects/ mpower-researcher-portal (accessed November 12, 2019). 4  For more information about Fox Insight, see https://www.michaeljfox.org/fox-insight (ac- cessed November 12, 2019). PREPUBLICATION COPY—Uncorrected Proofs

62 NEUROSCIENCE DATA IN THE CLOUD information exchange comprising multiple institutions, he and his col- leagues found that only 5 percent of the population had blood pressure recorded. He found that only nurses involved in specific research projects had gone to the trouble of entering this information in the records, raising questions about the usefulness of the dataset. Ferguson suggested one pos- sible solution to the problem of incomplete data entry: having scribes whose sole job is to follow a physician around and record data. Although this might be a very expensive solution, it could also reduce physician burnout, said Wahl. Jane Roskams raised the possibility of going back and looking retrospectively at EMR data. Indeed, said Sutherland, this might be a way to identify early signs of disorders such as PD, where the prodromal state is currently not well characterized. However, Wahl noted that older data tend to be of lower quality. Moreover, structured data may provide an incomplete assessment of patient state in part because these data are codified using ontological con- structs that may be severely limiting, said Wahl. For example, there is often no way to quantify a status change from one stage of disease to another, he said. He advocated augmenting and validating structured data with terms and concepts extracted from free text notes in the EMR. Another reason to use RWD data to understand the natural history of a disease is that many disorders are misdiagnosed or underdiagnosed, said Wahl. While symptomatology is an important component of how a neurodegenerative disease, its subtypes, and its progression are defined, symptoms reported by undiagnosed or misdiagnosed patients may not be captured in a disease-defined dataset. EMR data may also fail to capture patient-reported information, which is increasingly important for inform- ing regulatory approval and labeling, and for postmarketing surveillance, said Wahl. Incentivizing clinicians to participate in real-world studies will require understanding what brings value to clinicians, said Sutherland. Hill noted that providing physicians with digital tools may allow them to spend more time with patients and less time entering data into the EMR as they try to decipher the trajectory of the patient’s illness. Ruth Marinshaw noted that in developing tools such as this it is important to partner with the users of those tools as well as the data providers to ensure that the interface and approach bring value to all of them. She called this a “carrot-flavored stick” because it brings them value, but also requires them to put their data into the EMR system. Capturing the context and provenance of how RWD were generated is equally important, said Hill. Was the parameter measured during the day or at night? Was the participant well rested or sleepy? Was he or she tak- ing medications? Having patients record data or sharing data with patients may, however, alter behaviors and outcomes, noted Wahl. For example, PREPUBLICATION COPY—Uncorrected Proofs

REAL-WORLD DATA 63 Horgan noted that in the context of sleep, patients wearing devices to measure sleep parameters may become so obsessed about their sleep that their sleep is disrupted. While acknowledging the potential of large cloud-based databases to test hypotheses, Di Martino noted that the scale of these databases may make it difficult to clean up all the confounding variables. She also noted that in many developmental and psychiatric conditions, there are no objec- tive diagnostic standards and suggested that to make RWD data useful, diagnostic standards and validation of those standards are needed. PREPUBLICATION COPY—Uncorrected Proofs

PREPUBLICATION COPY—Uncorrected Proofs

Next: 11 Future Directions »
Neuroscience Data in the Cloud: Opportunities and Challenges: Proceedings of a Workshop Get This Book
×
Buy Paperback | $45.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The cloud model of data sharing has led to a vast increase in the quantity and complexity of data and expanded access to these data, which has attracted many more researchers, enabled multi-national neuroscience collaborations, and facilitated the development of many new tools. Yet, the cloud model has also produced new challenges related to data storage, organization, and protection. Merely switching the technical infrastructure from local repositories to cloud repositories is not enough to optimize data use.

To explore the burgeoning use of cloud computing in neuroscience, the National Academies Forum on Neuroscience and Nervous System Disorders hosted a workshop on September 24, 2019. A broad range of stakeholders involved in cloud-based neuroscience initiatives and research explored the use of cloud technology to advance neuroscience research and shared approaches to address current barriers. This publication summarizes the presentation and discussion of the workshop.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!