3

Opportunities for Real-World Data

Following the identification of stakeholder needs in the first workshop session, the second session was focused on answering a framing question: What can we learn from real-world data? The growing availability of rich clinical data provides opportunities to address a broad range of real-world questions on effectiveness and value. However, as noted by Simon, concerns regarding the quality of clinical data have impeded efforts to incorporate real-world data into the traditional clinical research paradigm. Panelists in this session discussed opportunities to leverage the “data exhaust” from clinical practice (e.g., data captured in electronic health records [EHRs] and claims and pharmacy databases during the course of clinical care) and mechanisms to overcome the challenges that arise when applying those data for the secondary purpose of research. The panelists also discussed the potential of data streams originating from mobile devices and other digital health technologies that capture data outside the clinical setting. Setting the tone for the session’s discussions, Marc Berger, vice president, Real-World Data and Analytics, Pfizer Inc., stressed that it is “not a question about [whether] [these] real-world data [are] good enough. It’s about how . . . we move to a learning health care system and use the data . . . for an appropriate purpose that drives us to where we want to get.” He reminded the audience that “real-world evidence is good evidence and people are using it every day to make decisions.”

LEVERAGING ELECTRONIC HEALTH RECORDS

The promise of a learning health system is dependent on the ability to digitally capture, aggregate, and analyze health data for research and quality improvement purposes. Over the past 15 years, significant progress has been made toward the vision set out in the 2001 Institute of Medicine report Crossing the Quality Chasm (IOM, 2001), which underscored the importance of a robust health information technology infrastructure, observed Jon White, deputy national coordinator for health information technology, Office of the National Coordinator for Health Information Technology (ONC). In 2015, 96 percent of hospitals and 78 percent of office-based physicians used certified EHR technology. Califf emphasized that it is important to take advantage of this infrastructure to move the evidence generation system to a much more efficient model and to answer questions that are critical for people to make the right decisions about their health and health care.

Several workshop participants discussed barriers that arise when using EHRs for research. Andrew Roddam, vice president and head of Real-World Evidence, GlaxoSmithKline, noted that EHRs might not contain all of the data that researchers want, so it is important to consider whether the EHR can be expanded to become the repository of all desired information

or, instead, to use what is there and then collect the missing information using simple data collection tools.

Other challenges noted by individual workshop participants included the following:

• missing data
• need for computable phenotypes
• lack of standardization (e.g., data schemes and data transfer protocols)
• interoperability issues with proprietary health information systems

Addressing the limitations of EHRs, Califf asked, “How much energy do you spend on the upfront regimentation of data collection versus curating data on the back end?” Several workshop participants noted a need for balance. Good evidence can come from back-end curation, although it may not be perfect, replied Sherman. This can help demonstrate the value of those data for other purposes, which can help drive improved data quality and collection for secondary use. Vallance suggested that some effort to improve quality of data entry on the front end is needed to improve back-end curation, citing as an example a study that found 120 different definitions of myocardial infarction. Califf observed that changing reimbursement practices may incentivize entry of more accurate data by providers, who will increasingly require such data to demonstrate the quality and value of care they are delivering.

The promise of EHRs inspires excitement, but also frustration, about the technology’s unfulfilled potential. White noted that providers report to ONC not that they want to return to paper-based records systems, but that EHR systems need to work better for them. ONC is actively working on many of the barriers that are frequently noted, he said, including lack of standards and interoperability issues. Certified EHR technology is now required for participation in the Medicare incentive program and the newly released quality payment program, and in October 2015, ONC released the final version of its interoperability roadmap, Connecting Health and Care for the Nation: A Shared Nationwide Interoperability Roadmap Version 1.0 (ONC, 2015). The private sector is also advancing opportunities to leverage EHR data for quality improvement and research, said Berger, citing as an example Pfizer’s use of natural language processing to create very rich datasets by mining the wealth of EHR data residing in free text notes.

THE POWER OF LINKING AND MINING DISPARATE DATA SOURCES

The linking of multiple datasets provides a richness of data that cannot be achieved with any single data source. Combining EHR data with claims and pharmacy data, for example, captures a more complete picture of the continuity of care for a patient and a record of that person’s interaction with the health care system, said David Dore, vice president, Epidemiology, and principal epidemiologist, Optum Life Sciences. Linking in data from other sources also may help to address data-quality issues by filling in missing data and validating data through checks for consistency across data sources. However, Califf pointed out, inconsistencies are not always an indicator of bad data. Inconsistencies may be real, reflecting different perceptions of different providers or variability in lab testing results, and may only be detectable by comparing across datasets. For example, it is possible to identify a patient population prescribed a particular drug using EHR data while claims data show that a certain percentage of those patients never filled the prescription. This has significant implications for any safety or effectiveness analyses conducted on those data and is a question that can only be answered with linked EHR and claims datasets, said Berger. The datasets that need to be linked depend on the question that must be answered, added Dore. One data source may be better at capturing certain data, but may miss others. This is why understanding the inherent biases of different datasets is important, cautioned Luca Foschini, co-founder and chief data scientist, Evidation Health. For example, claims datasets often tend to be more complete because payment serves as the incentive to enter data, but claims data have their own biases—for example, more expensive things are more likely to be captured there.

Patient-level linking of datasets remains a challenge when there are no unique patient identifiers. Although this is an area of significant interest and some work has been done in the private sector, federal efforts to implement unique patient identifiers are currently prohibited by law,¹ explained White. Other efforts to facilitate data linking and aggregation include the use of claims data to link patient records across EHR systems and the development of common data models, which map concepts from different data sources into a common format with common definitions. As discussed by two panelists in this workshop session, these methods have enabled the

___________________

¹ See Sec. 510, Consolidated Appropriations Act, 2016. Public Law 113, 114th Cong. (December 18, 2015): “None of the funds made available in this Act may be used to promulgate or adopt any final standard under section 1173(b) of the Social Security Act providing for, or providing for the assignment of, a unique health identifier for an individual (except in an individual’s capacity as an employer or a health care provider), until legislation is enacted specifically approving the standard.”

development of large linked datasets to support both public-sector research and private-sector analyses.

PCORnet’s Clinical Data Research Networks

The Patient-Centered Outcomes Research Institute (PCORI) supports health-related decision making by patients, providers, payers, and policy makers by generating and examining evidence on the effectiveness of various medical treatments. Russell Rothman, director, Center for Health Services Research, Vanderbilt University, described how the National Patient-Centered Clinical Research Network (PCORnet), funded by PCORI, is advancing real-world evidence research by leveraging existing electronic health data sources to support national comparative effectiveness studies and pragmatic clinical trials. In addition to its 20 patient-powered research networks, PCORnet consists of 13 clinical data research networks (CDRNs) representing more than 100 health care systems and organizations across the country. PCORnet currently has EHR data from more than 110 million patients, and CDRNs are also working to link EHR data to data from other sources, including claims, vital statistics, registries, state health data, Medicare and Medicaid, and private health plans, in an effort to capture a more complete picture of patients for research purposes.

Because it incorporates standardized data from different sources using a common data model, the PCORnet infrastructure can now be used to identify potentially thousands of patients across the networks with particular conditions, to conduct observational studies that follow patient cohorts over time, and for interventional clinical research, including comparative effectiveness trials. Rothman also described tools that have been developed for PCORnet to support clinical trials, including electronic processes for patient identification and recruitment, consenting, and collecting patient-reported outcomes. These tools, along with some administrative simplification, have enabled the conduct of large pragmatic trials with great efficiency, he said. Rothman cited the ADAPTABLE trial on optimal aspirin dosing for patients with coronary heart disease as an example of the potential of the PCORnet infrastructure for conducting faster, cheaper, and more informative clinical research in the real-world space. In this pragmatic trial, which is still ongoing, patients were identified, recruited, and consented electronically and randomized to baby or regular strength aspirin. Data for follow-up were captured from EHRs and claims, and from patients directly using electronic survey tools. “The front door for PCORnet is now open,” said Rothman, for investigators interested in running queries or using the network for observational or interventional research.

Development and Use of Centralized Data Repositories in the Private Sector

In the private sector, efforts to aggregate and analyze data from EHRs, claims, and other sources are driven, in part, by demands from provider networks as they try to control financial risks for managing patient populations. Dore outlined how Optum, part of UnitedHealth Group, compiles data from electronic records (including medical, claims, and pharmacy records) for provider networks into a centralized repository. Data are linked using encrypted data linkage methods so that patient-identifying information is not shared across parties. To address interoperability issues across different record systems within provider networks, Optum uses an intensive manual process to extract information; validates, maps, and normalizes it; and iterates it to get to a standardized data format. Following a series of data quality checks at the end of the process, the company has generated a centralized repository containing data for those patients within a particular provider network. That repository can then be used for a range of analytics, including predictive modeling, quality benchmarking, and risk stratification (e.g., identifying patients who have high risk of rehospitalization). This process can be scaled up so that data from many provider networks are aggregated under a single ontology, capturing more than 70 million patients in a single centralized dataset (see Figure 3-1). Dore said Optum is

in the process of onboarding other data, including those from clinical trials, registries, and wearables. He emphasized that, beyond supporting clinical decision making for provider networks, these data repositories also have value for clinical research and have been used for observational studies evaluating comparative effectiveness.

COLLECTING REAL-WORLD DATA OUTSIDE THE CLINICAL SETTING USING DIGITAL HEALTH TOOLS

Speaking on the opportunities to engage and collect real-world data directly from patients and consumers outside of the clinical setting, Foschini described the tremendous recent growth of digital health technology in the consumer space. Not only has there been a proliferation of devices on the market, but the measurement capability of these devices is also expanding. Collectively, he estimated, wearables and other consumer devices can now measure physiological parameters at a level that is approaching what might be seen in a hospital intensive care unit.

Because many mobile health devices are commonly worn throughout the day and sometimes even during sleep, excitement regarding their potential stems from the ability to capture data from the 99 percent of patient and consumer activity that occurs outside the health care setting. This allows researchers to track the progression of an individual over time at a much finer level of resolution than ever before. Although these devices can be used to compare pre- and postevent or intervention data at the individual level, it is also possible to develop population-level outcome measures. Foschini cited as an example the measurement of recovery of mobility following surgery. Using data from a mobile health device, it is possible to calculate a mobility index and compare postsurgery levels to baseline to determine the time to recovery of full mobility following surgery. With population-level data, an outcome of interest may be the time it takes for an individual who received the surgery to return to 90 percent of his or her presurgery mobility level; in addition, the impact of variables such as age on the outcome measure can be examined to identify individuals at higher risk of not regaining full mobility.

In the context of clinical trials, the broad, consumer-driven distribution of digital health devices across large and diverse populations has important implications for trial design. For example, said Foschini, these devices can enable virtual study recruitment, which has the potential to increase the efficiency of clinical trials and reach subpopulations that might not be reached through traditional recruitment practices. When considering their use for data collection, however, Foschini emphasized that investigators should remember that these devices will be used in unsupervised settings

and may therefore necessitate a more user-oriented approach than is typical in traditional trial design.

Although there is a great deal of interest in the emerging potential of digital health tools, as demonstrated by an exponential increase in the number of publications featuring analyses of data collected using these devices, a number of workshop participants raised questions about the reliability of data collected using these tools, both in terms of their accuracy and their ability to engage consumers over the long term. A lot of scientific work is needed to validate results from wearables and define wearable-oriented endpoints that will support regulatory approval, cautioned John Hernandez, head of Health Economics, Value, and Access, Verily Life Sciences.

CONSIDERATIONS FOR REALIZING THE POTENTIAL OF REAL-WORLD DATA

The ability to use real-world data to answer research questions regarding effectiveness and value is contingent on access to the full spectrum of health data and capability to transform the data into evidence using analytic tools. In discussions on realizing the potential of real-world data, two key themes emerged: partnering with patients and consumers, and investing in data science capabilities.

Partnering with Patients and the Public

A number of levers can be applied to realize the potential of real-world evidence, including certified EHR technology and regulations, but the fulcrum, said White, is patients and consumers, and specifically, their data and information. Several individual workshop participants commented that the research enterprise needs to do a better job of engaging those individuals as partners. Patients can be a source of important data not routinely collected for purposes of care—socioeconomic, cultural, and educational background factors—that significantly affect treatment outcomes. They can also help to link their own longitudinal care data (e.g., data from surgery and rehabilitation services), said Frankel, who suggested that proactively engaging patients and consumers to obtain such data needs to be part of a data strategy for any research study.

Several examples of patient engagement mechanisms were provided by workshop participants. Rothman described efforts at his institution to make it easy for patients to share their data and participate in research by offering research portals within patient portals. These portals can be used to upload information that could be used for research purposes or to enable patients to sign up to participate in research studies. Robinson Beale highlighted the success of PatientsLikeMe, a patient-powered effort to make data avail-

able for the purposes of finding similar patients and comparing outcomes of different treatments. More broadly, though, said Nigam Shah, associate professor of medicine, Stanford University, a culture of data sharing needs to be promoted to advance the public’s understanding that to benefit from a learning health system, patients need to contribute their data.

Investing in Data Science

Data are increasingly becoming an asset for health care providers, with incentives for leveraging “big data” coming from CMS and pay-for-performance opportunities. These drivers are also generating opportunities to apply big data to clinical research, but expertise is a key component to support the necessary aggregation and curation of data and analytics. Several individual workshop participants discussed the creation of a culture of data science within organizations and the importance of investment in data science experts to transform health care data into meaningful information. The health care industry is lagging behind others already adept at working with big data, like many of the dominant American corporations such as Amazon and Walmart, said Califf, who added that efforts are needed to recruit that talent into the health care industry.

This page intentionally left blank.