The workshop’s second session explored a variety of perspectives and experiences with virtual and digital health technologies in interventional and observational studies as well as in clinical care. It also highlighted potential opportunities to use digital health technologies to improve clinical trials of investigational products, the challenges of doing so, and best practices for designing and implementing a virtual clinical trial. Jenna Bollyky, vice president for clinical research and analytics at Livongo Health, and Joshua Denny, professor of biomedical informatics and medicine at the Vanderbilt University Medical Center, provided lessons learned from clinical care and observational studies, respectively. Insights on lessons learned from interventional virtual clinical trials were presented by Steven Cummings, director of the San Francisco Coordinating Center; Wendy Weber, acting deputy director at the National Institutes of Health’s (NIH’s) National Center for Complementary and Integrative Health; and Kimberly Hawkins, clinical sciences and operations project leader head at Sanofi Genzyme. The session ended with a panel discussion composed of Noah Craft, chief executive officer (CEO) of Science 37; Adrian Hernandez, vice dean for clinical research at the Duke University School of Medicine and faculty associate director of the Duke Clinical Research Institute; Jon White, deputy national coordinator for health information technology at The Office of the National Coordinator for Health Information Technology in the U.S. Department of Health and Human Services; and Josh Rose, vice president and global head of strategy for IQVIA. An open discussion, moderated by Kelly Simcox, head of the Americas, clinical study units, and clinical operations at Sanofi, followed the discussion.
Jenna Bollyky, Vice President for Clinical Research and Analytics, Livongo Health
Bringing Together Technology and Health Care Expertise
Launched in 2015, Livongo is a participant-centered digital health business that seeks to address the confusion, complexity, and cost of interacting with the health care system for those living with chronic diseases. Livongo initially focused on diabetes monitoring and prevention, but is now expanding its programs to address cardiovascular disease. Livongo has developed a platform, Applied Health Signals,1 which aggregates real-world patient health data from multiple sources, interprets these data using medical literature and clinical insights, applies data to solutions, and iterates until desired patient outcomes are achieved.
In addition to the data collected from digital health technologies, Livongo aggregates data on people’s health behavior, physical activity, and medical and pharmacy claims, with the aim of helping people lead healthy lives and spend less time worrying about their conditions. Livongo leverages a diverse team of data scientists, behavioral health specialists, certified diabetes educators, and physicians to interpret and apply these data in a way that is clinically meaningful to the patient and their treating physician.
Challenges of Managing Chronic Conditions Remotely
Bollyky described three challenges associated with remote management of chronic conditions: attribution of data, creation of meaningful data, and coordination of care. The first challenge involves ensuring that aggregate measurements are coming from the person being observed. As an example, Bollyky illustrated how a digital weight scale used by multiple people can create noisy data and make it difficult to determine which measurements should be attributed to the participant. This can be rectified by various approaches, such as by introducing capabilities in the digital weight scale that allow the participant to indicate when he or she is using it, or by collecting self-reported weight at registration from the participant and leveraging data science techniques to filter out obvious outlier data.
The second challenge Bollyky emphasized is the creation and provision of meaningful health signals to the participant in the context of the disease being monitored. Using Livongo’s glucose meter as an example, Bollyky discussed how this digital health technology can provide participants value in
terms of instant feedback on their blood glucose levels, measurement trends, and tailored recommendations based on those trends. For people living with diabetes, a very meaningful measurement is their glycated hemoglobin A1c (HbA1c) level.2 However, Livongo currently does not have the capability to take this measurement. As a result, it uses data science techniques to derive HbA1c values from collected blood glucose data and validates the reliability of estimated HbA1c values based on the medical literature (Ford et al., 2018). However, reliable estimates require that observed data have a high enough frequency to confer statistical power.
The third challenge Bollyky discussed is in regard to coordinating care for people with chronic conditions. While Livongo does not currently serve as a health care provider or write prescriptions for its members, it does have algorithms to identify when someone may not be on the right medication or the right dose of medication. Bollyky noted that Livongo’s goal is not to replace providers, but rather to support them. By giving its members health information, Livongo empowers them to talk to their providers. In addition to engaging providers, Livongo has ongoing relationships with pharmacy benefit managers and health plan managers, which enable more streamlined care coordination for participants.
Leveraging real-world evidence can offer health benefits in part, Bollyky observed, due to the Hawthorne Effect—the alteration of behavior by the subjects of a study due to their awareness of being observed (McCambridge et al., 2014)—which can pose complications in the research setting. Though using digital tools to support clinical trials may provide convenience, allow for more frequent assessments, and offer cost savings, it will be important to take into account how monitoring may impact the outcomes being measured.
Joshua Denny, Professor of Biomedical Informatics and Medicine, Vanderbilt University Medical Center
All of Us
2 Glycated hemoglobin A1c (HbA1c) is a gold standard measurement of long-term glycemic control. Similar to a blood glucose test, an HbA1c test measures how much glucose is bound to hemoglobin. This test can provide information on a person’s average level of blood sugar over the past 3 months (CDC, 2018; Leow, 2016).
Baseline5 as examples. The All of Us Research Program was launched by NIH nationally in May 2018 with the goal of enrolling at least 1 million diverse participants.6 Enrollment, he explained, occurs through two routes: (1) health care provider organizations, and (2) direct enrollment of volunteers. Currently, more than 200 sites enroll participants through health care providers or directly at numerous consumer health sites, such as Walgreens clinics, blood banks, Quest, EMSI,7 and QTC/Leidos.8 To ensure that similar information is collected for all individuals, All of Us uses a common interface for recruitment, a common process for consent (which includes sharing EHR data), and common modules for health surveys. A key part of All of Us, said Denny, is that participants will have access to their data and may be recontacted over time.
All of Us will aggregate data from a variety of sources, including EHRs, collected specimens, claims data, and data provided directly by participants. All of Us is piloting the Sync for Science mechanism,9 a national collaboration among EHR vendors, to receive medical records data directly from participants. To harmonize these disparate types of data, All of Us uses the observational medical outcomes partnership common data model (OHDSI, 2019).
Denny noted that All of Us will begin collecting genome-wide association study (GWAS) data on participants and eventually plans to add whole-genome sequencing data. The program also plans to pilot Fitbits, Apple Watches, and other wearables in addition to working to link participants’ geographical locations with data.
U.K. Biobank, an open-access, prospective study, recruited more than 500,000 men and women ages 40 to 69 from 2006 to 2010. Participants consent for a wide range of research and for long-term follow-up. U.K. Biobank takes extensive baseline assessments of its participants and links to them longitudinally through their health record.
U.K. Biobank currently has GWAS data on all participants and the results from a standard panel of biochemical assays, such as lipids and metabolites. Whole-exome sequencing for every participant is under way, as well as whole-genome sequencing and metabolomics assays. There are plans
6 At the time of the workshop, approximately 83,000 have completed all enrollment elements. Of those enrolled, more than 45 percent are non-white and more than 75 percent are considered to be from an underrepresented population in medical research.
for proteomic analyses, too. For a subset of participants, U.K. Biobank is collecting whole-body imaging data.
Project Baseline is a large-scale project being conducted by Verily, Duke University, Stanford University, and Google that will collect real-world and clinical data, as well as biospecimens and survey results from some 10,000 individuals. Project Baseline will also heavily leverage digital tools by providing participants with home sensors and wearables, Denny emphasized.
The Role of Cohorts in Supporting Clinical Research
The combination of data that emerges from cohorts, such as EHRs, survey data, and wearable data, can be leveraged to conduct a wide range of clinical research activities, including
- Phenotyping and genotyping research to identify new disease targets or enable pharmacogenomics discovery;
- Phenome-wide investigations to test for indications and adverse drug events; and
- Artificial intelligence research and clustering approaches to identify disease subtypes.
To illustrate the potential of cohorts in informing drug discovery, Denny provided examples of studies that used EHR and genomic data to identify new classes of drugs to treat patients with rheumatoid arthritis, validate adverse effects for an enzyme-metabolized drug, and identify a novel class of cholesterol-lowering drugs (see Box 3-1).
Large, diverse observational study cohorts, Denny continued, stand to accelerate clinical trials by generating basic discoveries, enabling direct and targeted recruitment of diverse populations and facilitating their assessment through existing cohort technology platforms, and enabling more intelligent trial design. The cohort landscape, Denny noted, is expanding and includes groups such as the China Kadoorie Biobank,10 the Electronic Medical Records and Genomics (eMERGE) Network,11 Estonian Genome Center,12 Kaiser Permanente’s Division of Research,13 Million Veterans Program,14
11 Available at https://www.genome.gov/27540473/electronic-medical-records-and-genomics-emerge-network (accessed April 16, 2019).
Steven Cummings, Director, San Francisco Coordinating Center
Cummings discussed what he termed D2P trials, which he defined as having no physical clinical sites, and thus no geographic limits on recruitment. Unlike the term “virtual” clinical trial, Cummings expressed that the term D2P trial more aptly captures the importance of building relationships with participants. Using examples of three D2P trials, Cummings provided lessons learned about the importance of incorporating the participant perspective and simplicity in trial design.
KALM, Cummings explained, was an Internet-based, randomized, placebo-controlled trial that examined whether two herbal products, kava and valerian root, were effective in helping individuals self-manage anxiety and insomnia, respectively (Jacobs et al., 2005). Launched by 1747, Inc., with venture capital investment from Lilly Ventures (then called e.Lilly Venture Fund) (Lilly, 2001), the paper-free trial was based out of one center in San Francisco. It used a participant-facing electronic data capture system, eConsent (use of multimedia on a digital platform to develop an interactive consent process), and a quiz to confirm that the participant understood the study (Grady et al., 2017). Kava and valerian root were sent via FedEx to participants with proof of identification confirmation. The trial design, Cummings continued, was simple, with fewer than 10 steps to enroll in the trial and be randomized.
Over 8 weeks, Cummings and his team screened 1,500 potential participants and randomized 391 participants from 45 states. While participant adherence was 83 percent, there were no differences in anxiety and sleep scores between those receiving the treatment or placebo. Because the trial was entirely electronic, Cummings’s team was able to analyze all of the data within 1 hour of the trial’s ending and return results of the trial to the participants within 1 day.
e.Lilly Venture’s upfront investment cost approximately $3,224 per participant. Due to the simplicity of conducting the trial (and relative cost-effectiveness), the parent company Eli Lilly adopted 1747’s technology for a trial of Cialis. However, it became increasingly complex and involved enrollment and consent at sites, resulting in the most expensive trial per
participant it had ever run to date, prompting Eli Lilly to abandon the trial model.
REMOTE was an Internet-based trial, launched by Pfizer Inc., designed to mimic a site-based trial for overactive bladder to test the efficacy of Detrol. REMOTE used similar identification verification and consent as in the KALM trial, was run from a single center, and used Web-based recruitment.
However, replicating a site-based trial was complex, largely attributed to protocol requirements and regulations that increased the burden of enrollment. For example, the protocol involved more than 90 interactions with participants to enroll and be randomized. Additionally, after signing eConsent, study staff were required to call the participant and read the full consent over the phone. Although REMOTE had received U.S. Food and Drug Administration (FDA) approval to ship the investigational product directly to participants’ homes, prescribing laws limited recruitment to nine states, some of which required physical examinations by a physician prior to dispensation. Furthermore, during the run-in period, participants were asked to carry a plastic container to measure urine volumes and enter this information into a digital health technology. If the participant made an entry error, there was no opportunity to correct it and that participant could be excluded from the trial.
Recruitment was also challenging, noted Cummings. While nearly 21,000 women viewed the online introduction to the trial, only 1,159 were deemed eligible to participate. Of those eligible, only 1.6 percent of those women, or 19 women, made it through enrollment and randomization (see Figure 3-1). Although the perception is that REMOTE failed because of recruitment issues, Cummings noted that if the protocol were simplified so that 25 percent of the interested and eligible women could have easily enrolled, the trial would have achieved its goal of having 283 women participate.
TOPAZ, funded by the National Institute on Aging in partnership with the Parkinson’s Foundation, is a planned trial that will start recruitment in 2019 and will test the efficacy of zoledronate19 to prevent fractures in Parkinson’s disease patients over age 65. TOPAZ aims to recruit 3,500
19 Zoledronate is an FDA-approved generic drug that increases bone density. Administered by intravenous infusion, one dose lasts for more than 2 years.
participants and, given this ambitious goal, will need to depend on nationwide recruitment unencumbered by sites. Therefore, the entire trial will be conducted from participants’ homes, making it easier for disabled or cognitively impaired individuals who may benefit most from participation.
There will be only three interactions for participants to enroll in the study. First, initial eConsent and eligibility screens will be collected online. Second, a teleneurology examination will be conducted to confirm the diagnosis of Parkinson’s disease in the participant. Third, a nurse will administer a finger stick at the participant’s home to assess kidney function, then provide either zoledronate or a placebo. Endpoints for the study are assessed by either surveying the participants’ EHRs, or following up by mail, email, or phone every 6 months to identify those who have had fractures.
Cummings said the methods for D2P trials are well established. Simplicity for the participant is essential, and recruitment from trusted communities and known providers may be more successful than solely Web-based recruitment. Such studies could reach participants in more states if prescribing laws were changed to allow for the shipment of study drugs without requiring physical examinations, said Cummings.
NIH’s Health Care Systems Research Collaboratory
Wendy Weber, Acting Deputy Director, National Center for Complementary and Integrative Health
The NIH Health Care Systems Research Collaboratory20 (The Collaboratory), explained Weber, aims “to strengthen the national capacity to implement cost-effective, large-scale research studies that engage health care delivery organizations as research partners.” In leveraging data already passively collected as part of the EHR, it will be important, Weber noted, to determine when it is possible to partner with health care systems to answer the questions on improving health. The Collaboratory refers to its approach as embedded pragmatic trials because they are set in the health care system based on how patients see their health care providers and receive care in the health care systems. The Collaboratory has completed nine trials involving sites across the country, initiated six trials in the spring and summer of 2018, and will fund an additional set of trials in September 2019.
The Collaboratory Coordinating Center assists the trials via working groups21 and releases lessons learned into The Collaboratory’s knowledge repository.22 Materials posted include how to design and conduct trials embedded in health systems and disseminate their results for diffusion into learning health systems, noted Weber. Using these lessons learned, The Collaboratory is able to help projects troubleshoot real issues as they arise, such as transitioning from International Statistical Classification of Diseases and Related Health Problems (ICD)-9 codes to ICD-10 codes in the middle of trials.
Leveraging the EHR
While the EHR provides a cost-effective resource of information collected during routine care, it will be important for investigators to consider what data are routinely captured to assess if those outcomes are useful for a study, noted Weber. Data collected consistently in the EHR include billed services, such as codes for procedures, hospital stays, medical visits, laboratory measures, and in some cases, medication fills. However, using EHRs poses its own challenges, such as blank data fields, a lack of information on services received at different clinics, and the inability to consistently capture patient-reported outcomes and adverse events. The biggest challenge, noted
21 The Collaboratory has five active working groups that focus on biostatistics and study design, electronic health records, health care systems interactions, patient-reported outcomes, and ethics and regulatory issues.
Weber, is that patient follow-up in a health care setting may not have the desired schedule of a clinical trial.
The principal investigators of the various Collaboratory trials have developed some solutions to these challenges. To fill in missing data in a pain study, for example, the project team augments data capture of the Brief Pain Inventory23 by emailing participants a link so they can complete the instrument themselves online. Those participants who did not respond to the email then received an automated call that enables them to enter in their pain scores using their touchtone phones. Failure to respond to the automated call triggers an in-person call to capture the data. The project team also bypassed the issue of missing data by conducting trend analysis rather than attempting to consistently collect data from all participants at the same time point.
Weber noted that every project associated with The Collaboratory first has to complete a year-long planning activity before starting the trial to demonstrate the ability to capture the needed data from the EHR, identify how much data will be missing, and assess the overall feasibility of conducting a trial using EHRs.
One lesson learned from The Collaboratory has been to expect the unexpected, said Weber. For example, while staff turnover is to be expected, the frequency of staff turnover can be a surprise. In some cases, systems launched new EHRs in the middle of a trial. Given that these are pragmatic trials, there is likely to be lower adherence to the interventions, so it is important to power a study for a smaller effect size. Systemic changes in health care systems can also create challenges, as can changes in treatment guidelines. The biggest lesson learned, though, was how much time it takes to get data out of the EHR and clean them so that they can be analyzed, said Weber. She emphasized the importance of periodic data checks to make sure the data needed to answer the study question are still being collected in the EHR.
The majority of data collected in EHRs, Weber noted, will be for billing, so research may not be high on a health system’s agenda. As a result, it is important for investigators to keep embedded trials simple, such that the endpoints under investigation do not add undue burden to patients and clinicians during routine health care visits. Given these constraints, embedded pragmatic clinical trials will work well when the outcome of interest is
23 The Brief Pain Inventory is a medical questionnaire used to assess the severity of pain and its impact on functioning among patients with pain from acute conditions, cancer, chronic disease, lower back pain, and osteoarthritis (MD Anderson Cancer Center, 2019).
captured in the EHR; when the health care system partner is interested in the study outcome; when the intervention is familiar, but being tested for a new indication; and when there are few ongoing, competing interventions or trials. If done correctly, embedded trials, Weber noted, can be less costly than a conventional trial. In fact, Weber noted that over a 5-year period, The Collaboratory spent approximately $4.5 million on clinical trials, with sample sizes ranging from 200 to nearly 200,000 participants. This is in contrast to conventional trials, which have recently been estimated to have a median cost of $19 million (Moore et al., 2018).
Decentralized Clinical Trials
Kimberly Hawkins, Clinical Sciences and Operations Project Leader Head, Sanofi Genzyme
While the public has expressed an interest in participating in clinical trials, recruitment and participation rates are low due to a lack of awareness of trials, the necessity to travel long distances to study sites, and the duration and number of clinical visits required. Emerging digital health technologies provide an opportunity to design decentralized clinical trials, which she called a disruptive approach to organizing the trial around the patient. In particular, Hawkins noted five opportunities that decentralized trials provide:
- Increased flexibility such that the burden of participation is reduced for both the patient and clinical trial sites;
- Increased participation of diverse patient populations by expanding access to those who may not reside near traditional academic centers, particularly for patients with rare diseases;
- Increased frequency of data collection and use of continuous data flows to more accurately and rapidly detect signals;
- Improved patient recruitment and retention; and
- Improved long-term follow-up to increase understanding of drug safety profiles and home-based dispensing.
Through work on decentralized clinical trials, Hawkins and her colleagues have identified a number of operational, ethical, regulatory, and management challenges.
Hawkins and her colleagues faced issues integrating the new types of data from digital health technologies into the standard datasets the
company was accustomed to handling. Mapping out the data flow, Weber continued, will be important for sponsors to do before implementing such technologies into clinical trials. While D2P shipments can be challenging in terms of maintaining the temperature across the supply chain or delays in delivery, it can offer a lot of value for patient populations that are highly mobile. Patient management must be well defined and documented, but must also be mindful of not adding unnecessary safeguards in a decentralized trial. Given these operational challenges, conducting a decentralized trial will likely require additional resources and skillsets to work appropriately with tech vendors.
Hawkins also noted that the increased use of eConsent and eSource (data that are initially recorded in electronic format or data that are collected digitally without the need to record data on a piece of paper first) (FDA, 2013) has worked well when working with larger institutions and central Institutional Review Boards (IRBs), but can be challenging when working with smaller institutions and local IRBs.
Regarding regulatory challenges, Hawkins said she has found regulatory agencies to be quite interested in working collaboratively to implement and pilot these new digital technologies in the context of clinical trials. An important concern, however, is endpoint validation using a specific digital health technology, which requires implementing a time to validate the technology into a clinical development plan.
Hawkins indicated that change management is likely the biggest challenge of adopting decentralized clinical trials. Decentralized clinical trials are disruptive to the status quo, noted Hawkins, and there are risks to integrating them into the development plan for a medical product. At Sanofi Genzyme, this requires educating teams about the decentralized approach, allowing them to experiment with new ideas, and having good backup plans knowing that some approaches will fail.
The best fits for decentralized clinical trials are likely products for which safety is well characterized or that can be ingested easily by patients in the home, suggested Hawkins. Another consideration, she noted, is to use decentralized clinical trials for therapeutic areas in which telemedicine is already well established.
Noah Craft, CEO, Science 37
Achieving the shift from organizing activities around the doctor to the patient will likely be a difficult undertaking, noted Craft. In his opinion, the impact of doing so will be profound and the resulting scientific discoveries will be more potent.
The technical challenges, however, will not be small, he added. Craft shared that when he was working with the innovation group at a large pharmaceutical company to help address the problem of slow, expensive, and burdensome clinical trials, he observed that neither academic medical centers, contract research organizations, nor technology companies were equipped to truly create a D2P clinical trials paradigm. The company he started, Science 37, created an end-to-end system that brings together doctors, technology, telemedicine, and most importantly, a direct-to-patient approach to clinical trials.
Craft relayed the primary lesson from the projects Science 37 has completed thus far—trial design should fit participant needs. Trial complexity and merely trying to retrofit a standard trial into an at-home trial are the two main challenges he has faced. An additional challenge is to better understand the science of how to engage patients. Based on his experiences so far, Craft suggested that there is not a universally effective approach to recruitment, so he cautioned against extrapolating lessons learned from recruitment in any single trial.
Adrian Hernandez, Vice Dean for Clinical Research, Duke University School of Medicine
Adrian Hernandez noted there is no one-size-fits-all approach to D2P trials. He agreed with Cummings and others who believe the word “virtual” is not appropriate given that these trials aim to get closer to people and form real relationships with them.
A theme in the presentations that struck Hernandez is that successful D2P trials focus on specific outcomes informed by patient preferences in terms of the kind of research people want to be part of, when they want to be part of it, and how they will be part of it. Having motivated participants is also important, said Hernandez, but the field also needs to develop the science of what makes participating in a trial valuable enough to keep participants engaged through the course of the study and beyond. Hernandez noted the need for a decision tree in terms of fit-for-purpose versus retrofitting an existing trial design to ensure achieving complete outcomes and deciding what to include in a D2P trial versus what to let go. A decision tree
could also help high-level leaders and regulators better understand various risks and trade-offs involved with D2P trials.
Jon White, Deputy National Coordinator, The Office of the National Coordinator for Health Information Technology, U.S. Department of Health and Human Services
Jon White pointed out that 96 percent of hospitals and 80 percent of physicians that participate in Medicare or Medicaid have certified health information technology systems (i.e., EHRs) in place that they use daily. The Office of the National Coordinator for Health Information Technology (ONC) received the statutory imperative in the 21st Century Cures Act to make EHR data available to patients without special effort, said White. Proposed rules have been issued from ONC and the Centers for Medicare & Medicaid Services that will lay out the federal government’s policy for how it will regulate those health information systems to make the data available through application program interfaces without special effort. White reported that his office is going to work on freeing health data and challenged workshop participants to more actively partner with clinical trial participants—not least of all because in the future participants will have their data and researchers will need to work directly with participants to study the data. Although it will be possible to get these data in other ways as well, his counsel was that “the right way, the virtuous way, to get these data is to embrace people and have them bring their data to you on their terms.”
Josh Rose, Vice President and Global Head of Strategy, IQVIA
Josh Rose noted that flexibility is key when interacting with participants. Some clinical trial participants want more in-person interaction while others prefer to use just the technology. He commented that his impression from working with FDA is that the agency is open to these types of virtual, D2P trials. Rose noted that he would like to see FDA develop positions to help guide the field through issues such as shipping medications directly to participants’ homes or determining whether a digital health technology can make a desired measurement as opposed to requiring an office visit. Regulators outside the United States look to FDA for guidance, he said, so FDA can play an important role in shaping the global regulatory environment regarding D2P trials.
Turning to the subject of investigators, Rose said it is important to recognize that there are investigators who are quite happy with the current system, and others, especially younger investigators, who are not interested in running a clinical site and might be quite amenable to adopting these
newer approaches. In fact, he believes this new paradigm can serve as a way to bring more young researchers into clinical studies.
Rose echoed Hawkins’s remarks about change management in that he has seen a great deal of interest and excitement among pharmaceutical company clinical teams only to see their enthusiasm stall because they do not want to be the first to go ahead with a large virtual trial. The solution, he offered, is to receive assurance from senior executives that failure will be acceptable because failure will still move knowledge forward.
Rose suggested that virtual trials are not solely about the technology, but rather about leveraging technology to be able to bring together the complex pieces of process and science in the context of virtual trials.
Cummings asked if there was a mechanism or organization to aggregate the experiences of researchers running D2P trials with regard to endpoints, recruitment, retention, and shipping study drugs, for example. Hernandez replied that one of the purposes of The Collaboratory is to serve as a knowledge repository; it is starting to put together this type of information in a manner that could establish more universal approaches to these trials.
Simcox asked Denny if All of Us will provide participant feedback in real time. According to Denny, All of Us is still in its building stages. While it can return survey data in near-real time to participants, it cannot do so with EHR data. Because All of Us is not the health care provider, it must first wait to receive EHR data and clean them before they are ready for research use. All of Us is working on building mechanisms so that EHR and other data, such as genomics data, can be returned to the participants.
One challenge of new technologies, said Cummings, is that they can collect large amounts of data relatively cheaply, but that leaves the challenge and expense of analyzing those data and dealing with the consequences of unexpected findings. He suggested that enthusiasm for collecting remote measurements of clinical trial participants should be tempered by the realization that analyzing the data in a meaningful way is not cost free.
John Gardinier, retired from the National Center for Health Statistics, noted his concern regarding data reliability arising from the inevitable errors in EHRs and the fact that wearable sensors can be unreliable. Denny replied that the key is to understand that although some of the data are unreliable, much of the data are reliable. By understanding that the data are imperfect, there are ways to get enough signal from bigger datasets to overwhelm the noise from imperfect data. He added that existing benchmarks are enabling better analysis of activity monitors and EHR data. Bollyky remarked that her company uses two-way digital health technologies to reach out to people and confirm reading accuracy.
Craft commented on the idea that technology will make every study better, faster, and cheaper all at once. He suggested that the benefits of technology will include reducing participant burden, speeding up medical product discovery and development, and significant cost savings. But these benefits will be realized over the course of decades. Currently, costs are shifting from participants (in terms of the burdens of participating) to the sponsor or to the owner of the intellectual property, and that is how it should be, he added. Instead of the participant having to spend time and money to get to a clinical site, trial sponsors will increasingly bear the cost of sending a nurse to the participant’s home, for example.
Turkan Gardenier, a statistician, remarked on the parallels between developing D2P approaches and personalized medicine. Rose replied that D2P approaches can help the field of precision medicine by increasing participation among populations traditionally underrepresented in clinical trials.
Lipset noted that the metric of success for his group at Pfizer Inc. is not how many pilots his team can run, but when those approaches are ready for scaling across the organization. From the workshop discussions, his impression is that scale is not going to occur until there are simplified protocols and better endpoints. Hernandez agreed and said it will take time to reach that state. Craft, however, remarked that learnings are accumulating quickly and that Science 37’s clients are shifting rapidly to design trials to fit patient needs. In his opinion, momentum in the field argues that these approaches are ready for scale.
Simcox asked if any of the panelists would comment on eConsent and the need to ensure the identity of a participant receiving a medication. Craft said his experience has shown that investigator–participant relationships are enhanced in D2P trials. Additionally, due to the use of digital health technologies by research staff and participants, it is harder to commit fraud in a virtual clinical trial than in a regular trial. Craft also noted that participants are not required to provide photo identification in a traditional clinical trial, and it is wrong to set the bar higher for D2P trials. Disenfranchised and underserved populations often do not have photo identification cards and cannot get them easily, said Craft. Hernandez agreed with Craft, but noted that some state regulations require authentication of those participating in these studies. Another issue is that people share phones and technology, which can make it difficult to understand who a digital health technology is taking measurements from over the course of a study. Rose also agreed with Craft, but added that there are certain areas where verification is important, such as ensuring that a medication does not go to a neighbor. The process does not need to be complicated, and there are ways of using telemedicine to ask a patient if they are the one receiving a medication. Rose also noted that technology can improve compliance with medication regimens by checking in with patients on a daily basis.