The workshop featured a briefing on ethical considerations related to digital health technologies (DHTs) from a behavioral science perspective by Camille Nebeker, director of the Research Center for Optimal Digital Ethics (ReCODE) at the University of California, San Diego. A regulatory perspective was provided by Amy Abernethy, principal deputy Commissioner at the U.S. Food and Drug Administration (FDA), during a fireside chat moderated by Jennifer Goldsack, executive director of the Digital Medicine Society. Nebeker and Abernethy discussed the categories and uses of DHTs, opportunities to leverage them for capturing real-world evidence, and considerations for moving the field forward in an effective, ethical, and safe manner.
CATEGORIES AND USES OF DIGITAL HEALTH TECHNOLOGIES
DHTs have provided new tools for researchers to collect data about people’s day-to-day activities and can facilitate the study of trial participant behavior in real time by using wearable and remote sensor technologies, mobile applications, and strategies like ecological momentary assessment.1 Nebeker provided an overview of DHTs, such as mobile devices, wearables, and other sensors, that can collect data about participants’ everyday lives using these new methods (see Box 2-1). Nebeker illustrated how each of these approaches has presented unique challenges due to the large volume, high-dimensional, and diverse data being
___________________
1 Ecological momentary assessment: A clinical psychology method that involves repeatedly sampling data on a participant’s behavior in real time, with the aim of reducing recall bias and increasing validity (Shiffman et al., 2008).
collected, including geolocation data, physiological measurements, and biometrics. Additionally, Health Insurance Portability and Accountability Act privacy rules may not apply, depending on the type of data, given that many of the data captured are not housed within electronic health records (EHRs).
An additional layer of complexity has been introduced by the popularity of commercial technologies and wellness apps designed for consumers to capture their own data. In early studies using DHTs, Nebeker said, researchers had full access to and control over the data collected because most technologies used were research-grade tools rather than commercial products. In subsequent years, study researchers began using commercially available health and wellness technologies. The challenge with commercial products, Nebeker said, is the degree to which the products are effective and reliable. Furthermore, the products’ terms and conditions and privacy policies often are not, for the most part, written in favor of the consumer or the researcher. Nebeker explained that these terms and conditions directly conflict with federal regulations for human research protections—specifically for release of liability, or a waiver of responsibility for harm a person may be subjected to through use of a product. While the commercial terms of service include a release of liability, federal regulations prohibit this from occurring in human subject research (Nebeker et al., 2017a).
Leveraging Digital Health Technologies for Drug Research and Development
Abernethy identified several opportunities to use DHTs across the drug research and development process. Although DHTs have value in pre-clinical research, such as for drug discovery and in silico trials, Abernethy focused her remarks on DHT use in pre- and post-market clinical evidence development. Within this broad use of DHTs, Abernethy highlighted three categories:
- Data collection support: DHTs, such as biosensors and remote monitoring technologies, can support the collection of study participant–level and operational data, such as information pertaining to efficacy and safety endpoints within clinical trials.
- Patient-centricity support: DHTs, such as telemedicine, can support patient centricity by reaching patients where they are, bridging gaps in data collection in between clinic visits through continuous data collection, and facilitating the collection of patient-reported outcomes.
- Data curation and trial management: DHTs can contribute to the conduct of clinical trials by offering trial management solutions in the clinic as well as by supporting data curation.
When considering how DHTs can be used in clinical research, it is important to take into account the traditional approach of collecting data
for evidence generation, Abernethy emphasized. Digital technologies and data curation could substitute or complement the development of evidence at different points throughout the process of conducting a clinical trial—from following a protocol to generating a dataset and making a decision based on the final results of the study. Rather than replacing traditional clinical trial data, data from DHTs can serve a complementary purpose. For instance, these data could support the generation of real-world datasets and provide longitudinal follow-up data, additional control data, and supplementary information for certain data points. She suggested that it would be useful to consider all features of a clinical trial to find opportunities for DHTs to contribute, either as a complement to the current approach or as the main component in conducting a trial. Abernethy observed that DHTs may drive changes in the infrastructure of clinical trials. A shift in this direction, she noted, has already begun with the advent of telemedicine as well as with the need to adapt the clinical trial infrastructure in the evolving landscape of coronavirus disease 2019 (COVID-19).2
Evaluation of Digital Health Technologies for Drug Development
“All datasets have warts; we just have to have a way of measuring and solving for data quality,” Abernethy said. When evaluating a DHT, she said the setting of use (e.g., consumer use, adjunct to clinical care, or within the context of a clinical trial) is an important consideration. If they are to be used to collect data on endpoints and biomarkers for clinical evidence generation, DHTs should undergo validity assessments (see Chapter 5 for additional information on analytical and clinical validity). Validity refers to the likelihood that the given output from the DHT will be able to measure the target endpoint (FDA, 2017). As such, validation is related to the endpoint at hand. For example, if the endpoint being assessed is a change in blood glucose, then the validation of a glucometer may involve cross-referencing measurements against other constructs to gain confidence that the sensor performs within that specific setting.
Defining and Optimizing Data Quality
DHTs used for clinical research are often held to a higher standard than currently available tools, Goldsack said. She asked Abernethy if there are strategies for defining and optimizing data quality—or at least identifying limitations so they can be mitigated—in a solution-oriented way that would not compromise regulatory standards. Speaking from her per-
___________________
2 See Chapter 1 for additional details on the impact of COVID-19 on the conduct of clinical trials.
sonal perspective—not as a representative of FDA—Abernethy emphasized the value of developing more standardized approaches to defining data quality across the real-world data space. The field, she observed, may need a consistent way of documenting data completeness, reliability, and different types of validity of individual data elements—specifically face validity, context validity, and construct validity. In the context of DHTs, data quality is shaped by the interrelationships between data outputs, the algorithm used to make sense of that information, and the final endpoint or measurement ultimately used for research analysis.
A structured approach to defining data quality could inform similarly structured approaches to improving data quality, Abernethy said, identifying three strategies that could help: (1) improving the instrumentation; (2) collecting additional data points to triangulate information so that the aggregate data more accurately represent the “truth”; and (3) using analytic methods, such as developing proxies and new analyses. When aggregating a real-world dataset, for example, it will be important to identify data gaps and determine which sources of new data or analyses could be developed to fill them in. A subsequent challenge, Abernethy said, will be determining if those changes have improved the quality of the dataset; use of a standardized assessment of data quality provides a mechanism to monitor the impact of sequential changes. Abernethy predicted that the process of continually improving data will increasingly be used in digital approaches to collecting and using data.
USING DIGITAL HEALTH TECHNOLOGIES TO CAPTURE REAL-WORLD EVIDENCE
Abernethy remarked that there is a tendency to associate DHTs with sensors and other wearable technologies. However, the 21st Century Cures Act3 catalyzed an increasing focus on understanding real-world data and evidence, including administrative claims data, EHR data, and data collected from DHTs. Commensurately, the language on DHTs has begun to shift from biosensors toward merging with the language of real-world evidence, as the two have sources that are often related. It is important to note that FDA characterizes data collected from biosensors during a study as clinical trial data and not real-world data because the data are not being collected in a real-world setting (FDA, 2020d).
Goldsack remarked that EHRs and claims data have traditionally been considered the primary sources of real-world data and evidence and inquired if new sources of data, such as from DHTs, could be combined with traditional sources to make the resulting body of real-world evidence
___________________
3 The full text of the 21st Century Cures Act can be found at https://www.congress.gov/bill/114th-congress/house-bill/34 (accessed May 17, 2020).
more valuable. Abernethy highlighted three critical features of data that are relevant when blending different sources of real-world evidence: data linkage, data quality, and longitudinality.
- Data linkage: No single dataset can provide all of the variables needed to answer all research questions. While this challenge is addressed in clinical trials by pre-specifying a narrow research question and all variables required to address the question to help ensure that the necessary information will be captured, real-world evidence may require linkages between different data sources to provide additional necessary variables. For example, patient-reported outcomes are often missing in the EHR dataset, and clinical variables are missing in administrative claims datasets. Information from biosensors and other digital health solutions could help fill these types of information gaps to build valuable real-world evidence.
- Data quality: Real-world evidence is based on data sources of varying quality, Abernethy said. Given that no dataset is perfect, it is important to better understand how to measure the data limitations and address them. While DHTs, like biosensors, often provide instrumentation data that may be less subjective and have more completeness and reliability than other types of datasets, this information is not immune to data quality limitations. As such, the quality of DHT data should be measured in the same manner as other types of datasets.
- Data longitudinality: Longitudinality, or the length of time a dataset covers, is a valuable feature of datasets, especially in the context of evolving real-world evidence. DHT data (e.g., from biosensors) are often longitudinal and less likely to be episodic and cross-sectional. However, as with every other kind of dataset, DHT data often have limitations. For instance, if patients do not wear a device consistently over the long term, this may lead to missing data and an incomplete assessment of endpoints.
ETHICAL, LEGAL, AND SOCIAL IMPLICATIONS OF DIGITAL HEALTH TECHNOLOGIES AND CLINICAL RESEARCH
Nebeker said that the ethical, legal, and social implications (ELSI) framework, which has been used in genomics4 to guide research and protect study participants, can be applied to research using DHTs. Ethical
___________________
4 See the Ethical, Legal, and Social Implications Research Program at https://www.genome.gov/Funded-Programs-Projects/ELSI-Research-Program-ethical-legal-social-implications (accessed June 7, 2020).
implications are rooted in the foundational principles set forth in the Belmont Report (1978),5 which established a moral framework for biomedical research ethics in the United States. In practical terms, these principles are primarily manifested through the consent process (respect for persons), an evaluation of harms in relation to potential benefits (beneficence), and mitigating undue burden (justice). In research ethics, the normative principle of beneficence holds that research activities should benefit society by contributing new knowledge. While there is no guarantee or expectation that study participants will directly benefit, the principle of beneficence is applied by evaluating the probability and magnitude of potential harms against the possible benefits of knowledge to be gained. Furthermore, this comparison of risks to benefits must consider how risks will be mitigated for the actual research participants, including an evaluation of not only the type of risk but also the intensity, duration, and severity of potential harms to the participant. In the legal and regulatory domain of the ELSI framework, Nebeker focused on regulations that included liability concerns, conflict of interest, human subject protections, and intellectual property laws. She noted that new privacy laws had recently been enacted in Europe (General Data Protection Regulation6) and California (California Consumer Privacy Act7). Social implications relate to the downstream impacts of DHTs, she said. Mistakes may be inevitable when exploring this new digital health frontier, she added, and it will be important to share resulting lessons learned for the entire DHT community to benefit.
Engaging Stakeholders in Digital Health Research
Nebeker described some of the challenges regarding the various stakeholders in digital health research and how expertise areas could be bridged. She suggested bringing together diverse teams to collectively think through the potential implications of those mistakes for different populations, each of which may be affected in different ways. A variety of stakeholders—including the end users of DHTs—should be engaged in this conversation from the outset. Stakeholders, such as DHT developers, academics, and citizen scientists are now conducting research from a
___________________
5 The full title is the Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research, Report of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, which was published in the Federal Register in 1979. The report can be accessed at https://www.hhs.gov/ohrp/regulations-and-policy/belmont-report/read-the-belmont-report/index.html (accessed May 15, 2020).
6 More information about the European General Data Protection Regulation is available at https://gdpr.eu (accessed May 17, 2020).
7 More information about the California Consumer Privacy Act is available at https://oag.ca.gov/privacy/ccpa (accessed May 17, 2020).
range of perspectives. These stakeholders have different goals and varied expertise and levels of formal training. Moreover, they operate within diverse regulatory environments and have different expectations of what is considered acceptable data to demonstrate that a product or process is effective and reliable (Nebeker, 2020) (see Figure 2-1). For example, those within academic, biotechnology, and pharmaceutical sectors tend to have extensive, highly focused training and typically conduct research that is heavily regulated and grounded in ethical principles. In contrast, community health workers have less extensive training, but are instrumental partners in bringing research into community settings. Those who are conducting citizen science or participant-led research are unregulated, may have little to no formal research training, and may be unfamiliar with applying the scientific method and research ethics. To increase community research capacity, Nebeker and colleagues have developed and continue to create educational programs8 to increase research literacy and awareness of ethical practices among people who are not formally trained as scientists (Grant et al., 2019; Nebeker and López-Arenas, 2016; Nebeker et al., 2020b).
SOURCE: As presented by Camille Nebeker, March 24, 2020.
___________________
8 For more information on ReCODE Health’s Building Research & Integrity Capacity course, see https://recode.health/about (accessed June 18, 2020).
Digital Health Decision Support Framework and Checklist
Nebeker and her colleagues developed a digital health decision-making framework and checklist9 to guide how institutional review boards (IRBs) and researchers think through the ethical considerations (e.g., privacy, access, and risk–benefit analysis) and better protect participants. Ultimately, this framework guides researchers through the cycle of human-centered design (planning, designing, developing, testing, releasing, and applying feedback) to ensure that the right data are used, the right technologies are being developed, the technologies are developed ethically, the beneficiary population is involved in the design process, and the resulting technology fits within the context of the participants’ lifestyle.
In developing the framework, Nebeker and her colleagues drew on related work on a clinical decision-making framework that helped clinicians protect their patients from privacy and data management risks when using mobile phone apps for mental health. Nebeker and the ReCODE Health team convened a focus group of ethicists, scientists, legal scholars, and regulatory experts to develop a survey that was deployed with behavioral scientists, and the responses were used to identify key domains of ethical principles for digital health: access and usability, privacy, risks and benefits, and data management (see Figure 2-2 for additional details)—each of which are anchored by the ethical principles of the Belmont Report (Nebeker et al., 2020a).
- Access and usability: This domain captures product design and whether end users are able to use the technology. Considerations include how a given product works and how that information is communicated to the user, the technology’s previous use within the target population, if accessory tools (e.g., smartphone or Internet access) are needed, and whether the product can be used in both the short and long term. Nebeker said that maintaining the participants’ long-term engagement with a DHT is emerging as a major challenge.
- Privacy: This domain centers on the personal information collected and the participants’ expectations that the information will be kept secure. If the information will be shared, then considerations include what information is collected, what is shared and why, and the degree of control afforded to the end user.
- Risks and benefits: This domain’s goal is to evaluate the types of possible risks as well as the extent of possible harm, severity,
___________________
9 The framework and checklists are available on the ReCODE Health website, available at https://recode.health (accessed May 16, 2020).
SOURCES: As presented by Camille Nebeker, March 24, 2020; from Nebeker et al., 2018.
- duration, and intensity. The assessment of risks and benefits is influenced by the evidence available to support the reliability of the DHT, risk mitigation strategies, and recognition of unknown risks.
- Data management: This domain addresses how data are collected, stored, and shared, as well as the extent to which the data are incorporated within other systems. Considerations include what data are collected, what data are needed to answer the question, why and how the data are shared, the end user’s control over the data, and data interoperability.
Uncertainties Related to Digital Health Technology Use
Nebeker outlined several uncertainties related to data management, governance, health care delivery, and informed consent that she believes should be addressed before the use of DHTs in research can move forward. In terms of data management, questions around data ownership, data anonymity, and the use of data de-identification as a potential solution may require more research. Given the variability in governance, the relevance and responsiveness of existing systems should be considered. These include conventions, norms, and regulations that currently vary across the different disciplines and sectors engaged in this work. In the context of using DHTs to deliver better health care, there remain uncertainties about
whether machine learning and artificial intelligence can improve the effectiveness of clinicians. Finally, it is important to determine if consent can truly be informed in studies using DHTs. Strategies are needed to enhance the consent process so that even people with relatively low technology and data literacy can understand their involvement in a study. It would also be helpful, Nebeker said, to create consent mechanisms that are less onerous to understand than commercial products’ terms and conditions of use. These uncertainties have prompted Nebeker and ReCODE Health colleagues to conduct studies on an array of considerations in the field of digital health research to bridge the gap between researchers and IRBs on risk assessment, develop a better understanding of concerns about participating in studies using DHTs, and gain further knowledge on terms and conditions participants accept when using DHTs. Nebeker provided further details about each category of research and information gaps to be filled (see Table 2-1).
ReCODE Health is currently partnering with a local retirement community to learn about barriers and facilitators to adopting DHTs; one lesson has been that education is needed on how DHTs work and how they can be beneficial (Wang et al., 2019). Nebeker emphasized that investing time to build trust and channels of communication with communities can enable participants to better understand how their data are used and shared. Goldsack remarked that DHTs have the potential to bring clinical
TABLE 2-1 Digital Health Technology Research and Information Gaps to Be Filled
| Category of Research | Research or Information Gaps |
|---|---|
| Institutional review board (IRB) consent analysis (Nebeker et al., 2015) |
|
| IRB focus groups (Nebeker et al., 2017a) |
|
| Participant surveys (Nebeker et al., 2016) |
|
| Participant digital divide (Nebeker et al., 2017b) |
|
| Participant terms and conditions (Das et al., 2018) |
|
SOURCE: As presented by Camille Nebeker, March 24, 2020.
trials and other research opportunities to communities that have historically been excluded (Khozin and Coravos, 2019) but noted that the digital divide also runs the risk of increasing disparities. Partnering with community health workers can help build trust within communities, Nebeker said, adding that individuals should understand how technologies are used, how data are managed, and who has access to their data. ReCODE Health shares the results of its studies with a broad audience, Nebeker said. In addition to sharing their research results, researchers using digital health technologies also describe how they identified and navigated ethical challenges on the ReCODE Health platform. In many cases, these researchers know more about the potential risks associated with digital health technologies than an IRB. As such, lessons learned from how information is shared with IRBs might help to increase collaboration across the entire research community Nebeker emphasized that the digital health research community needs to support all of its stakeholders in thinking through their respective roles and responsibilities, with the end goal of protecting people who participate in these studies.
CONSIDERATIONS FOR THE FUTURE
In order for digital health research to move forward in an ethical way, Nebeker suggested that the digital heath community be guided by the goal to “move purposefully and fix things” rather than the common refrain of “move fast and break things,” echoing remarks of the former U.S. chief data scientist, Dhanurjay Patil (Drinkwater and Schlesinger, 2019). DHTs can be leveraged to answer important and timely research questions, but the risks are great if this work does not proceed thoughtfully and responsibly. Although efforts to date have primarily focused on behavioral health promotion and disease prevention, Nebeker said that work is currently under way to adapt the ReCODE Health Digital Health Checklist for use by clinicians, IRBs, and other communities working on DHTs. ReCODE Health is also looking at ways to support developers who are creating digital health tools.
Tension Between Privacy and Public Health Needs
The requirements related to participant privacy are constantly evolving; for example, privacy protections are now regulated in California with the passage of the California Consumer Privacy Act, Nebeker said. In general, greater communication with participants about privacy would be beneficial. Tying privacy and risk of privacy loss to potential improvements in personal and public health could help participants make decisions about using DHTs, Nebeker said. She also suggested that people
may be more willing to trade their privacy if they know what they are getting back. Another consideration is the different preferences that older and younger adults may have regarding their privacy (Wang et al., 2019). In addition to sharing what is being learned from participants’ data, it is also important to help participants learn more about themselves. The concept of “return of value” refers to conveying information back to participants in a way that is meaningful to them and represents a true partnership. However, sharing information is a long-term engagement strategy, and it takes time to build the necessary trust. It can also be challenging to convey that information back to participants in a clear and useful way (Wang et al., 2020).
The COVID-19 pandemic has underscored the tension between data privacy and public health, Nebeker said. For example, discussions are taking place concerning the potential to use digital tools, such as cell phone tracking, to mitigate the spread of the virus or using wearable devices to improve state-level real-time surveillance (Radin et al., 2020; Servick, 2020). Even prior to the pandemic, overburdened health care systems were relying on large technology companies to sort and analyze large volumes of data, despite the lack of a uniform standard for avoiding the misuse of patient data (Ross and Brodwin, 2020). “We are living in a real-time experiment on balancing privacy and public health,” she said, and emphasized that public communication will be critical in forging a path forward in digital health in a way that is responsible and transparent. She added that social media platforms and other technologies could be used for public outreach and the dissemination of science-based information, with the aim of educating the public while also mitigating hype and misinformation.
Regulatory Strategies for Digital Health Technologies
Goldsack remarked that the speed of innovation in the DHT space may be escalating at a faster pace than regulatory strategies. Assuming that DHTs are evolving in a direction of providing more high-quality data, Abernethy said that it will be important to evaluate if the current regulatory approach is working before considering mechanisms that enable the regulatory strategy to keep pace and introducing flexibilities to allow innovation to occur. Across FDA, approaches are being piloted to identify and pre-ascertain best practices for developing software, algorithms, and DHTs that will lead to the highest-quality data outputs (FDA, 2019a). Those running these pilot programs are also coordinating with DHT developers to create approaches for technology development that are flexible enough to allow for innovation, but within an appropriate monitoring framework. Balancing innovation with validity expectations
will help ensure that the data outputs of DHTs can contribute to high-risk activities, such as clinical trials and clinical evidence development. A workshop participant reminded the audience about a resource that clarified requirements in DHT research and was released in 2020 by the Secretary of Health and Human Services’ Advisory Committee on Human Subject Protections.10
U.S. Food and Drug Administration’s Technology Modernization Action Plan
Abernethy provided an overview of FDA’s Technology Modernization Action Plan, which was released in September 2019.11 The plan was developed to enable FDA to be ready to respond in a flexible way to new, continuously updating, structured and unstructured data acquired from DHTs. In addition to being able to store and analyze this new form of data, FDA needed to be able to interface with the broader community about DHTs. The plan has three parts, the first of which is to ensure that FDA has an internal cloud-forward technology strategy ready to receive and analyze data in a secure and private way. The second part is to create a series of use cases to help propel FDA and the life sciences community forward in terms of thinking about and using these data. For example, FDA is collecting 7- to 15-day safety data for investigational new drugs through an application programming interface that directly provides FDA with ready-to-use structured digital information. The third part involves opening up FDA’s communication channels to the larger community of DHT innovators and stakeholders so that those stakeholders can understand what kinds of solutions are needed across the life sciences space. FDA’s complementary Enterprise Data Strategy was announced in January 2020, Abernethy added. Its aim is to crystallize FDA’s thinking concerning issues related to data sharing, standards, analysis, and “putting data to work.” Considerations may include how to use new capabilities, such as artificial intelligence and blockchain, within the agency and in an integrated way across the industry.
___________________
10 Information about these advisory documents is available at https://www.hhs.gov/ohrp/sachrp-committee/recommendations/sachrp-recommendations/index.html (accessed May 16, 2020).
11 For more information on FDA’s Technology Modernization Action Plan, see https://www.fda.gov/about-fda/reports/fdas-technology-modernization-action-plan (accessed June 19, 2020).
This page intentionally left blank.
