During the final session of the workshop, individual participants reflected on the workshop presentations and discussions and discussed actions that they believed were important to achieving progress in the areas discussed. The suggestions made by individual participants covered five broad thematic areas: learning-ready records, the availability of a network for the development of questions, science stewardship, facilitative oversight, and the presence of a learning culture.
Throughout the workshop, many speakers and participants who spoke commented on the key role of electronic health records (EHRs) in large simple trials (LSTs). Their use was often cited to be critical to carrying out large trials that are integrated into the care setting. However, Richard Platt, in particular, cautioned against the notion that they are a panacea, noting that in their current state they can be quite difficult to use.
This discussion pointed to more interoperable, learning-ready records as a priority for progress. Approaches to achieving learning-ready records included enabling all records to collect a minimum data set designed to support maximum improvements in health care and the more widespread use and piloting of minimal data resources already in existence. The identification of such a data set was suggested to be an important first step in this endeavor, which would require the engagement of multiple stakeholders as well as national leadership to consider issues such as balancing the burden of data collection with clinical burdens and work flows.
Some progress toward the electronic collection of minimal data sets is already under way, as Rebecca Daniels Kush of Clinical Data Interchange Standards Consortium (CDISC) detailed. CDISC’s work to facilitate the collection of data for clinical trials from EHRs was cited as critical progress toward making the maximum use of digital health data for research.
In addition to the collection of the appropriate information to support learning, it was highlighted by several workshop participants that spoke that effective data sharing will be crucial to obtaining the amount of data needed for LSTs. Data sharing of this magnitude will require technical as well as policy solutions.
Several participants pointed to the interoperability of data to be a critical component to ensure that data are comparable across sites. Approaches to the sharing of data, such as distributed networks, that do not require the data to leave the originating institution were highlighted to be particularly promising. These challenges that the participants who spoke identified, however, are far from resolved and will require sustained innovation and continued capacity development.
Some participants, particularly Tim Ferris, pointed to the move toward accountable care organizations, which require the collection, sharing, and use of more data, as well as a focus on outcomes, as potential levers for progress toward the achievement of more learning-ready records. Still, collaborative, national, and global efforts to continue progress on the challenges of interoperability, data sharing, and standardization were priorities that participants who spoke mentioned often.
NETWORK FOR QUESTION DEVELOPMENT
Throughout the workshop, several participants and speakers pointed to the importance of the use of appropriate methodological approaches to answer research questions. Much discussion was had about the sorts of questions that LSTs are best suited to answer, with the understanding that all approaches have advantages and disadvantages. Stating that much of the research currently being conducted does not produce the evidence needed to inform decision making and that many basic effectiveness questions are left unanswered, several participants noted the opportunity to strengthen the evidence base in a way that can inform population-level questions and, ultimately, individual care decisions through the broader use of LSTs. Participants noted that the development of a national capacity to conduct LSTs could be accompanied by an ongoing capacity to elicit, refine, and triage questions to be addressed.
Several factors influencing the prioritization of research questions were discussed. These included functions of the nature of the question, such as
the expected size of the effect and outcomes, in addition to issues of stakeholder engagement.
References were made to the recent efforts made by the Patient-Centered Outcomes Research Institute (PCORI) to elicit research topics from a wide range of stakeholders and how such efforts could be a potential model to obtain questions to be answered by LSTs. Joe V. Selby, from PCORI, noted that no single legitimate source for research questions exists. He highlighted that although patients are beginning to be engaged in these efforts, clinicians and health care systems, crucial stakeholders in this work, are currently the least prepared to participate.
Among the issues raised were the different and often contrasting interests held by different stakeholders, as was echoed for the health care system level during Alan S. Go’s presentation during the workshop, and the need to balance these interests against the importance of maintaining scientific rigor. Several participants highlighted the need for inclusive governance structures and policies to support dialogues about prioritization.
The continued development and innovation of methods for the conduct of LSTs were key issues of discussion among several workshop participants and speakers. Efforts to affix a single, rigid definition to LSTs were discouraged, reflecting both the evolving landscape and the acknowledgment that one size does not fit all. Robert Temple suggested that rather than focus on one research approach, a theme of the workshop was the need for simpler trials overall and that one potential use for LSTs is the follow-up of hypotheses generated from observational studies.
As the infrastructural and policy capacity to conduct LSTs develops, several participants highlighted that methodological innovation must keep pace but that scientific rigor must be maintained. A theme of Rachel E. Sherman’s presentation, that the focus should be on producing good-quality data that answers the question being asked rather than on the selection of a particular methodology for the sake of it, was echoed by other workshop participants. One of the challenges to this, highlighted by Ralph I. Horwitz, is the need to get a better sense of what needs to be done to get to credible evidence in the variety of diverse situations in which evidence from LSTs will be used.
Several of the challenges to the broader use of LSTs that workshop participants who spoke highlighted dealt with current reward structures within the research community. A number of speakers and individual discussants pointed to the need for incentives for value rather than incentives for volume of studies (in which volume covers both the number of studies and the amount of data collected), which are major contributors to the un-
necessary complexity of clinical studies. The current approaches, they said, have resulted in the conduct of many low-quality studies without sufficient thought being given to study efficiency, including the placement of limits on the amount of data collected, the value of the questions asked to key stakeholders, or the utility of the results.
Many participants discussed the need to develop metrics, potentially for use by research funders, to assess the progress of research efforts toward these goals and reward those that make effective use of resources and contribute meaningfully to a useful evidence base for care and decision making.
Much of the discussion around policy challenges to the broader use of LSTs in the United States centered on the need for clear and facilitative oversight. Several presentations touched on the underlying reasons for the current complexity and inefficiencies of the clinical research enterprise. These included discussions over the shift of the focus of clinical trials from acute to chronic diseases and the collection of more genetic data but also highlighted the lack of understanding between researchers and regulatory and oversight organizations.
In the case of research on regulated products, speakers and individual participants spoke of a sense that many of the data were collected in response to a perception that they could be required by regulatory agencies, such as the U.S. Food and Drug Administration (FDA), and that the unnecessary collection of such large amounts of data greatly increased the cost and complexity of clinical trials. This perception was countered by comments from FDA officials that such fears are largely unfounded. Many individual participants highlighted that for the streamlined approaches required by LSTs to be more widely used in the assessment of regulated products, this communication and understanding gap must be closed. Clear, specific guidance from FDA on the use of LSTs and streamlining of clinical trials were highlighted as important first steps in this effort. Other strategies discussed included better training of study case managers to break cycles of unnecessary or defensive data collection.
Discussions of the need for clear, facilitative oversight also extended to the bodies responsible for research protections, including sharing of data. Ruth R. Faden’s presentation highlighted the reasons that the current framework for ethical oversight is no longer suitable to meet the requirements of a learning health system, particularly as it deals with learning from the delivery of health care. She suggested that a new ethics framework that supports learning and protects patients will be necessary to show respect for patients and honor their role in research.
A number of individual participants highlighted the crucial need for a
research governance regime that meets these goals of protecting patients while allowing the kind of learning from care that LSTs can facilitate.
Informed consent was one specific area for potential improvement that several participants highlighted. Suggestions on ways to improve informed consent included improving and shortening consent forms, rethinking the requirement for informed consent in situations limited to the use of patient information, and leveraging electronic platform to obtain portable legal consent.
Discussions of data sharing and privacy oversight policies followed similar themes. Deven McGraw’s presentation and comments from several participants highlighted the misplaced focus on differentiation of operations and research as a way to guide oversight of data sharing. Alternative suggestions of ways to protect patient information while facilitating learning processes included the application of fair information principles and keeping in mind patient expectations about what their data will be used for.
A theme of the discussion on what it will take to achieve the greater use of LSTs that arose often throughout the workshop was the need to move toward a culture of continuous learning in which every clinical care encounter is an opportunity for learning. Comments and reflections on discussions about the technical challenges facing LSTs often concluded that although work remains to be done in these areas, the real barriers are those of culture, including resistance to the use of randomization.
Individual workshop participants highlighted the need for leadership from the top that reflects the principles of learning from the provision of clinical care, the need to embed this concept into the organizational mission, and the need to use efficient, integrated methods to develop evidence that is useful for decision making. Approaches to achieving the goals that were discussed included training of support staff and clinicians, alignment of incentives, and the creation of incentives when necessary to ensure that everyone, including clinicians and patients, has a business case for participation in research.
Some participants called for efforts to fully engage clinicians and patients at all levels of the learning process: question identification, study design, implementation, and dissemination. Specifically, the need to address aversions to participating in integrated trials by clinicians who might perceive these to be risky propositions was mentioned. The call for patient engagement included the need to reframe patients as consumers of research rather than research subjects, the need for transparent dialogues about the risks and benefits of participation in research, and the complementary ethical responsibilities of clinicians and systems to protect patients and of
patients to participate in the kinds of learning activities that will ultimately benefit them and others like them. The need to develop careful and respectful communication strategies was highlighted to be an important component of these engagement efforts.
Finally, several individual participants highlighted the need to educate the end users of the evidence produced by approaches like LSTs in an effort to build awareness of the advantages of research and maximize the impact and utility of the results.
