National Academies Press: OpenBook
« Previous: Appendix B: Workshop One Agenda
Suggested Citation:"Appendix C: Workshop Two Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×
Page 155
Suggested Citation:"Appendix C: Workshop Two Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×
Page 156
Suggested Citation:"Appendix C: Workshop Two Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×
Page 157
Suggested Citation:"Appendix C: Workshop Two Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×
Page 158
Suggested Citation:"Appendix C: Workshop Two Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×
Page 159
Suggested Citation:"Appendix C: Workshop Two Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×
Page 160

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

C Workshop 2 Agenda Workshop Two: Practical Approaches March 6–7, 2018 National Academy of Sciences Building, Room 120 2101 Constitution Ave. NW, Washington, DC 20418 The National Academies of Sciences, Engineering, and Medicine is convening a three-part workshop series examining how real-world evidence development and uptake can enhance medical product development and evaluation. The workshops will advance discussions and common knowledge about complex issues relating to the generation and usage of real- world evidence, including fostering development and implementation of the science and technology of real-world evidence generation and usage. Workshop One (September 19–20, 2017) focused on how to align incentives to support collection and use of real-world evidence in health product review, payment, and delivery. Incentives need to address barriers impeding the uptake of real-world evidence, including barriers to transparency. Workshop Two (March 6–7, 2018) will illuminate what types of data are appropriate for what specific purposes and suggest practical approaches for data collection and evidence use by developing and working through example use cases. Workshop Three (July 17–18, 2018) will examine and suggest approaches for operationalizing the collection and use of real-world evidence. DAY 1: March 6, 2018 8:30 a.m. Breakfast Available Outside Room 120 8:40 a.m. Welcome and Opening Remarks P REP UBLI CATI ON COP Y: UNCORR ECTED P ROOFS 155

156 EXAMINING THE IMPACT OF RWE ON MEDICAL PRODUCT DEVELOPMENT GREG SIMON, Workshop Series Co-Chair Investigator Kaiser Permanente Washington Health Research Institute SESSION I: WHEN CAN WE RELY ON REAL-WORLD DATA? Session Discussion Questions: When can we have confidence in EHR data from real-world practice to accurately assess study eligibility, key prognostic factors, and study outcomes? When can we have confidence in data generated outside of clinical settings (e.g., mobile phones, connected glucometers, connected blood pressure monitors)? When does adjudication or other postprocessing of real-world data add value? Moderator: Greg Daniel, Duke-Margolis Center for Health Policy Session Discussants JESSE BERLIN Vice President and Global Head, Epidemiology Johnson & Johnson ANDY BINDMAN Professor of Medicine University of California, San Francisco 9:00 a.m. Introduction and Background to Inform the Discussion: Novel Oral Anticoagulants in Comparison with Warfarin ADRIAN HERNANDEZ Vice Dean for Clinical Research Duke University School of Medicine 9:20 a.m. Open Discussion with Audience What questions can characterize the utility of any real-world data source and signal reliability before a study is performed (examples below)? o When is accuracy good enough to reasonably and consistently identify the right population? o When is accuracy good enough to reasonably and consistently assess the exposure or intervention? o When is accuracy good enough to reasonably and consistently assess the right outcome? o Are there any big safety issues that would be missed? P REP UBLI CATI ON COP Y: UNCORR ECTED P ROOFS

APPENDIX C 157 o Are there concerns about data collection or entry, particularly in relation to creating systemic bias? o When is expert adjudication necessary to confirm that the recorded data are reliable and/or reasonably complete? What information is needed to answer such questions? 10:40 a.m. BREAK (Workgroup Participants Gather to Synthesize Audience Feedback) 11:00 a.m. Workgroup Presents Synthesis of Audience Feedback SESSION II: WHEN CAN WE RELY ON REAL-WORLD TREATMENT? Session Discussion Questions: When conducting research in a real-world setting, are there situations that would require special guidance, knowledge, or experience in order for clinicians to adequately monitor participant safety and respond appropriately to adverse events? When does variation between comparison groups (socioeconomic, demographic, etc.); in treatment fidelity; in provider behavior and preferences; or in adherence yield a valid signal about real-world effectiveness, and when is it just noise? Moderator: Khaled Sarsour, Genentech Session Discussants MICHAEL HORBERG Executive Director, Research, Community Benefit, and Medicaid Strategy Executive Director, Mid-Atlantic Permanente Research Institute Kaiser Permanente Mid-Atlantic Permanente Medical Group GREG SIMON Investigator Kaiser Permanente Washington Health Research Institute ROBERT TEMPLE Deputy Director for Clinical Science Center for Drug Evaluation and Research U.S. Food and Drug Administration 11:15 a.m. Introduction and Presentation to Inform Discussion on Participant Monitoring: Study on Lithium for Suicidal Behavior in Mood Disorders IRA KATZ Senior Consultant for Program Evaluation U.S. Department of Veterans Affairs Office of Mental Health and Suicide Prevention 11:35 a.m. Open Discussion with Audience P REP UBLI CATI ON COP Y: UNCORR ECTED P ROOFS

158 EXAMINING THE IMPACT OF RWE ON MEDICAL PRODUCT DEVELOPMENT What conditions make self-monitoring and reporting acceptable? Does this vary for treatments at different stages of product development or with different baseline knowledge about use in varied patient types and treatment conditions? Can we draw any generalizable lessons about cases in which self-monitoring is acceptable and safe? 12:15 p.m. Introduction and Presentation to Inform Discussion on Signal Detection: Novel Oral Anticoagulants in Comparison with Warfarin 12:30 p.m. Open Discussion with Audience What conditions and training prepare clinical care providers to monitor patient safety outside a tightly controlled environment? How does this vary for treatments at different stages of product development or with different baseline knowledge about use in varied patient types and treatment conditions? How do you decide which variables require strict adherence to “protocol” and which can be allowed to vary? 1:00 p.m. BREAK (Lunch Available Outside Room 120) (Workgroup participants gather to synthesize audience feedback) 2:00 p.m. Workgroup Presents Synthesis of Audience Feedback SESSION III: WHEN CAN WE LEARN FROM REAL-WORLD TREATMENT ASSIGNMENT? Session Discussion Questions: When can we have confidence in inference from cluster-randomized or stepped-wedge study designs? Under what conditions can we trust inference from observational or naturalistic comparisons? How could we judge the validity of observational comparisons in advance, rather than waiting until we’ve observed the result? Moderator: Richard Platt, Harvard Medical School Session Discussants Rob Califf Vice Chancellor, Health Data Science, Duke University Verily Life Sciences DAVID MADIGAN Professor of Statistics Dean, Faculty of Arts and Sciences P REP UBLI CATI ON COP Y: UNCORR ECTED P ROOFS

APPENDIX C 159 Columbia University DAVID MARTIN Associate Director for Real-World Evidence Analytics U.S. Food and Drug Administration 2:20 p.m. Introduction and Presentation to Inform the Discussion: Health Care Database Analyses of Medical Products for Regulatory Decision Making SEBASTIAN SCHNEEWEISS Professor of Medicine and Epidemiology Harvard Medical School Brigham & Women’s Hospital 2:50 p.m. Open Discussion with Audience Random assignment is always preferable, but when is the cost (in time, money, infrastructure, patient exposure) truly necessary? How can we know that the effects from unmeasured confounders are not so large that they would change a decision based on information from an observational study? What are some of conditions under which there is more confidence in inference from non-randomized comparisons (examples of some conditions below)? o Expectation of large effects o Outcome proximal to treatment o High degree of similarity between comparison groups o Pathway from treatment to outcome is relatively clear, and without lots of complexity or reciprocal effects o Treatment allocation method is relatively transparent 3:40 p.m. BREAK 4:00 p.m. Open Discussion with Audience and Reflections from Panelists 5:00 p.m. ADJOURN WORKSHOP DAY 1 DAY 2: MARCH 7, 2018 8:30 a.m. Breakfast Available Outside Room 120 SESSION IV: SYNTHESIZING THE USE CASES Session Objectives: Discuss key considerations presented in each session on Day 1 Consider components of a potential “checklist” for using real-world evidence P REP UBLI CATI ON COP Y: UNCORR ECTED P ROOFS

160 EXAMINING THE IMPACT OF RWE ON MEDICAL PRODUCT DEVELOPMENT 9:00 a.m. Welcome and Recap of Day 1 GREG SIMON, Workshop Series Co-Chair Investigator Kaiser Permanente Washington Health Research Institute MARK MCCLELLAN, Workshop Series Co-Chair Director Duke-Margolis Center for Health Policy 9:20 a.m. Open Discussion with Audience of Outputs from Day 1 and Potential Components to a “Checklist” for Using Real-World Evidence 10:40 a.m. BREAK 11:00 a.m. Open Discussion with Audience of Outputs from Day 1 and Potential Components to a “Checklist” for Using Real-World Evidence 12:30 p.m. ADJOURN WORKSHOP DAY 2 Future Workshop Objectives WORKSHOP THREE: Examine and suggest approaches for operationalizing the collection and use of real-world evidence (July 17–18, 2018, Washington, DC) Applications for using real-world evidence to supplement traditional clinical trials, pragmatic/effectiveness trials, or routine clinical application. Mechanisms for determining which discrete types of real-world evidence could support regulatory decisions. Operational challenges and barriers for generating and incorporating real-world evidence in the context of a learning health system and how clinicians can best be involved in the collection and usage/utilization of real-world evidence. P REP UBLI CATI ON COP Y: UNCORR ECTED P ROOFS

Next: Appendix D: Workshop Three Agenda »
Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series Get This Book
×
Buy Paperback | $60.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Randomized controlled trials (RCTs) have traditionally served as the gold standard for generating evidence about medical interventions. However, RCTs have inherent limitations and may not reflect the use of medical products in the real world. Additionally, RCTs are expensive, time consuming, and cannot answer all questions about a product or intervention. Evidence generated from real-world use, such as real-world evidence (RWE) may provide valuable information, alongside RCTs, to inform medical product decision making.

To explore the potential for using RWE in medical product decision making, the National Academies of Sciences, Engineering, and Medicine planned a three-part workshop series. The series was designed to examine the current system of evidence generation and its limitations, to identify when and why RWE may be an appropriate type of evidence on which to base decisions, to learn from successful initiatives that have incorporated RWE, and to describe barriers that prevent RWE from being used to its full potential. This publication summarizes the discussions from the entire workshop series.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!