National Academies Press: OpenBook

Observational Studies in a Learning Health System: Workshop Summary (2013)

Chapter: Appendix B: Workshop Agenda

« Previous: Appendix A: Biographies of Workshop Speakers
Suggested Citation:"Appendix B: Workshop Agenda." Institute of Medicine. 2013. Observational Studies in a Learning Health System: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18438.
×

Appendix B

Workshop Agenda

OBSERVATIONAL STUDIES IN A LEARNING HEALTH SYSTEM

image

An Institute of Medicine Workshop
Sponsored by the Patient-Centered Outcomes Research Institute

image

A Learning Health System Activity
IOM Roundtable on Value & Science-Driven Health Care

April 25–26, 2013

National Academy of Sciences
2101 Constitution Avenue, NW
Washington, DC

Meeting Objectives

1.   Explore the role of observational studies (OSs) in the generation of evidence to guide clinical and health policy decisions, with a focus on individual patient care, in a learning health system;

2.   Consider concepts of OS design and analysis, emerging statistical methods, use of OSs to supplement evidence from experimental methods, identifying treatment heterogeneity, and providing effectiveness estimates tailored for individual patients;

3.   Engage colleagues from disciplines typically underrepresented in discussions of clinical evidence discussions; and

4.   Identify strategies for accelerating progress in the appropriate use of OS for evidence generation.

Suggested Citation:"Appendix B: Workshop Agenda." Institute of Medicine. 2013. Observational Studies in a Learning Health System: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18438.
×
 
Day 1: Thursday, April 25
 
8:00 am

Coffee and light breakfast available

 
8:30 am

Welcome, introductions, and overview

Welcome, framing of the meeting, and agenda overview

Welcome from the Institute of Medicine (IOM)

Michael McGinnis, IOM

Opening remarks and meeting overview

Joe Selby, Patient-Centered Outcomes Research Institute

Ralph Horwitz, GlaxoSmithKline

 
9:00 am

Workshop stage setting

image   Session format

image   Workshop overview and stage setting
Steve Goodman, Stanford University

Q&A and open discussion

image   Session questions:

image   How do observational studies contribute to building valid evidence to support effective decision making by patients and clinicians? When are their findings useful?
When are they not?

image   What are the major challenges (study design, methodological, data collection/management/analysis, cultural, etc.) facing the field in the use of observational study data for decision making? Please include consideration of the following issues: bias, methodological standards, publishing requirements.

image   What can workshop participants expect from the following sessions?

 
9:45 am

Engaging the issue of bias

Moderator: Michael Lauer, National Heart, Lung, and Blood Institute

image   Session format

image   Introduction to issue
Sebastian Schneeweiss, Harvard University

Suggested Citation:"Appendix B: Workshop Agenda." Institute of Medicine. 2013. Observational Studies in a Learning Health System: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18438.
×

image   Presentations:

image   Instrumental variables and their sensitivity to unobserved biases
Dylan Small, University of Pennsylvania

image   An empirical approach to measuring and calibrating for error in observational analyses
Patrick Ryan, Johnson & Johnson

image   Respondents and panel discussion:

image   John Wong, Tufts University

image   Joel Greenhouse, Carnegie Mellon University

Q&A and open discussion

image   Session questions:

image   What are the major bias-related concerns with the use of observational study methods? What are the sources of bias?

image   How many of these concerns relate to methods and how many relate to the quality and availability of suitable data? What barriers have these concerns created for the use of the results of observational studies to drive decision making?

image   What are the most promising approaches to reduction of bias through the use of statistical methods?
Through study design (e.g., dealing with issues of multiplicity)?

image   What are the circumstances under which administrative (claims) data can be used to assess treatment benefits? What data are needed from electronic health records to strengthen the value of administrative data?

image   What methods are best used to adjust for the changes in treatment and clinical conditions among patients followed longitudinally?

image   What are the implications of these promising approaches for the use of observational study methods moving forward?

 
Suggested Citation:"Appendix B: Workshop Agenda." Institute of Medicine. 2013. Observational Studies in a Learning Health System: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18438.
×
11:30 am

Lunch

Participants will be asked to identify, along with the individuals at their table what they think the most critical questions are for patient centered research outcomes in the topics covered by the workshop. These topics will then be circulated to the moderators of the proceeding sessions.

 
12:30 pm

Generalizing randomized controlled trial (RCT) results to broader populations
Moderator: Harold Sox, Dartmouth College

image   Session format

image   Introduction to issue
Robert Califf, Duke

image   Presentations:

image   Generalizing the right question
Miguel Hernán, Harvard University

image   Using observational studies to determine RCT generalizability
Eloise Kaizar, The Ohio State University

image   Respondents and panel discussion:

image   William Weintraub, Christiana Medical Center

image   Constantine Frangakis, Johns Hopkins University

Q&A and open discussion

image   Session questions:

image   What are the most cogent methodological and clinical considerations in the use of observational study methods to test the external validity of findings from RCTs?

image   How do data collection, management, and analysis approaches affect generalizability?

image   What are the generalizability questions of greatest interest? Or, where does the greatest doubt arise (age, concomitant illness, concomitant treatment)? What examples represent well-established differences?

Suggested Citation:"Appendix B: Workshop Agenda." Institute of Medicine. 2013. Observational Studies in a Learning Health System: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18438.
×

image   What statistical methods are needed to generalize RCT results?

image   Are the standards for causal inference from OSs different when prior RCTs have been performed? How does statistical methodology vary in this case?

image   What are the implications when treatment results for patients not included in the RCT differ from the overall results reported in the original RCT?

image   What makes an observed difference in outcomes credible? Finding the effect shown in the RCT in the narrower population? Replication in more than one environment? The confidence interval of the result? The size of the effect in the RCT?

image   Can subset analyses in the RCT, even if they are underpowered, be used to support or rebut the OS finding?

 
2:15 pm

Break

 
2:30 pm

Detecting treatment effect heterogeneity
Moderator: Richard Platt, Harvard Pilgrim Health Care Institute

image   Session format

image   Introduction to issue

David Kent, Tufts University

image   Presentations:

image   Comparative effectiveness of coronary artery bypass grafting and percutaneous coronary intervention Mark Hlatky, Stanford University

image   Identification of effect heterogeneity using instrumental variables
Anirban Basu, University of Washington

image   Respondents and panel discussion:

image   Mary Charlson, Cornell University

image   Mark Cullen, Stanford University

Q&A and open discussion

Suggested Citation:"Appendix B: Workshop Agenda." Institute of Medicine. 2013. Observational Studies in a Learning Health System: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18438.
×

image   Session questions:

image   What is the potential for OSs in assessing treatment response heterogeneity and individual patient decision making?

image   What clinical and other data can be collected routinely to affect this potential?

image   How can longitudinal information on changes in treatment categories and clinical condition be used to assess variations in treatment responses and individual patient decision making?

image   What are the statistical methods for time-varying changes in treatment (including cotherapies) and clinical condition?

image   What are the best methods to form distinctive patient subgroups in which to examine heterogeneity of the treatment response?

image   What data elements are necessary to define these distinctive patient subgroups?

image   What are the best methods to assess heterogeneity in multidimensional outcomes?

image   How could further implementation of best practices in data collection, management, and analysis affect treatment response heterogeneity?

image   What is needed for information about treatment response heterogeneity to be validated and used in practice?

 
4:15 pm

Summary and preview of next day

 
4:45 pm

Reception

5:45 pm

Adjourn

Day 2: Friday, April 26
 
8:00 am

Coffee and light breakfast available

 
8:30 am

Welcome, brief agenda overview, and summary of previous day
Welcome, framing of the meeting, and agenda overview

 
Suggested Citation:"Appendix B: Workshop Agenda." Institute of Medicine. 2013. Observational Studies in a Learning Health System: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18438.
×
9:00 am

Predicting individual responses
Moderator: Ralph Horwitz, GlaxoSmithKline

image   Session format

image   Introduction to issue

Burton Singer, University of Florida

image   Presentations:

image   Data-driven prediction models
Nicholas Tatonetti, Columbia University

image   Individual prediction
Michael Kattan, Cleveland Clinic

image   Respondents and panel discussion:

image   Peter Bach, Sloan Kettering

image   Mitchell Gail, National Cancer Institute

Q&A and open discussion

image   Session questions:

image   How can patient-level observational data be used to create predictive models of the treatment response in individual patients? What statistical methodologies are needed?

image   How can predictive analytic methods be used to study the interactions of treatment with multiple patient characteristics?

image   How should the clinical history (longitudinal information) for a given patient be utilized in the creation of rules to predict the response of that patient to one or more candidate treatment regimens?

image   What are effective methodologies for producing prediction rules to guide the management of an individual patient on the basis of their comparability to the results of RCTs, OSs, and archived patient records?

image   How can we blend predictive models, which can predict impact of treatment choices, and causal modeling, that compare predictions under different treatments?

10:45 am Break
 
Suggested Citation:"Appendix B: Workshop Agenda." Institute of Medicine. 2013. Observational Studies in a Learning Health System: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18438.
×
11:00 am

Conclusions and strategies going forward
Panel members will be charged with highlighting very specific next steps laid out in the course of workshop presentations and discussions or suggesting some of their own.

image   Panel:

image   Cynthia D. Mulrow, University of Texas

image   Jean R. Slutsky, Agency for Healthcare Research and Quality

image   Steven N. Goodman, Stanford University

image   Session questions:

image   What are the major themes and conclusions from the workshop’s presentations and discussions?

image   How can these themes be translated into actionable strategies with designated stakeholders?

image   What are the critical next steps in terms of advancing analytic methods?

image   What are the critical next steps in developing databases that will generate evidence to guide clinical decision making?

image   What are critical next steps in disseminating information on new methods to increase their appropriate use?

 
12:15 pm

Summary and next steps

Comments from the Chairs

Joe Selby, Patient-Centered Outcomes Research Institute

Ralph Horwitz, GlaxoSmithKline

Comments and thanks from the IOM

Michael McGinnis, IOM

 
12:45 pm

Adjourn

Suggested Citation:"Appendix B: Workshop Agenda." Institute of Medicine. 2013. Observational Studies in a Learning Health System: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18438.
×

Planning Committee

Co-Chairs

Ralph I. Horwitz, GlaxoSmithKline

Joe V. Selby, Patient-Centered Outcomes Research Institute

Members

Anirban Basu, University of Washington

Troyen A. Brennan, CVS/Caremark

Steven N. Goodman, Stanford University

Louis B. Jacques, Centers for Medicare & Medicaid Services

Jerome P. Kassirer, Tufts University School of Medicine

Michael S. Lauer, National Heart, Lung, and Blood Institute

David Madigan, Columbia University

Sharon-Lise T. Normand, Harvard University

Richard Platt, Harvard Pilgrim Health Care Institute

Burton H. Singer, University of Florida

Jean R. Slutsky, Agency for Healthcare Research and Quality

Robert Temple, U.S. Food and Drug Administration

Staff Officer

Claudia Grossmann

cgrossmann@nas.edu

202.334.3867

Suggested Citation:"Appendix B: Workshop Agenda." Institute of Medicine. 2013. Observational Studies in a Learning Health System: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18438.
×

This page intentionally left blank.

Suggested Citation:"Appendix B: Workshop Agenda." Institute of Medicine. 2013. Observational Studies in a Learning Health System: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18438.
×
Page 101
Suggested Citation:"Appendix B: Workshop Agenda." Institute of Medicine. 2013. Observational Studies in a Learning Health System: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18438.
×
Page 102
Suggested Citation:"Appendix B: Workshop Agenda." Institute of Medicine. 2013. Observational Studies in a Learning Health System: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18438.
×
Page 103
Suggested Citation:"Appendix B: Workshop Agenda." Institute of Medicine. 2013. Observational Studies in a Learning Health System: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18438.
×
Page 104
Suggested Citation:"Appendix B: Workshop Agenda." Institute of Medicine. 2013. Observational Studies in a Learning Health System: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18438.
×
Page 105
Suggested Citation:"Appendix B: Workshop Agenda." Institute of Medicine. 2013. Observational Studies in a Learning Health System: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18438.
×
Page 106
Suggested Citation:"Appendix B: Workshop Agenda." Institute of Medicine. 2013. Observational Studies in a Learning Health System: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18438.
×
Page 107
Suggested Citation:"Appendix B: Workshop Agenda." Institute of Medicine. 2013. Observational Studies in a Learning Health System: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18438.
×
Page 108
Suggested Citation:"Appendix B: Workshop Agenda." Institute of Medicine. 2013. Observational Studies in a Learning Health System: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18438.
×
Page 109
Suggested Citation:"Appendix B: Workshop Agenda." Institute of Medicine. 2013. Observational Studies in a Learning Health System: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18438.
×
Page 110
Next: Appendix C: Workshop Participants »
Observational Studies in a Learning Health System: Workshop Summary Get This Book
×
Buy Paperback | $40.00 Buy Ebook | $31.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Clinical research strains to keep up with the rapid and iterative evolution of medical interventions, clinical practice innovation, and the increasing demand for information on the clinical effectiveness of these advancements. In response to the growing availability of archived and real-time digital health data and the opportunities this data provides for research, as well as the increasing number of studies using prospectively collected clinical data, the Institute of Medicine's Roundtable on Value & Science-Driven Health Care convened a workshop on Observational Studies in a Learning Health System. Participants, including experts from a wide range of disciplines - clinical researchers, statisticians, biostatisticians, epidemiologists, health care informaticians, health care analytics, research funders, health products industry, clinicians, payers, and regulators - explored leading edge approaches to observational studies, charted a course for the use of the growing health data utility, and identified opportunities to advance progress. Workshop speakers and individual participants strove to identify stakeholder needs and barriers to the broader application of observational studies.

Observational Studies in a Learning Health Systemis the summary of the workshop. This report explores the role of observational studies in the generation of evidence to guide clinical and health policy decisions. The report discusses concepts of rigorous observational study design and analysis, emerging statistical methods, and opportunities and challenges of observational studies to complement evidence from experimental methods, treatment heterogeneity, and effectiveness estimates tailored toward individual patients.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!