CHARACTERIZING AND COMMUNICATING UNCERTAINTY IN THE ASSESSMENT OF BENEFITS AND RISKS OF PHARMACEUTICAL PRODUCTS: AN INSTITUTE OF MEDICINE WORKSHOP
February 12, 2014
U.S. Food and Drug Administration (FDA) Campus
White Oak, Maryland
BACKGROUND AND MEETING OBJECTIVES
There is increasing attention on the need for enhancing the evaluation and communication of the benefits and risks associated with pharmaceutical products, thereby increasing the predictability, transparency, and efficiency of pharmaceutical regulatory decision making. In 2006, the IOM’s Forum on Drug Discovery, Development, and Translation held a workshop to explore the complex trade-off between drug benefits and risks and to examine approaches for better quantifying this balance and informing the public and the medical community. Since that time, FDA has worked to develop an enhanced structured approach to the assessment of benefits and risks in drug regulatory decision making to better communicate this aspect of the human drug review process. FDA envisions that this framework will serve as a template for product reviews, as
well as a vehicle for explaining the basis for FDA’s regulatory decisions.1 FDA’s work in this area coincides with efforts by other regulatory agencies, academia, and the pharmaceutical industry.
As FDA’s draft PDUFA V Implementation Plan (the Plan) indicates, an extensive body of evidence informs regulatory decisions on the safety and efficacy of a proposed product, but in many cases, FDA must draw conclusions from imperfect data. Identifying and evaluating sources of uncertainty (e.g., absence of information, conflicting findings, marginal results) in a regulatory application is an important part of reviewers’ work; however, drawing conclusions in the face of uncertainty can be a complex and challenging task. Effectively communicating regulatory decisions necessarily includes explanation of the impact of uncertainty on decision making. Uncertainty may arise from many sources; however, two particular areas of uncertainty that could benefit from additional attention are (1) the translation of premarket clinical trial data to the postmarket setting in which an approved drug is used in a much wider patient population, and (2) new findings that emerge in a postmarketing setting where the basis for the finding comes from sources of varying levels of rigor.
This public workshop will address the opportunity to advance the development of more systematic and structured approaches to characterize and communicate (a) the sources of uncertainty in the assessment of benefits and risks; and (b) their implications for pharmaceutical regulatory decisions. Specifically, the workshop will explore potential analytical and communication approaches and identify key considerations on their development, evaluation, and incorporation into the assessment of benefits and risks in pharmaceuticals. This workshop will consider the entire drug development lifecycle, including premarket drug review and postmarket safety surveillance.
The workshop objectives are to:
- Discuss the challenges in applying more systematic approaches to characterizing and communicating uncertainty in the assessment of a drug’s benefits and risks.
- Identify potential systematic approaches to address uncertainty faced by regulators in the assessment of benefits and risks in pharmaceuticals, drawing from various scientific and regulatory disciplines and domains.
__________________
1 FDA’s structured approach to benefit–risk assessment in drug regulatory decision making is outlined in the Draft PDUFA V Implementation Plan [February 2013], available at http://www.fda.gov/downloads/ForIndustry/UserFees/PrescriptionDrugUserFee/UCM329758.pdf (accessed August 23, 2014).
- Identify possible principles, best practices, and resources that can facilitate the development, evaluation, and incorporation of such approaches in regulatory decision making.
- Explore principles and approaches to facilitate the communication about uncertainty in the assessment of benefits and risks with FDA stakeholders.
First Workshop in the Series
9:00 a.m. | Opening Remarks |
BARUCH FISCHHOFF, Workshop Co-Chair | |
Howard Heinz University Professor | |
Department of Social and Decision Sciences | |
Department of Engineering and Public Policy | |
Carnegie Mellon University | |
ROBERT RATNER, Workshop Co-Chair | |
Chief Scientific and Medical Officer | |
American Diabetes Association | |
9:15 a.m. | The Importance of Considering Uncertainty in Regulatory Decision Making (15 min.) |
JANET WOODCOCK | |
Director, Center for Drug Evaluation and Research | |
U.S. Food and Drug Administration | |
SESSION I: APPROACHES TO EVALUATE UNCERTAINTY: MAXIMIZING THE VALUE OF THE EVIDENCE |
|
Session Objectives: | |
Discuss potential methods (proven and yet to be tried) to identify and evaluate sources of uncertainty. What structured systematic approaches to evaluating uncertainties could be considered by regulators? |
|
Acknowledge and discuss challenges in both identifying and addressing uncertainty in drug regulation. |
9:30 a.m. | Background and Session Objectives (5 min.) |
BARUCH FISCHHOFF, Workshop Co-Chair | |
Howard Heinz University Professor | |
Department of Social and Decision Sciences | |
Department of Engineering and Public Policy | |
Carnegie Mellon University | |
9:35 a.m. | Key Sources of Uncertainty in the Assessment of Benefits and Risks of Pharmaceuticals and Associated Challenges (15 min.) |
TAREK A. HAMMAD | |
Executive Director, Epidemiology | |
Merck Research Laboratories | |
Merck & Co., Inc. | |
9:50 a.m. | Identifying and Evaluating Uncertainty |
Addressing Challenges Arising from the Completeness of Data Collection in Clinical Trials (15 min.) | |
DEBORAH A. ZARIN | |
Director, ClinicalTrials.gov | |
National Library of Medicine | |
National Institutes of Health | |
Identifying and Retaining Subgroups in Clinical Trials in the Context of Uncertainty About the External Validity of Clinical Trials (15 min.) | |
MICHAELA KIERNAN | |
Senior Research Scientist | |
Stanford Prevention Research Center | |
Stanford University School of Medicine | |
Research Methodologies to Reduce or Address Uncertainties in the Evaluation of Pharmaceutical Benefits and Risks (15 min.) | |
SEBASTIAN SCHNEEWEISS | |
Professor of Medicine and Epidemiology | |
Harvard Medical School |
2:05 p.m. | Discussion with Speakers and Audience (20 min.) |
Discussion Moderator: Lisa LaVange, Director, Office of Biostatistics, Office of Translational Sciences, Center for Drug Evaluation and Research, U.S. Food and Drug Administration | |
2:25 p.m. | BREAK (15 min.) |
SESSION IV: REGULATORY DECISION MAKING UNDER UNCERTAINTY |
|
Session Objective: | |
Discuss potential approaches from decision theory that could be used in the regulatory setting (e.g., case studies), acknowledging that approaches will vary in the context of the unique uncertainties presented and that ultimately, the regulator will need to decide. |
|
2:40 p.m. | Background and Session Objective (5 min.) |
BARUCH FISCHHOFF, Workshop Co-Chair | |
Howard Heinz University Professor | |
Department of Social and Decision Sciences | |
Department of Engineering and Public Policy | |
Carnegie Mellon University | |
2:45 p.m. | Public Policy in an Uncertain World: Analysis and Decisions in Pharmaceutical Benefits and Risks (15 min.) |
CHARLES F. MANSKI | |
Board of Trustees Professor in Economics | |
Northwestern University | |
3:00 p.m. | Approaches Suggested from Decision Theory to Support Regulatory Decision Making Under Uncertainty |
Approaches to Eliciting Values for Uncertain Choices (15 min.) | |
TIMOTHY MCDANIELS | |
Faculty of Science | |
Institute of Resources and Environment | |
University of British Columbia |
CHARACTERIZING AND COMMUNICATING UNCERTAINTY IN THE ASSESSMENT OF BENEFITS AND RISKS OF PHARMACEUTICAL PRODUCTS: AN INSTITUTE OF MEDICINE WORKSHOP
May 12, 2014
U.S. Food and Drug Administration (FDA) Campus
White Oak, Maryland
BACKGROUND AND MEETING OBJECTIVES
There is increasing attention on the need for enhancing the evaluation and communication of the benefits and risks associated with pharmaceutical products, thereby increasing the predictability, transparency, and efficiency of pharmaceutical regulatory decision making. In 2006, the IOM’s Forum on Drug Discovery, Development, and Translation held a workshop to explore the complex trade-off between drug benefits and risks and to examine approaches for better quantifying this balance and informing the public and the medical community. Since that time, FDA has worked to develop an enhanced structured approach to the assessment of benefits and risks in drug regulatory decision making to better communicate this aspect of the human drug review process. FDA envisions that this framework will serve as a template for product reviews, as well as a vehicle for explaining the basis for FDA’s regulatory decisions.2 FDA’s work in this area coincides with efforts by other regulatory agencies, academia, and the pharmaceutical industry.
As FDA’s draft PDUFA V Implementation Plan (the Plan) indicates, an extensive body of evidence informs regulatory decisions on the safety and efficacy of a proposed product, but in many cases, FDA must draw conclusions from imperfect data. Identifying and evaluating sources of uncertainty (e.g., absence of information, conflicting findings, marginal results) in a regulatory application is an important part of reviewers’ work; however, drawing conclusions in the face of uncertainty can be a complex and challenging task. Effectively communicating regulatory decisions necessarily includes explanation of the impact of uncertainty on decision making. Uncertainty may arise from many sources; however, two particular areas of uncertainty that could benefit from additional attention are (1) the translation of premarket clinical trial data to the postmarket setting in which an approved drug is used in a much wider patient popu-
__________________
2 FDA’s structured approach to benefit–risk assessment in drug regulatory decision making is outlined in the Draft PDUFA V Implementation Plan [February 2013], available at http://www.fda.gov/downloads/ForIndustry/UserFees/PrescriptionDrugUserFee/UCM329758.pdf (accessed September 18, 2014).
lation, and (2) new findings that emerge in a postmarketing setting where the basis for the finding comes from sources of varying levels of rigor.
This two-part public workshop series will address the opportunity to advance the development of more systematic and structured approaches to characterize and communicate (a) the sources of uncertainty in the assessment of benefits and risks; and (b) their implications for pharmaceutical regulatory decisions. Specifically, the workshop series will explore potential analytical and communication approaches and identify key considerations on their development, evaluation, and incorporation into the assessment of benefits and risks in pharmaceuticals. This workshop series will consider the entire drug development lifecycle, including premarket drug review and postmarket safety surveillance.
The workshop series objectives are to:
- Discuss the challenges in applying more systematic approaches to characterizing and communicating uncertainty in the assessment of a drug’s benefits and risks.
- Identify potential systematic approaches to address uncertainty faced by regulators in the assessment of benefits and risks in pharmaceuticals, drawing from various scientific and regulatory disciplines and domains.
- Identify possible principles, best practices, and resources that can facilitate the development, evaluation, and incorporation of such approaches in regulatory decision making.
- Explore principles and approaches to facilitate the communication about uncertainty in the assessment of benefits and risks with FDA stakeholders.
Second Workshop in the Series |
|
9:00 a.m. | Welcome and Opening Remarks |
BARUCH FISCHHOFF, Workshop Co-Chair | |
Howard Heinz University Professor | |
Department of Social and Decision Sciences | |
Department of Engineering and Public Policy | |
Carnegie Mellon University | |
ROBERT RATNER, Workshop Co-Chair | |
Chief Scientific and Medical Officer | |
American Diabetes Association |