Characterizing and Communicating Uncertainty in the Assessment of Benefits and Risks of Pharmaceutical Products: Workshop Summary (2014)

Characterizing and Communicating Uncertainty
in the Assessment of Benefits and Risks
of Pharmaceutical Products

Workshop Summary

Denise Caruso, Rebecca A. English, and Anne B. Claiborne, Rapporteurs

Forum on Drug Discovery, Development, and Translation

Board on Health Sciences Policy

INSTITUTE OF MEDICINE
        OF THE NATIONAL ACADEMIES

THE NATIONAL ACADEMIES PRESS

Washington, D.C.

www.nap.edu

THE NATIONAL ACADEMIES PRESS     500 Fifth Street, NW     Washington, DC 20001

NOTICE: The workshop that is the subject of this workshop summary was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine.

This activity was supported by contracts between the National Academy of Sciences and the U.S. Department of Health and Human Services (HHSN26300023 [Under Base #HHSN263201200074I] and HHSF22301026T [Under Base #HHSF223200810020I]), AbbVie Inc., American Diabetes Association, American Society for Microbiology, Amgen Inc., Association of American Medical Colleges, AstraZeneca, Burroughs Wellcome Fund, Critical Path Institute, Doris Duke Charitable Foundation, Eli Lilly and Company, FasterCures, Friends of Cancer Research, GlaxoSmithKline, Johnson & Johnson, March of Dimes Foundation, Merck & Co., Inc., Novartis Pharmaceuticals Corporation, Pfizer Inc., and Sanofi. The views presented in this publication do not necessarily reflect the views of the organizations or agencies that provided support for the activity.

International Standard Book Number-13: 978-0-309-31000-0
International Standard Book Number-10: 0-309-31000-8

Additional copies of this workshop summary are available for sale from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.

For more information about the Institute of Medicine, visit the IOM home page at: www.iom.edu.

Copyright 2014 by the National Academy of Sciences. All rights reserved.

Printed in the United States of America

The serpent has been a symbol of long life, healing, and knowledge among almost all cultures and religions since the beginning of recorded history. The serpent adopted as a logotype by the Institute of Medicine is a relief carving from ancient Greece, now held by the Staatliche Museen in Berlin.

Suggested citation: IOM (Institute of Medicine). 2014. Characterizing and communicating uncertainty in the assessment of benefits and risks of pharmaceutical products: Workshop summary. Washington, DC: The National Academies Press.

“Knowing is not enough; we must apply.
Willing is not enough; we must do.”      

                                                —Goethe

INSTITUTE OF MEDICINE
           OF THE NATIONAL ACADEMIES

Advising the Nation. Improving Health.

THE NATIONAL ACADEMIES

Advisers to the Nation on Science, Engineering, and Medicine

The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences.

The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. C. D. Mote, Jr., is president of the National Academy of Engineering.

The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Victor J. Dzau is president of the Institute of Medicine.

The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. C. D. Mote, Jr., are chair and vice chair, respectively, of the National Research Council.

www.national-academies.org

PLANNING COMMITTEE ON CHARACTERIZING AND COMMUNICATING UNCERTAINTY IN THE ASSESSMENT OF BENEFITS AND RISKS OF PHARMACEUTICAL PRODUCTS¹

BARUCH FISCHHOFF (Co-Chair), Carnegie Mellon University, Pittsburgh, PA

ROBERT E. RATNER (Co-Chair), American Diabetes Association, Alexandria, VA

MARGARET ANDERSON, FasterCures, Washington, DC

CHRISTOPHER P. AUSTIN, National Center for Advancing Translational Sciences, Bethesda, MD

PATRICK J. FREY, Center for Drug Evaluation and Research, Silver Spring, MD

TAREK A. HAMMAD, Merck & Co., Inc., North Wales, PA

GAVIN HUNTLEY-FENNER, Huntley-Fenner Advisors, Irvine, CA

CHARLES F. MANSKI, Northwestern University, Evanston, IL

PAUL J. SELIGMAN, Amgen Inc., Washington, DC

LANA R. SKIRBOLL, Sanofi, Washington, DC

BRIAN L. STROM, Rutgers, the State University of New Jersey, Newark

MYRL WEINBERG, National Health Council, Washington, DC

STEVEN WOLOSHIN, Dartmouth Medical School, Hanover, NH

IOM Staff

ANNE B. CLAIBORNE, Forum Director

REBECCA A. ENGLISH, Program Officer

ELIZABETH F. C. TYSON, Associate Program Officer

BARRET ZIMMERMANN, Senior Program Assistant (until July 2014)

ANDREW M. POPE, Director, Board on Health Sciences Policy

__________________

¹ Institute of Medicine planning committees are solely responsible for organizing the workshop, identifying topics, and choosing speakers. The responsibility for the published workshop summary rests with the workshop rapporteurs and the institution.

This page intentionally left blank.

FORUM ON DRUG DISCOVERY, DEVELOPMENT, AND TRANSLATION¹

JEFFREY M. DRAZEN (Co-Chair), New England Journal of Medicine, Boston, MA

STEVEN K. GALSON (Co-Chair), Amgen Inc., Thousand Oaks, CA

RUSS BIAGIO ALTMAN, Stanford University, CA

MARGARET ANDERSON, FasterCures, Washington, DC

HUGH AUCHINCLOSS, National Institute of Allergy and Infectious Diseases, Bethesda, MD

CHRISTOPHER P. AUSTIN, National Center for Advancing Translational Sciences, Bethesda, MD

ANN C. BONHAM, Association of American Medical Colleges, Washington, DC

LINDA BRADY, National Institute of Mental Health, Bethesda, MD

GAIL H. CASSELL, Harvard Medical School (Visiting), Carmel, IN

ANDREW M. DAHLEM, Eli Lilly and Company, Indianapolis, IN

JAMES H. DOROSHOW, National Cancer Institute, Bethesda, MD

GARY L. FILERMAN, Atlas Health Foundation, McLean, VA

MARK J. GOLDBERGER, AbbVie Inc., Rockville, MD

HARRY B. GREENBERG, Stanford University School of Medicine, CA

KATHY L. HUDSON, National Institutes of Health, Bethesda, MD

LYNN D. HUDSON, Critical Path Institute, Tucson, AZ

S. CLAIBORNE JOHNSTON, Dell Medical School, University of Texas, Austin

MICHAEL KATZ, March of Dimes Foundation, White Plains, NY

JACK D. KEENE, Duke University Medical Center, Durham, NC

RUSTY KELLEY, Burroughs Wellcome Fund, Research Triangle Park, NC

RONALD L. KRALL, University of Pittsburgh, Steamboat Springs, CO

FREDA C. LEWIS-HALL, Pfizer Inc., New York, NY

CAROL MIMURA, University of California, Berkeley

BRIGGS W. MORRISON, AstraZeneca, Summit, NJ

BERNARD H. MUNOS, InnoThink Center for Research in Biomedical Innovation, Indianapolis, IN

ELIZABETH (BETSY) MYERS, Doris Duke Charitable Foundation, New York, NY

JOHN J. ORLOFF, Baxter BioScience, Deerfield, IL

RAJESH RANGANATHAN, National Institute of Neurological Disorders and Stroke, Bethesda, MD

__________________

¹ Institute of Medicine forums and roundtables do not issue, review, or approve individual documents. The responsibility for the published workshop summary rests with the workshop rapporteurs and the institution.

ROBERT E. RATNER, American Diabetes Association, Alexandria, VA

MICHAEL ROSENBLATT, Merck & Co., Inc., Whitehouse Station, NJ

JAMES S. SHANNON, GlaxoSmithKline, Brentford, Middlesex, UK

ELLEN V. SIGAL, Friends of Cancer Research, Washington, DC

LANA R. SKIRBOLL, Sanofi, Washington, DC

BRIAN L. STROM, Rutgers, the State University of New Jersey, Newark

JANET TOBIAS, Ikana Media, New York, NY

JOANNE WALDSTREICHER, Johnson & Johnson, New Brunswick, NJ

JANET WOODCOCK, U.S. Food and Drug Administration, Rockville, MD

IOM Staff

ANNE B. CLAIBORNE, Forum Director

REBECCA A. ENGLISH, Program Officer

CHRISTOPHER J. DEFEO, Program Officer

ELIZABETH F. C. TYSON, Associate Program Officer

BARRET ZIMMERMANN, Senior Program Assistant (until July 2014)

KATHRYN HOWELL, Senior Program Assistant (as of August 2014)

ANDREW M. POPE, Director, Board on Health Sciences Policy

Reviewers

This workshop summary has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the National Research Council’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published workshop summary as sound as possible and to ensure that the workshop summary meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the process. We wish to thank the following individuals for their review of this workshop summary:

Patricia Furlong, Parent Project Muscular Dystrophy

Elaine K. Gallin, QE Philanthropic Advisors

Robert J. Meyer, University of Virginia School of Medicine

Michael Rosenblatt, Merck & Co., Inc.

Although the reviewers listed above have provided many constructive comments and suggestions, they did not see the final draft of the workshop summary before its release. The review of this workshop summary was overseen by Eli Y. Adashi, The Warren Alpert Medical School, Brown University. Appointed by the Institute of Medicine, he was responsible for making certain that an independent examination of this workshop

summary was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this workshop summary rests entirely with the rapporteurs and the institution.

Contents

ACRONYMS

1   INTRODUCTION

The Impact of Uncertainty on Regulatory Decision Making

2   IDENTIFYING AND CHARACTERIZING UNCERTAINTY

Key Sources of Uncertainty in Benefit–Risk Assessment and Associated Challenges

Reducing Uncertainty Through Maximizing the Value of Evidence

Methods to Address Uncertainty in the Postmarket Phase

Approaches to Assessing Internal and External Validity of RCTs

3   THE REGULATORS’ CHALLENGE

Unique Challenges of the Pharmaceutical Regulatory Setting

Communicating with the Public About Benefit, Harm, and Uncertainty

4   BASIC METHODOLOGIES AND APPLICATIONS FOR UNDERSTANDING AND EVALUATING UNCERTAINTY

Applying Decision Science in the Drug Review Process

Applying Decision Theory Approaches to Regulatory Decision Making

FDA Approaches to Decision Making

5   COMMUNICATING UNCERTAINTY

Overview of Risk Communication

Increasing Public Awareness and Understanding of Uncertainty

Communicating Uncertainty: FDA and Media Messages About Tysabri

Challenges and Opportunities to Improve Communication About Uncertainty

6   FINAL REFLECTIONS ON WAYS TO CHARACTERIZE AND COMMUNICATE UNCERTAINTY

Identifying and Mitigating Uncertainty Through Maximizing the Value of Evidence

Characterizing and Understanding Uncertainties

Eliciting Values from Stakeholders, Particularly Patients

Communicating Uncertainty About Benefit and Risk Assessments of Pharmaceutical Products

REFERENCES

APPENDIXES

A   WORKSHOP AGENDA

B   FDA CASE STUDIES

C   BIBLIOGRAPHY

D   PARTICIPANT BIOGRAPHIES

Boxes, Figures, and Table

BOXES

1-1   FDA PDUFA V Plan and the Characterization of Uncertainties in Benefits and Risks

1-2   Statement of Task for the Workshops

1-3   The Range of Sources of Uncertainty

1-4   Themes Observed by Workshop Co-Chairs

2-1   Considerations for Designing an Approach to Addressing Uncertainty

3-1   FDA Approaches to Inform Risk Communication

3-2   Exploring Patient Perspectives on Uncertainty Through PFDD

3-3   Decisions Made Under Uncertainty: Tysabri and Anoro Ellipta Case Studies

5-1   Suggested Routine Disclosures for Regulators

FIGURES

1-1   FDA benefit–risk framework

2-1   A system of proactively, prospectively designed interlocking randomized controlled trial (RCT) and observational studies to reduce uncertainty, retain patient populations, and gather information that would not be possible in an RCT or observational study alone

4-1   Example of inconsistent use of verbal probability terms from two standards produced by the same government department

5-1   Decision tree for Plan B use after suspected contraceptive failure

5-2   Tysabri prescribing information appearing on page 4 of the package insert

5-3   A Tysabri Drug Facts Box suggested by Woloshin and Schwartz to concisely highlight what was known, still being studied, and unknown at the time of Tysabri’s accelerated approval

5-4   Media response to 2004 Tysabri approval announcement

TABLE

3-1   Hierarchy of Scientific Evidence

Acronyms