CLINICAL PRACTICE GUIDELINES WE CAN TRUST
Robin Graham, Michelle Mancher, Dianne Miller Wolman, Sheldon Greenfield, and Earl Steinberg, Editors
INSTITUTE OF MEDICINE
OF THE NATIONAL ACADEMIES
THE NATIONAL ACADEMIES PRESS
THE NATIONAL ACADEMIES PRESS
500 Fifth Street, N.W.
Washington, DC 20001
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.
This study was supported by Contract No. HHSP23320042509XI between the National Academy of Sciences and the Department of Health and Human Services. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the organizations or agencies that provided support for this project.
Library of Congress Cataloging-in-Publication Data
Institute of Medicine (U.S.). Committee on Standards for Developing Trustworthy Clinical Practice Guidelines.
Clinical practice guidelines we can trust / Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, Board on Health Care Services, Institute of Medicine of the National Academies ; Robin Graham ... [et al.], editors.
p. ; cm.
Includes bibliographical references.
ISBN 978-0-309-16422-1 (pbk.) — ISBN 978-0-309-16423-8 (pdf)
1. Medicine—Practice—Standards—United States. 2. Total quality management—United States. I. Graham, Robin, 1952- II. Title.
[DNLM: 1. Practice Guidelines as Topic—standards—United States. 2. Total Quality Management—standards—United States. W 84.4 AA1]
Additional copies of this report are available from the
National Academies Press,
500 Fifth Street, N.W., Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area); Internet, http://www.nap.edu.
For more information about the Institute of Medicine, visit the IOM home page at: www.iom.edu.
Copyright 2011 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
The serpent has been a symbol of long life, healing, and knowledge among almost all cultures and religions since the beginning of recorded history. The serpent adopted as a logotype by the Institute of Medicine is a relief carving from ancient Greece, now held by the Staatliche Museen in Berlin.
Cover credit: Photograph by Mai Le.
Suggested citation: IOM (Institute of Medicine). 2011. Clinical Practice Guidelines We Can Trust. Washington, DC: The National Academies Press.
THE NATIONAL ACADEMIES
Advisers to the Nation on Science, Engineering, and Medicine
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council.
COMMITTEE ON STANDARDS FOR DEVELOPING TRUSTWORTHY CLINICAL PRACTICE GUIDELINES
SHELDON GREENFIELD (Chair), Donald Bren Professor of Medicine and Executive Director,
Health Policy Research Institute, University of California–Irvine
EARL PHILLIP STEINBERG (Vice Chair), Senior VP,
Clinical Strategy, Quality and Outcomes, WellPoint, Inc., and
President and CEO,
Resolution Health, Inc., Columbia, MD
ANDREW AUERBACH, Associate Professor of Medicine in Residence,
School of Medicine, University of California–San Francisco
JERRY AVORN, Professor of Medicine,
Harvard Medical School and
Division of Pharmacoepidemiology and Pharmaeconomics, Brigham and Women’s Hospital, Boston, MA
ROBERT S. GALVIN, Chief Executive Officer,
Equity Healthcare, and
Corporate Private Equity, The Blackstone Group
RAYMOND GIBBONS, Consultant in CV Disease and Internal Medicine,
Mayo Clinic, Rochester, MN
JOSEPH LAU, Director,
Tufts EPC, Tufts Medical Center, Boston, MA
MONICA MORROW, Chief,
Breast Service and
Anne Burnett Windfohr Chair of Clinical Oncology,
Memorial Sloan-Kettering Cancer Center, and
Professor of Surgery,
Weill Medical College of Cornell University, New York, NY
CYNTHIA D. MULROW, Senior Deputy Editor,
Annals of Internal Medicine, American College of Physicians, San Antonio, TX
ARNOLD J. ROSOFF, Professor of Legal Studies & Health Care Management,
The Wharton School;
Professor of Family Medicine and Community Health,
School of Medicine; and
The Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA
JOHN SANTA, Director,
Consumer Reports Health Ratings Center, Yonkers, NY
RICHARD N. SHIFFMAN, Professor of Pediatrics,
Yale School of Medicine, New Haven, CT
WALLY SMITH, Professor and Chair,
Division of Quality Health Care, and
Center on Health Disparities, Virginia Commonwealth University, Richmond, VA
WALTER F. STEWART, Associate Chief Research Officer,
Center for Health Research, Geisinger Health System, Danville, PA
ELLEN STOVALL, Senior Health Policy Advisor,
National Coalition for Cancer Survivorship, Silver Spring, MD
BRIAN L. STROM, Chair and Professor,
Department of Biostatistics and Epidemiology, University of Pennsylvania, Philadelphia, PA
MARITA G. TITLER, Professor and Associate Dean,
Office of Practice and Clinical Scholarship, University of Michigan School of Nursing, Ann Arbor, MI
ROBIN GRAHAM, Senior Program Officer, Study Director
DIANNE MILLER WOLMAN, Senior Program Officer (through December 2010)
MICHELLE MANCHER, Research Associate
ALLISON McFALL, Senior Program Assistant (through August 2010)
JILLIAN LAFFREY, Senior Program Assistant (starting July 2010)
ROGER HERDMAN, Director,
Board on Health Care Services
DAVID ATKINS, Director,
Quality Enhancement Research Initiative, Veterans Health Administration
RONEN AVRAHAM, Thomas Shelton Maxey Professor of Law,
University of Texas School of Law
JONATHAN D. DARER, Medical Director of Clinical Transformation, Innovation,
Geisinger Health System
MARTIN ECCLES, Professor of Clinical Effectiveness and The William Leech Professor of Primary Care Research,
Institute of Health and Society, Newcastle University
JEREMY GRIMSHAW, Director,
Clinical Epidemiology Program, Ottawa Hospital Research Institute
J.B. JONES, Research Investigator I,
Geisinger Center for Health
MORI KRANTZ, Associate Professor,
Cardiology, University of Colorado, Denver Health Medical Center
WILLIAM SAGE, Vice Provost for Health Affairs and James R. Dougherty Chair for Faculty Excellence in Law,
University of Texas–Austin
ANNE SALES, Adjunct Professor,
Department of Family Medicine, Faculty of Medicine and Dentistry, University of Alberta
HOLGER SCHÜNEMANN, Chair,
Department of Clinical Epidemiology & Biostatistics, McMaster University
PAUL SHEKELLE, Medical Center Director,
Quality Assessment and Improvement Program, RAND Health
DEAN SITTIG, Professor,
School of Biomedical Informatics, UT–Memorial Hermann Center for Healthcare Quality & Safety, University of Texas Health Science Center at Houston
LEIF SOLBERG, Director for Care Improvement Research,
HealthPartners Research Foundation
STEVE WOOLF, Director,
VCU Center on Human Needs,
Professor of Family Medicine,
Virginia Commonwealth University
This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the National Research Council’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report:
JESSE A. BERLIN, Vice President, Epidemiology, Johnson & Johnson Pharmaceutical Research and Development
RICHARD BERMAN, President and CEO, LICAS
JAKO BURGERS, Senior Researcher, IQ healthcare, Radboud University, Nijmegen Medical Centre
MARJORIE FUNK, Professor, Yale School of Nursing
ALAN M. GARBER, Henry J. Kaiser, Jr. Professor and Professor of Medicine, Director, Center for Health Policy, Director, Center for Primary Care and Outcomes Research, Stanford University
MARGUERITE A. KOSTER, Practice Leader, Technology Assessment and Guidelines Unit, Kaiser Permanente Southern California
GREG PAWLSON, Executive Vice President, National Committee for Quality Assurance
DAVID RANSOHOFF, Professor of Medicine, Clinical Professor of Epidemiology, Director, Clinical Research Curriculum (K30), University of North Carolina School of Medicine
CAROL SAKALA, Director of Programs, Childbirth Connection
VINCENZA SNOW, Medical Director, Vaccines, Specialty Care Medical Affairs, Pfizer, Inc.
SHOSHANNA SOFAER, Professor, and Robert P. Luciano Chair of Health Care Policy, Baruch College, The City University of New York
MARY E. TINETTI, Department of Internal Medicine/Geriatrics, Yale University School of Medicine
KATRIN UHLIG, Division of Nephrology, Tufts Medical Center
JAMES WEINSTEIN, Director, The Dartmouth Institute for Health Policy and Clinical Practice; President, The Dartmouth-Hitchcock Clinic; Peggy Y. Thomson Professor in the Evaluative Clinical Sciences
Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations nor did they see the final draft of the report before its release. The review of this report was overseen by ENRIQUETA C. BOND of the Burroughs Wellcome Fund, and MARK R. CULLEN of Stanford University. Appointed by the National Research Council and the Institute of Medicine, they were responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution.
Many factors enter into healthcare decisions. What alternatives are available? What does the evidence suggest about their potential benefits and harms? How firm is the evidence? Is there reason to adjust expectations based on a particular patient’s age, gender, race, comorbidities, or other attributes? How might different patient preferences affect the best choice for a particular patient? Are there any social, economic, or other practical considerations that could affect the results of a particular care option?
Clinical Practice Guidelines (CPGs) are intended to provide a systematic aid to making such complex medical decisions. When rigorously developed using a transparent process that combines scientific evidence, clinician experiential knowledge, and patient values, CPGs have the potential to improve many clinician and patient healthcare decisions, and enhance healthcare quality and outcomes.
The present state of CPG development has yet to fully meet this promise. At the request of the U.S. Congress, the Institute of Medicine (IOM) undertook this study to develop a set of standards for developing rigorous, trustworthy clinical practice guidelines. The proposed standards cover a number of elements essential to developing sound practice guidelines, including transparency; conflict of interest; guideline development group composition; CPG–SR intersection; establishing evidence foundations for and strength of recommendations; articulation of recommendations; external review; and updating. This report and the eight proposed stan-
dards it contains are intended to reinforce the work of numerous researchers, developers and users of guidelines. This report clarifies where evidence and expert consensus buttress best CPG development practices, and where there is still much to learn. We hope and expect these standards to be pilot-tested, assessed for reliability and validity, evaluated for effectiveness, and to evolve as science and experience dictate.
I want to thank the excellent committee who conducted this work, ably led by Sheldon Greenfield, chair, and Earl Steinberg, vice chair. The committee was assisted by dedicated IOM staff led by Robin Graham. A companion report will set out standards for conducting systematic reviews of comparative effectiveness research. I hope that these reports together will advance the state of the art of systematic review and clinical practice guideline development, and contribute to a more transparent, scientifically rigorous, and patient-centered healthcare system in the United States.
Harvey V. Fineberg, M.D., Ph.D.
President, Institute of Medicine
In the early 1990s, the Institute of Medicine (IOM) issued several reports on clinical practice guidelines (CPGs). In the ensuing years, CPGs and guideline development groups have proliferated enormously to the point that the Agency for Healthcare Research and Quality’s National Guideline Clearinghouse contains nearly 2,700 CPGs. Parallel growth in CPGs has occurred in other countries; the Guidelines International Network’s database currently lists more than 3,700 CPGs.
Although the numbers of CPGs and CPG developers have increased substantially, our understanding of the impact of CPGs on clinical practice and patient outcomes is limited. However, research has shown that CPGs have the potential to reduce inappropriate practice variation, enhance translation of research into practice, and improve healthcare quality and safety. CPGs also have had an important influence on development of physician and hospital performance measures. The data gathered from use of such measures have provided consumers with information on the quality of different healthcare providers and, in some instances, provided physicians and hospitals with an economic incentive to improve quality of care.
At the same time, there has been considerable concern expressed by physicians, consumer groups, and other stakeholders about the quality of the processes supporting development of CPGs, and the resulting questionable validity of many CPGs and CPG-based clini-
cal performance measures. Specifically, this concern extends from limitations in the scientific evidence base on which CPGs rely; a lack of transparency of development groups’ methodologies; conflict of interest among guideline development group members and funders; and questions regarding how to reconcile conflicting guidelines. In light of these challenges, and in response to the growing demand for insight into the quality of care being delivered in conjunction with rising healthcare costs and the strong indications of the need to improve clinical decision making and healthcare quality, a provision was included in the Medicare Improvements for Patients and Providers Act of 2008. It directed the IOM to form two separate, but related, committees to develop standards for systematic reviews (SRs) of comparative effectiveness research and for CPGs. If standards for development of valid SRs and CPGs were available, then clinicians and the public should have greater trust in standards-based CPGs and clinical performance measures founded on them. Standards for development of trustworthy CPGs additionally could foster the easier translation of guidelines into electronic forms of clinical decision support.
When the CPG committee was formed, we regarded the charge as more or less updating the state of the art based on accumulated experience and advances in thinking. As we delved into our work, however, we recognized that the rapid growth of CPG development efforts had resulted in substantial variation in CPG development processes. If CPGs were to have their intended impacts, there was a pressing need for standards regarding many dimensions of guideline development, including the potential for conflict of interest; the importance of transparency of the guideline development process; the appropriate type and level of patient and public input into the CPG development process; the need for clarity regarding the reasoning supporting each CPG recommendation; the approaches used to rate the quality of evidence underlying and strength of each CPG recommendation; the need to ensure that CPGs take account of patients with coexisting conditions; and the relationship between individuals who develop a guideline and those who perform SRs on topics relevant to the CPG. The committee found no existing set of standards that addressed all of the above elements or offered prospective guidance for developing high-quality, trustworthy CPGs. Thus, the committee proposes its own standards.
The diversity of talents and experiences of the committee members made our task more complicated and challenging than we had anticipated, but ultimately resulted in a highly thoughtful, rich report. Academicians from a variety of disciplines, experts from various types of stakeholder entities, and a diverse array of individ-
uals involved in guideline development and implementation participated in our deliberations and contributed to this report. More than 2,500 publications were reviewed by staff and committee members; a public forum was conducted for organizations that develop and want to use CPGs; and several papers were commissioned to enable the committee to gain as much perspective as possible. The extraordinary efforts of Robin Graham, Study Director, and Michelle Mancher and Dianne Wolman made the task possible.
The two of us express our great appreciation to the committee members and staff for their commitment, effort, dedication, and wisdom. The spirited discussions during meetings and the frequent communications between meetings all contributed to this report. We hope the committee’s findings and proposed standards and recommendations will foster trustworthy CPGs that increase quality of care and improve patient outcomes.
Sheldon Greenfield, Chair
Earl Phillip Steinberg, Vice Chair
Committee on Standards for Developing Trustworthy Clinical Practice Guidelines
The committee and staff are indebted to a number of individuals and organizations for their contributions to this report. The following individuals testified before the committee during public meetings or workshops:
William G. Adams, American Academy of Pediatrics
David Atkins, Veterans Administration
Michael Bettmann, American College of Radiology
Zobeida Bonilla, Our Bodies Ourselves
Kent Bottles, Institute for Clinical Systems Improvement
Cynthia Boyd, Physician Expert in Multimorbidity, Johns Hopkins Department of Medicine
Arleen Brown, Physician Expert in Health Disparities, UCLA Internal Medicine
Vivian Coates, National Guideline Clearinghouse/ECRI Institute
Joyce Dubow, AARP
Laura Fochtmann, American Psychiatric Association
Ted Ganiats, American Academy of Family Physicians
Alice Jacobs, American College of Cardiology and American Heart Association
Louis B. Jacques, Centers for Medicare & Medicaid Services
Richard Kahn, Former Chief Scientific and Medical Officer of the American Diabetes Association
Karen Kelly-Thomas, National Association of Pediatric Nurse Practitioners
Marguerite Koster, Kaiser Permanente Southern California
Sandra Zelman Lewis, American College of Chest Physicians
Joan McClure, National Comprehensive Cancer Network
Elizabeth Mort, Quality and Safety, Massachusetts General Hospital, Massachusetts General Physicians Organization
Jim Schibanoff, Milliman Care Guidelines
Nita L. Siebel, National Cancer Institute
Denise Simons-Morton, National Heart, Lung, and Blood Institute
Vincenza Snow, American College of Physicians
Katrin Uhlig, Tufts Medical Center, Boston
Public Forum Testifiers
Ethan Basch, American Society of Clinical Oncology
Christopher Bever, American Academy of Neurology
Terrie Cowley, The TMJ Association
Steven Findlay, Consumers Union
Merrill Goozner, Health Tech Review/GoozNews.com
David Paul Harries, International Spine Intervention Society
Belinda Ireland, BJC HealthCare
Norman Kahn, Council of Medical Specialty Societies
Lisa Mojer-Torres, Chair, Citizens’ Advisory Council for the Division of Addiction Services, State of New Jersey
Katherine Nordal, American Psychological Association
Jennifer Padberg, Infectious Diseases Society of America (IDSA)
William Rich, American Academy of Ophthalmology
Richard Rosenfeld, American Academy of Otolaryngology–Head and Neck Surgery
Kathleen Sazama, Society for the Advancement of Blood Management (SABM)
Aryeh Shander, Society for the Advancement of Blood Management (SABM)
Christopher Wolfkiel, American College of Occupational and Environmental Medicine (ACOEM)
Diana Zuckerman, National Research Center for Women & Families
We also extend special thanks to the following individuals who were essential sources of information, generously giving their time and knowledge to further the committee’s efforts:
Josh Adler, Vice President, Business Development, Archimedes, Inc.
Ned Calogne, Chief Medical Officer, Colorado Department of Public Health and Environment
Paul Chrisp, Associate Director, Accreditation, NHS Evidence, NICE
David Eddy, Founder, Archimedes, Inc.
Jeremy Grimshaw, Director, Clinical Epidemiology Program, Ottawa Hospital Research Institute
Richard Grol, Chair, Department of Quality of Care, Radboud University Nijmegen; Director, Centre for Quality of Care Research (WOK)
Russell Harris, Professor of Medicine, Adjunct Professor of Epidemiology, Division of General Medicine, University of North Carolina at Chapel Hill
Greg Pawlson, Executive Vice President, NCQA
David Ransohoff, Professor of Medicine, Clinical Professor of Epidemiology, Director, Clinical Research Curriculum, University of North Carolina School of Medicine
Holger Schünemann, Chair, Department of Clinical Epidemiology & Biostatistics, McMaster University
Funding for this study was provided by the Agency for Healthcare Research and Quality (AHRQ). The committee appreciates the opportunity and support extended by AHRQ for the development of this report.
Finally, many within the IOM were helpful to the study staff. We especially would like to thank Jillian Laffrey, a new member of our staff, for her great ingenuity and dedication, and the staff of the Systematic Reviews report for their insight and cooperation: Lea Binder, Jill Eden, Mai Le, and Laura Levit. Additionally, the staff would like to thank Clyde Behney, Greta Gorman, Cheryl Levey, William McLeod, Abbey Meltzer, Vilija Teel, and Lauren Tobias.
Boxes, Figures, and Tables
Recommended Attributes of CPGs,
Criteria for Inclusion in The National Guideline Clearinghouse (NGC),
Infectious Diseases Society of America Lyme Disease Guidelines (2006),
Colorectal Cancer Screening Guidelines (2008),
National Kidney Foundation’s Kidney Disease and Outcomes Quality Initiative Anemia Management Guidelines (2006),
Models of Interaction Between Clinical Practice Guideline (CPG) Groups and Systematic Review (SR) Teams,
Analytic framework and KQs,
Interpretation of the American College of Physicians’ Guideline Grading System,
Guidelines with Conflicting Recommendations,
Examples of Tools for Measuring the Quality of Clinical Practice Guidelines Development,
Selected Approaches to Rating Strength of Evidence and Clinical Recommendations,