National Academies Press: OpenBook
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×

FINDING WHAT WORKS IN HEALTH CARE

STANDARDS FOR SYSTEMATIC REVIEWS

Committee on Standards for Systematic Reviews of Comparative Effectiveness Research

Board on Health Care Services

Jill Eden, Laura Levit, Alfred Berg, and Sally Morton, Editors

INSTITUTE OF MEDICINE
OF THE NATIONAL ACADEMIES

THE NATIONAL ACADEMIES PRESS

Washington, D.C.
www.nap.edu

Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×

THE NATIONAL ACADEMIES PRESS
500 Fifth Street, N.W.
Washington, DC 20001

NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.

This study was supported by Contract No. HHSP23320042509X1 between the National Academy of Sciences and the Department of Health and Human Services. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the organizations or agencies that provided support for this project.

Library of Congress Cataloging-in-Publication Data

Institute of Medicine (U.S.). Committee on Standards for Systematic Reviews of Comparative Effectiveness Research.

Finding what works in health care : standards for systematic reviews / Committee on Standards for Systematic Reviews of Comparative Effectiveness Research, Board on Health Care Services, Institute of Medicine of the National Academies ; Jill Eden … [et al.], editors.

p. ; cm.

Includes bibliographical references.

ISBN 978-0-309-16425-2 (pbk.) — ISBN 978-0-309-16426-9 (pdf)

1. Medical care—Standards—United States. 2. Medical care—United States—Quality control. I. Eden, Jill. II. Title.

[DNLM: 1. Comparative Effectiveness Research—standards—United States. 2. Outcome and Process Assessment (Health Care)—United States. W 84.3]

RA399.A3I565 2011

610.28’9—dc23

2011017455

Additional copies of this report are available from the

National Academies Press,

500 Fifth Street, N.W., Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area); Internet, http://www.nap.edu.

For more information about the Institute of Medicine, visit the IOM home page at: www.iom.edu.

Copyright 2011 by the National Academy of Sciences. All rights reserved.

Printed in the United States of America

The serpent has been a symbol of long life, healing, and knowledge among almost all cultures and religions since the beginning of recorded history. The serpent adopted as a logotype by the Institute of Medicine is a relief carving from ancient Greece, now held by the Staatliche Museen in Berlin.

Suggested citation: IOM (Institute of Medicine). 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press.

Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×

“Knowing is not enough; we must apply.

Willing is not enough; we must do.”

—Goethe

INSTITUTE OF MEDICINE
OF THE NATIONAL ACADEMIES


Advising the Nation. Improving Health.

Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×

THE NATIONAL ACADEMIES

Advisers to the Nation on Science, Engineering, and Medicine


The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences.


The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering.


The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine.


The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council.


www.national-academies.org

Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×

COMMITTEE ON STANDARDS FOR SYSTEMATIC REVIEWS OF COMPARATIVE EFFECTIVENESS RESEARCH

ALFRED O. BERG (Chair), Professor,

Department of Family Medicine, University of Washington School of Medicine, Seattle, WA

SALLY C. MORTON (Vice Chair), Professor and Chair,

Department of Biostatistics, Graduate School of Public Health, University of Pittsburgh

JESSE A. BERLIN, Vice President,

Epidemiology, Johnson & Johnson Pharmaceutical Research and Development, Titusville, NJ

MOHIT BHANDARI, Canada Research Chair in Musculoskeletal Trauma,

McMaster University, Orthopaedic Research Unity, Clarity Research Group, at the Hamilton Health Sciences–General Site (joined October 2009; resigned July 2010)

GISELLE CORBIE-SMITH, Professor,

Social Medicine, Medicine and Epidemiology, Departments of Social Medicine, Medicine and Epidemiology, University of North Carolina–Chapel Hill, Chapel Hill, NC

KAY DICKERSIN, Professor of Epidemiology, Director,

Center for Clinical Trials and U.S. Cochrane Center, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD

JEREMY M. GRIMSHAW, Director,

Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada

MARK HELFAND, Director,

Oregon Evidence-Based Practice Center,

Professor of Medicine and Medical Informatics and Clinical Epidemiology,

Oregon Health and Science University, Portland, OR

VINCENT E. KERR, President,

Care Solutions, UnitedHealthcare, Fairfield, CT

MARGUERITE A. KOSTER, Practice Leader,

Technology Assessment and Guidelines Unit, Kaiser Permanente Southern California, Pasadena, CA

KATIE MASLOW, Scholar-in-Residence,

Institute of Medicine, Washington, DC

DAVID A. MRAZEK, Chair,

Department of Psychiatry and Psychology,

Professor of Pediatrics and Psychiatry,

College of Medicine, Mayo Clinic, Rochester, MN

CHRISTOPHER H. SCHMID, Director,

Biostatistics Research Center;

Professor of Medicine,

Tufts Medical Center and Tufts University, Boston, MA

ANNA MARIA SIEGA-RIZ, Professor of Epidemiology and Nutrition, Associate Chair of Epidemiology,

University of North Carolina Gillings School of Global Public Health, Chapel Hill, NC

Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×

HAROLD C. SOX, Editor Emeritus,

Annals of Internal Medicine, American College of Physicians of Internal Medicine, Hanover, NH

PAUL WALLACE, Medical Director,

The Permanente Federation, Kaiser Permanente, Oakland, CA

Study Staff

JILL EDEN, Study Director

LAURA LEVIT, Program Officer

LEA BINDER, Research Associate (through May 2010)

MAI LE, Research Assistant (starting September 2010)

ALLISON McFALL, Senior Program Assistant (through August 2010)

JILLIAN LAFFREY, Senior Program Assistant (starting July 2010)

ROGER HERDMAN, Director,

Board on Health Care Services

Consultants

JULIA KREIS, Harkness/Bosch Fellow in Health Care Policy and Practice,

Johns Hopkins Bloomberg School of Public Health, Baltimore, MD

DAVID MOHER, Senior Scientist,

Ottawa Hospital Research Institute (OHRI), Ottawa, Ontario, Canada

ALEXANDER TSERTSVADZE,

Ottawa Hospital Research Institute (OHRI), Ottawa, Ontario, Canada

SALLY HOPEWELL,

National Institute for Health Research, U.K. Cochrane Centre, Oxford, United Kingdom

BRYAN LUCE, Senior Vice President,

Science Policy, United BioSource Corporation, Chevy Chase, MD

RACHAEL FLEURENCE, Senior Research Scientist,

United BioSource Corporation, Chevy Chase, MD

Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×

Reviewers

This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the National Research Council’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report:

NAOMI ARONSON, Executive Director, Blue Cross and Blue Shield Association, Technology Evaluation Center

VIVIAN COATES, Vice President, Information Services and Technology Assessment, ECRI Institute

JOHN P. A. IOANNIDIS, C.F. Rehnborg Professor in Disease Prevention, Professor of Medicine, and Director, Stanford Prevention Research Center, Stanford University

ERIC B. LARSON, Executive Director, Group Health Research Institute

JOSEPH LAU, Director, Tufts Evidence-based Practice Center, Tufts Medical Center

Page viii Cite
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×

ART LEVIN, Director, Center for Medical Consumers

ALVIN I. MUSHLIN, Professor and Chairman, Department of Public Health, Weill Medical College, Cornell University

PAUL SHEKELLE, Director, Southern California Evidence-Based Practice Center, The RAND Corporation

RICHARD N. SHIFFMAN, Associate Director, Center for Medical Informatics, Yale School of Medicine

LESLEY STEWART, Director, Centre for Reviews and Dissemination, University of York

Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations nor did they see the final draft of the report before its release. The review of this report was overseen by ENRIQUETA C. BOND of the Burroughs Wellcome Fund, and MARK R. CULLEN of Stanford University. Appointed by the National Research Council and the Institute of Medicine, they were responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution.

Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×

Foreword

Knowing what works in health care is of highest importance for patients, healthcare providers, and other decision makers. The most reliable way to identify benefits and harms associated with various treatment options is a systematic review of comparative effectiveness research. Increasingly recognized for their importance, systematic reviews are now being sponsored and conducted by a number of organizations across the United States. When conducted well, a systematic review identifies, appraises, and synthesizes the available body of evidence for a specific clinical question. However, not all of these reviews meet the appropriate standards of quality and methodology. At the request of the U.S. Congress, the Institute of Medicine (IOM) undertook this study to develop a set of standards for conducting systematic reviews of comparative effectiveness research.

The report will have direct implications for implementation of the Patient Protection and Affordable Care Act of 2010. This law established the first nonprofit, public–private Patient-Centered Outcomes Research Institute (PCORI). PCORI will be responsible for setting methodological standards for clinical effectiveness research, including systematic reviews of research findings. I hope this study will support PCORI’s development of standards to ensure that systematic reviews meet a minimum level of objectivity, transparency, and scientific rigor. The IOM study should also help to inform other

Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×

public sponsors of systematic reviews of comparative effectiveness research.

To conduct this study, the Institute of Medicine convened a highly qualified committee with diverse backgrounds, ably led by Alfred Berg, chair, and Sally Morton, vice chair. The committee was assisted by dedicated IOM staff led by Jill Eden. This report draws on available evidence, review of expert guidance, and careful consideration of alternative standards according to specified criteria. While this report presents an initial list of standards for improving the quality of publicly funded systematic reviews, it also calls for continued investment in methodological research to identify better practices for future reviews. A companion report establishes standards for developing clinical practice guidelines. I hope these documents will help guide a robust systematic review enterprise for health in the United States.


Harvey V. Fineberg, M.D., Ph.D.

President, Institute of Medicine

February 2011

Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×

Preface

Page through any volume of a medical journal from the 1970s and read a clinical review. The authors are likely to be recognized as experts in the field, and the introduction will often open with “we reviewed the world’s medical literature,” moving on to reach clinical conclusions based as much on the experience and opinions of the authors as on the published evidence. Systematic literature searches, critical appraisal, quantitative meta-analysis, and documented pathways linking the evidence to reaching clinical conclusions were virtually unknown. Today’s explicit, scientifically rigorous, transparent, and publicly accountable systematic reviews (SRs) and clinical practice guidelines (CPGs) are the barely recognizable heirs to that earlier convention for giving clinical advice.

Enormous progress has been made by a large and growing international community of clinicians, methodologists, statisticians, and other stakeholders in developing SRs and CPGs, yet problems remain. There are many competing systems for evaluating and synthesizing evidence, and there are no internationally agreed-upon standards for how to conduct an SR or create a CPG. In the United States, the decades-old interest in SRs and CPGs among public and private agencies is receiving a boost from the highlighting of the importance of both in debates about healthcare reform; a specific provision in the Medicare Improvements for Patients and Providers Act of 2008 brought two Institute of Medicine (IOM) committees into being, aimed at setting standards for SRs and CPGs. Furthermore, in the United States there is enormous interest in and high expecta-

Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×

tions for the newly created Patient-Centered Outcomes Research Institute, whose authorizing legislation specifically names SRs and CPGs as important components in developing a national program of comparative effectiveness research.

As both SR and CPG reports indicate, the term “standard” is problematic. Our two committees found a sparse evidence base that directly evaluates alternative approaches to SRs and CPGs. The SR committee thus relied on available literature, expert guidance from organizations engaged in SRs, and its own criteria and internal discussions to propose a set of standards, recognizing that any such recommendations must be considered provisional pending further development of the evidence. Collectively the standards set a high bar that will be difficult to achieve for many SRs, yet the evidence and experience are not reassuring that it is safe to cut corners if resources are limited. The standards will be especially valuable for SRs of high-stakes clinical questions with broad population impact, where the use of public funds to get the right answer justifies careful attention to the rigor with which the SR is conducted. The best practices collected in this report should be thoughtfully considered by anyone conducting an SR. In the end the most important standard is to be transparent in reporting what was done and why. Importantly, the committee concludes with recommendations that the United States invest in a program to improve both the science of SRs (with attention to both scientific rigor and feasibility/cost) and the environment that supports them, including a process to update standards as the evidence improves.

Finally, one of the most professionally satisfying benefits of leading an IOM committee is the opportunity to work with committee members with an amazing breadth and depth of experience, and IOM staff whose anticipation and completion of the next steps always appears effortless. We are deeply grateful that this committee and staff have again demonstrated the process at its best.

In conclusion, the committee believes we are at an important juncture in the development of SRs and CPGs, and that timely investment in both will produce an excellent return in improving health care and patient outcomes. We hope our recommended standards will serve as a useful milestone as the United States joins international partners to advance the science and improve the environment for SRs and CPGs.


Alfred O. Berg, Chair

Sally C. Morton, Vice Chair

Committee on Standards for Systematic Reviews of Comparative Effectiveness Research

Page xiii Cite
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×

Acknowledgments

The committee and staff are indebted to a number of individuals and organizations for their contributions to this report. The following individuals provided testimony to the committee:


Naomi Aronson, Executive Director, Technology Evaluation Center, Blue Cross and Blue Shield Association

Kalipso Chalkidou, Director, NICE International, National Institute for Health and Clinical Excellence

Yngve Falck-Ytter, Assistant Professor of Medicine, Case Western Reserve University; Director of Hepatology, VA Medical Center, Cleveland

Rebekah Gee, Assistant Professor of Clinical Medicine, Department of Obstetrics and Gynecology, Tulane University, American College of Obstetricians and Gynecologists

Louis B. Jacques, Director, Coverage & Analysis Group, Office of Clinical Standards & Quality, Centers for Medicare & Medicaid Services

Sandra Zelman Lewis, Assistant Vice President, Health & Science Policy, American College of Chest Physicians

Virginia Moyer, Section Head, Academic General Pediatrics, Baylor College of Medicine, American Academy of Pediatrics

Edmund Pezalla, National Medical Director and Chief Clinical Officer, Aetna Pharmacy Management

Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×

Alan Rosenberg, Vice President of Technology Assessment, Wellpoint Health Networks

Carol Sakala, Director of Programs, Childbirth Connection

Gail Shearer, Former Director, Consumer Reports Best Buy Drugs, and Former Director, Health Policy Analysis, Consumers Union

David Shern, President and CEO, Mental Health America

David B. Wilson, Crime and Justice Group Cochair, Steering Committee, The Campbell Collaboration


We also extend special thanks to the following individuals who were essential sources of information, generously giving their time and knowledge to further the committee’s efforts:


Stephanie Chang, EPC Program Director, AHRQ

Julian Higgins, Senior Statistician, MRC Biostatistics Unit, Institute of Public Health, University of Cambridge

Alison Little, Medical Director, Drug Effectiveness Review Project

Rose Relevo, Research Librarian, AHRQ Scientific Resource Center

Karen M. Schoelles, Director, ECRI Institute Evidence-based Practice Center

Lesley Stewart, Director, Centre for Reviews and Dissemination


Funding for this study was provided by the Agency for Healthcare Research and Quality (AHRQ). The committee appreciates the opportunity and support extended by AHRQ for the development of this report.

Finally, many within the Institute of Medicine were helpful to the study staff. We especially would like to thank the staff of the Clinical Practice Guidelines We Can Trust report for their collaborative spirit: Robin Graham, Michelle Mancher, and Dianne Wolman. Additionally, the staff would like to thank Clyde Behney, Michelle O. Crosby-Nagy, Greta Gorman, Cheryl Levey, William McLeod, Abbey Meltzer, Vilija Teel, and Lauren Tobias

Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×

3

 

STANDARDS FOR FINDING AND ASSESSING INDIVIDUAL STUDIES

 

81

   

 The Search Process,

 

83

   

 Recommended Standards for the Search Process,

 

106

   

 Screening and Selecting Studies,

 

108

   

 Recommended Standards for Screening and Selecting Studies,

 

114

   

 Managing Data Collection,

 

117

   

 Recommended Standard for Extracting Data,

 

122

   

 Critical Appraisal of Individual Studies,

 

123

   

 Recommended Standards for Assessing the Quality and Relevance of Individual Studies,

 

137

4

 

STANDARDS FOR SYNTHESIZING THE BODY OF EVIDENCE

 

155

   

 A Note on Terminology,

 

157

   

 Recommended Standard for Assessing and Describing the Quality of a Body of Evidence,

 

172

   

 Qualitative Synthesis of the Body of Evidence,

 

173

   

 Recommended Standards for Qualitative Synthesis,

 

176

   

 Meta-Analysis,

 

177

   

 Recommended Standards for Meta-Analysis,

 

187

5

 

STANDARDS FOR REPORTING SYSTEMATIC REVIEWS

 

195

   

 Reporting Guidelines,

 

198

   

 Systematic Reviews Published in Journals,

 

202

   

 Recommended Standard for Preparing the Final Report,

 

203

   

 Recommended Standard for Report Review,

 

214

   

 Recommended Standard for Publishing the Final Report,

 

217

6

 

IMPROVING THE QUALITY OF SYSTEMATIC REVIEWS: DISCUSSION, CONCLUSIONS, AND RECOMMENDATIONS

 

223

   

 Improving the Science of Systematic Reviews,

 

225

   

 Improving the Environment for Supporting Systematic Reviews,

 

229

   

 Conclusions and Recommendations,

 

232

Page xviii Cite
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×

This page intentionally left blank.

Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×

Boxes, Figures, and Tables

Summary

Boxes

S-1

 

Charge to the Committee on Standards for Systematic Reviews of Comparative Effectiveness Research,

 

2

S-2

 

Recommended Standards for Initiating a Systematic Review,

 

6

S-3

 

Recommended Standards for Finding and Assessing Individual Studies,

 

8

S-4

 

Recommended Standards for Synthesizing the Body of Evidence,

 

10

S-5

 

Recommended Standards for Reporting Systematic Reviews,

 

14

Chapter 1

Boxes

1-1

 

Charge to the Committee on Standards for Systematic Reviews of Comparative Effectiveness Research,

 

20

1-2

 

Types of Comparative Effectiveness Research Studies,

 

21

1-3

 

Four Examples of the Use of Observational Studies in Systematic Reviews of Comparative Effectiveness Research,

 

22

1-4

 

Charge to the Committee on Standards for Developing Trustworthy Clinical Practice Guidelines,

 

27

1-5

 

Numbering System for the Committee’s Recommended Systematic Review Standards,

 

33

Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×

Tables

1-1

 

Statutory Requirements for the Patient-Centered Outcomes Research Institute,

 

25

1-2

 

Committee Criteria for Assessing Potential Standards and Elements for Systematic Reviews,

 

30

1-3

 

Examples of U.S. Governmental Organizations That Produce Systematic Reviews,

 

35

1-4

 

Examples of Private U.S. Organizations That Produce Systematic Reviews,

 

36

1-5

 

Examples of International Organizations That Produce Systematic Reviews,

 

37

1-6

 

Examples of Organizations That Use Systematic Reviews,

 

38

Chapter 2

Boxes

2-1

 

Recommended Standards for Initiating a Systematic Review,

 

48

2-2

 

International Committee of Medical Journal Editors Types of Conflict-of-Interest Disclosures,

 

52

2-3

 

Organizations and Groups Included in the Commissioned Paper on Consumer Involvement in Systematic Reviews,

 

57

2-4

 

Examples of Interaction Between Systematic Review (SR) and Clinical Practice Guideline (CPG) Teams,

 

62

Figure

2-1

 

Analytic framework for a new enteral supplement to heal bedsores,

 

70

Tables

2-1

 

Models of Interaction Between Systematic Review (SR) and Clinical Practice Guideline (CPG) Teams,

 

63

2-2

 

PICO Format for Formulating an Evidence Question,

 

68

Chapter 3

Boxes

3-1

 

Recommended Standards for Finding and Assessing Individual Studies,

 

84

3-2

 

Bibliographic Databases,

 

96

3-3

 

Subject-Specific Databases,

 

98

Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×

3-4

 

Grey-Literature Databases,

 

99

3-5

 

Study Selection Criteria for a Systematic Review of Therapies for Children with Autism Spectrum Disorders (ASD),

 

111

3-6

 

Types of Data Extracted from Individual Studies,

 

118

3-7

 

Cochrane Back Pain Group Criteria for Internal Validity of Randomized Trials of Back Pain,

 

134

3-8

 

Cochrane Risk of Bias Tool Domains,

 

135

3-9

 

USPSTF Criteria for Grading the Internal Validity of Individual Studies (RCTs and Cohort Studies),

 

137

Figure

3-1

 

Example of a flow chart,

 

115

Tables

3-1

 

Types of Reporting Biases,

 

88

3-2

 

Expert Suggestions for Conducting the Search Process and Addressing Reporting Bias,

 

95

3-3

 

Expert Suggestions for Screening Publications and Extracting Data from Eligible Studies,

 

109

3-4

 

Types of Bias in Individual Studies,

 

124

3-5

 

Characteristics of Individual Studies That May Affect Applicability,

 

127

Chapter 4

Boxes

4-1

 

Recommended Standards for Synthesizing the Body of Evidence,

 

158

4-2

 

Key Concepts Used in the GRADE Approach to Assessing the Quality of a Body of Evidence,

 

165

4-3

 

Key Concepts Used in the AHRQ Approach to Assessing the Quality of a Body of Evidence,

 

166

4-4

 

Research Trends in Meta-Analysis,

 

186

Figure

4-1

 

Forest plot,

 

183

Tables

4-1

 

Examples of Approaches to Assessing the Body of Evidence for Therapeutic Interventions,

 

161

4-2

 

Key Purposes of the Qualitative Synthesis,

 

174

Page xxii Cite
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×

Chapter 5

Boxes

5-1

 

Recommended Standards for Reporting Systematic Reviews,

 

197

5-2

 

Requirements for Research Funded by the Patient-Centered Outcomes Research Institute,

 

199

5-3

 

A History of Reporting Guidelines for Comparative Effectiveness Research,

 

201

5-4

 

Example of a Structured Abstract: Clinical Utility of Cancer Family History Collection in Primary Care,

 

206

5-5

 

Example of a Plain-Language Summary: Antenatal Corticosteroids for Accelerating Fetal Lung Maturation for Women at Risk of Preterm Birth,

 

208

5-6

 

Reporting Funding and Conflict of Interest: Selected Examples,

 

215

Tables

5-1

 

Topics to Include in the Methods Section,

 

210

5-2

 

Topics to Include in the Results Section,

 

212

5-3

 

EPICOT Format for Formulating Future Research Recommendations,

 

214

Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×
Page R1
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×
Page R2
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×
Page R3
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×
Page R4
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×
Page R5
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×
Page R6
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×
Page R7
Page viii Cite
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×
Page R8
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×
Page R9
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×
Page R10
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×
Page R11
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×
Page R12
Page xiii Cite
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×
Page R13
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×
Page R14
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×
Page R15
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×
Page R16
Page xvii Cite
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×
Page R17
Page xviii Cite
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×
Page R18
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×
Page R19
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×
Page R20
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×
Page R21
Page xxii Cite
Suggested Citation:"Front Matter." Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. doi: 10.17226/13059.
×
Page R22
Next: Summary »
Finding What Works in Health Care: Standards for Systematic Reviews Get This Book
×
Buy Paperback | $61.00 Buy Ebook | $48.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Healthcare decision makers in search of reliable information that compares health interventions increasingly turn to systematic reviews for the best summary of the evidence. Systematic reviews identify, select, assess, and synthesize the findings of similar but separate studies, and can help clarify what is known and not known about the potential benefits and harms of drugs, devices, and other healthcare services. Systematic reviews can be helpful for clinicians who want to integrate research findings into their daily practices, for patients to make well-informed choices about their own care, for professional medical societies and other organizations that develop clinical practice guidelines.

Too often systematic reviews are of uncertain or poor quality. There are no universally accepted standards for developing systematic reviews leading to variability in how conflicts of interest and biases are handled, how evidence is appraised, and the overall scientific rigor of the process.

In Finding What Works in Health Care the Institute of Medicine (IOM) recommends 21 standards for developing high-quality systematic reviews of comparative effectiveness research. The standards address the entire systematic review process from the initial steps of formulating the topic and building the review team to producing a detailed final report that synthesizes what the evidence shows and where knowledge gaps remain.

Finding What Works in Health Care also proposes a framework for improving the quality of the science underpinning systematic reviews. This book will serve as a vital resource for both sponsors and producers of systematic reviews of comparative effectiveness research.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!