The initial steps in the systematic review (SR) process define the focus of the complete review and influence its ultimate use in making clinical decisions. Because SRs are conducted under varying circumstances, the initial steps are expected to vary across different reviews, although in all cases a review team should be established, user and stakeholder input gathered, the topic refined, and the review protocol formulated. Current practice falls far short of recommended guidance1; well-designed, well-executed SRs are the exception. At a workshop organized by the committee, representatives from professional specialty societies, consumers, and payers testified that existing SRs often fail to address questions that are important for real-world healthcare decisions.2 In addition, many SRs fail to develop comprehensive plans and protocols at the outset of the project, which may bias the reviews (Liberati et al., 2009; Moher et al., 2007). As a consequence, the value of many SRs to healthcare decisions makers is limited.

The committee recommends eight standards for ensuring a focus on clinical and patient decision making and designing SRs that minimize bias. The standards pertain to: establishing the review team, ensuring user and stakeholder input, managing bias and conflict of interest (COI) for both the research team and users and stakeholders, formulating the research topic, writing the review protocol, providing for peer review of the protocol, and making the protocol publicly available. Each standard includes a set of requirements composed of elements of performance (Box 2-1). A standard is a process, action, or procedure for performing SRs that is deemed essential to producing scientifically valid, transparent, and reproducible results. A standard may be supported by scientific evidence; by a reasonable expectation that the standard helps to achieve the anticipated level of quality in an SR; or by the broad acceptance of the practice in SRs. Each standard includes elements of performance that the committee deems essential.


Unless otherwise noted, expert guidance refers to the published methods of the Evidence-based Practice Centers in the Agency for Healthcare and Research Quality Effective Health Care Program, the Centre for Reviews and Dissemination (University of York, UK), and the Cochrane Collaboration. The committee also consulted experts at other organizations, including the Drug Effectiveness Review Project, the ECRI Institute, the National Institute for Health and Clinical Excellence (UK), and several Evidence-Based Practice Centers (with assistance from staff from the Agency for Healthcare Research and Quality). See Appendix D for guidance.


On January 14, 2010, the committee held a workshop that included four panels with representatives of organizations engaged in using and/or developing systematic reviews, including SR experts, professional specialty societies, payers, and consumer groups. See Appendix C for the complete workshop agenda.

The National Academies of Sciences, Engineering, and Medicine
500 Fifth St. N.W. | Washington, D.C. 20001

Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement