These SR steps require meticulous execution and documentation to minimize the risk of a biased synthesis of evidence. Current practice falls short of recommended guidance and thus results in a meaningful proportion of reviews that are of poor quality (Golder et al., 2008; Moher et al., 2007a; Yoshii et al., 2009). An extensive literature documents that many SRs provide scant, if any, documentation of their search and screening methods. SRs often fail to acknowledge or address the risk of reporting biases, neglect to appraise the quality of individual studies included in the review, and are subject to errors during data extraction and the meta-analysis (Cooper et al., 2006; Delaney et al., 2007; Edwards et al., 2002; Golder et al., 2008; Gøtzsche et al., 2007; Horton et al., 2010; Jones et al., 2005; Lundh et al., 2009; Moher et al., 2007a; Roundtree et al., 2008; Tramer et al., 1997). The conduct of the search for and selection of evidence may have serious implications for patients’ and clinicians’ decisions. An SR might lead to the wrong conclusions and, ultimately, the wrong clinical recommendations, if relevant data are missed, errors are uncorrected, or unreliable research is used (Dickersin, 1990; Dwan et al., 2008; Glanville et al., 2006; Gluud, 2006; Kirkham et al., 2010; Turner et al., 2008).
In this chapter, the committee recommends methodological standards for the steps involved in identifying and assessing the individual studies that make up an SR’s body of evidence: planning and conducting the search for studies, screening and selecting studies, managing data collection from eligible studies, and assessing the quality of individual studies. The committee focused on steps to minimize bias and to promote scientifically rigorous SRs based on evidence (when available), expert guidance, and thoughtful reasoning. The recommended standards set a high bar that will be challenging for many SR teams. However, the available evidence does not suggest that it is safe to cut corners if resources are limited. These best practices should be thoughtfully considered by anyone conducting an SR. It is especially important that the SR is transparent in reporting what methods were used and why.
Each standard consists of two parts: first, a brief statement describing the related SR step and, second, one or more elements of performance that are fundamental to carrying out the step. Box 3-1 lists all of the chapter’s recommended standards.
Note that, as throughout this report, the chapter’s references to “expert guidance” refer to the published methodological advice of the Agency for Healthcare Research and Quality (AHRQ) Effective Health Care Program, the Centre for Reviews and Dissemination (CRD) (University of York), and the Cochrane Collaboration.