and nonsystematic. The reviews may have provided thoughtful, readable discussions of a topic, but the conclusions were generally not credible.

The following sections of the chapter describe the fundamentals of conducting a scientifically rigorous systematic review and then provide the committee’s findings on current efforts.


Although researchers use a variety of terms to describe the building blocks of a systematic review, the fundamentals are well established (AHRQ EPC Program, 2007; Counsell, 1997; EPC Coordinating Center, 2005; Haynes et al., 2006; Higgins and Green, 2006; Khan and Kleijnen, 2001; Khan et al., 2001a,b; West et al., 2002).4 Five basic steps (listed below) should be followed, and the key decisions that comprise each step of the review should be clearly documented.

Step 1: Formulate the research question.

Step 2: Construct an analytic (or logic) framework.

Step 3: Conduct a comprehensive search for evidence.

Step 4: Critically appraise the evidence.

Step 5: Synthesize the body of evidence.

The following sections briefly describe each of these steps in the process.

Step 1:
Formulate the Research Question

The foundation of a good systematic review is a well-formulated, clearly defined, answerable question. As such, it guides the analytic (or logic) framework for the review, the overall research protocol (i.e., the search for relevant evidence, decisions about which types of evidence should be used, and how best to identify the evidence), and the critical appraisal of the relevant evidence. The objective, in this first step, is to define a precise, unambiguous answerable research question.

Richardson and colleagues (1995) coined the mnemonic PICO (population, intervention, comparison, and outcome of interest) to help ensure that explicit attention is paid to the four key elements of an evidence question.5,6


Unless otherwise noted, this section draws from these references.


Personal communication, W. S. Richardson, Boonshoft School of Medicine, Wright State University, October 3, 2007.


A recent draft version of an AHRQ comparative effectiveness methods manual proposes expanding the PICO format to PICOTS, adding “t” for timing and “s” for settings (AHRQ, 2007a).

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement