Page 4

TABLE ES-1 Quality of Evidence




Evidence obtained from at least one properly randomized controlled trial.


Evidence obtained from well-designed controlled trials without randomization.


Evidence obtained from well-designed cohort or case-control analytical studies, preferably from more than one center or research group.


Evidence obtained from multiple time series with or without the intervention. Dramatic results in uncontrolled experiments (such as the results of the introduction of penicillin treatment in the 1940s) could also be regarded as this type of evidence.


Opinions of respected authorities, based on clinical experience, descriptive studies, and case reports, or reports of expert committees.

als, emphasize internal validity at the expense of external validity. That is, the sampling, data collection, and data analysis procedures are designed to support the strongest possible inferences about associations between independent and dependent variables (i.e., cause and effect) in a tightly controlled context. The best, strongest studies in tightly controlled situations may, however, lack generalizability to routine medical practice.

  • Treatment effectiveness studies, including the largest and most comprehensive outcomes studies, emphasize external validity often at the expense of internal validity. They may involve very large samples that are fully representative of the patients seen in routine clinical practice, but the studies may include confounding factors that weaken the inferences about cause-and-effect relationships.

The committee believes the results of a single, well-designed outcomes study (e.g., a cohort study or variation of care and outcome study) should be considered to be as compelling as the results of a single, well-controlled randomized trial in determining treatment effectiveness. An outcomes study will have few concerns about the generalizability of its findings to real-world settings (external validity) but perhaps some concerns about internal validity; RCTs will have the opposite pattern of strengths and weaknesses. For a detailed discussion of the various types of studies, see Chapter 3. If there were studies of both types available with similar results, the combined evidence would be quite powerful. Studies of the two types with conflicting findings would essentially cancel each other out and no conclusion could be drawn. Table ES-2 is organized to

The National Academies of Sciences, Engineering, and Medicine
500 Fifth St. N.W. | Washington, D.C. 20001

Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement