Inappropriate Statistical Analysis and Planning. On occasion, statistical analyses reported in published studies are not performed correctly, or the most appropriate statistical analyses are not performed. In other instances, statistical issues, such as statistical power to detect a difference between two arms of a study if one really existed, do not seem to have been adequately considered in planning the study, or had to be compromised for practical reasons (such as study cost or patient availability). As a result, the results reported in some studies are misleading and have a significant probability of being wrong. Investigators should report the statistical significance of their results, and provide 95 percent confidence intervals around group differences or main effects. In addition, if relative risks or odds ratios are reported (such as reporting that a particular outcome is twice as likely to occur with treatment A as with treatment B), the absolute rate with which the outcome occurs also should be reported.

Poorly Described Techniques. Diagnostic and therapeutic techniques are often employed using very specific protocols or techniques that affect the effectiveness or safety of the interventions. For example, different pulse sequences can be used in magnetic resonance imaging studies and different software might base comparisons of digitized mammography images on different calculations. Unless the technology under study, and the technologies to which it is being compared, are clearly described, it is not possible to meaningfully compare the results of one study to those of other studies of what appears to be the same technology. Without such descriptions it also may be difficult or even impossible to judge the relevance of the study results.

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement