Prior to recommending treatments, the committee reviewed and analyzed information on evaluating treatment effectiveness. Treatment effectiveness is defined as the benefit produced by a given treatment in day-to-day clinical practice in unselected patient populations that do not receive extra tests, education, or visits because of participation in a study. There is little formal evidence of treatment effectiveness for any medical treatment because relatively few true effectiveness studies have been conducted. There are, however, efficacy studies. Treatment efficacy is the benefit produced by a given treatment in tightly controlled, perhaps artificial, study conditions in which patients are carefully selected and may be more frequently observed, tested, and monitored than is typically the case in routine practice.
A number of study designs can provide varying levels of evidence of treatment efficacy. They include, from strongest to weakest:
Multiple well-designed randomized controlled trials (RCTs);
Single well-designed RCTs or multiple small RCTs;
Cohort study, particularly one with “multiple on/off” features;
Case-control study; and
Series of clinical observations or anecdotes.
In addition to the above designs, there is the technique of meta-analysis. Meta-analysis was developed to fit the situation in which study results are not fully consistent or there are multiple studies of differing degrees of design rigor. In meta-analysis the results of multiple studies are combined to yield an overall cross-study estimate of effectiveness.
In its review of clinical studies, the U.S. Preventive Health Services Task Force (USPHSTF) used strict criteria for selecting admissible evidence of effectiveness in grading the quality of evidence (see
Table ES-1). The task force gave greater weight to those study designs that, for methodological reasons, are less subject to bias and inferential error (USPHSTF 1996).
In evaluating treatments for Gulf War veterans, the committee chose to recommend as effective only those treatments with demonstrated efficacy using the highest level of evidence—the randomized controlled trial (Level I of the USPHSTF scale). However, in responding to its charge to identify and describe approaches to assessing treatment effectiveness, the committee has explored other alternatives. From the perspective of evaluating treatment effectiveness, there are two general classes of studies, each with strengths and weaknesses:
Treatment efficacy studies, including prospective randomized tri