Page 33

effect of a drug, for example, because the dose chosen for study was too low. The patients in a particular study may be unique in some way that makes them not representative of all patients to whom the treatment might be given in the future. For example, if all study subjects are middle-age white men, it is not clear whether the treatment would work in the same way for older or younger Asian women. Results that appear significant in a near-term follow-up may change with longer follow-up (e.g., a treatment shrinks tumors dramatically for two months, but the cancers recur and the patients die after 18 months). The most powerful evidence of treatment efficacy comes from the cumulative, consistent results of several RCTs, preferably in different patient populations and in different settings, and with extensive follow-up periods.

Other kinds of studies (i.e., quasi-experimental designs) can provide evidence of treatment efficacy, too. In situations where it is technically or ethically impossible to run concurrent control groups, a series of “off/on” periods of treatment in a single group of patients can be studied. In these studies treatment is administered to a single group of patients and then taken away. Evidence of efficacy is provided if the benefit is consistently seen when treatment is given and the benefit disappears when treatment is not given. This is a specific example of a before-after study design without controls. A single round of off/on provides very weak evidence for effectiveness unless results are unusual and dramatic, because many other things occurring at the same time as the treatment may have caused the result. Being able to repeat the effect over and over again strengthens the argument for the treatment, rather than something else, being the cause of the effect.

One could also use a cohort study. In this study design a large number of patients who receive a treatment are followed over time to observe a possible benefit and are compared to those who did not receive the treatment. It may offer strong evidence of treatment effectiveness if the group studied is particularly large so that other possible causes of an effect may be evaluated through statistical analysis, or if the result is unusually strong and/or consistent in the large group. For example, one might observe a lower rate of heart attacks in a large group of men taking an aspirin every day over a 10-year period compared to similar men who did not take it. One might challenge the results, though, and ask whether the men taking aspirin became more health conscious in general and also lost weight, drank less, quit smoking, or did something else that was actually the cause of the reduced rate of heart attacks. Being able to go into the database and find the effect in a subset of men who did not lose weight or quit smoking or drink less would offer an answer to the challenge.

Another possible study design is a case-control study. In this study design, patients are assigned to study groups based on the results of treatment rather than the treatment itself. One might, for example, identify

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement