ogy, legal status, and social stigma that preclude confident generalization to the drug domain.

Moreover, these studies only evaluated the self-selected group of patients who presented for treatment rather than the universe of sufferers in the community. For drug addiction, we would like to know the effects of treatment on all of those with the disorder, including those not presenting for treatment. Because the number of current treatment slots can only accommodate a fraction of those with the disorder, a critical policy question is whether the creation of additional slots is cost-effective. Finding the answer would require a control in the community randomized to no treatment whatsoever.

The meta-analytic data do suggest that nonrandomized trials don’t invariably inflate effect sizes. Lipsey and Wilson (1993) found no reliable differences in the effect sizes between experiments (mean=0.46) and quasi-experiments (mean=0.41). Shadish and his colleagues (Heinsman and Shadish, 1996; Shadish and Ragsdale, 1996), in more rigorous meta-analyses of data from a sample of the domains covered by Lipsey and Wilson, found that effect sizes tended to be significantly larger in randomized experiments, even after controlling for various confounding differences between experimental and quasi-experimental studies.

In this regard, a study by McKay et al. (1998) is relevant. These authors compared patients either randomly assigned to cocaine treatment and those who “self-selected” into the same treatment settings, finding “greater problem severity at intake among randomized patients coupled with greater improvements by 3-month follow-up relative to the nonrandomized patients” (McKay, 1998:697). The investigators argue that “randomized studies of treatment for cocaine abuse may produce somewhat larger estimates of improvement than what is observed in more typical treatment situations” (see Campbell and Boruch, 1975, for a relevant discussion).

Thus, it is our contention that randomized experiments with no-treatment controls provide more accurate estimates of the efficacy of drug treatment, not necessarily smaller estimates. We do not contend that such no-treatment controls are essential for testing possible improvements in treatment methods; randomized “Treatment A versus Treatment B” trials are a powerful mechanism for that goal. Rather, in the committee’s view, no-treatment control groups are necessary to provide the kind of information needed to support policy analyses of the effectiveness and cost-effectiveness of providing drug treatment and of expanding treatment access.

Bias due to incomplete compliance with randomized assignment. In some settings, the experimenter can encourage compliance with the treatment protocol, but some experimental subjects may not comply. Realistically, some proportion of clients in a no-treatment control group may seek out

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement