ELIMINATING INFERENTIAL ARTIFACTS AND ESTIMATING ABSOLUTE EFFECT SIZES

Considering the enormous challenges of conducting research in clinical settings, the randomized trials highlighted in the previous section are quite rigorous. They are a powerful source of information for improving drug treatment. But as designed they cannot provide robust estimates of the absolute magnitude and range of treatment effects for various types of clients (especially voluntary versus coerced clients), which are needed for use in cost-effectiveness comparisons, benefit-cost analyses, and simulation modeling of the potential benefits of scaling up or expanding the current treatment system.

The inferential benefits of randomization to experimental condition are well known; see Cook and Campbell (1979) for a comprehensive listing of threats to validity that are reduced or eliminated by randomization. (Note that design limitations create vulnerability to biased inference; they do not guarantee that biased inferences will occur. Whether any bias actually resulted is an empirical question.)

Here, we emphasize the various processes that can differentially bias selection into, or attrition out of, the treatment and control conditions of the study. When other factors are confounded with the treatment variations under study—e.g., addiction severity, motivation to change, life stresses and resources—it is not possible to directly estimate treatment effects by simply examining the difference between mean outcomes in each condition.

Many of these selection biases result from the causal forces that bring clients into treatment. The net directional effect of such biasing processes is rarely clear. Consider a nonexperimental study in which treatment clients are compared with a demographically matched sample of drug users not in treatment. On one hand, one might expect that those who seek and stick with treatment might be more motivated to give up their drug use (see DiClemente, 1999). On the other hand, many if not most clients are in drug treatment not because they voluntarily chose to be, but because they were either formally or informally coerced by a court, law enforcement agency, employer, spouse, or family member. For example, in the 1997 TEDS study, 34.9 percent of all admissions were referred by the criminal justice system (Substance Abuse and Mental Health Services Administration, 1999: Table 3.4). (We briefly examine the literature on coerced treatment below.)

Moreover, at least in the case of tobacco smoking, there is some evidence that smoking cessation clinics disproportionately see the hardest cases—those who were unable to quit smoking on their own (Schachter, 1982). Some selection biases involve client or setting characteristics that



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement