Can the Outcomes Research Literature Inform the Search for Quality Indicators in Substance Abuse Treatment?
A. Thomas McLellan, Mark Belding, James R. McKay, David Zanis, and Arthur I. Alterman
Penn-Veterans Affairs Center for Studies of Addiction, Philadelphia
Over the past 20 years, treatment researchers within the field of substance abuse have focused on questions of whether treatment is effective and which type of substance abuse treatment is most effective (Bale, 1979; Gerstein et al., 1994; Hubbard and Marsden, 1986; McLellan et al., 1992, 1994; Sells and Simpson, 1980). The data on these questions have been quite consistent, with well-substantiated evidence available from both controlled clinical trials and field studies, suggesting four important conclusions.
Many of the traditional forms of substance abuse treatment (e.g., methadone maintenance, therapeutic communities, outpatient drug-free treatment) have been evaluated multiple times and have been shown to be effective (Ball and Ross, 1991; Gerstein et al., 1994; Hubbard and Marsden, 1986; IOM, 1990a, b; McLellan et al., 1980; Simpson and Sells, 1982).
The benefits obtained from these treatments typically extend beyond the reduction of substance use to areas that are important to society, such as reduced crime, reduced risk of infectious diseases, and improved social function (Ball and Ross, 1991; Gerstein et al., 1994; IOM 1990a, b; McLellan et al., 1980).
Individuals who complete treatment or receive more days of treatment typically show more improvements than those who leave care prematurely (DeLeon, 1984; Gerstein et al., 1994; Hubbard et al., 1989; Sells and Simpson, 1980).
The costs associated with the provision of substance abuse treatment provide three- to sevenfold returns to employers, the health insurers, and society within
3 years following treatment (French et al., 1991; Gerstein et al., 1994; Holder and Blose, 1992).
HOW DO THESE RESULTS TRANSLATE INTO RECOMMENDATIONS FOR PROVIDING QUALITY TREATMENT?
Although the conclusions from this line of research are important and gratifying, they are not adequate to inform important clinical, economic, or health and social policy questions regarding the delivery of substance abuse treatment services. Knowledge that some of these treatments can work and that better outcomes are associated with longer treatment does not help to determine (1) which of the multiple elements in these multicomponent treatments are causally related to the favorable outcomes (i.e., the so-called active ingredients of treatment), (2) how long or at what level of intensity these ingredients should be delivered, or (3) the point at which the additional provision of these ingredients is no longer associated with increased benefits.
If the field of substance abuse treatment research is to help guide and inform the search for better, faster, less expensive treatments, it will be necessary to develop better, faster, less expensive means of evaluating the specific effects of substance abuse treatments. To respond to this need, the treatment field has begun to look for markers of or proxies for true outcomes that can be easily measured during the course of treatment (ideally as part of a management information system) and that have been associated with favorable outcomes following treatment. These early indicators of subsequent favorable outcomes have been called “quality indicators.”
Until now, these indicators have typically been developed by groups of clinicians and administrators who have simply selected indicators that have a clear, “face-valid” or intuitive link with longer-term outcomes. This common sense approach has had great appeal because the results have been measures that can be collected, analyzed, and reported rapidly and inexpensively, with the results being clear to patients, clinicians, and administrators alike.
Furthermore, because these indicators could be measured for individual patients and during the early course of an individual's treatment, they have the potential for use as early warning signs to correct inappropriate treatments. Because of their potential clinical and administrative value, systems of quality indicators have already been widely adopted by treatment providers, and there is a widespread effort to build the reporting of these measures into existing clinical or management information systems.
The existing and proposed quality indicators for the substance abuse field (e.g., American Managed Behavioral Healthcare Association and the National Committee for Quality Assurance) have been useful in identifying obvious problems in the conduct of treatment, in bringing the consumer perspective into the treatment setting, and in stimulating the treatment field toward greater self-
examination and self-evaluation. At the same time, there is some concern that these initial indicators may only identify extremely poor outcomes, bordering on malpractice (e.g., prescribing antipsychotic medications to patients without psychosis diagnoses). Furthermore, although the indicators have been developed to encourage improved clinical practices, some of them can be made to show apparent improvement through administrative action without actually changing treatment practices (e.g., an administrative decision not to readmit discharged patients within 1 month posttreatment to give the impression of a low 1-month relapse rate). Thus, the current indicators may be insufficient to differentiate subtler levels of treatment quality or to offer guidance for providers searching for more effective treatment methods.
WHAT QUALITIES ARE NECESSARY FOR AN IDEAL QUALITY INDICATOR?
Some of the problems with the existing quality indicators result from the lack of a clear rationale or conceptual basis for what would and would not constitute an indicator that is valid and useful. In our view, such an indicator would be a measure of a treatment process or a patient characteristic that can be recorded easily during treatment and that has been clearly associated with a favorable outcome.
It is important to examine the rationale for and subtleties of each of the components of this definition of an ideal quality indicator. First, the definition includes both treatment process factors and patient changes during treatment. Although the majority of current quality indicators focus on treatment practices, policies, and processes, it is potentially more practical and more informative to focus upon interim patient changes brought about during the course of treatment. The distinctions between these two types of potential indicators are important and are discussed at the end of this paper.
The definition also suggests that the measures that will ultimately serve as these indicators must be easily, inexpensively, and reliably made during the course of treatment at the individual patient level. Ease of measurement is essential if these indicators are to be used widely in standard clinical settings. Furthermore, these measures should be recorded at the individual patient level because early indications of favorable or unfavorable treatment progress could be extremely useful for clinical management of individual patients, again increasing the likelihood that they will be adopted and used regularly in the clinical setting. Moreover, indicators that are recorded at the individual patient level can always be aggregated through sampling to permit reporting at the program level. However, indicators that are based on the aggregate data from a treatment program can rarely be reduced to provide clinically significant information at the individual patient level.
Finally and most importantly, the true value of potential quality indicators rests ultimately upon the relationship of those indicators to treatment outcomes.
Although this may seem apparent, there has been, in fact, a lack of agreement about how to define and measure outcomes, which in turn makes it difficult to identify useful indicators. One way to approach this issue is to identify two separate and distinct stages of substance abuse treatment: detoxification-stabilization and rehabilitation. Each of these stages has distinct therapeutic goals, different treatment processes, and markedly different expectations with regard to outcome.
Thus, this paper will review the available outcomes research for the detoxification-stabilization stage and the rehabilitation stage of treatment. Each section will include a description of processes and therapeutic goals for the treatment stage, define outcomes on the basis of these therapeutic goals, discuss a strategy for reviewing the literature based on the outcomes definitions, and finally, present research findings pertinent to the identification of quality indicators within that stage of treatment.
The review includes only data from clinical trials, treatment matching program studies, or health services studies where the patients were adults who were clearly alcohol or drug (excluding tobacco) dependent by contemporary criteria, where the treatment provided was a conventional form of either detoxification or rehabilitation (any setting or modality), and where there were measures of either treatment processes or patient change during the course of treatment as well as posttreatment measures of outcome as defined later in the paper.
THE DETOXIFICATION OR ACUTE STABILIZATION STAGE
Before the advent of managed care strategies in the United States, the acute stage of substance dependence treatment was synonymous with hospitalization, regardless of whether the focus of the treatment was the medical detoxification of a true withdrawal syndrome (i.e., neuroadaptation, withdrawal symptoms, etc.) or simply the stabilization of physiological and emotional symptoms associated with the cessation of drug use that might not produce a bona fide withdrawal syndrome. Currently, detoxification from alcohol, opiate, barbiturate, or benzodiazepine use is generally the only type of treatment for which hospital admission may be warranted, and even the majority of these “true detoxifications ” now occur in outpatient or nonmedical settings.
However, this review includes both true detoxification as well as initial stabilization from the acute effects of drugs in which tolerance and withdrawal are less clearly documented (e.g., phencyclidine, LSD, marijuana, and even cocaine). The therapeutic settings, procedures, and goals are quite similar for both forms of these acute treatments, which seek to stabilize the patient medically and psychologically and to develop an effective discharge plan that includes continued rehabilitative care, almost always in an outpatient setting.
The acute stage of treatment is associated with lasting improvements only when there is continued rehabilitative treatment (IOM, 1990a, b). This associa-
tion is quite important in the development of indicators of treatment effectiveness and quality for this stage of treatment.
Goals of Detoxification and Stabilization
Patients and Treatment Settings
The detoxification and stabilization phase of treatment is designed for patients who have been actively abusing alcohol or street drugs, or both, and who are suffering physiological or emotional instability, or both. In cases of severe withdrawal potential or extreme physiological or emotional instability, detoxification-stabilization helps to prevent serious medical consequences of abrupt withdrawal, to reduce the physiological and emotional signs of instability, and to motivate necessary behavioral change strategies that will be the focus of rehabilitation. This stage of treatment may take place in inpatient settings, either a hospital or a nonhospital, residential setting, or in outpatient settings, such as in a hospital-based clinic or a residential or social setting.
Treatment Elements and Methods
Medications are available for both physiological withdrawal signs and for the temporary relief of acute medical problems associated with physiological instability (e.g., sleep medications, antidiarrheal medications, vitamins, and nutritional supplements). Motivational counseling is widely used to address shame and ambivalence, as well as to increase adherence with recommendations for continued rehabilitation.
Regardless of the setting, stabilization of acute problems is typically completed within 2 to 10 days, with the average being 3 to 5 days (Fleming and Barry, 1992). True detoxification is necessary only for cases of severe alcohol, opiate, benzodiazepine, or barbiturate use, although many cocaine-dependent and other drug-dependent patients suffer from significant physiological and emotional instability that precludes immediate participation in rehabilitation. The duration of the detoxification-stabilization process depends on the presence and severity of the patient's dependence symptoms as well as concurrent medical and psychiatric problems. Stays longer than 5 days are unusual and typically are due to conjoint medical or psychiatric problems or physiological dependence upon some forms of sedatives (e.g., alprazolam).
Key Findings for Detoxification and Stabilization Treatment
This section reviews research on treatment processes or patient changes during the course of detoxification and stabilization that have been associated with sustained reductions in physiological and emotional instability and, particularly, with continued engagement in the rehabilitation stage of treatment. As suggested earlier, there are recognized tolerance and withdrawal syndromes following the heavy use of alcohol, opiates, benzodiazepines, and barbiturates.
The standard detoxification strategy for barbiturate withdrawal was described more than a decade ago by Robinson and colleagues (Robinson et al., 1981). The majority of published work on detoxification strategies for alcohol and opiates has been reviewed in two former Institute of Medicine publications (IOM, 1990a, b). Much less has been written regarding detoxification procedures for benzodiazepine dependence, perhaps because dependence upon this group of drugs is much less prevalent.
Although cocaine “withdrawal” has been recognized in the Diagnostic and Statistical Manual, Fourth Edition (DSM IV), there is continued debate regarding the treatment and even the existence of a bona fide withdrawal syndrome following cocaine use (Satel et al., 1991; Weddington, 1992). At the same time, there is clear agreement that patients who have used cocaine or crack continuously over sustained periods, suffer two to five day periods of measurable physiological and psychiatric instability (Gawin and Ellinwood, 1988; Gawin and Kleber, 1986). For this reason, stabilization is included along with detoxification in this treatment category and was included with detoxification in the few available studies that have investigated factors associated with the acute stabilization of cocaine cessation.
Setting of Care: Medical or Nonmedical and Inpatient or Outpatient
Debate regarding the appropriate setting of care in which to detoxify alcohol-dependent patients has been substantial. Since the mid-1970s, medical settings such as residential treatment facilities or even outpatient treatment centers have conducted detoxification or stabilization treatments for alcohol, opiates, and more recently, cocaine. Although studies have not systematically compared social settings with medical settings for detoxification from alcohol dependence, there are reports of favorable outcomes in both (Naranjo et al., 1983; Whitfield et al., 1978).
In the presence of significant physiological signs of alcohol, opiate, benzodiazepine, or barbiturate withdrawal, however, the standard treatment includes medical supervision in either a hospital or an outpatient medical clinic (Fleming and Barry, 1992; IOM, 1990b). Although research is not extensive, medical settings are generally viewed as being more appropriate for detoxifications involving medical problems (particularly those with a history of seizures) and psychiatric problems (particularly for individuals with depression and at risk of suicide) and also when patients
have concurrent cocaine dependence. This last group of patients now makes up the majority of many clinical populations (DATOS, 1992; ONDCP, 1995).
Alcohol Detoxification. Within the framework of medically supervised alcohol detoxification, the relative effectiveness and costs of inpatient versus outpatient alcohol detoxification have been examined (Hayashida et al., 1989; Stockwell et al., 1986). In a study by Hayashida et al. (1989), chronic alcohol-dependent patients without histories of serious psychiatric or medical complications were randomly assigned to receive medically supervised alcohol withdrawal in either an inpatient or a day-hospital setting. On two of the outcome domains considered important for detoxification treatments (safe elimination of withdrawal signs and engagement in ongoing rehabilitation), the inpatient group showed significantly better performance, but the readdiction rates were less than 12 percent for both groups. Despite this statistically significant advantage for the inpatient setting, it was 10 times more costly than outpatient detoxification in an outpatient setting.
There may be some advantage to inpatient detoxification when a patient does not have the social or personal supports necessary to comply with the outpatient attendance requirements. However, despite somewhat lower retention rates for outpatient than for inpatient alcohol detoxification (Hayashida et al., 1989; Stockwell et al., 1991), outpatient detoxification may be more acceptable to a wider range of drinkers who wish to avoid the stigma of treatment in a designated detoxification (Stockwell et al., 1990).
Opiate Detoxification. Available evidence suggests that opiate detoxification with methadone can generally be accomplished in an outpatient setting under medical supervision with gradually reduced doses of methadone (Cushman and Dole, 1973; IOM, 1995a). However, completion rates for treatment of opioid dependence may be higher in inpatient than in outpatient detoxification programs (Gossop et al., 1986; Lipton and Maranda, 1983).
Cocaine and Crack Detoxiflcation. Few studies have examined the appropriate setting for the stabilization of physiological and psychiatric signs and symptoms associated with extended cocaine or crack use. The prevailing practice has been to attempt to stabilize all but the most severely affected patients through outpatient care (Higgins et al., 1994). Patients who are in the acute stages of cocaine cessation and who are more severely affected (medically or psychiatrically) are placed into a hospital if they have significant cardiac problems or significant psychiatric symptomatology or are at least placed in inpatient social settings for the first 3 to 5 days of treatment (Fleming and Barry, 1992).
The available literature is replete with accounts of early dropouts during the first 2 to 3 weeks of outpatient cocaine treatment (Alterman et al., 1994; Carroll et al., 1994; Higgins et al., 1993; Kang et al., 1991), with attrition rates ranging
from a low of 27 percent to a high of 47 percent in the first few weeks of care. As discussed below, it is reasonable to conclude that the patients with the most severe medical and psychiatric problems are most susceptible to drop out of treatment early.
Length of Stay and Criterion for Completion
Alcohol and Opiates. Several detoxification studies (Cushman and Dole, 1973; Hayashida et al., 1989; Senay et al., 1981) have measured detoxification as 3 consecutive days of abstinence from observable withdrawal signs or symptoms (opiate or alcohol), using standardized inventories of these physical measures. Lengths of stay for alcohol detoxifications vary from about 3 days to as long as 1 month. However, the great majority of detoxifications can be accomplished in 3 to 5 days (Fleming and Barry, 1992), and there is no evidence of greater effectiveness from extended stays.
In an early study by Cushman and Dole (1973), only 3 percent of 525 opiate-dependent patients who failed to provide an opiate-negative urine specimen following the outpatient detoxification (signifying at least 3 days of abstinence) were able to engage in the suggested abstinence-oriented rehabilitation program following detoxification. One hundred percent of these patients were readdicted to opiates at the 6-month follow-up.
Cocaine. A recent study of cocaine-dependent patients entering outpatient rehabilitation also offers some relevant information on the clinical importance of developing a criterion of successful completion. In a study of cocaine-dependent veterans, Alterman et al. (1996) found that the single best predictor of engagement in the rehabilitation process, and ultimately program completion (elimination of cocaine use verified by urinalysis), was the presence or absence of cocaine metabolites in the urine sample submitted upon admission to the program, signifying recent cocaine use. Of those patients without cocaine metabolites present in their urine on admission, 79 percent engaged in and completed the outpatient treatment, whereas only 39 percent of those with a positive urine sample on admission engaged and completed the outpatient treatment.
Potential Quality Indicators for Detoxification and Stabilization
The therapeutic goals of detoxification and stabilization are focused primarily on the amelioration and stabilization of the acute medical, psychiatric, or substance use symptoms that were out of control and thus responsible for preventing the patient from entering directly into rehabilitation. Thus, the goal of detoxification-stabilization is removal of the physiological and emotional instability that has impeded direct entry to rehabilitative treatment. Readiness for the rehabilitation stage of treatment should be assessed separately.
Detoxification can be said to have succeeded if shortly after discharge (i.e., 1 week to 1 month) the patient has:
shown significant reductions in physiological and emotional instability (at least to levels appropriate for rehabilitation entry),
has not had serious medical or psychiatric complications, and
has been integrated into and engaged in an appropriate rehabilitation program.
As summarized above, serious consequences can result from not addressing medical complications from alcohol detoxification, such as seizure history. Thus, a potential indicator for nonmedical or social detoxification settings could be the number of patients admitted with a history of medical complications, such as seizures or cardiac arrhythmias.
Given that alcohol-dependent and perhaps cocaine-dependent patients may not have the requisite personal or social resources to comply with the daily attendance requirements associated with outpatient detoxification regimens, one potential negative indicator could be the number of individuals in outpatient treatment who are homeless or who have previously failed outpatient detoxification.
Evidence suggests that it may be possible to set measurable thresholds for determining whether the detoxification has at least reduced the physiological and emotional symptoms that were the foci of treatment. This threshold may be importantly related to subsequent performance in rehabilitation treatment, at least for outpatient rehabilitation. Thus, a potential positive indicator of detoxification performance could be the number of patients who are discharged or transferred from acute care (detoxification or stabilization) who have had 3 consecutive days without withdrawal signs or symptoms. This might be measured by standard inventories of symptoms and signs or at least by breathalyzer and urinalysis measures.
Goals of Rehabilitation
Patients and Treatment Settings
Rehabilitation is appropriate for patients who are no longer suffering from the acute physiological or emotional effects of recent substance use and who need behavioral change strategies to regain control of their urges to use substances.
Rehabilitation can take place in inpatient settings, such as a hospital (which is very rare) or a residential setting (which is increasingly rare). More frequently, however, rehabilitation takes place in a hospital-based clinic or a residential or social setting.
Treatment Elements and Methods
The purposes of this stage of treatment are to prevent a return to active substance use that would require detoxification-stabilization; to assist the patient in developing control over urges to use alcohol or drugs, or both, usually through sustaining total abstinence from all drugs and alcohol; and to assist the patient in regaining or attaining improved personal health and social function, both as a secondary part of the rehabilitation function and because these improvements in lifestyle are important for maintaining sustained control over substance use.
Professional opinions vary widely regarding the underlying reasons for the loss of control over alcohol and drug abuse, for example, genetic predispositions, acquired metabolic abnormalities, learned, negative behavioral patterns, deeply ingrained feelings of low self-worth, self-medication of underlying psychiatric or physical medical problems, character flaws, and lack of family and community support for positive function. Thus, there is an equally wide range of treatment strategies and treatments that can be used to correct or ameliorate these underlying problems and to provide continuing support for the targeted patient changes.
Strategies have included such diverse elements as psychotropic medications to relieve “underlying psychiatric problems”; medications to relieve alcohol and drug cravings; acupuncture to correct acquired metabolic imbalances; educational seminars, films, and group sessions to correct false impressions about alcohol and drug use; group and individual counseling and therapy sessions to provide insight, guidance, and support for behavioral changes; and peer help groups (e.g., Alcoholics Anonymous [AA] and Narcotics Anonymous [NA]) to provide continued support for the behavioral changes thought to be important for sustaining improvement.
Typically, inpatient hospital-based forms of treatment last 7 to 11 days (ONDCP, 1995; White Paper, 1995). Nonhospital forms of residential rehabilitation are typically longer, ranging from 30 to 90 days; therapeutic community modalities typically range from 6 months to 2 years in. Outpatient forms of treatment (at least abstinence-oriented treatments) range from 30 to 120 days (ONDCP, 1995; White Paper, 1995).
Many of the more intensive forms of outpatient treatment (intensive outpatient and day hospital) begin with full or half-day sessions five or more times per week for approximately 1 month. As the rehabilitation progresses, the intensity of the treatment reduces to shorter-duration sessions of 1 to 2 hours delivered twice
weekly to semimonthly. The final part of outpatient treatment is typically called “continuing care” or “aftercare,” with biweekly to monthly group support meetings continuing (in association with parallel activities in self-help groups) for as long as 2 years. Maintenance forms of treatment are designed with an indeterminate length, with some intended to continue throughout the patient's life.
Although the majority of rehabilitation treatment programs in the United States are abstinence oriented, a significant number of rehabilitation programs maintain patients on a medication that is designed to either block the effects of the abusable drugs (e.g., disulfiram and naltrexone) or, in the case of opiates and nicotine, a medication that is designed to override the effects of the abusable drugs through the development of tolerance to a safer, more potent, and longer-acting form of the drug (nicotine patch, methadone, buprenorphine, levo-alpha-acetylmethadol [LAAM]). These maintenance approaches are quite similar to current strategies for ameliorating the physiological or emotional problems in individuals with other chronic medical conditions, such as long-term maintenance on antidepressant, antipsychotic, or other psychotropic medications for psychiatric patients; maintenance on beta-blockers and other normotensive agents for patients with hypertension; antiasthmatics for asthma sufferers; and insulin for diabetics.
The use of medications in general and maintenance medications in particular has been controversial because this general medical approach has often conflicted with the broader view that it is important to teach substance-dependent patients to live without a reliance on any type of medication. At the same time, a substantial amount of research has shown that these medications can be very effective in the rehabilitation of several forms of addiction (IOM, 1995a; O'Malley et al., 1992; Transdermal Nicotine Study Group, 1991; Volpicelli et al., 1992).
Among the most widely and thoroughly studied medications in the pharmacopoeia is methadone. Despite this fact, methadone, at least as a maintenance medication in the rehabilitation of opiate dependence, remains a controversial medication (IOM, 1995a). Compared with the medications used to treat other types of addiction, the medication is among the most tightly controlled and regulated, the chronicity and the severity of the patients' treatment problems are different from those of patients addicted to other drugs, and maintenance on methadone is often for 10 or more years, compared with 1 to 3 months maintenance for any other form of addiction medication.
This review, however, includes methadone maintenance, as well as maintenance with its long-acting form, LAAM, as part of the general category of rehabilitation treatments, because the psychosocial elements of methadone treatment and the overall rehabilitative goals of methadone treatment are quite similar to those for other forms of rehabilitation. Many of the same patient and treatment
variables that have been predictive of outcomes from other forms of rehabilitation are also predictive of the same outcomes from methadone maintenance treatments (McLellan et al., 1994).
Key Findings in the Rehabilitation Stage of Treatment
A variety of outcomes have been proposed from several perspectives, for example, cost offset, patient satisfaction, and abstinence. However, regardless of the specific setting, modality, philosophy, or methods of rehabilitation, the goals of all forms of rehabilitation are to:
maintain the physiological and emotional improvements that were to be initiated during detoxification, preventing relapse to redetoxification,
enhance and sustain reductions in or elimination of alcohol and drug use (most rehabilitation programs suggest a goal of complete abstinence), and
provide services and encourage behaviors that lead to improved personal health, improved social function, and reduced threats to public health and public safety.
For substance abuse treatment, particularly rehabilitative forms of treatment, to be worthwhile to society, outcomes must be lasting improvements in those problems that led to the treatment admission and that are important to the patient and to society. Each component of the definition will be explained below.
This definition of outcomes is restricted to improvements that can be shown to have an enduring or lasting quality (McLellan and Durell, 1995; McLellan et al., 1995; McLellan and Weisner, 1996). Because these disorders are chronic and relapsing, a “cure” for substance use disorders is not now achievable in most cases. In the case of extended outpatient abstinence-oriented treatments or maintenance treatments in which the patient is expected to remain in treatment for many months to many years, the expected improvements should be in evidence by at least the third month and should remain in evidence throughout the course of the maintenance period. The literature is replete with evaluation studies showing sustained improvements in important outcome domains at periods of 6 months to 1 year following treatment (Anglin et al., 1989; DeLeon, 1984, 1994; Finney et al., 1981; IOM 1990a, b; McLellan and Ball, 1995; Simpson and Savage, 1980).
The definition also is restricted to those improvements in problems that led to the treatment admission and that are important to the patient and to society. For the patient, and particularly for the many stakeholders, the effectiveness of
treatment will be measured in some significant part by the extended effects of treatment on the addiction-related problems that have limited personal health and social function in the patient and that may have become public health and public safety concerns, such as the risk of acquiring or transmitting infectious diseases or committing personal and property crimes. These are generally the precipitating factors leading to the treatment admission.
In this regard, achievement of the primary goal of reducing alcohol and drug use is necessary, but not always sufficient, to improve the addiction-related problems that are typically so prominent among individuals seeking treatment. Furthermore, without additional improvements in these associated problems, addiction treatment is not worthwhile either to the patient who undergoes it or to the society that supports it (McLellan and Ball, 1995; McLellan and Durell, 1995; McLellan and Weisner, 1996; McLellan et al., 1995).
Three domains are relevant to the rehabilitative goals of the patient and to the public health and safety goals of society. The first two domains are quite consistent with the primary and secondary measures of effectiveness typically used by the Food and Drug Administration (FDA) to evaluate new drug or device applications in controlled clinical trials and are quite consistent with the mainstream of thought regarding the evaluation of other forms of health care (Stewart and Ware, 1989). The final outcome dimension is more specific to the treatment of substance use disorders, because it acknowledges the significant public health and public safety concerns associated with addiction.
Sustained Reduction of Alcohol and Drug Use. Sustained reduction of drug and alcohol use is the foremost goal of treatment for substance dependence and is the primary outcome domain in this review, consistent with the FDA view. In this review, operational evidence for improvement in this domain includes both objective data from urinalysis and breathalyzer readings as well as patients' self reports of alcohol and drug use, when those reports were recorded by independent interviewers under conditions of privacy and impartiality.
Sustained Improvements in Personal Health and Social Function. Improvements in the patients' medical and psychiatric health and social function are important from a societal perspective, because these improvements reduce the problems produced by the disorders and thereby the expenses associated with their treatment. In addition, improvements in these areas are clearly related to maintenance of gains in the primary outcome area of reduced substance use. Within this review, evidence is included from measures such as general health status inventories, psychological symptom inventories, family function measures, and simple
measures of days worked and dollars earned, collected either directly from the patient via confidential self report or from independent medical or psychiatric evaluations and employment records.
Sustained Reductions in Threats to Public Health and Public Safety . The threats to public health from substance abusing individuals come from behaviors that spread infectious diseases, such as human immunodeficiency virus infection and AIDS. Specifically, the sharing of needles, unprotected sex, and trading sex for drugs are serious behaviors that have clearly been linked to addiction and are significant threats to public health. Within the review, sources of evidence include confidential self-reporting techniques or objective measures of the acquisition of AIDS, sexually transmitted diseases, tuberculosis, and hepatitis from laboratory tests, although the latter are rarely available.
Major threats to public safety include personal and property crimes committed by an individual under the influence of alcohol and drugs or for the purpose of obtaining alcohol or drugs and the irresponsible or dangerous use of automobiles or equipment by an individual under the influence of alcohol or drugs. In the studies reviewed, these behaviors were measured either by confidential interviews and questionnaires or by objective records of arrests and incarcerations.
Key Findings for Rehabilitation Treatment Outcomes
Studies are included in this review only if they measured one or more of the above three domains at 6 or more months following discharge from any form of rehabilitation treatment or 6 months or more following the initiation of a maintenance form of rehabilitation. This necessarily excludes results such as increased treatment retention, short-term improvements in symptom reduction, and patient satisfaction, because these do not represent lasting behavioral changes in any of the problem areas that are typically responsible for treatment initiation (McLellan and Weisner, 1996). Patient satisfaction has been found to be almost completely independent from most of the commonly accepted behavioral outcomes (e.g., reduced drug use, unemployment, and health care service utilization) (McLellan and Hunkeler, in press).
Summarized below are the most robust and well-replicated variables from two general categories: patient factors and treatment factors. Although a number of patient factors have been reliably related to posttreatment outcomes, very few of these, by themselves, are directly translatable into potential quality or performance indicators. However, any review of potential quality indicators should include variables that might be important as case mix adjusters, or factors that could be used to adjust two or more groups of patients in a comparative evaluation of factors that would likely affect outcome, independent of the treatment process.
Demographic Factors. Demographic factors are typically important predictors of the development of drug abuse problems (IOM, 1990b; Johnston et al., 1996; Wilsnack and Wilsnack, 1991). However, there is not much compelling evidence that race, gender, age, or educational level are consistent predictors of treatment outcome. A wide range of treatment outcome studies in the substance abuse rehabilitation field have found that demographic factors such as age, education, race, and even treatment history are relatively poorly related to outcome (as defined above) across the major rehabilitation modalities (Ball and Ross, 1991; Finney and Moos, 1992; McLellan et al., 1994; Rounsaville et al., 1987).
There may be some important exceptions. Pregnant and parenting women are an important subgroup of the larger patient population. For these individuals different features of treatment programs are required to allow them to gain access to treatment, as are different constellations of treatment services needed to address their often significant treatment problems (Gomberg and Nirenberg, 1993; Wilsnack and Wilsnack, 1993). More specifically, many of these women have been reluctant to get into standard treatments because of stigma and because of the absence of services for their children (Gomberg, 1989; Hagan et al., 1994; Weisner, 1993; Weisner et al., 1995).
Experimental programs have been created to meet these needs but there have been very few long-term outcome studies of specialized treatments for these women. The limited evidence suggests that the following may be valuable:
An inpatient or residential setting. This would offer protection from potentially aggressive spouses, because a large proportion of the women come from abusive relationships and because there may be few community resources and few opportunities for self support (Finnegan, 1991; Hagan et al., 1994).
The availability of general medical, obstetric-gynecologic, and psychiatric services. These women have been shown to have medical and psychiatric problems that are of much greater severity than those of their male counterparts (Gomberg, 1989; Hagan et al., 1994).
The availability of quarters and care for children are likely to be necessary for these women to be able to enter treatment (Finnegan, 1991).
Severity of Substance Abuse
Patients who have more serious drug dependence problems at the outset of treatment have been found to benefit less from standard treatment [Carroll et al., 1991, 1994, 1995; McKay et al., in press (a)]. This has been true of alcohol-dependent patients (Babor et al., 1988; Finney and Moos, 1992), opiate-dependent patients in therapeutic communities and in patients on methadone maintenance (Ball and Ross, 1991; DeLeon, 1994; Simpson et al., 1986), and cocaine-depen-
dent patients treated in outpatient and inpatient settings (Alterman et al., 1994; Carroll et al., 1991, 1994; McLellan et al., 1994). In all studies, a professional therapy (relapse prevention therapy) condition seemed to result in better outcomes than those from standard forms of peer counseling for highly cocaine-dependent patients.
Although the level of severity of substance use at the time of treatment admission predicts posttreatment substance use, it does not predict changes in the other domains of personal health and social function or public health and safety (Kosten et al., 1987; McLellan et al., 1984, 1994). Furthermore, the predictive relationship of level of severity of substance use to treatment response is not particularly robust, because it generally accounts for less than 10 percent of the total outcome variance (Babor et al., 1988; McLellan et al., 1994), even in the substance abuse domain.
Severity of Psychiatric Problems
Another general patient variable predicting treatment response and posttreatment outcome has been the chronicity and severity of the psychiatric problems presented by the patient at the start of treatment. Psychiatric problems have been measured by using many scales and interviews, and all have attempted to distinguish more enduring or chronic psychiatric symptoms from the acute and temporary effects of alcohol and drug withdrawal (Carroll et al., 1991, 1994, 1995; Kadden et al., 1990; McLellan et al., 1982, 1983a, b; Powell et al., 1982; Project MATCH Research Group, in press; Rounsaville et al., 1987; Woody et al., 1983, 1984, 1987).
For opiate-dependent patients on methadone maintenance, the psychiatric severity scale from the Addiction Severity Index (ASI), a general measure of the number and severity of psychiatric symptoms, has been found to be among the best predictors of 6-month substance use, personal health, and social adjustment in studies by McLellan and colleagues (1983a, b). Similar findings have been shown by Ball and Ross (1991) in their study of 6 methadone maintenance treatment programs and by studies of Kosten et al. (1987) and Rounsaville et al. (1982, 1983) of patients on methadone maintenance.
Measures of psychiatric severity have also been shown to be predictive of dropout and posttreatment substance use in studies of opiate-dependent and multiple-drug-dependent patients entering an inpatient therapeutic community setting (DeLeon, 1984, 1994). In a study by McLellan and colleagues (1984), the patients with high-severity psychiatric problems who stayed in therapeutic community treatment the longest actually showed the worst posttreatment status, suggesting that the therapeutic environment that had been demonstrably effective for the patients with problems that were not psychiatrically severe was actually countertherapeutic for the patients with high severity problems.
Poorer outcomes have been found for cocaine-dependent patients with
greater psychiatric pathologies. Similar results have been found for outpatient treatment (Carroll et al., 1991) as well as for treatment in a day-hospital and an inpatient rehabilitation setting (Alterman et al., 1994).
Among alcohol-dependent patients, there has been a great deal of evidence for the predictive power of general psychiatric symptomatology (Rounsaville et al., 1987). The severity of depression and anxiety (Powell et al., 1982; Schuckit et al., 1985, 1988, 1990) have been predictive of posttreatment drinking and posttreatment social adjustment among various samples of alcohol-dependent patients. More recently, findings from a multisite study of patient treatment matching sponsored by the National Institute on Alcohol Abuse and Alcoholism (NIAAA) (Project MATCH Research Group, in press) showed that the psychiatric severity scale from the ASI was a significant general predictor of posttreatment drinking and posttreatment social adjustment in a sample of more than 1,200 alcohol-dependent patients and three types of outpatient rehabilitation treatment.
Although the data relating the severity of psychopathology and posttreatment outcome are consistent, Schuckit and colleagues have argued cogently against overdiagnosing psychiatric symptoms, on the grounds that much of the serious psychopathology seen among alcohol-dependent patients at the time of admission for treatment is reduced following even 4 weeks of abstinence (Brown et al., 1991; Schuckit et al., 1990). There is evidence for this position among opiate-dependent patients and for patients following abstinence from cocaine.
Potential Quality Indicators
In summary, almost any general measure of severity of psychiatric symptomatology (i.e., the psychiatric severity scale from the ASI, total score on the Symptom Checklist 90, general pathology scale on the Minnesota Multiphasic Personality Inventory, number of diagnostic symptoms, etc.) can predict scores for almost all pertinent outcomes measures following standard rehabilitation treatments for alcohol, cocaine, or opiate dependence. In general, as the severity of psychiatric symptomatology increases, the likelihood of a successful outcome decreases.
This relationship appears to be attenuated by more professional forms of treatment, such as the addition of psychotropic medications to relieve depression symptoms or the provision of professional forms of family or individual psychotherapy. That is, there appears to be general evidence that previously detoxified and stabilized patients suffering from significant psychiatric symptoms such as depression, thought disorder, or anxiety perform better during treatment and have substantially better posttreatment outcomes when they receive psychotropic medications or professional therapies that would generally be prescribed to nondependent patients with these conditions.
Thus, a potential quality indicator of treatment might be the percentage of patients who are in rehabilitation for alcohol, cocaine, or opiate dependence, who have been diagnosed with major or intermittent depression, generalized anxi-
ety disorder, or other psychiatric condition, and who have been provided adjunctive pharmacotherapy (an appropriate antidepressant or anxiolytic) or professional psychotherapy during the course of their treatment.
Patient Motivation or Stage of Change
Motivation for treatment has traditionally been conceptualized and measured as the extent to which patients had entered treatment under their own free will, without external pressure from legal, family, or employment sources. Many studies have measured motivation in this way, with results that are generally quite consistent: performance during treatment and posttreatment outcomes are comparable for patients who are seemingly forced to enter a substance abuse treatment against their will, based on legal or work-related pressure (Anglin and Hser, 1990; Inciardi, 1988; Lawental et al., 1996; Roman, 1988) and for internally motivated patients (see IOM [1990a] for additional information on the history of coerced treatments). When motivation is conceptualized and measured in terms of the degree to which the patient has been coerced into treatment, it has not been an important predictor of treatment response.
However, motivation, as defined as “readiness for change” and conceptualized and measured in stages as suggested by Prochaska and DiClemente and associates (Prochaska et al., 1992), may be an important predictor of treatment response and treatment outcome. According to the model, behavior change occurs in a progression through five distinct stages, each of which is characterized by a different constellation of attitudes and behaviors. An individual in the precontemplation stage has no awareness of a problem and no desire to change. An individual in the preparation stage has made the decision to change and is already taking steps to do so. An individual in the maintenance stage has successfully made the desired change and is working to maintain that change.
Evidence supports the stages-of-change model. Stage of change may be measured by a brief questionnaire such as the University of Rhode Island Change Assessment (URICA) (McConnaughy et al., 1983) or by the use of an algorithm based on an individual's stated intentions regarding behavior change (DiClemente et al., 1991). Several studies have shown that stage classification can predict change in substance use behaviors for individuals both in and out of treatment across a variety of populations including smokers (DiClemente et al., 1991), heavy drinkers (Heather et al., 1993; Marlatt, 1988), and opiate users (Belding et al., in press).
The model postulates that progression through the stages is mediated by different types of activities or processes of change. The activities most conducive to behavior change vary according to stage. Thus, the model provides a way of identifying patients with different levels of motivation and outlines a way of tailoring interventions to match their stage of change. For example, different types of motivational enhancement therapies (MET) (Miller et al., 1994) can prepare a
patient in the early stages of change for subsequent interventions that are typically found in rehabilitation treatments.
Research concerning motivation or stage of change provides an important breakthrough in the treatment of substance-dependent patients. Specifically, if patients do not acknowledge that they have a problem needing treatment, they are likely not to respond to the type of interventions that are often the focus of rehabilitation treatments. However, for decades, precontemplators who are presumed to have an alcohol or drug problem have been forced into treatments and typically confronted about their denial (IOM, 1990a). Thus, the true contribution of this line of research appears to be evidence that forced acceptance can be not only inefficient but also counter therapeutic.
Potential Quality Indicators
A potential quality indicator could be the proportion of recently admitted patients shown to be precontemplators who have received a form of treatment (such as MET or at least a motivational interview) designed to change their motivational readiness. Evidence would suggest that individuals who are potential admissions to rehabilitation and who have not shown at least “preparation for change” status will not be good candidates for traditional forms of rehabilitation treatment. Accordingly, this indicator could be most reasonable for use in the initial detoxification-stabilization phase of treatment.
Employment, employability, and self-support skills often are a significant problem for rehabilitating substance abuse patients, and unemployed patients are more likely to drop out of treatment prematurely and to relapse to substance use early following treatment (Dennis et al., 1993; Platt, 1995). McLellan et al. (1981) found that patients who derived most of their income from employment showed more improvement and better 6-month outcomes in several outcomes domains, including not only employment but also drug use, legal and psychiatric problems, and employment, than similar patients who derived the majority of their income from unemployment or welfare.
Unemployment has been found to be a significant predictor of early relapse to opiate use among detoxified heroin-dependent males (Hall et al., 1981). Similarly, in a sample of primarily employed, multiple substance abusers entering private inpatient or outpatient, abstinence-oriented treatment programs, problems with employment (not getting along with the supervisor, dissatisfaction with present job and salary, etc.) were one of the most significant predictors of both posttreatment substance use as well as posttreatment personal health and social function, measured at the 6-month follow-up (McLellan et al., 1993).
Similar to the findings from studies of the severity of the pretreatment alco-
hol and drug use, findings from studies of pretreatment employment problems also indicate that those patients (opiate, cocaine, and mixed drug abusers) who have more severe employment and self-support problems also have poorer outcomes following treatment, as measured by a return to substance use and posttreatment self support. In summary, employment problems appear to be a general predictor of poor outcome across most treatment modalities and patient subgroups.
Family and Social Supports
Social support has been conceptualized in a wide variety of ways. These include participation in peer-supported treatments such as AA and NA, the availability of relationships that are not conflict-producing (McLellan et al., 1980, 1985), the level of patient investment in relationships, the level of psychological support from those relationships, and the level of support from those relationships for abstinence from the alcohol or drug use (Longabaugh et al., 1993, 1995).
Among alcohol-dependent patients, those who are members of families with significant dysfunction are more likely to drop out of outpatient treatment programs earlier (McLellan et al., 1983a, b, 1994), to relapse to drinking earlier following treatment (Finney and Moos, 1992), and generally to function poorly after treatment (McCrady et al., 1986; McKay et al., 1993; Moos and Moos, 1984). Patients on methadone maintenance typically return to their families after treatment, and those families have been found to show significant instability and social pathology. The level of social pathology in the family of origin is associated with the use of heroin during methadone treatment (Stanton, 1979; Stanton and Todd, 1982). The family relationship scale on the ASI predicts posttreatment drug use and general personal and social function among opiate-dependent patients in either inpatient therapeutic communities or outpatient methadone maintenance treatment programs (McLellan et al., 1983a, b).
A paradoxically negative relationship has been found between the reported number of available family and friends of the patient and relapse to cocaine use following treatment. For primarily African-American cocaine-dependent patients, the return to cocaine use was earlier if more friends and family had contact with the patient (Havassy et al., 1991, 1994). Many interactive variables may combine in important ways to define the nature and strength of the effect between a particular family and social constellation and a specific treatment (Longabaugh et al., 1995; Moos et al., 1990).
Overview. Comparatively few treatment variables have been shown to be predictive of outcome; only those for which evidence for their predictive value has been replicated are presented here. In contrast to the number of studies of patient factors, there are few studies of treatment setting, modality, process, and
service factors as predictors of outcome from substance abuse treatments. Perhaps this is because there have been many reliable and valid measures of various patient characteristics but still very few measures of treatment setting (Allison and Hubbard, 1982; Moos, 1974, 1987) or treatment services (McLellan et al., 1992). Developments such as the multisite NIAAA study of patient treatment matching (Project MATCH Research Group, in press) may begin to change this.
Treatment Setting. In the field of rehabilitation from alcohol dependence, several important studies have examined the role of treatment setting, generally showing that the setting of care might not be an important contributor to outcome (Alterman et al., 1994; McCrady et al., 1986). Reviews of the literature on inpatient and outpatient alcohol rehabilitation by Miller and Hester (1986) concluded that across a range of study designs and patient populations there was no significant advantage provided by inpatient care over that provided by outpatient care in the rehabilitation of alcohol dependence, despite the substantial difference in costs. One exception was a study of employed alcohol-dependent patients, which found that an inpatient program produced better outcomes than a very nonintensive form of outpatient treatment, largely Alcoholics Anonymous meetings (Chapman-Walsh et al., 1991).
In treatment of cocaine dependence differences in completion rates for inpatient and outpatient treatments do not appear to be related to longer-term outcome. Alterman et al. (1994) found that 89 percent of inpatients completed treatment, compared with a completion rate of 54 percent for day-hospital treatment. However, at 7 months posttreatment, both groups had made considerable improvements in their drug and alcohol use, family and social, legal, employment, and psychiatric problems. Abstinence rates for both groups were of 50 to 60 percent.
Similar findings have been reported in field studies of private substance abuse treatment programs treating primarily cocaine-dependent and cocaine-plus alcohol-dependent patients (McLellan et al., 1993; Pettinati et al., in press). In all of these studies, patients in outpatient treatment programs were less likely to complete treatment than those in inpatient programs, but those who did complete treatment showed equal levels of improvement and the outcomes in the two settings were comparable.
Attempts to formalize clinical decision processes regarding who should and who should not be assigned to inpatient and outpatient settings of care have had mixed results [McKay et al., 1992, 1994, in press(b)]. Partial support has been found for the predictive validity of the patient placement criteria of the American Society on Addiction Medicine (ASAM) [McKay et al., in press(b)], but research is in progress to evaluate whether the criteria can be effectively used to make decisions concerning placements to levels of care, defined by the amount and quality of medical supervision and monitoring.
Potential Quality Indicators
There may be an advantage for inpatient or residential forms of care for patients who may be less likely to complete treatment (e.g., homeless, more psychiatrically severe, and cocaine-dependent patients), but otherwise, there seem to be few differences in outcomes for inpatient and outpatient treatment. Thus, it would be useful to develop decision criteria that can effectively differentiate those patients who are not likely to show significant improvement from outpatient forms of treatment, such as the proportion of patients assigned to an outpatient treatment program who were found to meet ASAM criteria for placement in an inpatient or residential setting. Conversely, an indicator of clinical cost-effectiveness might be the proportion of patients assigned to inpatient hospitalization who were found to meet ASAM criteria for some form of outpatient treatment.
Length of Treatment and Adherence to Treatment
Virtually all studies of rehabilitation have shown that patients who stay in treatment longer or who attend the most treatment sessions have the best posttreatment outcomes (Armor et al., 1976; Ball and Ross, 1991; DeLeon, 1984, 1994; Simpson et al., 1986). Adherence with the suggestions of the health care provider (usually a physician for those with medical disorders or the counselor or treatment team for those with substance use disorders) has been found to be the single best predictor of continued favorable posttreatment function for substance abuse treatments as well as for other forms of chronic medical conditions, such as diabetes, hypertension, and asthma (McLellan and Durell, 1995; O'Brien and McLellan, 1996).
These relationships suggest that length of stay and patient adherence would appear to be exactly the type of measure that would be well suited to use as a quality indicator (e.g., the percentage of patients who have completed treatment or the percentage of patients who have attended 90 AA meetings in 90 days). However, such a measure would reflect an assumption that patients who enter treatment gradually acquire new motivation, skills, attitudes, knowledge, and supports over the course of their stay in treatment and that the gradual acquisition of these qualities or services is the reason for the favorable outcomes.
Patients who have been randomly assigned to receive a longer duration of treatment do not necessarily show better outcomes than those patients who have been randomly assigned to receive shorter treatments (Miller and Hester, 1986; Project MATCH Research Group, in press). Therefore, it is possible that better patients are likely to stay in treatment longer and to do more of what is recommended. If treatment gradually produces positive changes over time, it is clinically sound practice for patients to stay in treatment longer. On the other hand, if well-motivated, highly functioning, compliant patients enter treatment with the
requisite skills and supports necessary to do well, they might also benefit from a shorter treatment regimen.
When a high proportion of patients in a program stay in treatment for the recommended duration and comply with all treatment requirements, does that mean that the treatment processes, policies, and therapists have been selected to instill motivation and engage patients into behavioral change strategies, or that the program has been successful in selecting motivated and compliant patients who are likely to do well regardless of the services that they receive? The distinction between adherence and length of treatment needs further exploration.
Potential Quality Indicators
Length of stay and adherence with treatment recommendations are perhaps the two most easily measured aspects of treatment and are both suitable for inclusion in contemporary clinical management information systems. However, the measures are more appropriately viewed as descriptive rather than as predictive indicators of quality, because there are still questions regarding the meaning of these two measures with regard to outcome (See McKay et al., 1991).
Participation in AA and NA
AA is recognized as a social organization and not a formal treatment. However, AA has become synonymous with the last part of rehabilitation, aftercare. Virtually all alcohol dependence rehabilitation programs and most cocaine dependence rehabilitation programs refer patients to AA and/or NA programs, with instructions to get a sponsor, to attend “share and chair” at meetings, and to attend 90 meetings in 90 days as a continued commitment to sobriety. There has always been consensual validation for the value of AA and other peer-support forms of treatment, but relatively few studies have evaluated the contributions of AA and NA because of the anonymous quality of the groups.
Evidence shows that patients who have participated in AA, NA, or some other form of peer-support group, who have a sponsor, or who have participated in the fellowship activities have much better abstinence records than patients who have received rehabilitation treatments but who have not continued in AA [McKay et al., 1994, in press(a); Timko et al., 1994].
Although studies have generally suggested that the peer-support component of rehabilitation is valuable, it is also difficult to sort out the extent to which AA attendance constitutes an active ingredient of successful treatment or is simply a marker of treatment adherence and motivation (Vaillant, 1996). Infrequent or irregular attendance of AA meetings following discharge from residential treatment was associated with a poorer prognosis than either regular attendance or no attendance (McLatchie and Lomp, 1988).
Potential Quality Indicators
There is no doubt that peer-support groups have made an important contribution for a significant minority of patients. Participation in AA during and following rehabilitation seems to be an excellent marker for sustained reductions in alcohol and drug use as well as improved personal and social function for a significant, if still inexact, proportion of alcohol-dependent patients.
Thus, quality indicators for rehabilitation treatment (at least for alcohol-dependent patients) could be the proportion of patients who have acquired an AA sponsor and the proportion of patients in aftercare who have attended more than three AA meetings in the first month of treatment.
Therapists and Counselors Who Provide Treatment
A drug or alcohol abuse counselor or therapist can make an important contribution to the engagement and participation of the patient in treatment and to the posttreatment outcome. One example of the role of individual counseling was in a study of methadone patients who were randomly assigned to receive individual counseling plus methadone or methadone alone. Fifty-three percent of patients who received counseling showed sustained elimination of opiate use and 41 percent showed sustained elimination of cocaine use over the 6 months of the trial. In contrast, 68 percent of patients assigned to the no counseling condition failed to reduce their level of drug use (confirmed by urinalysis), and 34 percent of these patients required at least one episode of emergency medical care.
However, different outcomes are found for different therapists, including professional psychotherapists with doctorate-level training (Luborsky et al., 1985, 1986), experiential substance abuse counselors (McLellan et al., 1988; Miller et al., 1980), and different individual counselors within an alcohol treatment program (McCaul and Svikis, 1991). What distinguishes more effective from less effective counselors is not clear. A client-centered approach emphasizing reflective listening has been found to be more effective for problem drinkers than a directive, confrontational approach (Miller et al., 1993). In a review of the literature on therapist differences in substance abuse treatment, Najavits and Weiss (1994) concluded, “The only consistent finding has been that therapists' in-session interpersonal functioning is positively associated with greater effectiveness” (p. 683). Among indicators of interpersonal functioning are the ability to form a helping alliance (Luborsky et al., 1985, 1986), measures of the level of accurate empathy (Miller et al., 1980; Valle, 1981), and a measure of “genuineness,” “concreteness,” and “respect” (Valle, 1981).
Counselor certification is available from several sources throughout the country. These include the Rehabilitation Accreditation Commission; Certified Addictions Counselor (CAC) program, as well as professional certification from the ASAM, American Academy of Addiction Psychiatry, and the American Psycho-
logical Association. Currently, there is no evidence to show whether patients treated by certified addictions counselors, physicians, or psychologists have better outcomes than patients treated by noncertified individuals. This is an important gap in the existing literature, and results from such studies would be quite important for the licensing efforts and health policy decisions of many states and health care organizations.
NIAAA and the National Institute on Drug Abuse have sponsored a great deal of research aimed at developing useful medications for the treatment of substance-dependent persons. Great progress has been made over the past 10 years in the development of new medications and in the application of existing medications for the treatment of particular conditions associated with substance dependence and for particular types of substance-dependent patients. Because of the vast amount of research, this review includes some of the clearest results from the use of medications in the treatment of substance dependence, as well as citations for more comprehensive reviews of medications for interested readers. Two types of medications are discussed: agonist and antagonist or blocking medications.
Agonist Medications. Two agonist medications are in use in the treatment of drug dependence. Both of these medications work by direct occupation of receptors within the body to mimic the effects produced by the target drug. The most prevalent agonist is nicotine replacement in the form of gum or a skin patch, which has recently been approved for over-the-counter sales. Nicotine replacement is typically prescribed and used for relatively brief periods (1 to 3 months) as part of an abstinence-oriented program for nicotine dependence.
For more than 25 years, methadone has been an approved agonist medication for the maintenance treatment of opiate dependence. The long-acting form of methadone (48 to 72 hours in duration), LAAM, has recently received FDA approval and has been accepted by 16 states for use in the same way as methadone; that is, it is available only at methadone maintenance programs. Buprenorphine is a partial opiate agonist that has been widely used in Europe and the United States. It is thought to have some advantages over methadone in that it produces many fewer withdrawal symptoms and often produces none. At the time of this writing, it is not approved for use, although approval is expected shortly.
Among the most robust findings in the treatment literature is the relationship between the dose of methadone and the general outcome of methadone treatment: higher doses are more effective than lower doses (Ball and Ross, 1991; D'Aunno and Vaughn, 1995; IOM, 1995a). In a well-controlled, double-blind, multisite study, Ling et al. (1976) found that 100 milligrams per day was superior to 50 milligrams per day, as indicated by staff ratings of global improvement and by a drug use index comprising weighted results of opiate urine tests.
In a more recent randomized, double-blind study, Strain et al. (1993) compared 50 milligrams and 20 milligrams with a 0 milligram placebo-only group. They found orderly dose-response effects on treatment retention, and they found that 50 milligrams was more effective than 20 milligrams or 0 milligrams at decreasing opiate and cocaine use, as measured by urinalysis results. In a randomized double-blind comparison of moderate (40 to 50 milligrams) and high (80 to 100 milligrams) doses of methadone, Strain et al. (1996) found a significantly lower rate of opiate-positive urine specimens among patients receiving the high dose of methadone (53 percent vs. 62 percent). They concluded that although the higher dose was more effective, substantial opiate use can persist even among patients treated with 80 to 100 milligrams of methadone per day. There are many other studies of opiate agonist medications, but space limitations do not permit more detail here (see IOM, 1995a for additional information).
Potential Quality Indicator
If methadone is prescribed for purposes of maintenance rehabilitation as opposed to detoxification, the dose should be high enough to block the euphoric effects of street opiates (heroin) and the craving for opiates. Thus, a potential quality indicator would be the proportion of patients on methadone maintenance who have continued regular opiate use (as evidenced by past two or three opiate-positive urine specimens) but who have not had increases in their methadone or LAAM dose. Because it is a federal requirement that urine samples be obtained from patients on methadone maintenance, the data should be readily available at the clinic level.
Antagonist and Abuse Blocking Agents. Naltrexone is an orally administered opiate antagonist that blocks the actions of externally administered opiates such as heroin by competitive binding to opiate receptors. Naltrexone under the trade name Trexan® has been used for more than 20 years in the treatment of opiate dependence. More recently, naltrexone (marketed under a different trade name: Revia®) has been found to be effective in the treatment of alcohol dependence (O'Malley et al., 1992; Volpicelli et al., 1992).
Naltrexone at 50 milligrams per day has been approved by the FDA for use with alcohol-dependent patients, because independent studies have shown that it is a safe, effective pharmacological adjunct for reducing heavy alcohol use among alcohol-dependent patients. Its mechanism of action appears to be the blocking of at least some of the high produced by alcohol consumption, again through competitive binding of the opiate receptors (O'Malley et al., 1992; Volpicelli et al., 1992).
With regard to other medications designed to block the effects of an abused drug, disulfiram (Antabuse®) has been used the longest and most pervasively in the treatment of alcohol dependence. Although both disulfiram and naltrexone
can be used for extended periods, in practice they are generally prescribed for about 1 to 3 months as part of a more general rehabilitation program that includes behavioral change strategies (see the review by Anton, 1995). Many agents have been tried as blocking agents in the treatment of cocaine dependence, but there is still no convincing evidence that any of the various types of cocaine-blocking agents are truly effective for even brief periods of time or for even a significant minority of affected patients.
The use of opiate and alcohol antagonists or blocking agents is increasing as traditional addiction medicine physicians are becoming more comfortable with the prescription of adjunctive medications and as more substance dependence is treated by primary care physicians in office settings (Fleming and Barry, 1992). The past 10 years have witnessed innovation and discovery in this area, but the parameters that are most effective when using them are still not clear. Thus, some traditional addiction medicine physicians are reluctant to prescribe these medications unless therapy alone has been ineffective (IOM, 1995a, b).
The responsible and appropriate use of these antagonist or blocking medications in the treatment of substance dependence disorders may be among the most important topics for future research in the treatment field. Long-term studies are needed to evaluate the effects of these medications for various types of substance-dependent patients, as well as to determine the most appropriate and efficient mix of psychosocial and pharmacological services that will maximize the impact of rehabilitation.
Potential Quality Indicator
The available literature suggests that naltrexone can be very effective in the abstinence-oriented treatment of opiate dependence and that disulfiram and naltrexone can be effective as adjuncts in the treatment of alcohol dependence. A potential indicator of poor quality care for alcohol dependent patients could be the proportion of patients in abstinence-oriented rehabilitation for alcohol dependence who have continued to use alcohol (as evidenced by the past two or three positive breathalyzer readings) who have not been evaluated for naltrexone or disulfiram treatment.
The majority of patients admitted to substance abuse treatment have significant addiction-related problems in one or more areas such as medical status, employment and self support, family relations, and psychiatric function (McLellan and Weisner, 1996). As indicated above, the severity of these problems is generally predictive of the response during treatment as well as posttreatment outcome.
Studies over more than a decade have documented that strategies designed to direct and focus specialized services to these addiction-related problems can be ap-
plied in standard clinical settings and can be effective in improving the results of substance abuse treatment (McLellan et al., in press). Adding professional marital counseling (O'Farrell et al., 1985; McCrady et al., 1986; Stanton and Todd, 1982), psychotherapy (Carroll et al., 1994; Woody et al., 1995), and medical care (Fleming and Barry, 1992; Schonberg, 1988) produces clinically significant better outcomes from substance abuse treatment. However, interventions that have been developed to improve employment and self support among substance-dependent patients have had mixed results (French et al., 1992; Hall et al., 1981; Zanis et al., 1994).
The majority of adjunctive forms of therapy and services have been most clearly associated with improved personal health and social function following treatment and have been less related to reduced alcohol and drug use. In addition, and not surprisingly, these treatments have only been shown to be effective with those patients having more severe problems in the target area (i.e., matching effect); that is, if there has been no indication of a relatively severe problem in the target area, there has typically been no evidence that the provision of the target therapy is effective or worthwhile (Woody et al., 1984).
Potential Quality Indicators
Significant problems in the areas of employment, medical and psychiatric health, and family relations are thought to be impediments to treatment for substance-dependent patients in two ways. First, the presence of these problems often complicates the provision of standard substance abuse treatment, and second, these problems, if left unattended, can provoke a relapse to substance use even among well-motivated, abstinent individuals. For these reasons, the provision of treatments for these problems is seen as important both for the immediate purpose of retaining patients in treatment and for reducing the risk of a relapse. If these specialized services are potent, it follows that they will have a direct effect on symptoms in the target problem area (e.g., reduction of symptoms of depression) but an indirect and possibly delayed effect on the substance use problems (e.g., longer latency until relapse).
Although many substance abuse treatment programs do not have the resources to provide specialized treatment services, it is at least possible for these programs to perform an active referral to an appropriate agency or practitioner to attempt to access these services. Thus, potential quality indicators would be the proportion of patients who showed evidence of a significant psychiatric problem (by the criteria of DSM IV) who received sessions of specialized psychiatric or psychological care, and the proportion of patients reporting significant family problems who receive sessions of specialized couples or family therapy.
SUMMARY AND DISCUSSION
The search for quality indicators in the research literature has revealed five major challenges. They are summarized in this section.
The existing research on treatment outcomes has been disappointing with regard to informing the search for potential quality indicators.
Most of the outcomes studies in the current literature were conducted by clinical researchers, typically in controlled trials. The purpose of these studies was generally to determine whether the index treatment, when delivered under specified conditions to rather highly selected samples of patients, could effect the expected changes relative to standard or minimal treatment conditions.
However, most clinical trials reviewed here do not lend themselves to the identification of quality indicators. They often exclude important classes of patients (e.g., users of multiple substances and psychotic patients) and focus on very specific outcomes (e.g., abstinence from a single substance). Under such conditions, it is difficult to flexibly tailor treatments to individual patients, because studies call for strict adherence to experimental protocols. In clinical practice, when a patient fails to respond to one type of intervention, the sensitive clinician will alter the approach. Thus, the interventions that are compared in experiments may not reflect what happens in practice.
Research has effectively established that treatment can be effective, but there are only preliminary indications at this time about why treatment is effective or what it is about or within treatment that makes it effective. Treatment researchers are only now beginning to develop the measures and models that will be necessary for the exploration of questions regarding why treatment works. If the outcomes research field is really to inform the search for quality or performance indicators in substance abuse treatment, then it will be necessary to move beyond the question of whether treatment works to the question of how treatment works.
To accomplish this, researchers will need to make a methodology shift from the simple evaluation or comparison of treatment outcomes to the parametric study of the various types of treatment services and therapeutic processes delivered within those treatments and their relationship to the target outcomes. The methodology will require measurement of more than just the target outcomes at a posttreatment follow-up point. Careful recording of the treatment services and processes provided during treatment will be necessary, as will the concurrent monitoring of during-treatment changes in patient attitude, cognition, motivation, affect, and behavior that are the interim goals of these processes.
These types of dose-response or dose-ranging designs will ultimately permit the discovery of the important changes or therapeutic milestones that patients must achieve along their route to recovery and the active ingredients within a treatment that are responsible for those milestones and ultimately for lasting outcomes follow-
ing treatment. This is a line of research that has been called for by several within the field, but the present review has uncovered very few studies to date that have pursued this line of research. Thus, one message from this paper is a call to the treatment research field for more systematic work along this line of investigation.
Significant confusion and disagreement exist within the field on important and basic concepts that are essential for the identification of potential quality indicators. The most basic confusion has been on the definition of outcomes.
A reviewer of this field will get substantially different views about the outcomes of a substance abuse treatment depending on the perspective taken regarding what outcome is and when, how, and by whom it is measured. Consider three common perspectives on the evaluation of an outpatient treatment program. A quality assurance or service delivery evaluation of that treatment might conclude that the program had very good outcomes because there was no waiting for treatment entry and at discharge more than 80 percent of the patients were highly satisfied with their counselors and physicians. A clinical researcher, having interviewed a sample of patients at admission to the program and again 6 months following discharge, might conclude that the program had mixed outcomes because at the follow-up point only 50 percent of the patients were abstinent (the intended goal of the program) but there was a 70 percent reduction in the frequency of drinking and a 50 percent reduction in medical and psychiatric symptoms. Meanwhile, an economist or health policy analyst might have used Medicaid data tapes to compare the health services utilization rates of a sample of discharged patients 2 years prior to their treatment admission and 2 years following their discharge. The conclusion here might be that treatment had a very poor outcome because there had been no decrease in health care utilization from the pretreatment to the posttreatment period, and hence no cost offset to the public.
This example illustrates two points. First, these three common perspectives on outcomes have different purposes for their evaluations and different expectations regarding treatment. They measure different elements of the treatment process and the patient population and at different points in time. Second, these different measures of outcome are not well related to each other, and to the extent that quality indicators are expected to relate to outcomes, these different perspectives will suggest different quality indicators.
Two stages of treatment exist for substance-dependent patients: the acute care, detoxification-stabilization stage and the subsequent rehabilitation stage. Each stage has different expected outcomes.
For the detoxification-stabilization stage, favorable outcomes include the elimination of the signs and symptoms of physical and emotional instability asso-
ciated with the initial cessation of substance use and the motivation and engagement of the patient into continued rehabilitation. Treatment characteristics that were most closely associated with these outcomes were the inpatient setting of care (at least for patients likely to drop out prematurely) and treatment to the criterion of 3 consecutive days without withdrawal signs or symptoms.
For rehabilitative treatment to have an opportunity to succeed, the outcomes from the detoxification-stabilization stage would have to be achieved. An outcome from rehabilitative treatment should be lasting improvements in those problems that led to the treatment admission and that were important to the patient and to society. Three outcomes domains have been measured at least 6 months following treatment discharge: reduction in substance use, improvement in personal health and social function, and reduction in public health and safety problems.
Patient variables that had been reliably associated with better outcomes from rehabilitation included (1) low severity of dependence and psychiatric symptoms at admission, (2) “readiness for change” beyond the precontemplation stage of change, (3) being employed or self supporting, and (4) having family and social supports for sobriety. Treatment variables that have been reliably associated with better outcomes in rehabilitation included (1) staying longer in treatment and being more compliant with treatment recommendations, (2) having an individual counselor or therapist (particularly an effective one), (3) receiving proper medications, (4) participating in voucher-based, behavioral reinforcement interventions, (5) participating in AA or NA following treatment, and (6) having specialized services provided for adjunctive medical, psychiatric, or family problems.
Although none of these patient or treatment variables showed a completely unambiguous record of prediction outcomes, the findings have been replicated across more than one type of primary drug problem (alcohol, cocaine, or opiates) and in more than one evaluation. However, although some of the predictors identified (e.g., longer lengths of stay and greater adherence) are quite robust, there is no clear understanding of the basis for the predictive relationship.
No single rehabilitation modality or therapeutic process has yet been reliably associated with superior outcomes across all populations of patients. Furthermore, it was surprising that some of the treatment elements that are most widely provided in substance abuse treatment (e.g., group therapy) have not been associated with outcomes. Clearly, more research is needed to identify the “active ingredients” of treatment and the “minimal effective dose” of these ingredients.
The ability to identify potentially useful quality indicators relies on a clear understanding of the ways in which these indicators will ultimately be used.
Quality indicators can be used in two ways: (1) at the individual patient level to provide clinicians with early warning signs for poor outcome and thus allow for modification of the treatment plan, and (2) in the aggregate, to provide evaluators and regulators with rapid, easily collected, and face-valid indications of treat-
ment program performance, ultimately for purposes of interprogram comparisons and possibly for report cards.
Because a primary purpose for these indicators will be clinical decision support, they need to be measured at the individual patient level and to be collectable as early as possible in the course of treatment, using nonintrusive and rapid methods of data collection. These features are essential for the information to be relevant to clinical decision-making and, in turn, worthwhile for the clinicians who will ultimately be charged with recording these measures.
Not all variables identified as predictors of outcome will be useful in clinical decision-making, because many (e.g., gender and socioeconomic strata) cannot be modified in the course of treatment. At the same time, because a second purpose of these quality indicators will be to compare aggregated mean values between two treatment programs or among several patient subgroups, it will be important to have access to all variables that affect the outcomes of treatment independent of the treatment process. These “case mix adjusters” are important in any comparative study of outcomes or quality indicators to adjust the groups on variables that could have an independent effect on outcome, thus helping to provide a level playing field when the comparative evaluations are performed. Before any of the prospective quality indicators can be used in a comparative fashion, however, much more research comparing different case mix adjustment strategies and different combinations of predictor variables is needed.
Although both treatment process and patient change variables can serve as quality indicators, patient change variables are conceptually and practically much better.
Only two types of measures meet the practical and conceptual needs of the clinical, management, and regulatory groups that are interested in identifying quality indicators. The first of these are treatment elements, processes and practices: interventions or services that are done to or for the patient during treatment. The second of these are interim changes in patient status: aspects of the patient's affect, knowledge, motivation, and behavior that are presumed to be problematic in the patient at the start of treatment and are thus the direct focus of the treatment elements within rehabilitation.
The great majority of the quality indicators used thus far in the evaluation of substance abuse and mental health treatments have been treatment process indicators (counseling provided to urge smokers to quit, referral to outpatient care following inpatient discharge, etc.). Typically, they have been measured by staff notations in treatment charts. There is justification for using these process measures. First, because costs or charges are typically associated with the provision of treatment processes, these measures are generally available and accessible in clinical management information systems. Second, and more importantly, there are
clear indications from the outcomes literature that certain treatment processes and treatment elements are reliably related to outcomes.
This review has referred to these treatment process measures as secondary indicators of treatment quality for two reasons. The first reason is practical and based on the quality of available evidence. The simple notation in a chart that an activity, labeled as the appropriate or intended process, has been provided at some level of intensity, by someone with an unknown ability or training level, with no indication of its immediate effects, is not, by itself, the type of evidence that inspires confidence in the quality of that treatment.
The second reason for referring to even those treatment practices or treatment elements that have been reliably associated with outcomes as secondary indicators of quality relates to the level of inference that is available from such an association. No treatment element, service, or procedure produces a lasting outcome directly but, rather, produces an outcome through the production of at least one interim change in a patient's attitude, affect, knowledge, motivation, cognition, or behavior. For example, patients who attend rehabilitation following detoxification have better posttreatment outcomes than those who stop treatment following detoxification.
Thus, it can be said that the treatment process or the treatment practice of referring a patient to outpatient treatment is associated with a better posttreatment outcome. However, this association is only true when the referral has actually resulted in the patient 's attendance and participation in the rehabilitation. Actually, it is this interim change in the patient's behavior (attendance) rather than the process of referral that is most directly associated with the ultimate outcome, and the treatment practice is only associated with that outcome through its ability to produce that interim result.
There is another reason to use measures of interim changes in patient status (symptoms, signs, behaviors, etc.) instead of treatment process measures as quality indicators. The majority of patient status measures can be measured in a more valid, unbiased, and verifiable way than most treatment process measures. Thus, although it would be possible to check a patient chart for a note indicating the current dose of methadone (a secondary indicator of treatment quality), greater confidence would come from primary indicators, such as interim results in the form of reductions in observed withdrawal signs and negative urine screens.
Although these measures of interim change in patient status may be slightly more difficult to collect, most are not burdensome and are, again, directly associated with the focus of the treatment elements or interventions being applied. Specifically, given a patient status variable that has been reliably associated with treatment outcome (e.g., a high-severity psychiatric problem at admission) and a treatment process variable that has also been reliably related to outcome (e.g., provision of professional psychotherapy), the responsible clinician and clinical regulator will be better informed regarding the quality of the care provided to the patient by measuring changes in psychiatric symptomatology over the course of
treatment (e.g., a weekly change in Beck Depression Inventory) rather than measuring the number of therapy sessions provided during the course of treatment.
NEED FOR FURTHER RESEARCH
A new line of research is needed to address at least two central questions of effective and efficient treatment delivery.
What types of interim changes in patients should be effected during treatment to provide the highest probability of lasting gains following treatment?
Not all of the changes in patients' attitudes, affects, motivation, knowledge, and behavior that are the interim goals of substance abuse treatments are important for attaining favorable posttreatment outcomes. An important role for future treatment research will be to identify those interim patient changes that are reliably predictive of lasting benefits following treatment. These ultimately will be the quality indicators that the field is searching for.
Which treatment settings, modalities, and services provide the most potent and rapid ways of effecting the during-treatment changes that have been shown to be important predictors of lasting outcomes, and at what costs?
Not all of the treatment elements, services, or activities that are provided to patients in treatment will be appropriate or adequate to produce the interim patient changes that are desired. An important role for future treatment research will be to identify the active ingredients and the minimum effective dose of those ingredients that can effect the important interim changes in patients during the course of treatment.
Combinations of active ingredients will ultimately be translated into empirically derived clinical pathways and treatment guidelines. Because these combinations of proven effective treatment ingredients are compared for potency and duration of action as well as on the basis of their costs of delivery for both the provider and the patient, real estimates of the value and efficiency of treatments can be developed.
Allison M, Hubbard RL. 1982. Drug Abuse Treatment Process: A Review of the Literature. TOPS Research Monograph. Raleigh, NC: Research Triangle Press.
Alterman AI, McLellan AT, O'Brien CP, August DS, Snider EC, Cornish JC, Droba M, Hall CP, Raphaelson A, Schrade F. 1994. Effectiveness and costs of inpatient versus day hospital cocaine rehabilitation. Journal of Nervous and Mental Diseases 182:157-163.
Alterman AI, McKay J, Mulvaney F, McLellan AT. 1996. Prediction of attrition from day hospital treatment in lower socioeconomic cocaine-dependent men. Drug and Alchohol Dependence 40:227-233.
Anglin MD, Speckart GR, Booth MW, Ryan TM. 1989. Consequences and costs of shutting off methadone. Addictive Behaviors 14:307-326.
Anglin MD, Hser Y. 1990. Legal coercion and drug abuse treatment. In: Inciardi J, ed. Handbook on Drug Control in the United States. Westport, CT: Greenwood Press.
Anton RF. 1995. New Directions in the pharmacotherapy of alcoholism. Psychiatric Annals 25:353-362.
Armor DJ, Polich JM, Stambul HB. 1976. Alcoholism and Treatment. Santa Monica, CA: The RAND Corporation Press.
Babor TF, Dolinsky Z, Rounsaville BJ, Jaffe JH. 1988. Unitary versus multidimensional models of alcoholism treatment outcome: An empirical study. Journal of Studies on Alcohol 49(2):167-177.
Bale RN. 1979. Outcome research in therapeutic communities for drug abusers: A critical review 1963-1975. International Journal of the Addictions 14:1053-1074.
Ball JC, Ross A. 1991. The Effectiveness of Methadone Maintenance reatment. New York: Springer-Verlag.
Belding MA, Iguchi MY, Lamb RJ. In press. Stages and processes of change as predictors of drug use among methadone maintenance patients. Experimental and Clinical Psychopharmacology.
Brown SA, Irwin M, Schuckit MA. 1991. Changes in anxiety among abstinent male alcoholics. Journal of Studies on Alcohol 52:55-61.
Carroll KM, Rounsaville BJ, Gawin FH. 1991. A comparative trial of psychotherapies for ambulatory cocaine abusers: Relapse prevention and interpersonal psychotherapy. American Journal of Drug and Alcohol Abuse 17:229-247.
Carroll KM, Rounsaville BJ, Nich C, Gordon LT, Wirtz PW, Gawin F. 1994. One-year follow-up of psychotherapy and pharmacotherapy for cocaine dependence: Delayed emergence of psychotherapy effects. Archives of General Psychiatry 51(12):989-997.
Carroll KM, Nich C, Rounsaville BJ. 1995. Differential symptom reduction in depressed cocaine abusers treated with psychotherapy and pharmacotherapy. Journal of Nervous and Mental Disease 183:251-259.
Chapman-Walsh D, Hingson RW, Merrigan DM, Morelock Levenson S, Cupples,A, HeerenT, Coffman GA, Becker CA, Barker TA, Hamilton SK, McGuire TG, Kelly CA. 1991. A randomized trial of treatment options for alcohol-abusing workers . The New England Journal of Medicine 325:775-782.
Cushman P, Dole VP. 1973. Detoxification of rehabilitatied methadone-maintained patients. Journal of the American Medical Association 226:747-752.
D'Aunno TJ, Vaughn T. 1995. An organizational analysis of service patterns in drug abuse treatment . Journal of Substance Abuse 16:123-131.
DATOS (Drug Abuse Treatment Outcome Study). 1992. Drug abuse treatment outcome study. NIDA Research Monograph No. 237. Rockville, MD: National Institute on Drug Abuse.
DeLeon G. 1984. The therapeutic community: study of effectiveness treatment. NIDA Research Monograph. Publication No. 84-1286. Rockville, MD: National Institute on Drug Abuse.
DeLeon G. 1994. Therapeutic communities: Toward a general theory and model. In: Tims FM, De Leon G, Jainchill N, eds. NIDA Research Monograph. Washington, D.C.: National Institute on Drug Abuse.
Dennis ML, Karuntzos GT, McDougal GL, French MT. 1993. Developing training and employment programs to meet the needs of methadone treatment clients. Evaluation Program Planning 16:73-86.
DiClemente CC, Prochaska JO, Fairhurst SK, Velicer WF, Velasquez MM, Rossi JS. 1991. The process of smoking cessation: An analysis of precontemplation, contemplation, and preparation stages of change. Journal of Consulting and Clinical Psychology 59:295-304.
Finnegan LP. 1991. Treatment issues for opioid-dependent women during the perinatal period. Journal of Psychoactive Drugs 23:191-199.
Finney JW, Moos RH, Chan DA. 1981. Length of stay and program component effects in the treatment of alcoholism: A comparison of two techniques for process analyses. Journal of Consulting and Clinical Psychology 49:120-131.
Finney JW, Moos RH. 1992. The long-term course of treated alcoholism: II. Predictors and correlates of 10-year functioning and mortality. Journal of Studies on Alcohol 53:142-153.
Fleming MF, Barry KL, eds. 1992. Addictive Disorders. St. Louis: Mosby Yearbook Primary Care Series.
French MT, Rachal JV, Hubbard RL. 1991. Conceptual framework for estimating the social cost of drug abuse . Journal of Health and Social Policy 2:1-22.
French MT, Dennis ML, McDougal GL, Karuntzos GT, Hubbard RL. 1992. Training and employment programs in methadone treatment: Client needs and desires. Journal of Substance Abuse Treatment 9:293-303.
Gawin FH, Kleber HD. 1986. Abstinence symptomatology and psychiatric diagnoses in cocaine abusers . Archives of General Psychiatry 43:107-113.
Gawin FH, Ellinwood EH. 1988. Cocaine and other stimulants: Actions, abuse, and treatment. The New England Journal of Medicine 318:1173-1182.
Gerstein DR, Johnson RA, Harwood HJ, Fountain D, Suter N, Malloy K. 1994. Evaluating Recovery Services: The California Drug and Alcohol Treatment Assessment (CALDATA). Fairfax, VA: Lewin-VHI and National Opinion Research Center at the University of Chicago.
Gomberg ESL. 1989. Alcoholic women in treatment: Early histories and early problem behaviors . Advances in Alcoholism and Substance Abuse 8:133-147.
Gomberg ESL, Nirenberg TD, eds. 1993. Women and Substance Abuse. Norwood, NJ: Ablex Publishing Corporation.
Gossop M, Johns A, Green L. 1986. Opiate withdrawal: Inpatient versus outpatient programmes and preferred versus random assignment. British Medical Journal 293:103-104.
Hagan TA, Finnegan LP, Nelson L. 1994. Impediments to comprehensive drug treatment models for substance abusing women: Treatment and research questions. Journal of Psychoactive Drugs 26:163-171.
Hall SM, Loeb P, LeVois P, Cooper J. 1981. Increasing employment in ex-heroin addicts. II: Methadone maintenance sample. Behavioral Medicine 12:453-460.
Havassy BE, Hall SM, Wasserman DA. 1991. Social support and relapse: Commonalities among alcoholics, opiate users, and cigarette smokers. Addictive Behaviors 16(5):235-246.
Havassy BE, Wasserman D, Hall SM. 1994. Social relationships and cocaine use in an American treatment sample . Addiction 18:145-161.
Hayshida M, Alterman AI, McLellan AT, O'Brien CP, Purtill JJ, Volpicelli JR, Raphaelson AH, Hall CP. 1989. Comparative effectiveness of inpatient and outpatient detoxification in patients with mild-to-moderate alcohol withdrawal syndrome. The New England Journal of Medicine 320(6):358-365.
Heather N, Rollnick S, Bell A. 1993. Predictive validity of the Readiness to Change Questionnaire. Addiction 88:1667-1677.
Higgins ST, Budney AJ, Bickel WK, Hughes JR, Foeg FE, Badger GJ. 1993. Achieving cocaine abstinence with a behavioral approach. American Journal of Psychiatry 150:763-769.
Higgins ST, Budney AJ, Bickel WK, Foerg FE, Donham R, Badger GJ. 1994. Incentives improve outcome in outpatient behavioral treatment of cocaine dependence. Archives of General Psychiatry 51:568-576.
Holder HD, Blose JO. 1992. The reduction of health care costs associated with alcoholism treatment: A 14-year longitudinal study. Journal of Studies on Alcohol 53(4):293-302.
Hubbard RL, Marsden ME. 1986. Relapse to use of heroin, cocaine and other drugs in the first year after treatment. In: Relapse and Recovery in Drug Abuse. NIDA Research Monograph No. 72. Rockville, MD: National Institute on Drug Abuse.
Hubbard RL, Marsden ME, Rachal JV, Harwood HJ, Cavanaugh ER, Ginzburg HM. 1989. Drug Abuse Treatment: A National Study of Effectiveness. Chapel Hill: University of North Carolina Press.
Inciardi JA. 1988. Some considerations on the clinical efficacy of compulsory treatment: Reviewing the New York experience. In: Leukefeld CG, Tims FM, eds. Compulsory Treatment of Drug Abuse: Research and Clinical Practice . NIDA Research Monograph No. 86. Rockville, MD: National Institute on Drug Abuse.
IOM (Institute of Medicine). 1990a. Broadening the Base of Treatment for Alcohol Problems. Washington, DC: National Academy Press.
IOM. 1990b. Treating Drug Problems. Vols. 1 and 2. Washington, DC: National Academy Press.
IOM. 1995a. Federal Regulation of Methadone Treatment. Washington, DC: National Academy Press.
IOM. 1995b. Development of Medications for the Treatment of Opiate and Cocaine Addictions: Issues for the Government and Private Sector. Washington, DC: National Academy Press.
Johnston LD, O'Malley PM, Bachman JG. 1996. National Survey Results on Drug Use From the Monitoring the Future Study. NIDA Publication No. 96-4027. Washington, DC: National Institute on Drug Abuse.
Kadden RM, Cooney NL, Getter H, Litt MD. 1990. Matching alcoholics to coping skills or interactional therapies: Post treatment results. Journal of Consulting and Clinical Psychology 57:698-704.
Kang SY, Kleinman PH, Woody GE, Millman RB, Todd TC, Kemp J, Lipton DS. 1991. Outcomes for cocaine abusers after once-a-week psychosocial therapy . American Journal of Psychiatry 148:630-635.
Kosten TR, Rounsaville BJ, Kleber HD. 1987. Multidimensionality and prediction and treatment outcome in opioid addicts: 2.5-year follow-up. Comprehensive Psychiatry 28:3-13.
Lawental E, McLellan AT, Grissom GR, Brill P, O'Brien CP. 1996. Coerced treatment for substance abuse problems detected through workplace urine surveillance: Is it effective? Journal of Substance Abuse 8(1):115-128.
Ling W, Charuvastra C, Kaim SC, Klett J. 1976. Methadyl acetate and methadone as maintenance treatments for heroin addicts. Archives of General Psychiatry 33:709-720.
Lipton DS, Maranda MJ. 1983. Detoxification from heroin dependency: An overview of method and effectiveness. Advances in Alcohol and Substance Abuse 2:31-55.
Longabaugh R, Beattie M, Noel N, Stout R, Malloy P. 1993. The effect of social investment on treatment outcome. Journal of Studies on Alcohol 54:465-478.
Longabaugh R, Wirtz PW, Beattie MC, Noel NE, Stout R. 1995. Matching treatment focus to patient social investment and support: 18-month follow-up results. Journal of Consulting and Clinical Psychology 63:296-307.
Luborsky L, McLellan AT, Woody GE, O'Brien CP. 1985. Therapist success and its determinants. Archives of General Psychiatry 42:602-611.
Luborsky L, Crits-Christoph P, McLellan AT, Woody GE. 1986. Do psychotherapists vary much in their effectiveness? The answer within four outcome studies. American Journal of Orthopsychiatry 56(4):501-512.
Marlatt GA. 1988. Matching clients to treatment: Treatment models and stages of change . In: Donovan DM, Marlatt A, eds. Assessment of Addictive Behaviors. New York: Guilford Press. Pp. 474-483.
McCaul M, Svikis D. 1991. Improving client compliance in outpatient treatment: Counselor-targeted interventions. NIDA Research Monograph No. 106. Rockville, MD: National Institute on Drug Abuse. Pp. 204-217.
McConnaughy EA, Prochaska JO, Velicer WF. 1983. Stages of change in psychotherapy: Measurement and sample profiles . Psychotherapy: Theory, Research and Practice 20:368-375.
McCrady BS, Noel NE, Abrams DB, Stout RL, Nelson HF, Hay WM. 1986. Comparative effectiveness of three types of spouse involvement in outpatient behavioral alcoholism treatment. Journal of Studies on Alcohol 47:459-467.
McKay JR, Alterman AI, McLellan AT. 1991. Does achieving goals of addiction treatment predict better outcomes? Alcoholism: Clinical and Experimental Research 15:383.
McKay JR, McLellan AT, Alterman AI. 1992. An evaluation of the Cleveland Criteria for inpatient treatment of substance abuse. American Journal of Psychiatry 149:1212-1218.
McKay JR, Longabaugh RC, Beattie M, Maisto S, Noel N. 1993. Changes in family functioning during treatment and drinking outcomes for high and low autonomy alcoholics. Addictive Behaivors 18:355-363.
McKay JR, Alterman AI, McLellan AT, Snider EC. 1994. Treatment goals, continuity of care, and outcome in a day hospital substance abuse rehabilitation program. American Journal of Psychiatry 151:254-259.
McKay JR, McLellan AT, Alterman AI, Cacciola J, Rutherford M, O'Brien CP. In press(a). Predictors of participation in aftercare sessions and self-help groups following completion of intensive outpatient treatment for substance abuse. Journal of Studies on Alcohol.
McKay JR, Cacciola J, Alterman A, McLellan AT, Wirtz P. In press(b). An initial evaluation of the psychosocial dimensions of the Americant Society on Addiction Medicine criteria for inpatient versus intensive outpatient substance abuse rehabilitation. Journal of Studies on Alcohol.
McLatchie BH, Lomp KG. 1988. Alcoholics Anonymous affiliation and treatment outcome among a clinical sample of problem drinkers. American Journal of Drug and Alcohol Abuse 14:309-324.
McLellan AT, Ball JC. 1995. Is methadone treatment effective? Background Paper prepared for IOM Committee on Federal Regulation of Methadone Treatment. In: IOM. Federal Regulation of Methadone Treatment. Washington, DC: National Academy Press.
McLellan AT, Durell J. 1995. Evaluating substance abuse and psychiatric treatments: Conceptual and methodological considerations. In: Sederer L, ed. Outcomes Assessment in Clinical Practice. New York: Williams and Wilkins.
McLellan AT, Weisner C. 1996. Achieving the public health potential of substance abuse treatment: Implications for patient referral, treatment “matching” and outcome evaluation . In: Bickel W, DeGrandpre R, eds. Drug Policy and Human Nature. Philadelphia: Wilkins and Wilkins.
McLellan AT, Hunkeler E. In press. Relationships between patient satisfaction and patient performance in addiction treatment. American Journal of Psychiatry.
McLellan AT, Luborsky L, O'Brien CP, Woody GE. 1980. An improved diagnostic instrument for substance abuse patients: The Addiction Severity Index. Journal of Nervous and Mental Diseases 168:26-33.
McLellan AT, O'Brien CP, Luborsky L, Woody GE, Kron R. 1981. Are the addiction-related problems of substance abusers really related? Journal of Nervous and Mental Diseases 169(4):232-239.
McLellan AT, Woody GE, Luborsky L, O'Brien CP, Druley KA. 1982. Is treatment for substance abuse effective? Journal of the American Medical Association 247:1423-1427.
McLellan AT, Luborsky L, Woody GE, Druley KA, O'Brien CP. 1983a. Predicting response to alcohol and drug abuse treatments: Role of psychiatric severity. Archives of General Psychiatry 40:620-625.
McLellan AT, Luborsky L, Woody GE, O'Brien CP, Druley KA. 1983b. Increased effectiveness of substance abuse treatment: A prospective study of patient-treatment “matching”. Journal of Nervous and Mental Diseases 171(10):597-605.
McLellan AT, Griffith J, Childress AR, Woody GE. 1984. The psychiatrically severe drug abuse patient: Methadone maintenance or TC. American Journal of Drug and Alcohol Abuse 10(1):77-95.
McLellan AT, Luborsky L, Cacciola J, Griffith J. 1985. New data from the Addiction Severity Index: Reliability and validity in three centers. Journal of Nervous and Mental Disease 173:412-423.
McLellan AT, Woody GE, Luborsky L, Goehl L. 1988. Is the counselor an "active ingredient" in substance abuse rehabilitation? Journal of Nervous and Mental Disease 176:423-430.
McLellan AT, Alterman AI, Woody GE, Metzger D. 1992. A quantitative measure of substance abuse treatments: The treatment services review. Journal of Nervous and Mental Disease 180:100-109.
McLellan A, Grissom G, Durell J, Alterman AI, Brill P, O'Brien CP. 1993. Substance abuse treatment in the private setting: Are some programs more effective than others? Journal of Substance Abuse Treatment 10:243-254.
McLellan AT, Alterman A, Metzger DS, Grissom GR, Woody GE, Luborsky L, O'Brien CP. 1994. Similarity of outcome predictors across opiate, cocaine, and alcohol treatments: Role of treatment services. Journal of Consulting and Clinical Psychology 62(6):1141-1158.
McLellan AT, Alterman AI, Woody GE, Metzger D, McKay JR, O'Brien CP. 1995. Great Expectations: A review of the concepts and empirical findings regarding substance abuse treatment. In: Alcohol and Substance Dependence. London, England: Royal Task Force on Substance Dependence.
McLellan AT, Grissom G, Brill P. In press. Improved outcomes from treatment service “matching” in substance abuse patients: A controlled study. Archives of General Psychiatry.
Miller WR, Hester RK. 1986. Inpatient alcoholism treatment: Who benefits? American Psychologist 41:794-805.
Miller WR, Taylor CA, West JC. 1980. Focused versus broad-spectrum behavior therapy for problem drinkers . Journal of Consulting and Clinical Psychology 48:590-601.
Miller WR, Benefield RG, Tonigan JS. 1993. Enhancing motivation for change in problem drinking: A controlled comparison of two therapist styles. Journal of Consulting and Clinical Psychology 61:455-461.
Miller WR, Zweben A, DiClemente CC, Rychtarik RG. 1994. Motivational Enhancement Therapy Manual: A Clinical Research Guide for Therapists Treating Individuals With Alcohol Abuse and Dependence . Rockville, MD: National Institute on Alcohol Abuse and Alcoholism.
Moos RH. 1974. Evaluating Treatment Environments. New York: Wiley.
Moos RH. 1987. The Social Climate Scales: A Users Guide. Palo Alto, CA: Consulting Psychologists Press.
Moos RH, Moos B. 1984. The process of recovery from alcoholism. III: Comparing family functioning in alcoholic and matched control families. Journal of Studies on Alcohol 45:111-118.
Moos RH, Finney JW, Cronkite RC. 1990. Alcoholism Treatment: Context, Process and Outcome. New York: Oxford University Press.
Najavits LM, Weiss RD. 1994. Variations in therapist effectiveness in the treatment of patients with substance use disorders: An empirical review. Addiction 89(6):679-688.
Naranjo CA, Sellers EM, Chater K, Iversen P, Roach C, Sykora K. 1983. Nonpharmacologic intervention in acute alcohol withdrawal. Clinical Pharmacology and Therapeutics 34:214-219.
O'Brien CP, McLellan AT. 1996. Myths about the treatment of addiction. Lancet 347:237-240.
O'Farrell TJ, Cutter HS, Floyd FJ. 1985. Evaluating behavioral marital therapy for male alcoholics: Effects on marital adjustment and communication from before to after treatment . Behavior Therapy 16(2):147-167.
ONDCP (Office of National Drug Control Policy). 1995. Pulse Check: National Trends in Drug Abuse. Washington, DC: U.S. Government Printing Office.
O'Malley SS, Jaffe AJ, Chang G, Schottenfeld RS. 1992. Naltrexone and coping skills therapy for alcohol dependence: A controlled study. Archives of General Psychiatry 49:881-887.
Pettinati HM, Belden PP, Evans BD, Ruetsch CR, Meyers K, Jensen JM. In press. The natural history of outpatient alcohol and drug abuse treatment in a private health care setting. Alcoholism: Clinical and Experimental Research.
Platt JJ. 1995. Vocational rehabilitation of drug abusers. Psychological Bulletin 117:416-435.
Powell BJ, Pennick EC, Othemer E, Bingham SF, Rice AS. 1982. Prevalence of additional psychiatric syndromes among male alcoholics . Journal of Clinical Psychiatry 43:404-407.
Prochaska JO, DiClemente CC, Norcross JC. 1992. In search of how people change: Applications to addictive behaviors . American Psychologist 47(9):1102-1114.
Project MATCH Research Group. In press. Matching alcoholism treatments to client heterogeneity. Project MATCH posttreatment drinking outcomes. Journal of Studies on Alcohol.
Robinson GM, Sellers EM, Janecek E. 1981. Barbiturate and hypnosedative withdrawal by a multiple oral phenobarbital loading dose technique. Clinical Pharmacology and Therapeutics 30:71-76.
Roman P. 1988. Growth and transformation in workplace alcoholism programming. In: Galanter M, ed. Recent Developments in Alcoholism. Vol.11. New York: Plenum. Pp. 131-158.
Rounsavile BJ, Weissman MM, Crits-Christoph K, Wilber C, Kleber H. 1982. Diagnosis and symptoms of depression in opiate addicts: Course and relationship to treatment outcome. Archives of General Psychiatry 39:151-156.
Rounsaville BJ, Glazer W, Wilber CH, Weissman MM, Kleber H. 1983. Short-term interpersonal psychotherapy in methadone-maintained opiate addicts. Archives of General Psychiatry 40:630-636.
Rounsaville BJ, Dolinsky ZS, Babor TF, Meyer RE. 1987. Psychopathology as a predictor of treatment outcome in alcoholics . Archives of General Psychiatry 44:505-513.
Satel SL, Price LH, Palumbo JM, McDougle CJ, Krystal JH, Gawin F, Charney DS, Heninger GR, Kleber HD. 1991. Clinical phenomenology and neurobiology of cocaine abstinence: A prospective inpatient study. American Journal of Psychiatry 148:1712-1716.
Schonberg SK, ed. 1988. Substance Abuse: A Guide for Health Professionals. Elk Grove Village, IL: American Academy of Pediatrics.
Schuckit MA.. 1985. The clinical implications of primary diagnostic groups among alcoholics . Archives of General Psychiatry 42:1043-1049.
Schuckit MA, Monteiro MG. 1988. Alcoholism, anxiety and depression. British Journal of Addiction 83:1373-1380.
Schuckit MA., Irwin M, Brown SA. 1990. History of anxiety symptoms among 171 primary alcoholics. Journal of Studies on Alcohol 51:34-41.
Sells SB, Simpson DD. 1980. The case for drug abuse treatment effectiveness, based on the DARP research program. British Journal of Addiction 75:117-131.
Senay EC, Dorus W, Showalter CV. 1981. Short-term detoxification with methadone. Annals of the New York Academy of Science 362:16-21.
Simpson D, Savage L. 1980. Drug abuse treatment readmissions and outcomes. Archives of General Psychiatry 37:896-901.
Simpson DD, Sells SB. 1982. Effectiveness of treatment for drug abuse: An overview of the DARP research program. Advances in Alcohol and Substance Abuse 2:7-29.
Simpson DD, Joe GW, Lehman WEK, Sells SB. 1986. Addiction careers: Etiology, treatment, and 12-year follow-up outcomes . Journal of Drug Issues 16:107-121.
Stanton MD. 1979. The client as family member. In: Brown BS, ed. Addicts and Aftercare. New York: Sage Publications.
Stanton MD, Todd T. 1982. The Family Therapy of Drug Abuse and Addiction. New York: Guilford Press.
Stewart RG, Ware LG. 1989. The Medical Outcomes Study. Santa Monica, CA: The RAND Corporation Press.
Stockwell T, Bolt E, Hooper J. 1986. Detoxification from alcohol at home managed by general practitioners . British Medical Journal 292:733-735.
Stockwell T, Bolt L, Milner I, Puch P, Young I. 1990. Home detoxification for problem drinkers: acceptability to clients, relatives, general practioners and outcome after 60 days. British Journal of Addiction 85:61-70.
Stockwell T, Bolt L, Milner I, Russell G, Bolderston H, Hugh P. 1991. Home detoxification from alcohol: Its safety and efficacy in comparison with inpatient care. Alcohol and Alcoholism 26:645-650.
Strain EC, Stitzer ML, Liebson IA, Bigelow GE. 1993. Dose-response effects of methadone in the treatment of opioid dependence . Annals of Internal Medicine 119:23-27.
Strain EC, Bigelow GE, Liebson IA, Stitzer ML. 1996. Moderate Versus High Dose Methadone in the Treatment of Opioid Dependence . Poster session presented at the annual meeting of the College on Problems of Drug Dependence. San Juan, Puerto Rico. June.
Timko C, Moos RH, Finney JW, Moos BS. 1994. Outcome of treatment for alcohol abuse and involvement in Alcoholics Anonymous among previously untreated problem drinkers. Journal of Mental Health Administration 21:145-160.
Transdermal Nicotine Study Group. 1991. Transdermal nicotine for smoking cessation. Journal of the American Medical Association 266:3133 -3138.
Vaillant GE. 1996. The Natural History of Alcoholism. Cambridge, MA: Harvard University Press.
Valle S. 1981. Interpersonal functioning of alcoholism counselors and treatment outcome. Journal of Studies on Alcohol 42:783-790.
Volpicelli JR, Alterman AI, Hayashida M, O'Brien CP. 1992. Naltrexone in the treatment of alcohol dependence. Archives of General Psychiatry 49:876-880.
Weddington WW. 1992. Cocaine abstinence: “Withdrawal” or residua of chronic intoxication? American Journal of Psychiatry 149:1761-1762.
Weisner C. 1993. Toward an alcohol treatment entry model: A comparison of problem drinkers in the general population and in treatment. Alcoholism: Clinical and Experimental Research 17(4): 746-752.
Weisner CM, Greenfield TK, Room R. 1995. Trends in treatment for alcohol problems in the US general population, 1979-1990. American Journal of Public Health 85(1):55-60.
White Paper. 1995. Effectiveness of Substance Abuse Treatment. Washington, DC: U.S. Department of Health and Human Services.
Whitfield CL, Thompson G, Lamb A, Spencer V, Pfeifer M, Browning-Ferrando M. 1978. Detoxification of 1024 alcoholic patients without psychoactive drugs . Journal of the American Medical Association 239:1409-1410.
Wilsnack SC, Wilsnack RW. 1991. Epidemiology of women's drinking. Special issue: Women and substance abuse. Journal of Substance Abuse 3:133-157.
Wilsnack SC, Wilsnack RW. 1993. Epidemiological research on women's drinking: Recent progress and directions for the 1990s. In: Gomberg ESL, Nirenberg TD, eds. Women and Substance Abuse. Norwood, NJ: Ablex Publishing Corporation. Pp. 62-99.
Woody GE, Luborsky L, McLellan AT, O'Brien CP, Beck AT, Blaine J, Herman I, Hole A. 1983. Psychotherapy for opiate addicts: Does it help? Archives of General Psychiatry 40:639-645.
Woody GE, McLellan AT, Luborsky L. 1984. Psychiatric severity as a predictor of benefits from psychotherapy . American Journal of Psychiatry 141(10):1171-1177.
Woody GE, McLellan AT, Luborsky L, O'Brien CP. 1987. Twelve-month follow-up of psychotherapy for opiate dependence. American Journal of Psychiatry 144:590-596.
Woody GE, McLellan AT, Luborsky L, O'Brien CP. 1995. Psychotherapy in community methadone programs: A validation study . American Journal of Psychiatry 152(9):1302-1308.
Zanis DA, Metzger DS, McLellan AT. 1994. Factors associated with employment among methadone patients. Journal of Substance Abuse Treatment 11:443-447.