Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Evaluating Prevention Program Effects As we have seen, the theoretical foundations for prevention are based on three principal approaches: (1) the risk factor approach, implemented mainly in the primary grades to affect predisposing factors; (2) the develop- mental approach, which concentrates on the socially reinforcing properties of classrooms and family environments; and (3) the social learning ap- proach, working in junior high and middle schools mainly to alter enabling factors, such as skills and motivation to resist media and peer influence. All three approaches use the school as the basic vehicle through which prevention efforts flow, although the stronger examples of each type of intervention recognize and seek to make positive use of the fact that schools exist within the context of family and community. There is a related movement toward the use of mass communications media as an educational channel. Much of the theoretical foundation for social influence approaches is transferable to mass media, although commu- nications experts view the media fundamentally as a supplemental or ampli- fying rather than the primary carrier of persuasive communications regard- ing health-related behavior. The empirical research picture is not as tidy as the theoretical concepts. For one thing, differences that are sharp and clear in theoretical abstraction become blurred in the details of application. Most actual school-based prevention curricula, of whatever theoretical inspiration, include a number of the following components: 76
EVALUATING PREVENTION PROGRAM EFFECTS 77 · Technical information about drugs and the consequences of use, · Instruction on techniques for making decisions about drug use, · Clarification of values to help put decisions about drug use in per- spective, Instruction in stress management techniques, Exercises to enhance self-esteem, Social learning to enhance self-efficacy, Instruction in setting goals and working to implement them, Life skills training to assist students in resisting drug use, Resistance skills training to help students resist pressures, direct and indirect, to use drugs, · Making a pledge publicly not to use drugs, · Instruction in how to set norms for one's age-graded peers and self, · Instruction in how to provide assistance to one's peers, and Identification of and encouragement to seek alternatives to drug use. . The empirical challenge has been to sort out the critical elements from the adventitious ones, find the best time to begin intervening, select the optimal programmatic sequence and emphasis, identify the most conducive agents of transmission, and divine the most effective ways to prepare those agents for the task. The prevention research field is substantial enough, and of long enough standing, that a number of large-scale, meticulously conducted research evalua- tions of preventive interventions have been completed; numerous research reviews and collections of reviews have been published (see, for example, Goplerud, 1991; Bell and Battjes, 1987; Kumpfer, 19873. Several well- defined prevention programs have been very widely disseminated. But for various reasons, the transitions from publication of major results, to compi- lation of definitive reviews, to wide dissemination of practices have been less than ideal. Indeed, reading the prevention research literature brings to mind the Cheshire cat in Lewis Carroll's Wonderland: lines of work seem to resolve into vivid conclusions, which then fade away in a few critical turns of the page. The will to believe on the part of implementers and program sponsors alike seems stronger than the evidence supports. With this forewarning, we begin the chapter by recounting widely cited recent meta-analyses of research findings on preventive interventions. To give more concrete meaning than we think can yet be gained from these synthetic reviews, we then analyze (1) a series of curricula that use cogni- tive and behavioral approaches in relatively limited-scale experimental in- terventions; (2) completed large-scale experimental studies using social in- fluence programming; (3) prominent work now in progress probing social influence and developmental interventions; and (4) the special role of mass media as channels for prevention communications.
78 PREVENTING DRUG ABUSE: WHAT DO WE KNOW? META-ANALYSES OF PREVENTION INTERVENTIONS Meta-analysis refers to techniques developed by behavioral statisticians for quantitatively integrating the findings from various studies. These tech- niques have been described and debated in a number of recent books (e.g., Wachter and Straf, 1990) and widely used in the past decade. As Bangert- Drowns (1988:245), one of the authors reviewed below, points out, meta- analysis has two potentially major advantages over more traditional narra- tive reviews of the scientific literature. First, it adheres to a precisely defined metric of outcome that is comparable across studies: the intervention's effect size, defined as the difference between the average (mean) scores on an outcome measure of the experimental and control groups, divided by their standard deviation. Second, meta-analysis uses reproducible statistical tests to examine relations between effect sizes and characteristics of the studies being reviewed. Two types of meta-analysis have been applied to interventions to pre- vent drug abuse. Tobler (1986) first employed "classic" meta-analysis (Glass et al., 1981~. This method sweeps together methodologically loose as well as rigorous studies, on the grounds that evaluations of methodological strength differ, and even weak studies contain some increments of information. It treats separately each of the different numbers of outcome items collected in different studies, thus allowing some studies disproportionate weight. Bangert-Drowns (1988) employed"study effect" meta-analysis to examine school-based substance abuse education. The advantage of the study-effect approach is that it is more selective, excluding studies with serious method- ological flaws, and it weights each study equally when average effect sizes are calculated. In a reanalysis, Tobler (1989) applied the more restrictive inclusion criteria and weighting used by Bangert-Drowns and then extended the new analysis by focusing on characteristics of the 10 most effective programs. Tobler I Tobler (1986, 1989) included 143 programs in her first widely cited meta-analysis. Four criteria were used to include a program in the meta- analysis: · Use of quantitative outcome measures including mediating variables; · Presence of control or comparison groups (however, in many cases these were supplied by Tobler post hoc); . Students in grades 6-12 as recipients of intervention; · Prevention as a goal of the intervention (i.e., assisting young people in developing attitudes, values, behavior, and skills that may reduce the likelihood of drug use).
EVALUATING PREVENTION PROGRAM EFFECTS 79 Each program was coded for 17 different content items, which were then mapped into five program types: · Knowledge-only, meaning purely informational programs about drug effects; · Affective-only, meaning largely nondrug-specific curricula to enhance self-esteem or general competency skills (see further discussion below); · Knowledge-plus -affective ; · Peer programs (which means that some program element focuses on peer interaction, either as a teaching method or as a transmitter of drug behavior this does not necessarily mean training in peer resistance skills), and . Alternatives, generally meaning that the subjects were treated out- side a conventional school environment. In all, 63 variables (e.g., outcome measures, client characteristics, meth- odological issues, program implementation, etc.) that could affect program success were coded. Tobler estimated effect sizes for program success based on outcome variables for drug knowledge; drug attitudes and values; behavioral skills (i.e., decision making, assertiveness, refusal, etc.), in terms of learning the skills and, separately, reporting instances of using them; and self-reported drug use. Tobler (1986) found that the average effect size for change in knowl- edge (0.52) was nearly double the effect for desired change in nondrug behaviors (0.27), skills development (0.26), and self-reported drug use (0.24~. The effect size for attitudinal change was the lowest among the outcomes assessed (0.18~. Knowledge-only programs had measurable effects on knowledge but negligible effects on attitudes and self-reported drug use. Affective- only programs were, in Tobler's analysis, ineffective across all outcome measures. Knowledge-plus-affective programs had a very modest average effect size on drug use (0.15~. Peer programs had the most marked effect on self-reported drug use (0.40~. Alternative programs, which were highly intensive and targeted on high-risk adolescents, were midway between. Tobler's analysis suggests that a significant effect on drug knowledge and attitudes can occur without significant parallel changes in drug use. The analysis also suggests that there are no significant differences in drug use outcome effects between urban and suburban populations and between junior and senior high students. Tobler II Tobler's original report was critically reviewed by Bangert-Drowns (1988~. He noted that an unreported number of the evaluations included in Tobler's analysis were not located in the peer-reviewed literature and, for this and
80 PREVENTING DRUG ABUSE: WHAT DO WE KNOW? other reasons, there were far too many methodologically weak reports in the pool of studies analyzed. Moreover, he noted that an unspecified number of the studies did not include sufficient information to calculate effect sizes, which had led Tobler to improvise various unspecified imputation proce- dures. Finally, he noted that the overall results gave very disproportionate weight to a small group of studies with large numbers of outcome measures. (Note: this is also problematic from a statistical significance testing stand- point insofar as the samples are not independent.) Tobler (1989) subsequently reanalyzed 91 of the 143 prevention inter- vention programs included in the original meta-analysis. The weakest stud- ies were evidently excluded. This new analysis was based solely on the self-reported drug use outcome and computed only one effect size for each program. The effect sizes for knowledge-only, affective-only, and knowl- edge-plus-affective programs were all insignificant at 0.07 or less. The effect size on drug use outcome for peer programs was 0.42; for alternatives it was 0.20. These results were quite similar to those originally reported. Tobler achieved further specification by focusing on the "top-10" (highest effect sizes) peer programs. Tobler found certain commonalities here, in particular an emphatic focus on group interaction and delivery of the inter- vention by mental health professionals or counselors rather than regular teachers or peer leaders. The most successful programs for those of junior high age stressed the acquisition of skills, particularly refusal skills, al- though there was evidence of efficacy for broad-spectrum (decision making, competency, life) skills as well. The top peer programs among those of high school age featured well-structured group discussions that maintained an emphasis on drugs. Tobler notes that individual sessions often aug- mented the group sessions. Based on these results of scrutinizing the top 10, Tobler reanalyzed the data from the 91 programs and found that overall effect sizes for mental health professionals or counselors were at least twice the effect size for health education specialists, peer leaders, teachers, college students/others, and a combination of mental health professionals or counselors and teach- ers. Tobler (1989:19) noted: "The success of the peer programs is not dependent on the leader but is enhanced by the presenter.... Mental health professionals or counselors were represented almost entirely in the peer strategies. This combination produced the highest average effect size (0.80~. When peer leaders or teachers were used in the peer strategies, their aver- age effect sizes were equivalent (0.31~." As clear as these results appear, direct scrutiny of the top-10 programs yields ambiguities and obstacles to generalization that neither Bangert-Drowns's nor Tobler's reanalysis addresses. One cardinal point is that Tobler's ge- neric use of the term drug includes cigarettes and that 4 of the top-10 peer programs (and an uncertain number of others in the sample) focused exclu
EVALUATING PREVENTION PROGRAM EFFECTS 81 sively on cigarettes; only 3 of the 10 included measures of alcohol, mari- juana, or other drugs. Half of the top 10 did not use an experimental design involving random assignment. Just 2 of the 10 studies drew representative samples of students experimentally assigned to treatment and control condi- tions, and in both of those studies the interventions (and outcomes reported) are specifically on cigarette smoking. In addition, program subjects were not generally followed up for long; only two had a follow-up period beyond 1 year. Despite Tobler's selectivity, the general methodological rigor and relevance of the studies included remains low. Although this would not in itself invalidate the results, a closer look at a handful of the top-10 pro- grams stipulated by Tobler, those available in peer-reviewed venues and not restricted to cigarette smoking, provides a revealing perspective on the meta- analytic results. One of these programs was reported by Sorensen and Jaffe (19754. It involved a total of 10 adolescents who were self-recruited to a 14-week, once-a-week "drug group" organized by a paraprofessional staff member in a storefront community youth center. Four recruits stopped participating after one or two sessions (three after a confrontation over coming to group meetings while intoxicated or in an otherwise disruptive condition); these four were used as the control group. The other six participants reported lifetime use of 9 drugs, while early departees averaged 14 drugs. These results yield an effect size of 0.71; they were, however, pastiest data. No pretest data had been collected to ascertain whether control and treatment groups had different drug experiences even before the intervention, which the reasons given for the creation of the "dropout" control group certainly suggest. In a second top-10 program, Wunderlich et al. (1974) reported on a procedure instituted in a juvenile court, in which short-term group therapy was prescribed for adolescents and their parents. The treated group of 100 parent-child cases comprised juvenile drug offenders 14-19 years old (aver age age 16.6), three-fourths of whom had been detained specifically on drug charges; 85 percent of their parents participated in 12-week parent groups (which were separate from those for the adolescents). The 100 comparison cases were juvenile offenders 9-18 years old (average age 15), 62 percent of whom had been detained for nondrug-related felony offenses and 33 percent for the status offense "in need of supervision." All of the comparison group were referred to detention centers, forestry camps, or juvenile services pro- bation. At 2-year follow-up, the comparison group (although nearly two grades younger) had left school more often (25 versus 15), been rearrested more often on nondrug offenses (41 versus 11), and had more drug rearrests (3 versus 2) the last statistic yielding an effect size of 0.62. In Tobler's third top-10 study, Chambers and Morehouse (1983:84-85) reported on a school-based student assistance program in which counse
82 PREVENTING DRUG ABUSE: WHAT DO WE KNOW? lore with master's degrees conducted individual, family, and group ses- sions for students. The program was publicized by counselor presenta- tions in classrooms and letters sent home. According to the authors, "Ex- cept for students referred . . . because they were caught using alcohol or drugs, participation is voluntary." About 70 percent of the students in the program were classified as alcohol or drug abusers. How many of these were mandatory referrals is unclear. Details of the evaluation procedure are quite sketchy in the published report, and it is not clear what com- prised a control group. Based presumably on unpublished data, Tobler calculated an effect size of 0.94. The fourth top-10 study was the only one of the four employing a persuasively equivalent control group of reasonable size against which the program effect size was inferred, and it was the only one using a sample of students generalizable to most school settings (although not to the general population of students). In this study, Horan and Williams (1982) reported an experiment in which the least assertive one-third of girls and boys, re- spectively, in an 8th grade cohort were randomly assigned to three condi- tions: active treatment, "placebo" sessions (both types administered by master's-level counselors), or no treatment. The students were tested just prior to the intervention, immediately after the intervention, and again at a 3-year follow-up. The active treatment consisted of five 45-minute sessions of assertion training over a 2-week period, each session involving three new exercises (one of which was a peer-pressure-to-use-drugs type of stimulus) and live modeling, role-playing, and correction of the assertive response. The placebo sessions were comprised of discussions of assertiveness, peer pressure, and drug use-but no modeling or role-playing. There were no pre-post assertiveness effects in the placebo or control groups, and no 3-year differences between placebo and controls in their use of alcohol and marijuana or hard drugs. The active training group, how- ever, gained significantly in pre-post assertiveness, and at 3-year follow-up they reported three times as many total refusals and one-third as many total episodes of using drugs; however, the many zero reports and high variance in quantities marginalized the statistical significance of these results. In summary, of the four top-10 peer programs reported in accessible, refereed publications, only the one (Horan and Williams) engenders scien- tific confidence on the basis of a sound design and here, the result for which effect size was calculated was statistically suspect. Even more trouble- some than the prevailing methodological defects is the fact that these inter- ventions are not, by and large, drug prevention programs as the term is generally understood. Admission to three of the four programs just re- viewed required substantial levels of drug-related problems to begin with; even the fourth program was quite selective, excluding two-thirds of stu- dents. Calling these interventions prevention rather than treatment or reha
EVALUATING PREVENTION PROGRAM EFFECTS 83 bilitation is difficult to justify. The fact that counseling professionals pro- duced better results would certainly be expected if the programs were in fact therapeutic rather than prophylactic interventions. Tobler's results in favor of peer programs that is, interventions refer- ring to peer interaction as a teaching or therapeutic method may be con- sidered suggestive to the degree that where there is smoke, even smoke amplified by mirrors, there may be fire. There is certainly a marked con- trast between the positive peer results and the uniformly negative results found with three other types of interventions. Nevertheless, when closely examined, the fruits of Tobler's meta-analysis can be considered imagina- tive and provocative but hardly persuasive concerning the question of how effective prevention programs may be. Other Meta-Analyses Bangert-Drowns studied a selection of educational programs much more tightly screened than Tobler's. He limited the analysis to studies meeting the following stringent criteria: the programs had to be conducted in schools with "traditional students", tobacco-only programs were excluded; a no- treatment control had to be used that was shown not to be significantly different before treatment from the experimental group; and the original data had to be reported in sufficient detail to permit unambiguous calcula- tion of effect sizes. Under these selection criteria, only 33 programs were admitted to the meta-analysis. Most were knowledge-only or knowledge-plus-affective pro- grams, in Tobler's terms, and most used teachers to deliver the intervention. In all, 4 were in elementary schools, 12 in junior high or middle schools, and 17 in high school or college. Slightly over half the interventions fo- cused exclusively on alcohol education, and half were of 5 weeks' duration or less. The evaluations employed three outcome criteria: knowledge about substances (alcohol or drugs); attitudes toward substances, their use, and abuse; and behavior with regard to substances. Of them, 26 evaluations measured knowledge, 18 measured attitudes, and 14 measured behavior; only 3 studies measured all three criteria (Bangert-Drowns, 19881. Effects on knowledge were highest (average effect size 0.76), effects on attitudes were lower (0.34), and effects on behavior were lowest (0.12), not differing significantly from zero. No identified study feature had a consis- tent differential effect on knowledge. However, two program features dif- ferentially affected attitudinal results: the mode of delivery, with lecture- only as the weakest mode, and the use of peer leaders, which had significantly higher average effects in the desired direction compared with adult-led condi- tions, a result differing from Tobler's. Two study features reliably related to behavioral outcomes were the year of publication (the more recent the
84 PREVENTING DRUG ABUSE: WHAT DO WE KNOW? techniques, the higher the effects) and whether the students volunteered rather than being forced to participate (volunteers had higher effects). Bangert-Drowns's sample of evaluations were skewed toward higher grades, toward an alcohol focus, and toward programs of very short dura- tion compared with Tobler's selections. Since only a handful of the evalua- tions included knowledge, attitude, and behavior in the same design, con- clusions about the relative effect sizes must be viewed with caution. Bangert-Drowns's exclusion of studies with significant pretreatment differ- ences between experimental and control groups does not clarify what con- stituted significant initial nonequivalence in particular, whether statistical controls over initial conditions were accepted. Another consideration of the comparison of methods using meta-analysis is whether the grouping of studies in each category of intervention method (e.g., cognitive, cognitive-plus-affective) constituted a homogeneous set as measured by the pretest or pastiest effect sizes. If not, those studies with extreme effect sizes (outliers) should be removed from the group comparison. Bruvold and Rundall (1988) published a meta-analysis and theoretical review of 19 school-based tobacco- and alcohol-oriented intervention stud- ies. The 19 studies all utilized a control or comparison group and met 5 design criteria. The analysis contrasted the "rational" prevention theory of Fishbein and Ajzen (1975) with the social reinforcement and learning theory of Bandura (1977), the social norms/problem behavior theory of Jessor and Jessor (1977), and the developmental theory of Rosenberg (19791. Interventions based on the traditional rational teaching model had a significantly greater effect on knowledge than did the other models. How- ever, other interventions had greater positive impact on attitudes and to- bacco and alcohol behavior than the rational model. Bruvold and Rundall suggest that a threshold change in knowledge is necessary for behavior change, but attitude changes (in the desired direction) do not necessarily follow from knowledge changes. A combination of new knowledge and attitude changes is more certain to produce behavioral results. Traditional didactic approaches are less effective than other means social reinforce- ment, normative, or developmental approaches in generating the sequence of attitudinal and behavioral changes. Bruvold and Rundall concluded (1988: 72-73~: "If an individual receives peer praise and support for refusing cigarettes, the individual will become fully convinced that such refusals lead to peer praise and support, a desirable outcome ...." Interventions targeted at self-esteem enhancement, if they appropriately followed the te- nets of this theory, would be directed at providing the individual more constant and explicit feedback from significant other peers. A meta-analysis by Hansen et al. (1990) was based on 85 distinct co- horts of subjects. The results reveal that sample retention decreases over time: the mean proportion of subjects retained in the analyses decreased
EVALUATING PREVENTION PROGRAM EFFECTS 85 from 81.3 percent to 67.5 percent from follow-ups taken at 3 months and 3 years, respectively. There was considerable variability in the rates of attri- tion between studies. The greatest drop in sample retention was found to occur during the first year of investigation. The authors concluded that researchers should interpret their results in light of the rate of attrition and should further their efforts to reduce the rate of attrition. The results of Project ALERT, discussed later in this chapter, are particularly subject to this conclusion. Summary Tobler's, Bangert-Drowns's, and Bruvold and Rundall's results con- verge on the general ineffectiveness of knowledge-only, affective-only, and knowledge-plus-affective programs in affecting alcohol or drug use behav- iors. Hansen et al. (1990) provide a warning on long-term effects due to the attrition of subjects over time. Tobler and Bangert-Drowns diverge on what kind of trainers seem best to induce informational or attitudinal change, but these results may simply reflect the different kinds of programs analyzed. Tobler's review suggests that programs oriented toward peer relationships gain in efficacy, but it leaves open the question of what this advantage consists of and whether it actually applies to drug prevention programs among general student populations. The strength of Tobler's meta-analysis is its overview of different program types, but the strongest conclusion is difficult to regard as applicable to prevention programs at all. These results suggest that we need to examine studies of prevention interventions that employ much more tightly defined contents and more careful scientific de- signs than appear typical among the types of studies that carry so much weight in some of the meta-analyses. A good set of cases in point for preferred studies are the Life Skills Training Program, several studies using a cognitive-behavioral approach, and the Napa Drug Abuse Prevention Project. In each case, the research involved programs with discrete modular charac- teristics, applied to full grade cohorts, within closely controlled experimen- tal protocols. THREE PROGRAMS MEETING TOBLER'S CRITERIA Life Skills Training Program Life Skills Training (LST) is a middle-school curriculum with three components (Botvin and Wills, 1985; Botvin and Eng, 1982; Botvin et al., 1983): e Substance-specific information and refusal skills training;
86 PREVENTING DRUG ABUSE: WHAT DO WE KNOW? · A personal skills component to improve critical thinking and respon- sible decision making, help cope with anxiety, and learn principles of self- improvement; and . Improvement of nonverbal and verbal communication skills for so- cial encounters including dating, conversation, and assertiveness. The Life Skills Training program was implemented with booster ses- sions among a predominantly white sample of 7th grade students who were followed up in grades 8 and 9 (Botvin et al., 19901. Using a randomized block design, schools were assigned to receive one of three programs: (1) the LST program with formal provider training and implementation feed- back, (2) LST with videotaped provider training and no feedback, or (3) no treatment. Program outcomes showed significant reduction in smoking and marijuana use in both experimental conditions at the first and second year follow-up. The program did not have significant effects on drinking fre- quency or amount, although at second year follow-up there was a signifi- cant effect on the frequency of getting drunk for the experimental groups who received videotape teacher training. The effect was strongest for ciga- rette use; this is not surprising, as the intervention was originally designed for smoking prevention. These findings provide the most rigorous test of the LST approach and demonstrate the effectiveness of LST in reducing substance-using behaviors among youth in grades 7 to 9. There is further evidence of short-term efficacy of the LST approach to drug prevention. The program reduced the proportion of smoking among a sample of black urban youths by 56 percent in a 3-month pastiest (Botvin et al., 1989a). A skills training program for smoking prevention was tested in a predominantly Hispanic population, preliminary evidence supported the efficacy of the program (Botvin et al., 1989b). These findings suggest that a preventive approach with some short-term effectiveness in white middle- class populations may be generalizable to minority populations. There is some evidence of long-term efficacy using the LST approach with regard to cigarettes. Smoking prevention integrated into a primary cancer center prevention strategy with diet modification revealed that the rate of initiation of cigarette smoking was significantly lower in treatment schools 6 years following the intervention (Walter et al., 1989~. In grade 4 no smokers were present in the intervention and nonintervention groups; the rate of initiation of cigarette smoking was 73.3 percent less (3.5 versus 13.1 percent) among youths in the intervention schools than those in noninter- vention schools; and the effect was stronger for males than for females. An 8-year follow-up study in Finland (North Karelia Youth Project) revealed that the positive short-term effects on smoking prevalence found immedi- ately following the intervention and in a 4-year follow-up diminished by the 8-year follow-up. The difference in smoking prevalence, however, remained
EVALUATING PREVENTION PROGRAM EFFECTS 87 significant for the community-wide and direct program schools in compari- son to control schools (Vartiainen et al., 1990~. The effect was stronger for the community-wide program than for the direct program approach imple- mented in schools without community-wide activities. Almost one-half of youth in control schools (47 percent) smoked, compared with 37 percent of those in the direct program schools and 31 percent in the community-wide program schools. In summary, the research findings support short- and long-term efficacy of Life Skills Training with respect to knowledge, attitudes, and cigarette smoking outcomes of youth in different ethnic populations and in schools and community settings. The Cognitive-Behavioral Approach Positive effects on knowledge, attitudes, and behavior have been re- ported for the cognitive-behavioral approach to drug prevention. In a study of 5th and 6th graders (N = 741), the self-control skills group reported less weekly cigarette smoking and revealed better scores on measures of com- munication, self-instruction, self-praise, cigarette refusals, and noncompli- ance to smoke cigarettes at the 15-month follow-up (Gilchrest et al., 19864. Similar results were found in a smaller study (N = 65) of 6th graders in two middle schools. The average number of cigarettes smoked per week was lower at poshest, 6-month, 12-month, and 24-month follow-ups (Schinke et al., 1986~. This approach has been found effective in populations of Native American youths with posttest and 6-month follow-up results showing posi- tive outcomes on measures of substance use knowledge, attitudes, and self- reported rates of tobacco, alcohol and drug use (Schinke et al., 1988~. Similarly, Native American youths receiving the intervention revealed better knowl- edge scores on drug effects and interpersonal skills for managing pressure to use drugs and reported lower rates of alcohol, marijuana, and inhalant use (Gilchrist et al., 1987~. The cognitive-behavioral approach to drug abuse prevention has fo- cused exclusively on school-based programming. Compared with the social influences and life skills approaches, the efficacy of the cognitive-behav- ioral approach has been established on relatively small sample sizes. The long-term efficacy of the cognitive-behavioral approach has not been estab- lished, nor has its full potential been exploited. The approach has recently been applied to the prevention of HIV infection among African American and Hispanic youths (Schinke et al., 1990~. The Napa Drug Abuse Prevention Project The Napa project, implemented in Napa County, California, was de- signed to evaluate the effectiveness of seven school-based substance abuse
88 PREVENTING DRUG ABUSE: WHAT DO WE KNOW? prevention strategies. Four of the strategies were in-service teacher training courses that focused on classroom and individual factors thought to influ- ence attitudes toward school, self-esteem, and the development of social competencies. None of these four courses addressed the topic of drug use per se. The in-service teacher training strategies were designed to improve the classroom management skills of teachers and to provide a more positive and socially rewarding learning environment within the classroom; these achievements, in turn, would presumably affect drug-oriented attitudes and behaviors. The four strategies included: Magic Circle, in which teachers were trained to lead structured small- group discussions on particular topics designed to improve students' com- munications skills and their understanding of themselves and others (grades 3-4~. Effective Classroom ManagementlElementary, in which teachers were taught various communication skills, discipline techniques, and self-concept enhancement techniques (grades 4-6~. Effective Classroom Mana~ementlJunior Highs in which communica- tion, discipline, and sell-concept enhancement skills were adapted for teaching in the junior high environment (grades 7-9~. Jigsaw, in which teachers were taught to organize classrooms into co- operative learning groups of five or six students in which each student teaches an essential piece of the regular curriculum to the other group mem- bers (grades 4-63. ~7 Two alternative strategies were offered as elective academic courses to junior high school students. In these courses, students were taught certain skills and provided opportunities for helping peers or younger children. These courses, too, did not address the topic of drug use; instead, they sought to strengthen self-concepts and to teach social competencies. Cross-Age Tutoring, in which students tutored younger children on a regular basis in reading and other academic subjects (grades 8-91. Operating a School Store, in which students ran a school store on cam- pus selling school supplies and snacks, while learning relevant business skills in a related academic course (grades 8-91. The final strategy was a course in Drug Education. This course taught social competencies and drug information to 7th graders. In the final ver- sion of the 12-session (45 minutes per session) course, taught by a health educator, students learned Maslow's (1980) framework for understanding motivation, learned a systematic decision-making process, analyzed tech- niques used in commercial advertising, learned assertiveness skills for deal- ing with peer pressure, and practiced setting personal goals. Toward the end of the course, students were also provided information about tobacco,
EVALUATING PREVENTION PROGRAM EFFECTS 89 alcohol, and marijuana in response to their written questions. Students applied social skills in considering drug use issues. It should be noted that the Drug Education curriculum incorporates virtually all of the elements identified as important to the social influence approach (e.g., the Project ALERT lessons; see Chapter 2 and below), although in a somewhat differ- ent sequence. One or more separate evaluations of each strategy were conducted dur- ing the course of the Napa project, covering variables of classroom and school environment, personal satisfaction among students, perceived peer norms and behavior, and specific drug attitudes, beliefs, knowledge, and behavior. In some studies, two or three strategies were applied to the same group of students over 2- or 3-year periods. Schaps and colleagues (1984), the evaluators, found that none of the six nondrug-specific strategies evalu- ated in the Napa project produced a pattern of positive outcomes. There were no effects on attitudes or perceptions of self, peers, classroom climate, or the school experience; no effects on attendance or academic achieve- ment; no significant changes in perceptions of peer group norms or drug- related attitudes or behaviors. The Drug Education curriculum showed some short-term positive effects on 7th grade girls in one of two studies (but not on 8th-grade girls), and no positive results for boys in either grade. The one set of positive effects was no longer discernible at the 1-year follow-up. Schaps and associates concluded that these prevention strategies were ineffective because some were inadequately or inconsistently implemented and others were based on inadequate theories of prevention. For example, process evaluation showed that Effective Classroom Management skills were infrequently used by teachers, apparently because these skills were incom- patible with routine teaching practices and styles. In even the most "exem- plary" Jigsaw classrooms, there were no effects on students compared with controls, nor were any effects observed in fully operationalized Cross-Age Tutoring or School Store programs. The Napa results were further con- firmed in a meticulous study by Hansen and colleagues (1988), who con- ducted a 2-year trial with experimental and control groups comparing an affective education curriculum with a social influences curriculum. They too found that affective intervention was clearly less effective. In fact, students receiving the affective education intervention had even higher rates of substance use at follow-up than those in the control group. A plausible explanation for these findings was that none of the affective-type interven- tions was based on a clear vision or model of the specific pathways to drug use the predisposing, enabling, and reinforcing elements. Mild nondrug- specific curriculum augmentation on a few grade levels simply may not reach deeply enough into the lives of children, especially those most at risk, to change the likely trajectories of their behavior. Instead of viewing this as
9o PREVENTING DRUG ABUSE: WHAT DO WE KNOW? ruling out the school as a site for intervention, some have looked to evi- dence from more comprehensive efforts at school modification; we return to this line of research later in the chapter. DO LARGE-SCALE SOCIAL INFLUENCE PROGRAMS WORK? The results of the meta-analyses by Tobler and Bangert-Drowns, rein- forced by the Napa project and the work of lIansen and colleagues, suggest that modular, nondrug-specific affective programs are not effective. They leave open the possibility that modular programs recognizably conforming to the social influence approach may be effective. A series of large-scale programs that is, programs applying to cohorts of large sample sizes- have appeared in recent years. It is instructive to give detailed attention to two of these programs that have sustained high methodological standards, one focusing strictly on cigarette smoking prevention, the other on all of the gateway drugs: the Waterloo study in east-central Canada and Project ALERT in the western United States. Waterloo In two school districts in Waterloo, Ontario, 22 schools volunteered to participate in a study of an antismoking intervention using social influence. Eleven were assigned, mostly at random, to the experimental condition; the others served as controls. The study students were in grades 6-8 and pro- vided questionnaire and saliva samples at pretest, immediate poshest, and at end of the 6th, beginning and end of the 7th, and end of the 8th grades. Total attrition plus absenteeism was less than 10 percent per year; 67 per- cent of students provided data at all six data points (Flay, 1985; Flay et al., 1989~. Pretest differences between the two groups were minimal, and no group differences were observed for baseline smoking behavior. The program was found to have its major effect on those students in the experimental schools who were judged at high initial risk because they had parents, siblings, or friends who smoked. For example, among high-risk students who had never tried smoking prior to pretest, by the end of 8th grade, 67 percent of those in the program group versus 22 percent of controls still had never smoked, 6 percent of the program group versus 39 percent of controls were experi- mental smokers, and none of the program students versus 6 percent of controls were regular smokers (Best et al., 19841. Program effects varied over time. The greatest immediate effects were on those already experi- enced with smoking, but there were later effects on those with no or little . . . smoking experience at pretest. In a 6-year follow up, 90 percent of the subjects were tracked in the
EVALUATING PREVENTION PROGRAM EFFECTS 91 11th and 12th grades, and 80 percent provided data (Flay et al., 1989). Significant program effects that had been observed earlier (including fol- low-ups only one or two grades earlier) had decayed into statistical insig- nificance by the latter high school years. Whether the dependent variable is smoking at all or smoking heavily at follow-up, program effects were no longer significant. The best predictors of smoking status at the 6-year follow-up were: (1) pretest smoking behavior and (2) the pretest level of social risk (as defined above) for becoming a smoker. Flay and colleagues (1989:1374) conceded: "The lack of significant preventive effects by grade 12 raises the question of the value of the social influences approach for smoking prevention." (High school follow-up data from a very similar study in Minnesota closely parallel the results observed in Waterloo- Murray et al., 1989.) The researchers caution, however, against an interpretation of the latest findings as evidence that prevention efforts have no effect on results 6 years later, when the proper conclusion should be that results obtained early had disappeared 6 years later in the absence of sustained or repeated efforts. The decay of effects over a 6-year period might not be ineluctable; this process might be counteracted by modest intermediary efforts; for example, brief booster sessions in the early high school years may be sufficient to perpetuate the substantial early effects; this would be in line with advances over the 10 years since the Waterloo study was initiated with respect to understanding essential components of effective prevention programs. Such enhancements warrant at least the working hypothesis that more current versions of social influence program- ming could produce more durable effects, a hypothesis in line with Bangert- Drowns's finding that more recent techniques seem to produce greater ef- fects. In addition, broader social norms and policies have grown more supportive of nonsmoking; students in the control schools may have been endogenously exposed to many elements of the social influence approach in recent years, thus catching up with their experimental peers and obscuring the effects of the older program. Finally, even if smoking rates had equal- ized at a point in time 6 years later, the delayed onset of smoking in the treated cohort could have a valuable delayed effect, namely, a greater ten- dency to quit smoking in early and middle adulthood. Project ALERT Project ALERT was an $8 million study funded by the Conrad N. Hilton Foundation and conducted by a research team at RAND Corporation. It tested the effects of an education program focused explicitly on resistance to cigarettes, alcohol, and marijuana, using a carefully designed and ex- ecuted experimental procedure to evaluate the effects of the curriculum. The researchers recruited 30 junior high and middle schools in eight coop
92 PREVENTING DRUG ABUSE: WHAT DO WE KNOW? crating school districts in Oregon and California, which spanned urban, suburban, and rural locations and included large variations in ethnic and socioeconomic characteristics. The researchers administered questionnaires with a variety of baseline test items and collected saliva to test for nicotine metabolite in the entire cohort of 7th grade students in the 30 schools, generating a baseline group of 6,527 students. Each school was randomly assigned (with some restric- tions to ensure balanced samples) to one of three conditions. In 10 of the experimental schools, health educators taught the 8-session ALERT curricu- lum to the 7th grade classes and the following year delivered three booster sessions in 8th grade. In another 10 schools, trained "teen leaders" assisted the health educators in delivering these lessons. In the remaining 10 schools there was no intervention, although previously instituted information-type drug prevention programs continued during the study period in four of these schools. The research group monitored implementation to be certain the experi- mental sessions took place and all lesson elements were delivered (actually 92 percent were delivered in monitored sessions). The researchers adminis- tered follow-up retest items concerning alcohol, marijuana, and cigarette use at 3, 9, and 15 months after baseline, the last follow-up coming after the booster sessions were completed. About 59 percent of the baseline sample completed all three retests and were included in the analysis of outcomes- a rather disappointing follow-up rate for such a meticulously conceived study. Logistic regression was used to control for baseline differences among students and schools and isolate the treatment effect of the two experimen- tal approaches. Based on earlier research, particularly the results of Chassin (1984) and Botvin, Resnick, and Baker (1983), the authors (Ellickson et al., 1988; Ellickson and Bell, 1990) hypothesized that: · The program would affect cigarette and marijuana smoking more than alcohol use; · Teen leaders would improve the effectiveness of the curriculum; · Booster sessions would enhance or better preserve program effects; and The program would be more effective for students who were nonus- ers or "experimenters" than those reporting regular use. The latter hypothesis about differential effects according to subgroup, although opposite to the Waterloo findings, was held so powerfully that all of the outcome analyses were divided to report effects on three levels of risk groups per baseline drug use. For cigarettes and alcohol, respectively, the risk groups were defined as follows:
EVALUATING PREVENTION PROGRAM EFFECTS · Current users (past-month), · Ever-users (but only once or twice, and not currently), and · Never-users; In the case of marijuana, the baseline risk groups were as follows: . Marijuana ever-users, · Marijuana never/cigarette ever-users, and Marijuana never/cigarette never-users. . 93 The authors abstract the results of the study in the following rather positive light, albeit with a caveat (Ellickson et al., 1988:vii-viii; cf. Ellickson and Bell, 1990~: Project ALERT effectively prevented or reduced cigarette and marijuana use among young adolescents in our sample. The rate of marijuana initia- tion in the Project ALERT schools was one-third lower than that in the control schools. Regular and daily smoking by students who had not experimented with cigarettes before being exposed to the program were reduced by as much as 50 to 60 percent. These effects were reinforced or enhanced by the booster sessions offered in eighth grade. The program was equally effective in schools with substantial minority populations and in predominantly white schools. However, Project ALERT is by no means a panacea that would eliminate adolescent drug use. While it was initially successful against alcohol, the early gains in alcohol prevention had eroded by the time students reached eighth grade. Also, it was not effective with previously confirmed ciga- rette smokers, who actually smoked more after being involved in the pro- gram. This boomerang effect, however, was strictly limited to cigarette smoking. These very carefully phrased results focus on particular subgroups and particular individual transitions. Curiously, there is no report of the overall effect of the curriculum, taking all subgroups together; that is, it is unclear how the cohort as a whole responded, even though the treatments were delivered en bloc to students in all subgroups, and the authors do not pro- pose a mechanism for presorting students according to the analytically de- fined risk groups. These findings reinforce a concern about results reported by Botvin and associates (1982, 1983, 1989a) for their Life Skills Training curriculum for junior high school or high school sophomore students: although data were generally collected for students who had initiated smoking prior to receiv- ing the intervention, these groups are systematically excluded from Botvin and colleagues' analyses. The published data are not complete enough to reconstruct fully the main (whole-group) effects of Project ALERT, but they do permit a fairly
94 PREVENTING DRUG ABUSE: WHAT DO WE KNOW? good approximation of overall outcomes in each of the three conditions for each drug. When these estimated main effects are taken into account, a more conservative statement of the findings seems appropriate. In the un- treated control schools, at the final test point in grade 8, current use of alcohol, cigarettes, and marijuana (at least once in the past month) was reported by 48 percent, 22 percent, and 13 percent, respectively, of 8th grade students. In the experimental schools, at the same point in time- after completion of the 2-year curriculum- the comparable figures for the adult-taught and adult/teen-taught schools were 46 and 48 percent for alco- hol, 22 and 23 percent for cigarettes, and 10 and 11 percent for marijuana. Reports of relatively intensive alcohol, cigarette, and marijuana use yielded similar patterns but at lower percentages of the 8th grade classes: the control versus experimental group comparisons are 5.8 versus 5.6/6.3 percent (weekly alcohol), 2.9 versus 3.3/4.0 percent (daily cigarettes), and 2.0 versus 1.8/1.4 percent (weekly marijuana). The implied cell sizes for weekly marijuana are in the 20-30 range, and the p of this and other differ- ences is certainly above .10. All of these 8th-grade consumption rates, and the 7th grade baselines, are well above national and regional norms derived from retrospective reports by high school seniors. It appears, then, that the ALERT curriculum had virtually no net effect either on alcohol use (the effect was consistently lacking across all baseline subgroups) or on cigarette smoking (here the positive and negative effects in two of the three subgroups cancelled each other out, and there was no net effect in the third subgroup, comprising half the students). The net effect of the curriculum on marijuana use a reduction in the neighborhood of one- sixth to one-fourth is large enough to draw attention (the probability of avoiding a type I error is almost certainly acceptable, because cell sizes reach 110 and 1401. But methodological considerations warrant particular caution concerning the effects of the curriculum on marijuana use in this study. The overall sample attrition was 41 percent from baseline, and the attrition of baseline marijuana ever-users was the highest for any preinterven- tion characteristic 61 percent missing at follow-up. In this light, a 2-3 percent difference in posttest marijuana use among those remaining in the sample cannot be considered definitive. Despite the positive indications yielded by earlier studies and the strong theoretical underpinnings of social influence programs, two of the largest trials, representing best practice at the time they were designed, conducted by excellent research teams, have yielded results that must be viewed as discouraging. While these trials, inaugurated some years ago, do not neces- sarily represent the most advanced work that might be fielded today, they give a good indication of the degree to which school-based prevention pro- gramming is operating on the basis of plausibility and good faith rather than research knowledge. To make these points even more emphatically, it is
EVALUATING PREVENTION PROGRAM EFFECTS 95 worthwhile to examine the state of knowledge concerning the effectiveness of the two most widely disseminated school-based programs today: DARE (Drug Abuse Resistance Education), a social-influence program, and Here's Looking at You, a series of curriculum packages originally based on affec- tive education, which now incorporates most of the social influence model as well. DARE The 17-week DARE curriculum was developed in 1984 as a joint effort between the Los Angeles Police Department and the Los Angeles Unified School District. It is based to a large extent on the SMART curriculum (see Hansen et al., 1988) and thus resembles the social influence curriculum being delivered as STAR in Kansas City and I-STAR in Indianapolis (see Pentz et al., 1989b) (described in a later section). The DARE program is delivered primarily to 5th or 6th graders, although there are K-3 as well as junior high/middle school components. At the present time, DARE is of- fered in over 450 cities throughout the United States and in several other countries; in some states the program is statewide. Several unique aspects of the DARE program should be noted. The curriculum is taught by uniformed police officers instead of teachers. Of- ficers receive an intensive, 80-hour, structured training course covering the curriculum contents and associated teaching techniques. Officers are taught in all their training to "go by the book," and this instruction, along with the paramilitary character of police training in general, provides a greater prob- ability of strict fidelity to the curriculum. (Process evaluations by Clayton et al., 1991, on one site revealed strong fidelity to the curriculum and excellent classroom skills among police officers delivering the curriculum.) Implementation of the DARE program requires a working relationship be- tween the school system and the police department in a community, an arrangement that has potential benefits for the community at large. Published evaluations of the DARE program are scarce at this point in time. DeJong (1987) evaluated the program with 7th grade students in Los Angeles. His findings suggest that DARE students accepted significantly fewer offers to use drugs and reported significantly lower levels of sub- stance use than nonequivalent control group students. DeJong's study was an after-only design and there was no random assignment into treatment and control conditions. An evaluation of the DARE program in North Carolina (Ringwalt et al., 1990) focused on pre-posttest differences for students attending 10 ran- domly selected DARE schools and 10 randomly selected control schools. The DARE program had no significant effect on self-reported drug use, intentions to use drugs, or self-esteem. Significant differences in the appro
96 PREVENTING DRUG ABUSE: WHAT DO WE KNOW? priate direction did appear in attitudes toward drug use in general and to- ward specific drugs, in perceptions of peers' attitudes toward drug use, in assertiveness, and in recognizing media influences to use drugs. Paine and Bohlander (1988) examined pre-posttest differences between DARE and non-DARE students controlling for type of school (rural, paro- chial, inner-city, and suburban). Self-reported use of drugs was not exam- ined. Self-esteem and attitudes toward the police were significantly differ- ent between all DARE and all non-DARE students. However, when controls for types of school were instituted, the differences were not uniformly sig- nificant. There were significant differences between treatment and control students in perceived external control, which remained when statistical con- trols for type of school were implemented. The DARE curriculum did have the desired effect of producing significantly greater scores on peer resis- tance skills. An evaluation of DARE conducted among 400 inner-city youth in Nashville (Faire, 1989) found no support for the effectiveness of the DARE curricu- lum in changing peer resistance or positive attitudes toward drugs. In this study, DARE students had significantly more negative attitudes toward the police than the non-DARE students at posttest. Clayton and associates (1991) are engaged in a five-year, cohort se- quential study of the effectiveness of DARE in Lexington, Kentucky. In the first cohort, 23 schools were randomly assigned to receive the DARE cur- riculum; 8 schools were randomly assigned to the control condition (i.e., students received the standard health curriculum, which contained a drug education unit). In the remaining cohorts, all students received the DARE curriculum. A difficulty from a diffusion standpoint (see Best et al., 1988) is that, once a school system makes a commitment to implement a preven- tion intervention, it is virtually impossible to convince them to allow for control schools. In a pre-posttest analysis, Clayton et al. (1991) found statistically significant effects on general attitudes toward drugs, negative attitudes toward specific drugs (e.g., cigarettes, alcohol, marijuana), and on a scale measuring peer relationships. Expected differences in self-esteem and peer pressure resistance were not observed, although for the latter the results were close to the conventional criterion of statistical significance. There were no statistically significant differences in self-reported drug use, although this may have been affected by low base rates, a problem endemic to this type of research. Here's Looking at You The original version of the Here's Looking at You program was de- signed to help young people find responsible ways of dealing with alcohol in their environment by relying on decision-making skills. The curriculum
EVALUATING PREVENTION PROGRAM EFFECTS 97 was aimed at enhancing knowledge about alcohol, self-esteem, coping, and decision making (Hochheimer, 1981; Hawkins et al., 1986~. Here's Looking at You Two is a modification of the original curriculum and was integrated into a variety of different subjects (versus health classes only). Its objectives were expanded toward helping adolescents (grades 9- 12) make responsible decisions regarding the use of alcohol and drugs. The curriculum, consisting of 20 lessons, attempts to help students gain basic information about alcohol and drugs, to be able to express their feelings, and to understand their values and behavior in relation to alcohol and other drugs. This particular intervention was adopted in schools throughout the United States. The current version, Here's Looking at You 2000, was introduced in 1986. This intervention is composed of 150 lessons to be spread out over grades K-12. The goals are to provide information on substances, to de- velop social skills, and to encourage bonding to school, family, and commu- nity. The component on drug information is focused on the gateway drugs (tobacco, alcohol, and marijuana). Building social skills is focused on making friends and staying out of trouble and alerts students to the risk of having drug-using friends (Rogers et al., 1989~. The initially encouraging evaluation of Here's Looking at You by Mauss and colleagues (1981) included experimental and control schools and cov- ered 3 years. The most positive results occurred for students first exposed to the curriculum in grade 6 who continued to receive it through grade 8. Enhanced self-esteem, knowledge about alcohol, and decision-making skills were sustained across the duration of the study. A persistent impact was reported for problem drinking (that is, adverse consequences precipitated by drinking episodes), although not for the quantity and frequency of drinking. A notable finding was that peers, parents, and religiosity had a stronger predictive power over outcome variables than exposure to the curriculum. The evaluators noted that students receiving the curriculum appeared to "lose ground less rapidly to peer influence" than those who were in the control schools. This was expected to influence rates of experimental use of drugs. On the basis of the evaluation, Mauss et al. (1981) recommended that prevention interventions should occur prior to middle school, before the establishment of drinking-related attitudes and behavior, and that greater emphasis should be laid on intervention components focused on peers and parents (see Hawkins et al., 1986~. The Here's Looking at You curriculum was reexamined by Hopkins et al. (1988) over 3 years in a sample of 6,808 students in grades 4-12 from five school districts in the Northwest. Data were collected at pretest, posttest, and annual follow-up periods from students attending experimental and control schools. No statistically significant impact on measured outcomes was
98 PREVENTING DRUG ABUSE: WHAT DO WE KNOW? noted either in the short term or long term. Hopkins et al. (1988) concluded that this curriculum is not effective in preventing alcohol use or abuse. Swisher and his colleagues (1985) evaluated Here's Looking at You Two among predominantly white, middle-class 8th graders in experimental and control schools in Pennsylvania. The two schools had a total of 869 students and were similar in distribution on gender and other characteris- tics. Use of tobacco, marijuana, and stimulants was significantly lower in the school receiving the prevention intervention. The differences approached statistical significance for use of liquor and depressants. There were lower rates of drinking to become drunk and or "high" in the experimental school. The actual frequency of drinking was not lower among students in the experimental school, but the reported amount used per occasion was lower. Subsequently, the Here's Looking at You Two curriculum was evaluated by Green and Kelly (1989) among 1,698 experimental and 1,005 control students in elementary, middle, and high school grades in five school dis- tricts. Pretest, posttest, and follow-up data indicate a significant increase in knowledge about alcohol and drugs at the elementary and middle levels, but less effect on underlying attitudes about use and use itself. Few significant differences emerged with regard to key targets of this curriculum (e.g., self- esteem, decision making, coping skills). The authors suggest lack of fidel- ity in implementation as a possible explanation. In summary, research results on Here's Looking at You are inconsistent. The only replicated result is a more "responsible" use of alcohol among treated students-they don't drink less, but they appear to drink in ways that are less damaging to themselves. The contradictory results in other respects are not explained, and the great expansion of the size and scope of the program package and its widespread diffusion to schools around the country seems entirely out of proportion to the evidence of its usefulness in preventing drug problems. RESEARCH IN PROGRESS Developmental Interventions The developmental approach to intervention outlined in Chapter 2 in- volves deep-seated shifts in the organization of the school, designed to produce- far more than simple reductions in drug incidence and prevalence. This intervention is part of a larger movement of school reform. Higher overall educational achievement is part of its goal structure, but more as an expected consequence of prosocial achievement than as an end in itself. Two outstanding research programs employ interventions approaching the theoretically indicated degree of comprehensiveness: the Seattle Social Development Project and the Child Development Project in Northern Cali
EVALUATING PREVENTION PROGRAM EFFECTS 99 fornia. Each of these programs involves substantial training of teachers and a parent program. The Seattle Social Development Project is a longitudinal field experi- ment in an economically and ethnically diverse population of public school students (Hawkins and Catalano, 1987~. The theoretical model underlying the program hinges on social bonding. The program is designed to promote bonding with school and family by recognizing and rewarding prosocial behavior, increasing communication and positive interaction with adults and other children at home and in school, and improving school performance. The program teaches parents how to promote children's academic and social development and helps teachers establish a classroom environment more conducive to learning and self-control. Practices such as interactive teach- ing and cooperative learning are intended to increase opportunities for all students to succeed academically. Students and teachers in 12 elementary and middle schools were as- signed randomly to treatment and control conditions in 1981 and have been followed longitudinally since then. Fifth grade students compared with controls had more positive attitudes toward school, more positive attach- ments to family members and teachers, and more discussion of problems at home with parents. Seventh grade students had higher academic achieve- ment, fewer suspensions and expulsions, and less self-reported use of drugs at school. The Child Development Project is a comprehensive elementary cur- riculum implemented in a suburban middle-class school district in 1982 and a heterogeneous urban district in 1988. The program includes class- room, schoolwide, and family components designed to promote prosocial behavior by building a caring community within the school, making the curricula more accessible and engaging, and building family bonds. Ma- jor classroom elements include a literature-based reading program that highlights core values and interpersonal understanding, a cooperative learning strategy that aims to build internalized motivation and satisfaction, and a disciplinary approach that emphasizes relationship-building, rational ex- planation, and mutual problem-solving rather than contingent rewards and punishments. The suburban program has been extensively evaluated. Observational data in classrooms show the expected program effects on spontaneous prosocial behavior (Solomon et al., 1988) and capability to resolve hypothetical con- flicts (Battistich et al., 1989~; lower loneliness and anxiety, higher peer acceptance, and more democratic beliefs; and higher reading comprehen- sion in tests of higher-order thinking skills. The program has not been in effect long enough for statistically sufficient levels of drug behaviors to emerge so that differences between treatment and control schools can be assessed.
100 PREVENTING DRUG ABUSE: WHAT DO WE KNOW? Midwestern Prevention Project This particular project, initiated in 1984 in Kansas City and known as Project STAR, is being replicated in Indianapolis as I-STAR. The data published to date are from Kansas City. The overall design in Kansas City is a 6-year study. Pentz and associates (1989a) describe the components as resistance skills training and environmental support of nonsmoking and nondrug use through the use of school, parent, and community organization pro- grams, health policy changes, and mass media programming. To date, how- ever, the only program components in effect (or reported on) are "a 10 session youth educational program on skills training for resistance of drug use, 10 homework sessions involving active interviews and role-plays with parents and family members, and mass media coverage" (Pentz et al., 1989b: 3260~. During a 16-month program period, a total of 16 television, 10 radio, and 30 print media events for the project were broadcast over the metropolitan area. The sampling design was complicated. The subjects consisted of the entire 6th and 7th grade cohort in 16 schools, and 25 percent of the cohort, sampled randomly by classrooms, in 34 schools. However, "seventy per- cent of the sample was tracked by grade cohort (cross-sectional sampling of available students in the cohort, including new incoming students who might not have received the intervention; average n = 3371~; the remaining 30% was tracked by individual (n = 1607~" (Pentz et al., 1989b:3262~. In 1985 and 1986, 6 of the 50 schools were no longer active in the study and 2 schools missed data collection because of scheduling conflicts. This left 42 schools in the reported final sample. Of these, 8 had been assigned randomly to either program or control conditions, 20 had agreed to reschedule existing programming and were assigned to receive the interven- tion, and 14 were either unable or unwilling to reschedule existing program- ming and thus were designated as control schools (along with the 4 ran- domly assigned to the control condition). In the control schools, the classroom intervention was delayed by a year? and thus was delivered to the cohorts after those sampled by the researchers. Using as a base the 1,607 students who constituted the individuals spe- cifically tracked, Pentz et al. (1989b:3262) estimated that 3.1 percent had provided no follow-up at any time after baseline and 84 percent were as- sessed at both baseline and 1-year follow-up. In a report using data from all 42 schools, Pentz et al. (1989b) reported prevalence rates for use of ciga- rettes, alcohol, and marijuana that were significantly lower at 1-year fol- low-up in the intervention condition relative to the delayed intervention condition. This held true regardless of whether controls were implemented for race, grade, socioeconomic status, or urbanicity. The effects were 17 versus 24 percent for cigarette use, 11 versus 16 percent for alcohol use in
EVALUATING PREVENTION PROGRAM EFFECTS 101 the past month, and 7 versus 10 percent for marijuana use in the past month. The net increase in prevalence of drug use in the intervention schools was one-half the net increase observed in the delayed-intervention (control) schools. Using complete data from 1,244 subjects from the STAR project in Kansas City, Dwyer et al. (1989) reported 8 versus 18 percent in the 1-year follow-up prevalence rates for past-week cigarette smoking, mixed evidence of an effect on marijuana use, and no evidence of an effect on alcohol use. At 2 years, 12 percent of treatment versus 19 percent of control students reported smoking in the week preceding the data collection. There were also program effects across different levels of cigarette use ranging from no current use to use of one pack or more per day at the 2-year follow-up. Further 3-year follow-up findings reveal that the prevention programs were effective in reducing tobacco and marijuana use and in reducing the prevalence of drug use in youth identified at high and low risk (Johnson et al., 1990~. The authors conclude that a comprehensive community-based approach to drug abuse prevention is effective in preventing the onset of substance abuse, the benefits of which are accrued by high- and low-risk populations. Five-year follow-up results from Kansas City were released in June 1990; 24 percent of the treated students reported smoking cigarettes in the preceding month compared with 32 percent of the control students. Some 36 percent of students who received the STAR curriculum reported having used alcohol in the preceding month compared with 50 percent of the con- trol students. The results regarding past month use of marijuana were 14 percent for those students who had received STAR compared with 20 per- cent who had received the curriculum taught to control students. In sepa- rate analyses from eight schools in which students were tracked over time, 1.6 percent of the treatment students compared with 3.7 percent of the control students reported past month use of cocaine, including crack. The Midwestern Prevention Project is one of the most ambitious drug abuse intervention efforts undertaken to date. It aims to utilize a whole- community approach, and it targets different institutional spheres of influ- ence (e.g., schools, family, media, community institutions) for a consistent message. Unlike many prior and contemporary efforts at drug abuse pre- vention interventions, results from the Midwestern Prevention Project in Kansas City indicate solid, statistically significant effects on all three gate- way drugs: cigarettes, alcohol, and marijuana. These effects seem to have persisted for up to 5 years following the intervention. These are the most unequivocal results produced by any social influences (or any other kind of) prevention program to date. However, the multifaceted nature and com- plexity of the project has created a number of potential methodological confounds and concerns. Most generally, the sampling and selection process was neither random
102 PREVENTING DRUG ABUSE: WHAT DO WE KNOW? nor clearly described. While the investigators have attempted to address questions about sampling by pointing to evidence of initial equivalence of experimental and control groups, the tests for initial equivalence reported to date have been limited to several demographic characteristics and baseline rates of reported use. It is possible that sampling bias is obscured by presentation of data for individuals and that what is crucial for assessing concerns about sampling are data at the school level. Furthermore, concerns exist about implementation. The investigators indicate reasonably strong fidelity to the curriculum, but the process data reported to date are sparse, more often based on assertion than measure- ment. The investigators report on the number of media events that occurred with regard to the Midwestern Prevention Project. However, no data on actual exposure to the media events nor evidence of attention and response to these messages has been presented. The investigators also report an exceedingly high estimate of parental involvement in the homework assign- ments that are an integral part of the curriculum, but have not indicated how these data were obtained or their validity or reliability checked. Summary The major work in progress reported here on developmental programs and the most recent large-scale study of a social influence curriculum have not yet progressed to the point of changing the generally restrained position of earlier research regarding the effectiveness of known school-based pre- vention methods. The recent shift in research focus from within-classroom interventions to broader school reforms is consistent with the growing rec- ognition of the need to support educational interventions on the drug prob- lem with broader policy and environmental changes and to engage parents, community, and other social forces. MASS MEDIA AND DRUG ABUSE PREVENTION Rogers and Storey (1987:818-821) indicate that ". . . a minimal defini- tion of a communication campaign would have to include four characteristic features: (1) a campaign is purposive; (2) a campaign is aimed at a large audience; (3) a campaign has a more or less specifically defined time limit; and (4) a campaign involves an organized set of communication activities." By this definition, the sporadic efforts through the mass media since 1954 to reduce smoking have been a series of campaigns rather than a cohesive campaign. They have been impressive in their correspondence to the peaks and troughs in the more than 40 percent decline in adult male smoking since the first surgeon general's report on smoking and health (U.S. Department of Health, Education and Welfare, 1964~.
EVALUATING PREVENTION PROGRAM EFFECTS 103 Each drug abuse prevention campaign essentially attempts to inform individuals in the audiences about drugs, persuade them about the dangers and consequences of use or abuse, and to mobilize overt behavioral change (e.g., never to start or to quit). But like the smoking campaigns, they must be viewed in the larger historical context of various media activity as well as news events and program initiatives at all levels of national, state, and local organization. A number of objectives attach to any communications campaign. Vari- ous models have been developed to describe the intermediate variables and to explain how the objectives are achieved. In the context of marketing new products, Ray (1973) described a cognitive, affective, conative hierar- chy of communication effects. McGuire's (1968) model includes the con- cepts of attention, comprehension, yielding, retention, and action. Fishbein and Ajzen's (1975) model includes belief, attitude, intention, and behavior. Rogers (1983) describes the decision process about innovation in terms of knowledge, persuasion, decision, and confirmation stages. There have been several eras in views about communications campaigns. The first era involved a somewhat naive view of the monolithic influence of the media. As Rogers and Storey (1987:831) indicate: "In the early eras of communication campaigns there was frequent reliance on mass media alone to accomplish campaign objectives. The shifting conceptualization of com- munication effects and of the communication process had led to recognition that communication operates within a complex social, political, and eco- nomic matrix, and that communication could not be expected to generate effects all by itself." The more recent recognition that communication campaigns and media effects are embedded within a broad and varied range of other stimuli and forces has moved this field beyond simple topologies. Roberts and Maccoby (1985:543) note: "Recognition of the multidimensionality of media effects has led to more complex conceptualization of effects, including not only consideration of their nature (e.g., cognitions, attitudes, behaviors), but also such dimensions as time, unit of analysis, degree of content specificity (e.g., a specific behavior versus a class of behaviors), and type of impact (e.g., establishing, changing, or stabilizing a response). Finally, there is a growing theoretical attention to identification and elaboration of the role of third variables in the media-effects relationship." Research has been conducted on at least two levels of media effects. One of these is broad in scope and has generally been targeted at the na- tional level seeking evidence of change in awareness, knowledge, attitudes, behavioral intention, and behavior. A second level at which research has been conducted is at the development and implementation level, which is generally more basic and formative in nature. In a 1983 report, Flay and Sobel reviewed the state of the art regarding
104 PREVENTING DRUG ABUSE: WHAT DO WE KNOW? the role of mass media in preventing adolescent drug abuse. They conclude (1983:17~: "An overwhelming majority of mass media drug abuse preven- tion programs have failed to change behavior. One obvious reason for this is that most tpublic service announcements] campaigns literally fail to even reach the audience.... Another reason for the failure of most PSA cam- paigns has probably been heavy reliance on information and fear messages. . . . Another problem with anti-drug-abuse campaigns was the tendency for PSAs to be directed to unidentifiable audience segments." Flay and Sobel (1983:18-22) identify three macro-level mediators of suc- cess and failure in drug abuse prevention campaigns. They are whether pro- gram dissemination occurs at propitious times, selectivity or individual predis- positions to attend to the message, and whether the message is boosted with interpersonal communications regarding the issue or problem of concern. Flay et al. (1983) evaluated a smoking prevention program targeted at junior high school students and their families. It was a multifaceted pro- gram that involved five 5-minute segments on the early evening news hour, a 5-day classroom curriculum with an emphasis on skills for resisting social influences, home and family activities coordinated with the school-based and media intervention, followed by a series of five 5-minute segments on smoking cessation. They found greater effects on program than control students with regard to knowledge, attitudes, behavioral intention, and ac- tual reported smoking behavior. Flay (1987) reviewed evaluations of 40 mass media programs or cam- paigns designed to influence cigarette smoking. Programs and campaigns that were informational and motivational in nature did affect awareness, knowledge, and attitudes. Extensive national campaigns providing informa- tion on consequences of smoking and encouraging attempts to quit have had measurable effects on smoking prevalence and quitting rates. For example, Warner (1989) claims that, in the absence of antismoking campaigns, adult per capita consumption of cigarettes in 1987 would have been an estimated 79 to 89 percent higher than the levels actually measured. Flay (1987) reports mixed results from programs and campaigns designed to promote specific behavioral attempts to quit smoking. In the drug abuse field, there has been a concerted effort to ensure the priority of drug abuse on the public policy agenda via the "Just Say No" campaigns and the focus on cocaine and the cocaine hotline of the National Institute on Drug Abuse. These campaigns seem to have had the desired impact in terms of exposure and coverage as well as behavioral responses measured by hotline calls (see Forman and Lachter, 1989; Shoemaker et al., 1989~. Currently, the Partnership for a Drug-Free America is conducting a major antidrug campaign. The stated goal is to help "unsell" illegal drug use in the United States. Two strengths of this campaign are the voluntary
EVALUATING PREVENTION PROGRAM EFFECTS 105 involvement and commitment of the advertising industry in developing the messages and the extensive formative research that has provided a founda- tion for the messages and the campaign. The evaluation of effectiveness of these campaigns involves matched samples of persons interviewed annually for three consecutive years (1987, 1988, 1989~. It should be noted that these are not longitudinal samples; different persons were interviewed each year. The samples (children ages 9-12, teenagers ages 13-17, and adults) were recruited in malls or central locations via a technique known as the mall intercept method. The college students were interviewed in central campus locations on 130 college cam- puses. There is an attempt to weight the samples with census data on age. In addition, there is a partitioning of the data into high-media-exposure and low-media-exposure areas determined by the extent of media exposure of Partnership messages. None of the methods or data have yet been peer-reviewed. However, Black (1989) has reported that there are positive and statistically significant impacts on both attitudes and self-reported behavior overall, and that "teen- agers appear to be the most resistant to advertising messages in general, although changes have shown marked improvement over the three years of the study. It is harder to link this effect to the advertising in 1989 than it was in 1988." There are observed differences in the predicted direction with regard to effects on knowledge, attitudes, and behavior in the high- media versus the low-media areas. Although it has received a great deal of public and media attention, there are significant limitations in the evaluations of the Partnership for a Drug-Free America campaign. First, the data are generally gathered from persons who are intercepted in shopping malls and other central gathering places. In spite of the fact that the evaluators attempt to adjust the sample to census data, adjustment is made only for age. This is simply not an adequate sampling methodology for testing the effectiveness and impact of such a campaign. With regard to college students, there are representatives from 130 campuses. However, again the data were collected from conve- nience samples of students found in common areas. Second, in comparing self-reported drug use among mall intercept samples in high-media with those in low-media areas, there is a leap in inference from media exposure (a universalistic and macro-level measure) to self- reported drug use (an individual-level measure). This is an example of what has been called the ecological fallacy, attributing changes in indi- vidual behavior obtained from independent samples to a macro-level change or variable. Third, the data are presented in only a descriptive and, for the most part, univariate format. More sophisticated statistical analyses would allow for a more realistic appraisal of the impact of the messages on atti- tudes and behavior.
106 PREVENTING DRUG ABUSE: WHAT DO WE KNOW? Donohew et al. (199Ob:25) provide a succinct summary of the practical difficulties involved in achieving behavioral change using television spots. "Anti-drug PSAs, which have become an increasingly popular tool in pre- vention campaigns, are submerged in an often overwhelming clutter of pro- gramming and product advertisements and must be capable of: (1) immedi- ately attracting the attention of target audience members, and (2) motivating these viewers to attend to the remainder of the message. In addition, such messages require relatively high levels of information processing intensity and/or involvement to achieve informational and persuasive goals. Adding to these problems, motives for watching television ordinarily do not include exposure to advertising or PSAs." Donohew et al. (1990a) started their research program with a laboratory setting and the measurement of psychophysiological responses to certain kinds of media messages. The key variables in their research have been sensation seeking and the sensation value of the message. The assumption behind their research has been that people high on sensation seeking are at greater risk for drug use than low sensation seekers. On this premise, prevention messages should help move the more vulnerable group toward recreations, lifestyles, and occupations that compete more effectively with drugs in satiating these needs. Although amusement parks, active sports or exercise participation, and highly mobile or nonroutinized jobs are not sure- fire antidotes or immunizations against drugs, research on sensation seeking argues for the potential value of these types of alternatives. Donohew et al. (199Oa:22) indicate that: "If the goal of the media campaign is modest behavioral change, such as inducing young adults to call a hotline for drug-related information or to put them in touch with face- to-face intervention programs, then this study offers clear guidelines for designing messages to reach nondrug users with different needs for sensa- tion. For high sensation seeking non-users, a group whose members are particularly at risk to become users, a message which stresses exciting alter- natives to drug use and is high in sensation value clearly is more likely to be effective than one which stresses peer resistance skills and is lower in sensation value.... Low sensation seeking non-users, on the other hand, appear to be much more influenced by a message which stresses peer resis- tance skills and is low in sensation value than by a message which features exciting alternatives to drug use and is high in sensation value." A major strength of this project is that formative evaluation is an inte- gral part of the process. The fact that this study began in the laboratory and used reasonably large samples as well as physiological responses to mes- sages provides a solid foundation for subsequent expansions of effort. In addition, identification of sensation seeking as the primary characteristic to be used in targeting messages is unique in that most efforts at audience segmentation have focused on sociodemographic factors.
EVALUATING PREVENTION PROGRAM EFFECTS 107 However, there are some potential limitations that should be mentioned. First, this project relies heavily on sensation seeking as the most important predictor of drug use. Donohew and his colleagues found a relatively strong and general relationship between sensation seeking and use of various drugs. However, there are questions in the existing literature about this relation- ship. For example, Huba et al. (1981) found that the several dimensions of sensation seeking are differentially related to use of different drugs. There- fore, the relationship may be specific according to drug, not a general one between the sensation seeking construct and all drug use. In another analy- sis, Newcomb and gentler (1988) examined the relationship of alcohol use and delinquency, controlling for the underlying personality factor of sensa- tion seeking. The hypothesis was that the alcohol-delinquency relationship would be spurious because of the influence of sensation seeking. This was not the case. Second, the practice by Donohew et al. (199Oa) of grouping subjects into high and low sensation-seeking categories on the basis of a median split ignores the alleged multidimensionality of the construct. This characteristic of a potential audience also will make it difficult to pinpoint a population for purposes of audience segmentation. Third, the use of mes- sages that are either high on sensation value or low on sensation value, while related to drug use propensity in the laboratory, may be less differen- tiating in the real world amidst all the clutter that accompanies the delivery of the message as well as the context within which the message is received. Fourth, the data are either sparse to nonexistent regarding the relationship of sensation seeking to something other than initiation of drug use (e.g., continuation, maintenance, progression within classes of drugs, progression across classes of drugs, regression, cessation, and relapse). All research efforts have limitations, however; the research by Donohew and his colleagues constitutes a viable model for the process of linking messages to targeted audiences. This is an important concept. Roberts and Maccoby (1985:542) describe this orientation: "Current thinking also views the power of the media as highly conditional, depending on a variety of contingent and/or contributory third variables. Chaffee (1977) discusses a trend away from research concerned with demonstrating effects on only 10 percent of a full population toward studies suggesting a 100 percent effect on a specifiable group that may comprise only 10 percent of the population. In other words, recent models posit powerful media effects limited by speci- fiable (and empirically demonstrable) conditions." The goal then should be a closer linkage of the formative process of developing messages with exist- ing research on the etiology of drug use and abuse. Rogers and Storey (1987:836-840) have identified nine features of ef- fective mass media campaigns against drug abuse: · Widespread exposure to campaign messages is a necessary ingredient in a communication campaign's effectiveness.
108 PREVENTING DRUG ABUSE: WHAT DO WE KNOW? · The mass media can play an important role in creating awareness- knowledge, in stimulating interpersonal communication, and in recruiting individuals to participate in campaign activities. · Interpersonal communication through peer networks is very impor- tant in leading to and maintaining behavior changes. . The perceived credibility of a communication source or channel en hances the effectiveness of a communication campaign. . Formative evaluation can improve the effectiveness of campaigns by producing messages that are specific to the desired behavior change. · Campaign appeals that are socially distant from the audience mem- ber are not effective. · Campaigns promoting prevention are less likely to be successful than those with immediate positive consequences. · Audience segmentation strategies can improve campaign effective- ness by targeting specific messages to particular audiences. Audience seg- mentation strategies have focused almost entirely on sociodemographic char- acteristics or on specific drugs such as cocaine. The next step will be to use the existing literature on risk and protective factors to identify individuals at higher risk for drug use or abuse and target messages specifically at these groups. In fact, this is the basis for the Donohew et al. (1990a' emphasis on high sensation seeking, an identified risk factor for drug use. · Timeliness and accessibility of media and interpersonal messages . . . can contribute to campaign success. These characteristics of effective messages provide an important begin- ning point for future research on media effects on drug use and abuse and in antidrug prevention campaigns. However, the guiding principle of the en- tire enterprise has been identified by Roberts and Maccoby (1985:544~: ". . . the ubiquity and assumed homogeneity of media content can make one forget that meanings are not in messages, but in people." There are at least three lessons that have been learned from prior re- search. First, media alone are much less effective than media messages employed in the context of a broad campaign that includes the use of inter- personal channels. Second, there is a need for targeting or audience seg- mentation. Third, there is a strong need for formative research in message and campaign design. The media are only one tool in the hands of those concerned with drug abuse prevention, albeit a very important one. CONCLUSIONS AND RESEARCH NEEDS The only way to determine if something really works is to try it, in a way that permits objective evaluation. Systematic testing and evaluation are essential to progress in reducing drug abuse. A clear majority of the
EVALUATING PREVENTION PROGRAM EFFECTS 109 research published as evaluations of the effectiveness of preventive inter- ventions is methodologically weak. Corrections of their weaknesses is not a matter of applying rigid formulae. It requires patient commitment to attracting quality researchers to the field; applying stringent requirements to publications and research grants, and urging other research sponsors, col- laborators (such as school administrators), reviewers, and publication edi- tors to attend to them; developing and supporting appropriate research train- ing; and attending to socioenvironmental aspects and data quality control elements of proposed research. Social Influence Research Much of the work on social influence approaches to interventions com- pleted to date has focused on preventing or delaying onset of use of the gateway drugs. But preventing or delaying onset is only part of the drug using and abusing continuum. The entire continuum of transitions in drug use (e.g., initiation, continuation, progression, regression, cessation, relapse) constitute the proper focus of attention for prevention interventions (Clayton, 1992). A number of major methodological issues need to be addressed directly and critically. The first of these is attrition. While attrition rates are often reported, the analyses usually show demographic and pretest differences on gross drug use measures between those who remained in the studies and those who dropped out of the study by treatment condition. Very few researchers examine or report such differences on the major predictive or mediating variables. The attrition rates in longitudinal studies on preven- tion fall below rates achieved by large-scale national studies such as High School and Beyond, Monitoring the Future, and the National Education Longitudinal Study. It is essential that panel studies meet attrition stan- dards that amount to the state of the art in survey research. A second major methodological issue is statistical power. Most evalua- tions of prevention interventions suffer from a lack of statistical power to detect differences. When significant main effects fail to emerge, it is com- mon to make too much of subgroup differences. A third major method- ological issue concerns contamination. In the United States, it is virtually impossible to find a true no-treatment control group. However, none of the studies describes the prevention intervention received by the so-called con- trol group members. Clayton and Cattarello (1990) have identified a series of implementa- tion issues concerning social influence curricula that should receive atten- tion. Standards for reporting implementation information scarcely exist, and too often it is virtually impossible to say what actually occurred in an intervention (Moskowitz, 1989~. A set of standards such as those discussed
0 PREVENTING DRUG ABUSE: WHAT DO WE KNOW? by Clayton and Catalano should be promulgated as an evaluation criterion for NIDA-sponsored intervention research. It is clear that different curricula are being used by different research- ers, but explicit comparisons are just beginning to be made (see Rogers et al., 1989~. Comparing the efficacy of different social influence programs, for example, may be comparing the proverbial apples and oranges until standard descriptive frameworks and measures of what actually occurs in the classroom are developed, tested, and widely used. There may be differ- ent pedagogical strategies employed even by teachers or trainers working from the same book in adjacent classrooms delivering, presumably, the same lessons. These differences may be systematically related to the different roles occupied by those delivering the curriculum; to date the evidence on role effects (peer leaders, health educators, classroom teachers, counselors or mental health professionals, substance abuse specialists, police officers) is a raft of inconsistencies. A methodological initiative is needed to de- velop a gauge of trainers' ability to communicate content accurately, intelli- gibly, and in ways consistent with the theory intended to be applied by the intervention. The skills being taught in these interventions require different levels of ability for abstraction and specificity. There is little research on how a single curriculum teaching such skills affects youth in a grade cohort who have attained different developmental levels. There are few instances in which research projects have measured actual individual student exposure to the prevention intervention, to see whether exposure level is connected to measured outcomes; if there is an individual dose-response effect, this would substantially raise our confidence that outcomes and program features were indeed causally linked. It is also important to find some way to assess the degree to which curriculum is embedded in the school milieu. Finally, we add our concern to that of Kozlowski et al. (1989:455) and Flay (1985) about advocacy for social influence interventions in the ab- sence of even moderately compelling evidence of efficacy. Ethnicity and Gender: The Neglected Dimensions Most school-or~ented drug prevention programs are based on mainstream, one-size-fits-all cultural assumptions. Data on differential effects by ethnicity of recipient students have been notably scarce in evaluations of major school- based prevention programs; either no disaggregation by ethnicity is pro- vided by the evaluators, or the sample sizes for which data are available are too small for any differential zero-order or partially controlled effects to be statistically discernible. Differentiation of effects by gender is more com- mon, and there are glimmers of evidence that prevention programs are more effective with girls. One might assume that girls are a population that is
EVALUATING PREVENTION PROGRAM EFFECTS 111 more compliant to begin with, but assumptions will not take the place of . . . c .eeper investigation. An important exception to the rule of not reporting ethnic results is Graham and colleagues (1990), who evaluated short-term program effects of a social influence program (SMART) among Los Angeles 7th graders. There was a significant program effect for Asian students, nonsignificant positive effects for Hispanic and black students, and null effects for white students. There were group differences with respect to different drugs. Moreover, virtually all positive program effects were among girls, suggest- ing that gender role norms interact strongly with ethnic group differences. Koepke et al. (1990) found that in a cigarette smoking prevention and cessation program addressed to middle school students and their parents in San Diego and Los Angeles counties (where one-fifth of all Hispanics in the United States reside), black parents and children were more likely than white, Hispanic, or Asian ones to assess the parents as potentially effective in preventing their children from smoking, but Hispanic parents were more likely to implement "say no" teaching. School-based drug prevention programs that are not based on one-size- fits-all models but rather on culturally specific tailoring do exist, and these have been described to some extent in the literature. Most are adjunctive to a community-based program; however, none has been satisfactorily evalu- ated to date using well-designed outcome measures to test their effective- ness (Orlandi, 1986~. There have been some evaluations among commu- nity-based prevention programs; for example, Schinke and colleagues (1988) pilot-tested a program of culturally specific training in competence skills using random-assignment designs among bicultural Native Americans liv- ing on reservations; they reported desired effects on drug use prevalence. We will return to issues of cultural and community specificity in the appen- dix, where we provide a more elaborate discussion of how research can be structured to yield more usefully articulated results taking these specifici- ties into account. Generic Interventions Substance abuse is nested within a range of other high-risk activities, and it is highly plausible that progress in reducing the most serious levels of illicit drug use will require broad rather than narrow interventions. There is evidence throughout the prevention literature that training in resistance skills and information on health risks may be relatively puny if not counterpro- ductive forces in the lives of many high-risk children, compared with other problems that may overwhelm them. In studies relevant to substance abuse, intensive interdisciplinary programs that deal with more central aspects of their behavior show promise to meaningfully improve the prospects of these
112 PREVENTING DRUG ABUSE: WHAT DO WE KNOW? young people, although the specific effects on drug problems are as yet unknown. REFERENCES Bandura, A. 1977 Social Learning Theory. Englewood Cliffs, N.J.: Prentice Hall. Bangert-Drowns, R.L. 1988 Effects of school-based substance abuse education: a mete analysis. Journal of Drug Education 18(3):243-264. Battistich, V., D. Solomon, M. Watson, J. Solomon, and E. Schaps 1989 Effects of an elementary school program to enhance prosocial behavior on children's cognitive social problem-solving skills and strategies. Journal of Applied Devel- opmental Psychology 10: 147-169. Bell, C.S., and R. Battjes 1987 Prevention Research: Deterring Drug Abuse Among Children and Adolescents. NIDA Research Monograph 63. Rockville, Md.: National Institute on Drug Abuse. Best, J.A., B.R. Flay, S.M.J. Towson, K.B. Ryan, C.L. Perry, K.S. Brown, M.W. Kersell and J.R. d'Avernas 1984 Smoking prevention and the concept of risk. Journal of Applied Social Psychol- ogy 14(3):257-273. Best, J.A., S.J. Thomson, S.M. S anti, E.A. Smith, and E.S. Brown 1988 Preventing cigarette smoking among school children. Annual Review of Public Health 9: 161 -201. Black, G.S. 1989 The Attitudinal Basis of Drug Abuse. The Third Year. Report prepared for the Partnership for a Drug Free America. Gordon S. Black Corporation, Rochester, New York. Botvin, G.J., and A. Eng 1982 The efficacy of a multicomponent approach to the prevention of cigarette smok- ing. Preventive Medicine 11:199-211. Botvin, G.J., and T.A. Wills 1985 Personal and social skills training: cognitive-behavioral approaches to substance abuse prevention. Pp. 8-49 in C.S. Ball and R. Battjes, eds., Prevention Research: Deterring Drug Abuse Among Children and Adolescents. NIDA Research Mono- graph 63. Rockville, Md.: National Institute on Drug Abuse. Botvin, G.J., N.L. Resnick, and E. Baker 1983 The effects of scheduling format and booster sessions on a broad spectrum psychosocial approach to smoking prevention. Journal of Behavioral Medic~ne 6(4):359-379. Botvin, G.J., H.W. Batson, S. Witts-Vitale, V. Bess, E. Baker, and L. Dusenbury 1989a A psychosocial approach to smoking prevention for urban black youth. Public Health Reports 12(3):279-296. Botvin, G.J., L. Dusenbury, S. James-Oritz, and J. Kerner 1989b A skills training approach to smoking prevention among Hispanic youth. Journal of Behavioral Medicine 12(3) :279-296. Botvin, G.J., E. Baker, L. Dusenbury, S. Tortu, and E.M. Botvin 1990 Preventing adolescent drug abuse through a multi-modal cognitive-behavioral ap- proach: results of a 3-year study. Jo7~rnal of Consult~ng and Clinical Psychology 58(4):437-446. Bruvold, W.H., and T.G. Rundall 1988 A mete analysis and theoretical review of school based tobacco and alcohol inter- vention programs. Psychology and [Iealth 2:53-78.
EVALUATING PREVENTION PROGRAM EFFECTS 113 Chaffee, S.H. 1977 Mass media effects: new research perspectives. Pp. 210-241 in D. Terror and L. Nelson, eds., Communication Research-A Half Century Appraisal. Honolulu: East-West Center Press. Chambers, J., and E. Morehouse 1983 A cooperative model for preventing drug and alcohol abuse. National Association of Secondary School Principals Bulletin 81-87. Chassin, L. 1984 Chapter in P. Karoly and J. Steffen, eds., Adolescent Behavior Disorders. Lexing ton, Mass.: Lexington Books. Clayton, R.R. 1992 Transitions in drug use: risk and protective factors. Pp. 15-51 in M. Glanz and R. Pickins, eds., Vulnerability to Drug Abuse. Washington, D.C.: American Psycho- logical Association. Clayton, R.R., and A. Cattarello 1990 Prevention intervention research: the challenges and opportunities. In C. Leukefeld and W. Bukoski, eds., Drug Abuse Prevention Intervention Research: Method- ological Issues. Rockville, Md.: National Institute on Drug Abuse. Clayton, R.R., A. Cattarello, L.E. Day, and K.P. Walden 1991 Persuasive communications and drug prevention: an evaluation of the D.A.R.E. program. In H. Sypher, L. Donohew, and W. Bukoski, eds., Persuasive Communi- cation and Drug Abuse Prevention. Boston: Erlbaum. DeJong, W. 1987 A short-term evaluation of project DARE (Drug Abuse Resistance Education): preliminaryindicationsofeffectiveness. JournalofDrugEducation 17(4):279- 294. Donohew, L., E. Lorch, and P. Palmgreen 1990a Sensation seeking and targeting of televised anti-drug PSAs. In L. Donohew et al., eds., Persuasive Communication and Drug Abuse Prevention. Hillsdale, N.J.: Lawrence Erlbaum and Associates. Donohew, L., P. Palmgreen, E. Lorch, and W.F. Skinner 1990b Personal communication. Dwyer, J.H., D.P. MacKinnon, M.A. Pentz, B.R. Flay, W.B. Hansen, E.Y.I. Wang, and C.A. Johnson 1989 Estimating intervention effects on longitudinally observed health behaviors: the Midwestern Prevention Project. American Journal of Epidemiology 130:781-795. Ellickson, P.L., and R.M. Bell 1990 Drug prevention in junior high: a multi-site longitudinal test. Science 247:1299- 1305. Faine, J.R. Ellickson, P.L., R.M. Bell, M.A. Thomas, A.E. Robyn, and G.L. Zellman 1988 Designing and Implementing Project ALERT: A Smoking and Drug Prevention Expe'-ience. Santa Monica, Calif.: Rand Corporation. 1989 D.A.R.E. in Nashville Schools. Western Kentucky University Social Research Laboratory, Bowling Green, Kentucky. Faine, J.R., and E. Bohlander 1988 Drug Abuse Resistance Education: An Assessment of the 1987-88 Kentucky State Police DARE Program. Western Kentucky University Social Research Labora- tory, Bowling Green, Kentucky. Fishbein, M., and I. Ajzen 1975 Belief, Attitude, Intention, and Behavior: An Introduction to Theory and Re- search. Reading, Mass.: Addison-Wesley.
114 PREVENTING DRUG ABUSE: WHAT DO WE KNOW? Flay, B.R. 1985 What we know about the social influences approach to smoking prevention: re view and recommendations. Pp. 67-112 in C.S. Bell and R. Battjes, eds., Preven tion Research: Deterring Drug Abuse Among Children and Adolescents. NIDA Research Monograph No. 63. Rockville, Md.: National Institute on Drug Abuse. Flay, B.R. 1987 Mass media and smoking cessation: a critical review. American Journal of Public Health 77(February):153-160. Flay, B.R., and J.L. Sobel 1983 The role of mass media in preventing adolescent substance abuse. Pp. 5-35 in T.J. Glynn et al., eds., Preventing Adolescent Drug Abuse: Intervention Strategies. NIDA Research Monograph 47. Rockville, Md.: National Institute on Drug Abuse. Flay, B.R., C.A. Johnson, W.B. Hansen, L.M. Grossman, J.L. Sobel, and L.M. Collins 1983 Evaluation of a School-Based, Family-Oriented, Television-Enhanced Smoking Prevention and Cessation Program: The Importance of Implementation Evalua- tion. Paper presented at the joint meeting of Evaluation Network and the Evalua- tion Research Society, Chicago. Flay, B.R., K. Koepe, S.J. Thomson, S. Santi, A. Best, and S.K. Brown 1989 Six year followup of the first Waterloo school smoking prevention trial. American Journal of Public Health 79(10):1371-1376. Forman, A., and S.B. Lachter 1989 The National Institute on Drug Abuse cocaine prevention campaign. Pp. 13-20 in P.J. Shoemaker, ea., Communication Campaigns About Drugs: Government, Me- dia, and the Public. Hillsdale, N.J.: Lawrence Erlbaum Associates. Gilchrist, L.D., S.P. Schinke, J.K. Bobo, and W.H. Snow 1986 Self-control skills for preventing smoking. Addictive Behaviors l l :169-174. Gilchrist, L.D., S.P. Schinke, J.E. Trimble, and G.T. Cvetkovich 1987 Skills enhancement to prevent substance abuse among American Indian adoles- cents. International Journal of the Addictions 22:869-879. Glass, G., B. McGaw, and M. Smith 1981 Meta-Analysis in Social Research. Beverly Hills, Calif.: Sage Publications. Goplerud, E., ed. 1991 A Practical Guide to Substance Abuse Prevention in Adolescence. OSAP Preven- tion Monograph 8. DHHS Pub. No. (ADM)91-1725. Washington, D.C.: U.S. Government Printing Office. Graham, J.W., C.A. Johnson, W.B. Hansen, B.R. Flay, and M. Gee 1990 Drug use prevention programs, gender and ethnicity: evaluation of three seventh- grade Project SMART cohorts. Preventive Medicine 19(3):305-313. Green, J.J., and J.M. Kelly 1989 Evaluating the effectiveness of a school drug and alcohol prevention curriculum: a new look at "Here's Looking at You Two." Journal of Drug Education 19(2):117- 132. Hansen, W.B., C.A. Johnson, B.R. Flay, D. Phil, J.W. Graham, and J. Sobel 1988 Affective and social influences approaches to the prevention of multiple substance abuse among seventh graders: results from Project SMART. Preventive Med~cine 17(2): 135-154. Hansen, W.B., N.S. Tobler, and J.W. Graham 1990 Attrition in substance abuse prevention research. Evaluation Review 14(6):677 685. Hawkins, J.D., and R.F. Catalano 1987 The Seattle Social Development Project: Progress Report on a Longitudinal Pre
EVALUATING PREVENTION PROGRAM EFFECTS 115 vention Study. Paper presented at the National Institute on Drug Abuse Science Press Seminar, Washington, D.C. Hawkins, J.D., D.M. Lishner, and R.F. Catalano, Jr. 1986 Childhood predictors and the prevention of adolescent substance abuse. Pp. 75- 125 in C.L. Jones and R.J. Battjes, eds., Etiology of Drug Abuse: Implications for Prevention. NIDA Research Monograph No. 56. Rockville, Md.: National Insti- tute on Drug Abuse. Hochheimer, J.L. 1981 Reducing alcohol abuse: a critical review of education strategies. Pp. 286-335 in M.H. Moore and D.R. Gerstein, eds., Alcohol and Public Policy: Beyond the Shadow of Prohibition. Washington, D.C.: National Academy Press. Hopkins, R.H., A.L. Mauss, K.A. Kearney, and R.A. Weisheit 1988 Comprehensive evaluation of a model alcohol education curriculum. Journal of Studies on Alcohol 49(1):38-50. Horan, J.J., and J.M. Williams 1982 Longitudinal study of assertion training as a drug abuse prevention strategy. American Educational Research Journal 19(3) :341-35 1. Huba, G.J., J.A. Wingard, and P.M. gentler 1981 A comparison of two latent causal variable models for adolescent drug use. Jour- nal of Personality and Social Psychology 40:180-193. Jessor, R., and S.L. Jessor 1977 Problem Behavior and Psychosocial Development: A Longitudinal Study of Youth. New York: Academic Press. Johnson, C.A., M.A. Pentz, M.D. Weber, J.H. Dwyer, N. Baer, D.P. MacKinnon, W.B. Hansen, and B.R. Flay 1990 Relative effectiveness of comprehensive community programming for drug abuse prevention with high-risk and low-risk adolescents. Journal of Consulting and Clinical Psychology 58(4):447-457. Koepke, B.L., B.R. Flay, and C.A. Johnson 1990 Health behavior and minority families: the case of cigarette smoking. Family Community Health 13(1):35-43. Kozlowski, L.T., R.B. Coambs, R.G. Ferrence, and E.M. Adlaf 1989 Preventing smoking and other drug use: let the buyer beware and the interven- tions be apt. Canadian Journal of Public Health 80:452-456. Kumpfer, K.L. 1987 Special populations: etiology and prevention of vulnerability to chemical depen- dency in children of substance abusers. Pp. 1-73 in B. Brown and A. Mills, eds., Youth at High Risk of Substance Abuse. DHHS Pub. No. (ADM)87-1537. Rockville, Md.: National Institute on Drug Abuse. Maslow, A. 1980 Motivation and Personality. New York: Harper and Row. Mauss, A.L., R.H. Hopkins, R.A. Weisheit, and K.A. Kearney 1981 A Longitudinal Evaluation of the "Here's Looking at You" Alcohol Education Program. Final Report. Social Research Co., Pullman, Washington. McGuire, W.J. 1968 Personality and susceptibility to social influence. Pp. 1130-1187 in E.F. Borgatta and W.W. Lambert, eds., Handbook of Personality Theory and Research. Chi- cago: Rand McNally. Moskowitz, J.M. 1989 Guidelines for reporting outcome evaluation studies of HPDP programs. In M.T. Braverman, ea., Evaluating Health Promotion Programs. San Francisco: Jossey Bass.
16 PREVENTING DRUG ABUSE: WHAT DO WE KNOW? Murray, D.M., P. Pirie, R.V. Luepker, and U. Pallonen 1989 Five- and six-year follow-up results from four seventh-grade smoking prevention strategies. Journal of Behavioral Medicine 12:207-218. Newcomb, M.D., and P.M. gentler 1988 Consequences of Adolescent Drug Use. Beverly Hills, Calif.: Sage Publications. Orlandi, M.A. 1986 Community-based substance abuse prevention: a multicultural perspective. Jour- nal of School Health 56()9):394-401. Pentz, M.A., D.P. MacKinnon, B.R. Flay, W.B. Hansen, C.A. Johnson, and J.H. Dwyer 1989a Primary prevention of chronic diseases in adolescence: effects of the Midwestern Prevention Project on tobacco use. American Journal of Epidemiology 130:713- 724. Pentz, M.A., J.H. Dwyer, D.P. MacKinnon, B.R. Flay, W.B. Hansen, E.Y.I. Wang, and C.A. Johnson 1989b A multicommunity trial for primary prevention of adolescent drug abuse. Journal of the American Medical Association 261:3259-3266. Ray, M. 1973 Marketing communication and the hierarchy of effects. Pp. 147-176 in P. Clarke, ea., New Models for Communication Research. Beverly Hills, Calif.: Sage Publi cations. Ringwalt, C., S. Ennett, and K. Holt 1990 An outcome evaluation of Project D.A.R.E. Personal correspondence, March. 1. Roberts, D., and N. Maccoby 1985 Effects of mass communication. Pp. 539-598 in G. Lindzey and E. Aronson, eds., Handbook of Social Psychology, Vol. 2: Special Fields and Applications. New York: Random House. Rogers, E. 1983 Diffusion of Innovations, 3rd ed. Rogers, E.M., and J.D. Storey New York: Free Press. 1987 Communication campaigns. Pp. 817-846 in C.R. Berger and S.H. Chaffee, eds., Handbook of Communication Science. Beverly Hills, Calif.: Sage Publications. Rogers, T., B. Howard-Pitney, and B.L. Bruce 1989 What Works? A Guide to School-Based Alcohol and Drug Abuse Prevention Curricula. Health Promotion Research Center. Palo Alto, Calif.: Stanford Center for Research in Disease Prevention. Rosenberg, M. 1979 Conceiving the Self. New York: Basic Books. Schaps, E., J. Moskowitz, J. Malvin, and G. Schaffer 1984 The Nap a Drug Abuse Prevention Project: Research Findings. DHHS Publica- tion No. ADM(84)1339. Rockville, Md.: National Institute on Drug Abuse. Schinke, S.P., L.D. Gilchrist, R.F. Schilling, W.H. Snow, and J.K. Bobo 1986 Skills methods to prevent smoking. Health Education Quarterly 13(1):23-28. Schinke, S.P., G.J. Botvin, J.E. Trimble, M.A. Orlandi, L.D. Gilchrist, and V.S. Locklear 1988 Preventing substance use among American-Indian adolescents: a bicultural com- petence skills approach. Journal of Counseling Psychology 35~87-90. Schinke, S.P., A.N. Gordon, and.R.E. Weston 1990 Self-instruction to prevent HIV infection among African-American and Hispanic- American adolescents. Journal of Consulting and Clinical Psychology 58(4):432- 436. Shoemaker, P.J., W. Wanta, and D. Leggett 1989 Drug coverage and public opinion, 1972-1986. Pp. 67-80 in P.J. Shoemaker, ea.,
EVALUATING PREVENTION PROGRAM EFFECTS 117 Communication Campaigns About Drugs: Government, Media, and the Public. Hillsdale, N.J.: Lawrence Erlbaum Associates. Solomon, D., M.S. Watson, K.E. Delucchi, E. Schaps, and V. Battistich 1988 Enhancing children's prosocial behavior in the classroom. American Educational Research Journal 25:527-554. Sorensen, J., and S. Jaffe 1975 An outreach program in drug education: teaching a rational approach to drug use. Journal of Drug Education 5(2):87-96. Swisher, J.D., C. Nesselroade, and C. Tatanish 1985 Here's Looking at You Two is looking good: an experimental analysis. Humanis- tic Education and Development (Mar):111-119. Tobler, N.S. 1986 Meta analysis of 143 adolescent drug prevention programs: quantitative outcome results of program participants compared to a control or comparison group. Jour- nal of Drug Issues 16(4):537-567. Tobler, N.S. 1989 Drug Prevention Programs Can Work: Research Findings. Unpublished manu- script, School of Social Welfare, State University of New York, Albany. U.S. Department of Health, Education and Welfare 1964 Report of the Advisory Committee to the Surgeon General on Smoking and Health. Public Health Service. Washington, D.C.: U.S. Department of Health, Education and Welfare. Vartiainen, E., U. Fallonen, A.L. McAlister, and P. Puska 1990 Eight-year follow-up results of an adolescent smoking prevention program: the North Karelia Project. American Journal of Public Health 80(1):78-79. Wachter, K.W., and M.L. Straf, eds. 1990 The Future of Meta-Analysis. New York: Russell Sage Foundation. Walter, H.J., R.D. Vaughan, and E.L. Wynder 1989 Primary prevention of cancer among children: changes in cigarette smoking and diet after six years of intervention. Journal of the National Cancer Institute 81(13):995-999. Warner K. 1989 Effects of an anti-smoking campaign: an update. American Journal of Public Health 79(February):144-151. Wunderlich, R., J. Lozes, and J. Lewis 1974 Recidivism rates of group therapy participants and other adolescents processed by a juvenile court. Psychotherapy: Theory, Research and Practice 2(3):243-245.