Designing, Conducting, and Analyzing Programs Within the Preventive Intervention Research Cycle
Commissioned papers for this chapter were prepared by H. Kraemer and K. Kraemer and by S. Fawcett and colleagues and are available as indicated in Appendix D.
Successful science benefits from cumulative progress, and the field of prevention of mental disorders is no exception. The previous chapters have detailed the progress to this point, including the diverse lessons that can be taken from other areas in health research. It is apparent from the review in Chapter 7 that an encouraging number of well-designed research programs on the reduction of risk factors associated with the onset of mental disorders do exist. The task over the next decade will be to enlarge that body of work into a prevention science by instituting rigorous standards for designing, conducting, and analyzing future preventive intervention research programs. By adhering to such standards, prevention can achieve the credibility and validity necessary for its interventions to reduce the incidence of mental disorders.
Only rigorous standards can lead to an enrichment or expansion of the knowledge base essential for prevention efforts. Outcomes from trials built on such standards can serve to refine hypotheses and concepts related to risk and protective factors. The model building and hypothesis testing inherent in prevention research can elucidate pathways taken by individuals as they move toward or away from the onset of a mental disorder, as well as intervening mechanisms and brain-behavior-environment interactions that result in mental disorders or avert their occurrence, even in individuals at very high risk. In addition,
empirical validation of preventive interventions can usefully inform and broaden clinical practice. Epidemiological evidence, for example, can suggest causal factors that can best be tested in a preventive intervention research trial, which may, in turn, suggest molecular or behavioral mechanisms for further study.
THE PREVENTIVE INTERVENTION RESEARCH CYCLE
Just as the development of prevention into a science requires a series of rigorously designed research programs for its collective progress, so an individual research program requires a series of carefully planned and implemented steps for its success. Figure 10.1 presents the committee's concept of how these steps build upon another in the preventive intervention research cycle. The process proceeds in much the same sequence as it has in the report to this point. The first step is to identify and define operationally and reliably the mental disorder(s) or problem. The second step is to consider relevant information from the core biological and behavioral sciences and from research on the treatment of mental disorders, and to review risk and protective factors associated with the onset of the disorder(s) or problem, as well as prior physical and mental disorder prevention intervention research. The investigator then embarks on designing and testing the preventive intervention, by conducting rigorous pilot studies and confirmatory and replication trials (the third step) and extending the initial positive findings in large-scale field trials (the fourth step). If the trials are successful, the researcher facilitates the dissemination and adoption of the program into community service settings (the fifth step). Most of the research programs presented as illustrations in Chapter 7 are at the third step.
Although the review processes that constitute the first and second steps in Figure 10.1 are considered to be part of the preventive intervention research cycle, the original studies in these areas, with the exception of the previous studies on the prevention of mental disorders or problems, are not. For the individual researcher, it is the activities in the third and fourth steps that constitute preventive intervention research per se. Likewise, it is not the community service program and its evaluation but the facilitation by the investigator of the program 's widespread dissemination and adoption (the fifth step) that is part of the research cycle. The knowledge exchange processes that operate between the researcher and the community at this step are discussed in more detail in Chapter 11. (In this report, the term community refers not just to a community as a whole, but also to an element within a community, such as a school, health care clinic, advocacy group, or neighborhood.)
The final steps in the cycle, represented by the feedback loop, are to review the results of any subsequent epidemiological studies to determine if the prevention program actually resulted in reductions in incidence of the targeted problem or disorder(s) and to respond to community representatives regarding their research interests and suggestions for further work.
Each step in the cycle is outlined below. Sections later in the chapter present a host of issues relevant primarily to the research activities in steps three and four—including methodological issues pertaining to experimental design, sampling, measurement, and statistics and analysis, as well as documentation issues. Cultural, ethical, and economic issues that require attention throughout the cycle are also presented.
In this discussion the terms preventive intervention program and preventive intervention trial are carefully delineated. The preventive intervention program is the activity or activities that are provided to the target population (e.g., home visitation with mothers and their infants or a substance use resistance training curriculum delivered to school children by their teacher). The preventive intervention trial is the research component designed with experimental protocols to evaluate and validate the success of the intervention program. Preventive intervention research program is the inclusive term for the program plus the trials.
Identification of the Problem or Disorder(s) and Review of Information Concerning Its Extent
The first step in the preventive intervention research cycle is to identify the disorder, cluster of disorders, or problem that is to be the target of the intervention. Knowledge regarding the diagnostic criteria and course of the disorder, as well as its incidence and prevalence, can be helpful in determining whether a preventive intervention for a particular disorder is warranted. Problems that are appropriate targets for intervention can include those such as child maltreatment that are serious social problems in their own right but are also risk factors associated with the onset of mental disorders. At this step in the research cycle, the investigator also considers the personal, social, and economic costs associated with the suffering and disability resulting from the problem or disorder.
Further, because prevention research almost always touches the community in some way, even at its earliest stages, a partnership in project planning between the researcher and the community is highly desirable. Questions to ask at this point include: Is the particular problem or disorder a matter of concern within the social unit—
community, school, neighborhood, mental health service agency—where the research would be carried out? Would the community be responsive to the development of a research program to address such concerns? Giving the community a voice in defining the problem and in formulating the research program and procedures can be done in many ways, such as by having a representative from the community, perhaps a delegate from a service agency, participate with the research team on an ongoing basis (Kelly, Dassoff, Levin, Schreckengost, and Altman, 1988; Weiss, 1984; Snowden, Muñoz, and Kelley, 1979; see also the commissioned paper by Fawcett, Paine, Francisco, Richter, and Lewis, and commentaries by Gallimore and Rothman, available as indicated in Appendix D.)
Review of Risk and Protective Factors and Relevant Information from the Knowledge Base
Information regarding the concept of risk reduction and how it can be applied in research programs on the prevention of mental disorders can be obtained from a review of prevention programs in physical health (see Chapter 3). Knowing specifics about the predisposing biopsychosocial risk factors and environmental and personal protective factors that converge and interact to determine the onset of any mental disorder is critical for decisions that are made about the nature and targets of any preventive intervention strategy. To acquire this knowledge, the investigator can access a panoply of research disciplines, including molecular biology; behavioral, population, and molecular genetics; gene-environment interactions; neuroscience; developmental, experimental, and social psychology; sociology; behavior analysis; cognitive science; developmental psychopathology; and population and developmental epidemiology (see Chapter 4 and Chapter 5). The investigator next examines what is known regarding the relevant risk and protective factors affecting the onset of the disorder(s) or problem(s) of interest (see Chapter 6). This review provides information that will be useful later in choosing a theoretical model that specifies the mechanism or processes through which these factors have effects. In addition, the review can reveal information on sociodemographic or biological characteristics that may be helpful in targeting a population at risk, as well as identify modifiable risk or protective factors as potential targets for preventive intervention. A review of the relevant publications on prior preventive intervention research programs (see Chapter 7) is another essential step to take before designing the research program. Finally, some of the most important information about protective factors has come from research
on the treatment of mental disorders (see Chapter 8). Treatments to strengthen the social support network and social competence of an individual afflicted with a mental disorder, for example, have consistently been shown to improve that person's outcome. This points toward preventive interventions to reinforce these protective factors and thereby diminish the likelihood of stress-induced initial onset of illness.
Pilot Studies and Confirmatory and Replication Trials
Once the pertinent information has been reviewed, the investigator can begin the process of designing, conducting, and analyzing the research program. Initially, a small-scale, rigorously designed pilot study is done in a carefully controlled setting, often within a community institution, to test methods and procedures. A pilot study is exploratory in nature, and many alterations in design are made. Then the investigator applies the methods and procedures that appear to be successful to a larger population in a confirmatory trial to determine the efficacy of the research program, efficacy being “the extent to which a specific intervention, procedure, regimen, or service produces a beneficial result under ideal conditions” (Last, 1988). If a research program proposes to change risk or protective factors and does so, but the targeted factors are not causal, then the program will lack efficacy, failing to prevent the mental disorder even if it succeeds in altering the risk factor. Thus a well-controlled confirmatory trial can provide relevant data to confirm or deny the causal roles of hypothesized risk and protective factors. Finally, if the results from the confirmatory trial are encouraging, the same methods and procedures are applied in a replication trial to ensure continued efficacy. The Prenatal/ Early Infancy Project (Olds, Henderson, Tatelbaum, and Chamberlin, 1988, 1986), discussed in Chapter 7, is an example of a research program that is now being replicated in a new location.
At this third step in the cycle, the investigator faces a number of decisions. The first of these is the choice of a theoretical model to guide the preventive intervention program. With this model in place, the form of the intervention program itself can be designed. The features of the program—including such things as intervention techniques and site—are chosen here, although they may be adjusted somewhat in step four, when the program is applied in a large-scale field trial. Intervention program design issues are distinct from the methodological issues involved in designing the research component of the program, a task that is encountered both in this step and in step four and thus is discussed as a cross-cutting issue later in this chapter. When the design work is done, the processes of recruiting and training interveners and
identifying and securing the cooperation of appropriate participants can begin. Then the studies or trials are conducted. Thorough documentation of all these choices and the reasons for them is essential to subsequent analysis, both in this step and in step four. This is discussed as a cross-cutting issue below.
Choosing a Theoretical Model to Guide the Intervention Program
To prevent the targeted disorder or problem, the investigator chooses a theoretical model based on the available body of knowledge that addresses one or more of the following factors:
The presence of risk factors and absence of sufficient protective factors correlated with the disorder that may be both causal and malleable, that is, can be altered through intervention.
The mechanisms that link the presence of risk factors and the absence of protective factors to the initial onset of symptoms (which may involve gene-environment interactions).
The triggers that activate these mechanisms (including stressful life events, physical illness, and developmental changes).
The processes that mediate the triggering event and the onset of symptoms.
The processes that occur once symptoms have developed. Ideally, these processes can be attenuated through indicated preventive interventions before they cross the threshold criteria for diagnosis of the disorder.
The choice of a theoretical model stems not only from formulations of risk and protective factors, mechanisms, triggers, and processes, but also from analysis of interventions. Whether a particular theoretical approach can guide prevention strategies depends on the data supporting it. Practically, most current evidence is limited to assessment of risk and protective factors, although there is considerable speculation regarding mechanisms and triggers. Therefore, for now, basing preventive interventions on the risk reduction model, that is, on theories involving the reduction of risk factors and/or enhancement of protective factors, is the most productive strategy. This may ultimately lead to studies on incidence of disorders. No matter which theoretical model is used, the ultimate goal of reducing the incidence of mental disorders is the same.
Designing the Form of the Intervention Program
The intervention program is made up of (1) the activity or activities that are provided to the targeted population, such as an educational
curriculum, supportive counseling, and child care, at a planned frequency and for a set amount of time; (2) the psychological, biobehavioral, educational, organizational, or social techniques and procedures —sometimes called change technologies—used; and (3) the site in which the intervention takes place.
Theoretical and technological factors are closely intertwined and affect the choice of the intervention activities and change technologies. For example, educational interventions require teaching techniques known to work. Specific teaching techniques may work well with certain groups but not with others. If the instructors are not able to teach the participants the skills that are thought to decrease the probability of the disorder or problem, the theory cannot be tested, nor the intervention implemented. Interventions may thus have to be redesigned for different groups to address how they learn; language, educational level, cultural background, rural versus urban setting, and generational cohort will need to be considered. In addition to learning theory, intervention activities and change technologies may draw heavily on operations research, social psychology, behavioral modification technology, and a variety of other fields. They may include the use of biological-pharmacological, educational, or skills-building programs, environmental change strategies, new social policies, and regulations or laws.
A variety of questions are typically addressed at this stage in the prevention research process, such as: Is this intervention acceptable and feasible for the targeted population? Has consideration been given to ethical concerns, cultural factors, and linguistic differences? Have issues of access been addressed, including potential barriers in the host institution or community and dissemination of information regarding the availability of the intervention? In addition, questions about intervention intensity (that is, the frequency and length of intervener-participant contacts), the feasibility of administering the intervention to a group instead of individuals, and the use of special technologies such as video tapes, computer-aided learning, and specialized medical techniques are addressed at this stage.
Preventive interventions, in general, should be short enough to be practical, yet intensive and long-lasting enough to be effective. Obviously, it is best if they are not too costly, but the more relevant issue is whether the potential benefits justify the cost. With the possible exception of certain structural interventions, such as helping a participant secure a job, brief interventions usually cannot be expected to have long-term effects in preventing major disorders. Attempts to change behavior or instill certain skills and to sustain these changes over time require intensity of effort, not only from investigators, but also from participants.
Finally, it is useful to obtain information regarding how well the program and its component parts have been received. Feedback to the prevention researcher in this stage can come from the participants in the studies and trials as well as from community leaders (Krueger, 1988; Manoff, 1985).
Recruiting and Training Interveners
The choice of the interveners can be crucial to the success of the preventive intervention program. Sometimes the interveners are professionals; often they are not. Frequently, they have a natural relationship with the participant—such as being a teacher, parent, doctor, or neighbor (see Chapter 7). Careful selection, provision of initial training and ongoing supervision, payment of a salary, a reasonable workload, and involvement, as appropriate, with the interdisciplinary research team, help ensure high quality and low attrition of interveners.
Identifying and Securing Cooperation from Appropriate Participants
The researcher next decides for whom the intervention is appropriate. In general, the less expensive and the less likely to have any unintended adverse side effects the intervention is, the more widely it can be implemented (universal). As the intervention becomes more expensive, and as it becomes more potent, it becomes increasingly important for ethical as well as economic reasons to focus its implementation to reach the population most at risk (selective, then indicated). (See Chapter 2 for a discussion of population groups.) However, this is not to say that universal interventions are inexpensive to deliver. An intervention with even a low cost per participant becomes a large expense when delivered to thousands of participants. However, these delivery costs may be more than offset by the savings realized when disorders are prevented, especially if an entire lifetime of disability and expensive treatment can be avoided.
One crucial element in identifying appropriate participants is the current understanding of the nature of the problem or disorder (reviewed in steps one and two of the research cycle), in part because individuals who already have the disorder in question must be excluded from the preventive intervention and individuals who are at especially high risk should be included. For most mental disorders, genetic predispositions have only a probabilistic influence on the manifestation of the illness. The onset of a disorder often depends on the nature of the interaction between genetic predisposition and
environment. Therefore, if genes related to mental disorders are eventually identified, individuals with these genes may be particularly appropriate participants in prevention trials for indicated interventions.
Another crucial element is information about who in the population is at risk for the disorder or problem. This information comes not only from risk studies but also from treatment research (reviewed in step two). For example, a high incidence of a particular disorder within a population group identified by age, gender, or culture provides clues about whom to target. Finally, a knowledge of the developmental periods of risk and the ages of onset (from epidemiological studies reviewed in step two) is also valuable for decisions regarding when to intervene.
The investigator next develops a plan to successfully engage the targeted participants. These participants, by definition, do not have a problem that they are necessarily motivated to cure or relieve. There is no way of ascertaining whether any one individual in an at-risk group will develop the disorder if the intervention is not received. Therefore potential participants may not be willing to participate. Influential members of the community can often help by providing access to the targeted group and gaining their cooperation. The investigator can then inform the potential participants not only about any risks involved, but also about how the intervention may be useful to them. Incentives for participation, such as payment for interviews, video tapes of children, printed educational materials, and free transportation, are often presented at this time.
Noncompliance and attrition are major issues in prevention research programs. The intervention potentially can have its largest effects on participants who are receptive to its aims, participate in all intervention sessions, follow through on requests, and continue with the program until it is completed. But participants who do not comply may be those at the highest risk. Efforts to promote compliance are essential to well-designed interventions. One way to sustain participation is to shape the intervention so it is sensitive to the local culture and customs of the targeted group. For example, it is useful to uncover the targeted population's daily routine—including their daily tasks, their values and goals, and their culturally prescribed rules, norms, and scripts —as well as the motives, feelings, and meanings they may associate with the intervention (Gallimore, Goldenberg, and Weisner, in press; O'Donnell and Tharp, 1990). Making participation easy by crafting interventions congruent with these elements, and relevant to people 's lives, will increase participation.
Large-Scale Field Trials
Large-scale field trials offer an opportunity to expand preventive intervention programs found to be efficacious in initial confirmatory and replication trials to large-scale field conditions. Here also, the benefits and costs of the intervention can be more realistically assessed. These trials help to assess the generality of the efficacy of the program with different personnel, participants, settings, cultures, and conditions. A large amount of research in the field of social innovation and organizational change has addressed these questions. Such trials may require involvement with community service agencies or organizations of various kinds, including social service agencies, mental health clinics, primary health care clinics, schools, and day care centers—all here referred to for convenience as organizations —and will definitely require the involvement of many more interveners. Therefore the investigator, although still theoretically in charge, can lose some control over the fidelity of the implementation unless considerable attention is paid to the details regarding the delivery of the intervention and the recording of data.
Experience tells us that research in naturalistic settings can be beset with complications and failure (Hiltz, 1974). The changes in personnel at this point are often the crucial element. Poor communication and operational tensions between researchers and the organization 's personnel are common (Hood, 1990). These problems can lead to certain unwelcome results: personnel may fail to follow through in filling out forms or keeping records, may slow down the work, may provide false or misleading information, may circumvent established procedures, and may even sabotage or move to terminate the project (Hiltz, 1974). Furthermore, researchers are not ordinarily trained to deal with the interorganizational and interpersonal complexities of large-scale field trials.
Projects involving collaborative work between researchers in universities and institutes, on the one hand, and organizational personnel, on the other, can be brought into focus through the lens of interorganizational theory. Hasenfeld and Furman (in press) have specifically suggested the use of the following principles to facilitate these research relationships:
the problem and purpose should be manifest for both parties from the start,
the benefits for each party and the reasons for participating should be clear, and
the expectations and costs should be explicit.
In addition, a more satisfactory exchange may occur at this step if a community representative was involved earlier. Such a process helps to protect the community interests and engenders commitment for participation in the field trials. Rossi (1977) pointed out, in addition, that results of projects are unpredictable and often equivocal and can result in disappointment or bitterness within the community organization. Organizational personnel typically are skeptical at the beginning, but once work is in progress they may develop high and unrealistic expectations about results, especially when their investment of time and effort is substantial. Taking time initially to jointly establish feasible objectives for both entities can minimize this kind of dissonance.
The researcher can enhance the prospects for a cooperative relationship by selecting a compatible organization whose concerns and activities are conducive to those involved in empirical study (Alkin, 1985). Good matches can be found in organizations that
are open to innovation,
have been involved in research before,
use information and research regularly in decision making,
encourage staff to take courses (providing released time or tuition support), and
include a number of staff who teach or have taught courses.
Personnel are then given information on research objectives and methods. Training sessions can include orientation meetings, workshops, special seminars, and informal discussions.
Interorganizational collaboration is also a consideration in the selection of the investigator's own staff. Shadish, Cook, and Leviton (1991) have proposed a set of attributes to look for in recruiting such individuals, including the ability to function in complex, uncertain and ambiguous situations, a programmatic leaning, negotiation and communication skills, flexibility, and the ability to respond rapidly to requests.
Additional guidelines to aid in the shaping of productive interorganizational exchange can be derived from the experiences of a team of intervention researchers with extensive experience (Schilling, Schinke, Kirkham, Meltzer, and Norelius, 1988). They advise that researchers
approach and orient the organization at least six months in advance,
invite suggestions from the organization on research objectives and procedures,
gear operations, if possible, to tangibly benefit the organization 's program,
make procedures compatible with organizational processes,
specify costs to the organization openly and clearly,
indicate personnel time demands, client risk, and potential liability,
provide ongoing recognition to personnel for effort and accomplishments,
provide ongoing feedback through progress reports, and
help implement intervention products in the organizational setting.
Even for the most carefully designed preventive intervention, a single randomized field trial is not likely to result in an innovation prototype ready for large-scale implementation in the community. Furthermore, critical research questions concerning such issues as the plausibility of causal hypotheses regarding the role of risk or protective factors and the assessment of subsequent rates of disorder may require multiple generations of preventive trials. Both multiple generations and multiple sites may be required to determine the “active ingredients” in the intervention, that is, to distinguish the core elements, which must be included to ensure fidelity when a program is adopted by a community, from the less essential features, or adaptable characteristics (see Chapter 11 for further discussion of this critical issue).
After efficacy has been established in large-scale field trials, a final trial is needed to determine the program's effectiveness (as distinct from its efficacy), that is, “the extent to which a specific intervention, procedure, regimen, or service, when deployed in the field, [emphasis added] does what it is intended to do for a defined population ” (Last, 1988). For this trial the investigator turns the carefully tuned intervention program over to the organization that hopes to run it, but leaves the research component in place. This stage in the research cycle is frequently not achieved, but the Centers for Disease Control and Prevention is currently planning to test the Infant Health and Development Program (see Chapter 7 and program abstract, available as indicated in Appendix D) for effectiveness in field trials. Convincing documentation of the program's effectiveness (its efficacy already having been established) would be likely to lead to widespread dissemination of the program.
Facilitation of Large-Scale Implementation of the Preventive Intervention Program in the Community
When researchers and community organizations work together at all stages of the program, they can avoid the problem of “manifest” but not “true” adoption of an innovative preventive intervention. Rappaport, Seidman, and Davidson (1979) have shown what can happen when a
community “adopts” an intervention program shown by research to be efficacious and effective, but modified by community organizations in a manner that produces unexpected negative consequences for the recipients. This need not happen, but ensuring a positive large-scale implementation effort requires considerable knowledge and attentiveness to the concepts of core elements and adaptable characteristics. At this step, the investigator can provide a manual describing the program to guide implementation. The investigator can also facilitate a decision by the organization to include an ongoing evaluation component in the program. (See Chapter 11 for a description of other ways to facilitate the exchange of knowledge between investigator and community organization.)
It is essential to delineate explicitly the goals at the outset of designing the research component of a preventive intervention research program. For example, is the goal to reduce an occupational, social, educational, family, or personal risk factor—such as child abuse, marital stress, unemployment, or aggressive behavior? Is it also to enhance protective factors? Is the goal to intervene with mechanisms, triggers, and processes related to the onset of disorder? In addition to these goals, the ultimate goal of preventing or delaying the development of a full-blown mental disorder(s) should be explicitly stated even though at this stage that may not be the goal of the preventive intervention itself.
The goals influence the types of research methodology that will be used, as well as the answers to methodological questions encountered in steps three and four of the preventive intervention research cycle. Questions concerning the structure and duration of the trial and follow-up period, sampling, measurement, and statistics and analysis are considered here, including: What are the characteristics of the population to be used in sampling? How large should the sample be? What are the methods to be used to produce and measure changes in the targeted risk and protective factors in the population? What are the methods to be used to measure changes over time in the incidence of the targeted disorder(s) or problem?
Structure and Duration of the Trial and Follow-up Period
Structure of the Trial
The randomized controlled trial, in which members of a population are randomly allocated into experimental and control groups, usually is
the preferred experimental design in research studies, and it provides the most rigorous means for hypothesis testing available in preventive intervention trials as well. Random assignment helps to ensure that the participants' responses are unbiased estimates of what the average responses would have been if all members of the population could have been assigned to one of the two groups. Frequently, when trials are designed, there are conceptual or hypothesis-driven reasons to include more than one intervention for evaluation. Hence a particular prevention trial may have more than one experimental group, and therefore random assignment is made to multiple groups.
Randomized control groups are particularly important in selective and indicated prevention trials, especially if the targeted groups have been chosen carefully enough to ensure that they are at very high risk. In such trials the interventions are working against the probabilities associated with the “natural” course of the pathological process. If this course is increasingly negative, the results of the intervention can appear to be an increase in problems that did not exist before the intervention. On the other hand, some problems are self-limiting, and positive results may be due to the passage of time rather than the intervention. A finding of lower incidence of disorder in the experimental group as compared to higher incidence in the control group is the best way of documenting the effect of a preventive intervention.
Although randomized controlled trials remain the optimal design for preventive intervention trials, quasi-experimental time series designs can sometimes permit investigators to capitalize on policy or regulatory changes and conduct natural experiments in the real world, as, for example, with the Intervention Campaign Against Bully-Victim Problems (Olweus, 1991) reviewed in Chapter 7. Campbell (1991) has described a number of policy-oriented interventions that can be analyzed by using interrupted time series and regression, discontinuity analyses.
The logic of the interrupted time series analysis is relatively straightforward. The independent variable, in this case the preventive intervention, is expected to produce a change in the group under observation. The intervention “interrupts” a series of baseline observations at a specified point. If the intervention does indeed have an effect, the time series preceding it should differ from the time series subsequent to it. In a treatment study, for example, Liberman and Eckman (1981) used an interrupted time series design to assess the impact of two brief inpatient interventions on suicide ideation and attempts. Both treatments markedly reduced suicide attempts when two years following intervention were compared with two years prior to hospitalization. Such simple
quasi-experimental time series designs cannot rule out alternative explanations for a change in the variable of interest. Thus it is more desirable to use controlled, experimental forms of time series, such as multiple baseline, multiple schedule, reversal, withdrawal, and multi-element designs (Barlow and Hersen, 1973) or to combine time series with nonequivalent control group designs, as Cook and Campbell (1979) suggest.
Obviously, one cannot have as much confidence in the results of quasi-experimental designs as true experiments. Therefore quasi-experimental designs should be used only when it is not possible to randomize. An important problem with nonequivalent control group designs is that researchers simply may not be able to create control groups that are similar enough to the experimental group.
Several other problems severely restrict the conclusions that can be drawn equally from both true experiments and quasi-experiments. Consider, for example, instances in which, despite efforts by the research team, participants in a control group know that they are not receiving the desired intervention. The control group, as an “underdog,” may be motivated to reduce or reverse the expected effect of the intervention (Cook and Campbell, 1979), the so-called “John Henry effect.” On the other hand, members of a control group may be demoralized in knowing that they are not receiving the desired intervention, and there may be a decrement in their performance. Because control groups are often intact groups that may interact with one another, such as students in a particular classroom or residents of a particular block or neighborhood, their proximity increases the likelihood that they will act in concert or develop similar perceptions of the experiment.
Duration of the Trial and Follow-up
The length of the intervention as described above—short enough to be practical and yet long enough to be effective—governs the length of the preventive intervention trial as well. In addition, because a decrease in the incidence of a disorder is the major long-term goal, participants should be followed longitudinally in prospective designs. Therefore follow-up periods can be quite lengthy, but this is a complex issue. The longer the duration of follow-up, the greater the power—that is, the statistical capacity to be able to demonstrate a significant result—may be to detect the efficacy of the program in showing short-term as well as long-term positive effects. This is not, however, necessarily so. If multiple factors are involved in the onset of a disorder, lengthy
follow-up provides more opportunity for uncontrolled factors to influence outcome.
Long time frames also may be necessary in order to get beyond the age of risk of onset. Preventive intervention trials cannot prove that a particular disorder has been permanently prevented; they can only provide evidence that the onset of the disorder has been delayed for as long as the trial proceeds. A participant still free of the targeted disorder at the end of the trial's follow-up period may have the onset of the disorder one day later, unless the trial has followed the participant completely through the age at risk. For many mental disorders, however, risk of onset continues through the life span.
The longer the trial and follow-up, however, the greater the cost and difficulty, and the longer the delay in obtaining answers to the research questions. Consideration of cost, of course, plays a crucial part in the setting of the follow-up time. Funding sources are understandably reluctant to fund research programs that will take 10 to 20 years to produce results. But the current practice of short-term support is especially limiting in regard to research on prevention of mental disorders.
Lengthy follow-up periods that delay the reporting of results have disadvantages in terms of scientific practice as well. As time goes on, the importance attached to certain research questions changes. Also changing are the methods of measurement, diagnostic criteria, and other issues that must be taken into consideration. Therefore a balance must be found between the gain in power and precision resulting from long-term follow-up and the loss in relevance and quality of content or substance that may be incurred. This also suggests that data should be kept on symptoms and behaviors because the definition of disorders is subject to change.
The practical limitations placed on the duration of a preventive intervention trial and follow-up in part can be dealt with by timing the implementation carefully. Selecting participants who are moving into their period of highest risk for the onset of the disorder, a period of critical developmental challenge and maturation, or a period of high responsiveness to protective effects, permits detection of effects that are sufficiently large and immediate.
The choice of the sample from the targeted population has methodological repercussions. The major problem in using selective and indicated preventive interventions is the identification of the high-risk group. Obviously, it is critical that the definition of high risk be a valid
one. The sensitivity and specificity of screening tests are used in the determination of risk status. Sensitivity is the proportion of truly diseased persons in the screened populations who are identified as diseased by the screening test. Specificity is the proportion of truly nondiseased persons who are so identified by the screening test. Sensitivity and specificity of identification criteria tend to “see-saw”; that is, the cost of having high specificity is usually low sensitivity. It is possible to develop criteria with high sensitivity, but the result will be a far looser definition of what constitutes high risk.
It is useful to obtain samples that span the full range of gender and culture among individuals at risk. Since passage of the National Institutes of Health Revitalization Act of 1993, such representative samples are legally required. A trial of a preventive intervention that excludes women produces results that do not necessarily generalize to women; one that excludes minorities may yield results that do not necessarily generalize to minorities. The principle is clear: If one excludes any group from a trial, the results of that trial cannot be assumed to generalize to that group. For this reason, in implementing indicated interventions in which there are stringent inclusion and exclusion criteria, the effects of these restrictions on the generalizability of the results of the trial to the population at large must be carefully considered. In locations that include large segments of non-English-speaking individuals, studies must include assessment and interventions in the appropriate language (Muñoz and Ying, 1993; Maccoby and Alexander, 1979).
The use of a universal population presents methodological problems of a different sort. Such a population is typically diverse and may include participants who are not receptive to the program for a variety of reasons. The heterogeneity, combined with the low incidence rates of the disorder likely in such a population, creates a situation in which very large sample sizes are necessary to detect any indication of efficacy or effectiveness. Because the effects may seem quite small, the clinical or policy significance of the prevention program may be underestimated. Uniform implementation of both intervention and measurement protocols in universal interventions across entire communities may be difficult. Such problems reduce statistical power, either by increasing the heterogeneity of response or decreasing the reliability of the response measures.
For a trial using a universal population, the investigator can plan secondary analyses to focus on those subgroups likely to be most at risk and most receptive to the intervention, in order to generate results comparable to those that would be obtained with the selective or
indicated populations. Thus the investigator can have both generalizability of results and the possibility of specific a priori subgroup analyses of special interest. The costs and difficulties of such studies are, however, often substantial.
Another method for analyzing subgroups is stratification, that is, the separation of a sample into several subsamples, sometimes along a continuum, according to specified criteria such as age or level of education. Stratification can be useful when factors measurable at baseline, that is, at the beginning of the intervention, are believed to correlate strongly with onset of the disorder or problem. In such a situation, greater power can be achieved when samples are stratified before randomization (thus creating subsamples) and participants from the subsamples are randomized to the intervention and control groups.
If there are certain subgroups of the population of special interest, naturalistic sampling will likely yield too small a sample size to draw definitive conclusions that can be generalized to each subgroup separately. To answer questions about subgroups adequately, the investigator should ensure that there is adequate power through oversampling of these groups. Alternatively, it is often desirable to design a separate trial for each subgroup with a sample size adequate to answer the questions.
Although the processes of focusing attention on the questions that require answers, selecting the appropriate measures (and thus excluding the rest), deciding when and how often to measure, choosing the best measurement techniques or instruments, and taking steps to ensure the reliability and validity of the selected measures are perhaps the most tedious and difficult parts of a trial, these are also among the most essential procedures in determining its success or failure.
What to Measure
Careful selection of primary outcome measures is essential to the success of a preventive intervention trial. These are usually the measures of changes in the theorized mediating variables, including risk and protective factors, that are assumed to be responsible for the reduction in risk. They may be psychological outcomes such as measures of precursor signs or symptoms, social outcomes such as reduction of poverty, or biological outcomes such as reduction of the incidence of low birthweight. Finally, in the prevention of mental disorders, it is particularly desirable for programs explicitly to include measures of the
incidence of mental disorders. One benefit of including incidence measures is that they may reveal that a risk factor that was found to be malleable was not causal, thus contributing to the knowledge base about etiology.
Evidence of risk reduction is used as the primary outcome measure, that is, the measure used most often to document the results of the trial, in part because it is available first. For example, in the Perry Preschool Program, a selective preventive intervention with preschoolers that was intended to increase their intellectual and social development (see Chapter 7), the long-term results, including measures of such factors as how long the children stayed in school, took many years to be documented. However, short-term measures of behavioral problems, such as lower rates of aggression, lying, and stealing, served as early evidence of the success of the intervention. Documentation of the changes in risk and protective factors for the child could have been extended for a more complete picture of mediating variables. Although disorder incidence measures were not included in the Perry Preschool Program, it is one of the few research programs to have included long-term outcome measures of any sort.
Measures of process are also appropriate. Because prevention programs typically have many components, it can be difficult to determine which one accounts for the success of the intervention. However, it may be possible to examine, on a more exploratory level, the impact of the different components by gathering data that are descriptors of process. Measures of process are selected to reflect certain characteristics of the participants, program, activities, change technologies, and so on, and of the interaction of these, that might help to generate hypotheses as to why and how the program might work. For example, what is it that the intervener actually said and did during home visits, or how do teachers respond to participants' aggressive outbursts at school? This set of measures can include consideration of risks, costs, inconvenience, and dosage effects. Measures that reflect different theoretical models can be chosen to elucidate most salient elements of the intervention.
Measures of compliance are another type of process measure. Participants at the outset of a preventive intervention trial may simply refuse to participate. If they agree to participate, they may not comply, in whole or in part, with the procedures and activities of the prevention program. It is useful to document the extent of compliance of individual participants throughout the trial. If the program does not prove to be efficacious, it is important to gain insights as to why participants did not comply, as a basis for consideration in the design of future prevention efforts.
A primary goal of a trial is efficacy, but process measures may provide information that can be useful in improving the design of a subsequent prevention effort. Furthermore, such data provide documentation of the fidelity of the program, that is, the extent to which the components of the program as designed were actually delivered and received by the participants.
When and How Often to Measure
Random assignment does not yield groups that are identical on all baseline variables. Therefore an extensive collection of baseline information, including targeting variables, is necessary. Without this, the investigator's ability to draw firm conclusions about what would have happened in the experimental group in the absence of the intervention is compromised. The baseline information is also needed to determine eligibility for the program, to ensure that the elements in the prevention program are not already in place in the participants ' environment before the experiment, to describe the population to which the results might be expected to generalize, to document the success of the randomization procedures, and, in secondary analysis, to detect those subgroups for which there is differential outcome. If there are dropouts or missing data, baseline information is also necessary to investigate the possibilities of resulting sampling bias. Figure 10.2 shows the points on the time line for the trial and follow-up at which the baseline and other measures are taken.
After the baseline assessment, the greater the frequency of observation, the more precise the measurement of onset and course. Outcome measures on each participant should be taken frequently enough to determine the timing of short-term effects. Long-term outcome measures taken at follow-up to determine incidence should be continued past the mean age of onset for the disorder. Frequent follow-up can bind the participants more closely to the program and promote receptivity and compliance. However, too great a frequency of observation (particularly when the assessments are difficult, long, tiresome, stressful, or invasive) may annoy the participants and produce the opposite effect. The quality of information (validity and reliability) may suffer; dropouts may increase. Clearly, some balance must be achieved.
Which Measuring Techniques and Instruments to Use
The selection of outcome measures for use at baseline, over the short term, and during follow-up includes consideration of the relative value
and use of continuous (that is, a scaled or dimensional response) measures and the more usual categorical (that is, a number of nonhierarchial responses) measures. Almost always, a variable can be measured with either technique, but the yield is different. When they are appropriate, continuous measures can increase statistical power, but the crucial issue in deciding on the outcome measures in a trial of a prevention program is that of selecting the most valid and reliable measures available.
The most likely strategy to detect effects and to be cost-effective includes both categorical and continuous measures. For example, the intervention may produce a modest reduction in the incidence of new cases that meet the designated diagnostic criteria for a particular disorder or in the existence of specific risk factors (categorical measures), but may produce a very great reduction or attenuation in the severity and duration of risk factors, including precursor symptoms (continuous measures). Also, even if the intervention failed to prevent the onset of a disorder, it might reduce the severity, duration, or disability of the disorder (continuous measures). An example from infectious disease may illuminate this point. If an antibiotic taken prophylactically to prevent traveler's diarrhea produced only a modest reduction in the
frequency or incidence of a threshold for diagnosing the enteritis (e.g., presence of bacteria in stool with at least one episode of diarrhea with or without abdominal discomfort), but produced a very large reduction in the severity of the diarrhea (i.e., reduced frequency and amount of loose stools as well as reduced frequency and severity of abdominal discomfort), then the verdict of that prevention trial might be that the antibiotic was indeed useful in prophylaxis. A similar argument could be made for the effectiveness of fluoridated water in preventing the number and severity of caries; the effectiveness of a cognitive behavior therapy preventive intervention on the depth, duration, and disability of major depressions; and the effectiveness of family- and school-based educational and skills training programs with children at risk for conduct disorder on the incidence and severity of subsequent delinquency, substance abuse, and antisocial behavior.
Whether the chosen measures are categorical or continuous, they should display high internal consistency and construct validity based on earlier psychometric analyses and research as well as high reliability with different assessors. With the advent of the DSM-III and DSM-III-R, certain comprehensive diagnostic instruments that can elicit all the signs and symptoms of mental disorders have come into general use and provide a means for improving the reliability and replicability of diagnosis. Diagnostic interviews such as the Diagnostic Interview Schedule (DIS), the Present State Examination, the Schedules for Clinical Assessment in Neuropsychiatry (SCAN), and the Structured Clinical Interview for DSM-III-R (SCID) (all of which can provide continuous and categorical measures) can improve the detection of symptoms. They also lead to operational criteria for improving the accuracy of rating the presence or absence of symptoms or disorders. When diagnosticians are trained in the use of these structured instruments, they become more consistent, systematic, and precise—thereby enhancing the reliability, validity, and power of the preventive intervention trial. In addition, the work groups responsible for producing DSM-IV and ICD-10 have purposely interacted in their development of diagnostic criteria, and these classification systems are coming closer together.
How Many Measures and Instruments to Use
Measures should be carefully chosen and relatively independent. Every variable measured yields both signal and noise. Multiple noisy measures (unreliable measures) of the same signal used separately add no signal to the system, only noise. The signal is merely repeated along with the noise. For this reason, in any set of highly correlated measures,
the investigator should either select the best and delete the rest or combine them into one measure. This strategy not only reduces the number of analyses, thus diminishing the risk of false positive results, but also diminishes the risk of false negative results, because combining multiple measures of the same signal frequently results in “tuning out” much of the noise and thus “tuning in” the signal, resulting in a combined measure that is more reliable.
Having many measurements and diagnostic assessments may compromise the quality control of the measurements. When there are only a few crucial measurements on which the success of a trial depends, the investigator can spend a great deal of time and effort to select the best instruments, provide adequate training and orientation to the assessors, and institute adequate quality control procedures. But if hundreds of variables are collected, expedient measures of limited validity become a temptation, and the consistency and care in assessing each variable may be compromised. The fatigue of both participants and assessors can further impair the quality of measurements. What is sometimes called a “rich” data set that contains a large number of variables may, on closer inspection, be rich only in noise, not in signal.
Very large data sets require more staff effort to maintain and analyze. A large number of variables will not be helpful if key variables that will reflect the hypotheses are inadequately measured. Investment in quality control of key variables that includes error checking, detection, and correction procedures is critical to achieve a valid result.
How to Ensure Reliability and Validity of Measures
Reliability “refers to the degree to which the results obtained by a measurement procedure can be replicated,” and validity “is an expression of the degree to which a measurement measures what it purports to measure” (Last, 1988). Seldom do diagnostic procedures in any area of medicine have a reliability coefficient above 80 percent. Many diagnostic procedures in common use have reliability coefficients between 40 and 60 percent. The issue of reliability of diagnosis in psychiatry has certainly received far more attention than has reliability of diagnosis in most other fields of medicine. But the principle remains: unreliability tends to attenuate power, necessitating larger sample sizes (Kraemer, 1979); therefore, it is especially important to develop reliable methods.
A common error made in addressing the issue of validity is to collect many poor measurements in the hope that these will somehow make up for the absence of one highly valid measure. But multiple poor measurements that do not accurately assess the construct of interest can lead
to false positive or false negative results. To add to the confusion, multiple outcome measures can produce contradictory findings, making it impossible to draw any conclusions at all. In addition, lack of sensitivity to cultural variations in meaning can confound the validity of measurement; for example, among the Navajo the concept of “home ” includes the extended family, whereas in the mainstream culture it is restricted to the nuclear family. If a measure includes only nuclear family members, it will miss an essential part of Navajo life and therefore be less valid.
To ensure validity, outcome measures ideally should be assessed “blind” to the group to which the participants have been assigned. Measurement and assessment procedures that include any subjective component may be affected by the assessor's knowledge of group membership. Thus a certain response pattern, when observed in a participant known to be in the experimental group, may be assessed differently from the same response pattern observed in a participant known to be in the control group. This phenomenon introduces measurement bias and compromises the validity of the results. A quality control check on raters' blindness can be done by administering a questionnaire to raters at several times during the prevention trial, asking them to make guesses about the assignment of the participants.
As is the case in many randomized controlled trials, however, it is simply not possible to blind all assessors to the group membership of the participants. When the measure is based on self-report, it is often not possible to blind the participants to their own group membership. This situation places a premium on measures that are objective. It also makes the implementation of training and orientation procedures for assessors, and quality control procedures such as periodic reliability testing of the assessors over the course of the study, more vital to the validity of trials of prevention strategies than might otherwise pertain.
Adherence to the measurement protocols of the research program, for both experimental and control groups, adds to validity and reliability. Requirements for such adherence to protocol are often viewed as a rigidity that runs counter to good clinical care, and maintaining these protocols is difficult over the course of a long-term study. Such requirements are often seen as a challenge to the morale and commitment of the researchers, particularly to those who are also clinicians. Special efforts must be made both to inform all research colleagues of the necessity for such adherence and the consequences of deviations from protocol in terms of the validity and power of the results, and to ensure the enthusiastic participation and commitment of all participants to the goals of the study.
Statistics and Analysis
Strategies for Data Analysis
Randomized controlled trials in which participants are followed longitudinally inevitably entail collection of a great deal of data, no matter how parsimonious the investigator has been in choosing and pruning the type and frequency of measures and instruments. Many statistical methods for analyzing these data exist. For categorical data, the most familiar of these methods are logistic regression, log-linear modeling, and discriminant analysis.
For continuous measures, methods for the analysis of repeated measures are required. Considerable interest has been generated recently by the use of random effects regression models as alternatives to repeated measures analysis of variance and covariance or MANOVA designs (Gibbons, Hedeker, Elkin, Waternaux, Kraemer, Greenhouse et al., in press; Laird and Ware, 1982). In this methodology a separate curve is fit to each participant's response data, using a few clinically interpretable parameters to define the mathematical model for the curve. Unlike the more familiar repeated measures analysis of variance designs, these methods are relatively tolerant of missing data, irregular follow-up, and dropout. Moreover, because these approaches use scaled response data, they can be more powerful than approaches using binary indicators of disorder applied in the same context.
Latent structural equation modeling is a statistical tool for examining the relationships among multiple variables. This statistical methodology is available to clinical researchers as part of major software statistical packages. It permits simultaneous testing of complex multivariate hypotheses and may have considerable promise as a tool for exploratory data analysis. However, its utility in formal statistical hypothesis testing is less clear, because the validity of the statistical tests depends on the correctness of strong assumptions about multivariate distributions and on the existence of a clear theoretical model (i.e., Fergusson, Harwood, and Lloyd, 1991).
As more data regarding age of onset are gathered, the preferred analytic strategy for comparing incidence rates across groups is likely to be survival analysis. Survival analysis is a flexible and powerful statistical method for analyzing incidence of illness when time to onset is known. Like the more familiar contingency table methods based on counts of numbers of participants who have onset of the disorder during some follow-up interval, survival analysis can be used to compare the risk of becoming ill in two or more groups. Indeed, because survival
analysis depicts incidence across the whole follow-up period, it provides a more detailed picture of outcome. Both parametric and nonparametric approaches are widely available, the former being more powerful when the distributional assumptions are valid, and the latter being less restrictive and more familiar. Survival analysis is typically more powerful than simple counts of incidence during a specified period, particularly when base rates are low. The methodology adjusts for participants who are lost to follow-up. As with regression analysis, covariates can be analyzed, including both main effects and interactions.
The probability of a participant's surviving through a period of risk without developing a disorder may change as the duration of the intervention and follow-up increases. This changing probability is called the survival function. For example, the longer a participant proceeds through the period of risk for a disorder, the lower the probability for developing the disorder. Statistical methods of survival analysis are being used in treatment trials and epidemiological studies of onset and natural history (Elandt-Johnson and Johnson, 1980).
Typically, there are individual differences in the susceptibility among participants in any group—based on risk and protective factors —and these differences are reflected in different survival function shapes. Some participants may be essentially immune to the disorder, and, at the other extreme, some may already be experiencing the precursor signs or symptoms of the disorder at the initiation of the trial. Survival function curves begin at 100 percent and either decrease or plateau as participants succumb to the disorder. Thus it is possible to identify individual differences among participants, as well as to detect differences in effects of the experimental and control conditions.
Groups of participants with high survival curves (i.e., close to the 100 percent level of survival) are “low risk,” and those with low survival curves are “high risk,” but these are relative terms, with no precise definition. Participants selected from the general population are likely to be “low risk,” and participants selected because they have risk factors such as a family history of the disorder are likely to be “high risk” in terms of lifetime risk. In the real world, an individual participant's survival curve is a hypothetical construct that cannot actually be seen. However, the average survival curve for any group of participants can be estimated and provides information about delay of onset in that group. For example, Figure 10.3 presents three survival curves using new data from the Five Cities Program of Cardiovascular Risk Prevention (see Chapter 3 and the commissioned paper by Kraemer and Kraemer, available as indicated in Appendix D). These survival curves demonstrate the reported onset of smoking in three male birth cohorts: A: (1901
to 1940); B: (1941 to 1960); and C: (1961 to 1980). The last cohort was born after the publicity and widespread education following the Surgeon General's recommendation against smoking for health reasons. The change in the distribution of onset of smoking is clear. The onset of smoking came later, particularly in cohort C, and the lifetime prevalence (indicated by the plateau value) of smoking decreased in the later cohorts, leaving more nonsmoking “survivors” and a higher plateau value.
For a given group, the point in time at which its curve reaches the 50 percent point is the median survival time or median onset time. Thus the median age of onset of smoking for cohort A was about 20 years, and for cohort B, about 22 years. For cohort C, the median time is not less than 25 years (at which point 65 percent have survived without smoking), and there may be no median age of onset overall, for fewer than 50 percent may have the onset of smoking during their lifetime.
An interesting aspect of survival analysis is the hazard function, which is the probability of becoming ill at each point in time. Analyses of changes in risk over time may be particularly sensitive indicators of a program's efficacy and effectiveness. The hope is that the intervention program might begin to exert an effect at its inception and gradually build to its full effect as it is fully implemented with desired impacts on the participants' risk and protective factors. The hazard function curve quantifies the probability per unit time that a participant who has survived up to a particular time will have the onset of the disorder in the very short ensuing time interval.
By restricting consideration to those who have survived up to a particular point in time, the investigator can control for factors before that point that have already exerted their effects. By restricting the consideration of the hazard function curve to a short time period, factors that exert their influence on onset of disorder during that time interval only can be identified. Whereas the survival function curve must be either constant or decreasing downward from its initial 100 percent level, the hazard function curve can take any shape at all. It may be flat, it may increase as it does for disorders associated with aging, or it may decrease as it does for disorders primarily associated with infancy. Depending on the natural history and risk periods for the disorder, hazard curves may grow, recede, or have one or several peaks.
Whereas survival and hazard functions can illuminate the changes in incidence of mental disorders among participants in a prevention program, impacts of the program on the severity of the disorders that do develop among participants for whom the prevention program failed, such as the degree of impairment or disability, relapse pattern, or duration of episodes, require the use of prevalence assessments to highlight the differences between the experimental and control groups.
Currently, however, it may not be practical or feasible to obtain valid measures of time to onset for survival analysis. For an insidiously developing disorder, such as schizophrenia, the time to onset may be difficult to ascertain and, at least from the point of view of analyzing a prevention research program, relatively unimportant. If survival methods cannot be used, random effects regression models permit the best use of incomplete follow-up data for participants and help avoid some of the problems of sample bias associated with low retention rates during a trial. However, such problems do not disappear; every missing data point or dropout from the study costs some degree of power.
The Unit of Analysis and Statistical Power Consideration
When interventions are delivered to groups rather than to individuals, the appropriate unit of analysis is the group. Because there are typically far fewer groups than there are individual participants, use of the group as the unit of analysis may appear to result in a major sacrifice in power. However, power is not totally determined by the degrees of freedom, that is, the number of independent comparisons that can be made between the members of a sample. Power is more strongly affected by the size of the effect. Therefore groups can be used as the unit of analysis, and “two-stage” statistical models can be used for that purpose (Gibbons et al., in press).
The number of individuals or groups necessary to detect clinically significant effects with sufficient power is dependent on the design of the trial. The number may vary from two cities per group to tens of thousands of individuals, so choices made in sampling (such as whether entire communities or individuals are being studied), as well as choices made in measurement (such as whether categorical or continuous measures are used), can have an effect on the power achieved.
Power calculations should precede the initiation of a preventive intervention trial to determine the requisite sample size (Muñoz, 1993). For example, for a universal preventive intervention trial targeting the general population with a short follow-up period to measure the onset of a disorder that has a low baseline frequency and unreliable diagnosis, having one million participants may not yield adequate power to detect statistically significant effects. On the other hand, for a trial of a potent selective preventive intervention sampling a relatively high risk population and using frequent, repeated measurements that are valid and reliable, with a long follow-up period and good retention of subjects, a sample size of 50 per group might be adequate. It is important to keep in mind that false positives will always be more frequent with small samples than with large ones. The issue, then, is not only how many participants to use, but also how to design the trial to get the greatest power within the limits of the trial's feasibility. Once issues of feasibility and likelihood of effects being found are determined, standard power calculations (Cohen, 1988) can be used to determine the number of participants that are needed.
For the committee's examination of preventive intervention research programs, it compiled a list of criteria, which appear in Chapter 7, to be
used in identifying-research programs of particular merit. In documenting research programs in the future, the investigator may find these guidelines useful. But an even higher standard will be desirable in the next decade of preventive intervention research. For example, efforts will need to be made to assess costs and benefits in a realistic way (see the section on economic issues below).
When the research program has been completed, the design, sampling, measurement, and analytic decisions should be specified in the peer-reviewed literature and manuals in sufficient detail that they can be replicated by others. The background and rationale are also relevant. When the results have been analyzed, the statistical methods used should be reported in such a way that the proper inferences can be made about the effectiveness of the prevention program. Descriptive statistics can be used to describe the groups at baseline and to demonstrate the randomization of the groups. If the sample was stratified, descriptive data can be presented for each stratum.
Some details should be presented about how many participants were recruited, how many screened, how many passed and failed that screening (and why), how many consented (and why refusals occurred), how many of those who consented were actually randomized (and why some were omitted), and, of those randomized, how many entered their assigned groups (and why others did not). Of those who entered the randomized groups, how many completed the follow-up (and why did others not)? Of those who dropped out of the experimental and control groups, how long did they last in the protocol? What baseline factors were associated with dropout, and were they the same in the experimental as in the control group? How many of those in the experimental and control groups complied with the protocol (and why did others not)? In short, any information on the sample pertinent to sampling bias, measurement bias, or any other type of bias should be presented so that readers can judge how convincing the results are. A flowchart format can sometimes present these data clearly and efficiently.
There should be brief descriptions of the protocols for recruitment, retention, experimental and control delivery, and of measurement. Documentation of the quality of measurement (reliability or validity) is always valuable in aiding judgments of the results.
The estimated survival curves or hazard curves (or both), in addition to simple summary statements of statistical significance, are valuable in assessing the size and hence the clinical or policy importance of statistically significant results. They are also essential in assessing whether nonsignificant results are the result of low power and thus worth further pursuit or the result of ineffective preventive intervention
and thus not worth further consideration. If the secondary results prove informative, these might be documented with separate survival or hazard curves for subgroups found substantially different in response or for subgroups substantially different in terms of process (such as compliant versus noncompliant subjects).
ISSUES OF CULTURE, ETHNICITY, AND RACE
As discussed earlier in this chapter, the success of preventive interventions —whether at the level of the individual, family, community, or nation —depends heavily on the contexts in which they are delivered. Clearly, anticipating the social and cultural elements of these contexts and accounting for such elements in terms of content, format, staffing, and implementation are critical to subsequent outcomes. Given the cultural diversity that characterizes this country, no discussion of the current and future status of preventive intervention research is complete without systematic attention to culture, ethnicity, and race (Muñoz, Chan, and Armas, 1986).*
Throughout the preventive intervention research cycle, investigators must be sensitive to the attitudes, values, beliefs, and practices of the cultural groups with whom they are working, as matters of good science and therapeutic leverage, as well as professional ethics (Kavanagh and Kennedy, 1992; Locke, 1992; Vega, 1992; Galanti, 1991). However, they must strive for more, namely, a set of skills and a perspective that have become commonly known as cultural competence (Isaacs and Benjamin, 1991; Cross, Bazron, Dennis, and Isaacs, 1989; Lefley, 1982).
Sensitivity and competence can be conceptualized as existing along a continuum (Orlandi, 1992). Cultural sensitivity is the awareness of a body of important information relevant to the population(s) of interest, which should inform the entire research process, from defining the sampling frame, through negotiating access, to actual intervention and dissemination of results. Such sensitivity can be, and typically is, learned through formal, didactic means and by familiarity with the rapidly growing literatures. It is a necessary but insufficient condition for cultural competence.
Competence is achieved through personal experience, either closely supervised practice or actual immersion in the field, which leads to the
*This section is based, in part, on discussions at a one-day meeting, convened by the Institute of Medicine, that focused on preventive intervention research issues related to special populations. (See Appendix C for participants at this meeting.)
acquisition and mastery of the skills needed to fit intervention to context. Being competent involves employing the means by which to improve the probability that this fit will occur, that the prospective participants will embrace it, that the gains will be truly useful and valued, and that their diffusion to other potential beneficiaries will be maximized.
The evidence is increasingly clear in regard to the link between cultural competence and the success (or failure) of preventive intervention research and programming (Orlandi, 1992; Neighbors, Bashshur, Price, Selig, Donabedian, and Shannon, 1991; Manson, 1982). Thus some have argued for extensive retooling of graduate curricula in the social, behavioral, and health sciences to emphasize such skills (Bolek, Debro, and Trimble, 1992). Though crucial to the evolution of sound preventive intervention research over the long haul, an immediate emphasis should be to encourage investigators to obtain first-hand experience and to collaborate closely with other professionals, perhaps themselves members of the communities of interest. Those who accept this challenge will undergo a process of discovery, gaining insight into the equivalents between their own cultural framework (professional as well as personal in origin) and that of the people with whom they work. Ultimately, cultural competence specific to prevention will emerge through the conduct of preventive interventions themselves, critical reflection on what does and does not work, and constructive exchanges intended to capitalize on success and reduce the likelihood of future failure.
The committee identified a number of points throughout the preventive intervention research cycle at which issues of cultural competence become especially salient and must be addressed. Several examples illustrate the nature and frequency of such occurrences:
Forging relationships between researchers and community. Prevention research, typically conceptualized within and springing from academic settings, needs to be married with indigenous efforts, particularly those mounted by leaders of the communities of interest. The notion of prevention strikes a resonant chord in most ethnic minority communities. Considerable activity already is under way that is consonant with the intrinsic goals of prevention and with the quest for more knowledge about how best to achieve these goals. Sometimes those community leaders may themselves function as senior investigators. More often, they broker interactions between researchers and participant communities, facilitating the accommodations required to blend their separate strengths and resources. In order for this process to work,
traditional power relationships must be transformed. Mutual respect, appropriate responsibility, equity in decision making, and shared commitment to negotiating differences are central to that transformation. Lacking these conditions, the ensuing power differential can become a major barrier to the research process.
Identifying risks, mechanisms, triggers, and processes. Attempts to understand risk and protective factors, triggers, and processes regarding the onset of a disorder or problem should allow for the possibility of alternative explanation and circumstances among cultural groups (Neighbors, 1990). For example, risk factors can be unique to a specific population. Consider Levy and Kunitz's (1987) inquiry into suicide among the Hopi. They observed that rates of suicide are high not only among Hopi in “progressive villages” and off-reservation border-towns, but also in traditional villages. Specifically, Hopis at increased risk for suicide include the children of parents who entered into traditionally disapproved marriages, such as across tribes, mesas, and even clans of disparate social status. The labeling of parents as “deviant” in this regard stigmatizes their children, thereby engendering a distinct series of stressors. Typical inquiries about marital status would not have arrived at this discovery, missing an important, systematic risk for suicide and related mental health problems.
Employing relevant theoretical frameworks. Choosing theory to guide the intervention entails more than attending to the presumed links between cause and effect. It also must accommodate the relationship between what participants will (or are expected to) learn and those things valued by them. Different groups of people have different attributional styles, even different assumptions about consequences and costs. Numerous illustrations of the kinds of problems that can be caused by these differences can be found in Paul's (1955) book Health, Culture, and Community: Case Studies of Public Reactions to Health Programs. The best is the description by Wellin (1955) of what it took to mount a successful health intervention program in Peru. The problem was dysentery and typhoid; the solution was for people to boil their own water. The difficulty, however, was that the Peruvian community in question lacked an understanding of germ theory. Moreover, the interveners never anticipated that obtaining wood to fuel the fires necessary to boil water was difficult and costly. Analogous examples, both more current and closer to home, include similar disjunctions between conventional intervention models and health beliefs about hypertension among African-Americans (Dressler, 1987), intravenous drug use and HIV infection among African-Americans and Hispanics (Page, Chitwood, Smith, Kane, and McBride, 1990; Singer, Flores,
Davison, Burke, Castillo, Scanlon, and Rivera, 1990), and alcohol abuse among Native Americans (Thurman, Jones-Saumty, and Parsons, 1990; Walker, Walker, and Kivlahan, 1988).
Preparing the content, format, and delivery of preventive interventions. Individuals and groups are adapted to ethnocultural niches, defined in familial, social, political, and economic terms. To ignore the historical and evolving nature of these niches is to court failure, if not disaster, for prevention research. There is ample documentation that unintended negative effects can accrue from cultural insensitivity and incompetence (McCord, 1978). Thus considerable effort should be expended to inform intervention efforts along these lines. Pilot work and pretesting ought to accompany all attempts to transfer prevention technology across cultural boundaries. In this regard, ethnography stands out among the available methods (Trotter, Rolf, Quintero, Alexander, and Baldwin, in press; Gilbert, 1990; Montagne, 1988). For example, anthropological field work with Native Hawaiian children revealed that peer assistance was important in children's daily activities, that learning occurred most often in “child-constructed” contexts, and that children were seldom individually directed and monitored by adults (Gallimore et al., in press). These observations shaped a remedial school program, aimed at improving the educational attainment of Native Hawaiians, that encouraged peer teaching in independent learning centers in which groups of three to seven children studied together. This design of the educational environment resulted in higher frequencies of on-task behavior, peer assistance, and work completion in contrast to other classroom designs (Tharp and Gallimore, 1988). Hence the infusion of local knowledge permits social validation of intervention goals and procedures, enhances compatibility with valued ends, and increases participation.
Adopting appropriate narrative structures and discourse. What people are willing to discuss, how they talk about it, and with whom they share certain matters vary in important ways across ethnic minority and cultural groups (Kleinman, 1988, 1980). For example, there often is a great deal of discomfort in Hispanic families with respect to sexual issues (Marin and Marin, 1991). Thus an AIDS prevention program intended to reach Hispanics must recognize that discomfort talking about sexual behaviors is normative and approach the topic carefully or risk losing audience participation. How individuals refer to and discuss a given disorder or problem also can differ markedly by ethnicity and culture. “Down in the dumps,” “feeling blue,” and “feeling low”—idioms, metaphors, and labels commonly used in white, middle-class America to refer to depression—do not have the same currency among
Asian-Americans (Kinzie, Manson, Do, Nguyen, Bui, and Than, 1982) or Native Americans (Manson, 1993; Manson, Shore, and Bloom, 1985). Likewise, there are differences across ethnic groups in regard to the choice of person in whom to confide specific thoughts or feelings. African-Americans are much less open to discussing issues of shame and anger stemming from racial discrimination with persons whom they perceive as unlikely to have shared similar experiences and who are thus, presumably, less empathic (Neighbors et al., 1992; Jones and Matsumoto, 1982; Jones, 1978). For slightly different reasons, many Native Americans will not volunteer their beliefs about witchcraft as the cause of mental health problems, in part because such matters typically are kept secret from outsiders and, when disclosed, often meet with disdain or ridicule (Manson et al., 1985).
Tapping critical decision-making processes. The means by which decisions are processed and the locus of responsibility for decision making can be quite different from one ethnic or cultural group to another. For example, nearly one third of Native American tribes explicitly invest a great deal of authority in women, largely as a function of the matrilineal structure of their social organization. Indeed, in some of these tribes, at the time of marriage, men will relocate from their parental homes to those of their wives and their wives ' mothers. Decisions, then, that bear directly on the economics of the household, the allocation of family time and resources, and other life-style matters typically are more the responsibility of the women, of both generations, than the men. Though their role has been slightly eroded, family councils among Hmong refugees, who relocated to the United States after fleeing from Vietnam, remain central to conflict resolution and mediation of domestic disputes (Norton and Manson, 1993; Bloom, Kinzie, and Manson, 1985). Imagine the probability of success of preventive interventions that ignore, or even run counter to, such decision-making processes.
Determining points of intervention leverage. Just as it shapes decision-making processes, social structure also influences access to people and defines points of leverage for subsequent intervention. A home visiting model for prevention, akin to that employed by Olds (see Chapter 7), would need, for example, among the Navajo, to take into account seasonal household migration and concomitant shifts in caregiving responsibilities (Dinges, 1982). At winter residences, maternal grandmothers, who produce many of the crafts for market, assume a central role in childrearing as a consequence of their sedentary activities. By contrast, during summer months, and being free from school, older female siblings are responsible for caring for younger children at the
large, extended family camps located deep in the foothills, close to traditional grazing lands. Not only does the configuration of the family, and the corresponding roles, change, but so do developmental tasks for young children. Projecto Bienestar, as described by Vega and colleagues (see Chapter 7), which focused on Mexican-American women at high risk of depression, paid close attention to similar elements of household composition and residential patterns, facilitating access to these individuals and enhancing their participation in the intervention.
Recognizing social networks and natural helpers. Projecto Bienestar (Vega, Valle, and Kolody, submitted for publication; Vega and Murphy, 1990; Vega, Valle, Kolody, and Hough, 1987) also underscores the importance of the resources and strengths that exist in all communities, but which are accorded special prominence in ethnic minority groups. As noted in Chapter 7, natural helpers (Servidoras) were employed to deliver the intervention. This approach was congruent with the local social ecology, reduced the social distance that often separates participant from intervener, minimized errors due to cultural incompetence, and increased the likelihood that intervention skills and knowledge would be dispersed beyond the intended audience and continue to be delivered past the funding of the program. This same lesson is evident in reports of prevention programming among Southeast Asian refugees (Bliatout, Rath, Do, Kham One, Bliatout, and Lee, 1985; Le Xuan and Bui, 1985; Lum, 1985; True, 1985), Asian-Americans and Pacific Islanders (Murase, Egawa, and Tashima, 1985), other Hispanic populations (Szapocznik and Kurtine, 1993; Bestman, 1986), and Native Americans (Manson and Brenneman, in press; DHHS, 1990).
Seeking fidelity of implementation. Many factors impinge on the fidelity with which an intervention is delivered, as it moves from highly controlled tests of efficacy to effectiveness and from application in one community to another. An investigator struggling with this issue asks (or is asked): Does what has been done or must be done in order to deliver the intervention compromise its integrity, and, thus, its comparability to the original model? Service providers are seldom concerned with such questions; their mandate is to “make it work” in terms of the clients' best interests. No simple answers emerge for the researcher. Clearly, the researcher has license to use the cultural equivalents of the settings, units of intervention, key constructs, attributional processes, means of instruction, and reinforcement strategies that were used in the original model. But he or she then must carefully document the nature of these equivalences, how they were determined, and what forms they take subsequently in the intervention. In many instances, the adaptations will pose little threat to fidelity. For example, supplementing
written curricula with audio-taped versions for literacy-limited older Native American participants in a cognitive-behavioral intervention to prevent depression (Manson and Brenneman, in press) represents a minor deviation from the “Coping with Depression Course” developed by Lewinsohn, Clarke, and Hoberman (1989). Delivering the course through the continuing adult education division of tribally controlled colleges (Gallagher and Thompson, 1983) is another minor adaptation of the intervention. However, reworking the language of the intervention by substituting local idioms and metaphors for depression and drawing analogies between progressive relaxation techniques and indigenous forms of meditation carry the adapted intervention another step away from the original model. Here again, extensive documentation of effectiveness is essential.
Replicating interventions across diverse and changing populations. The cross-cultural literature on prevention illustrates the great diversity among and within ethnic minorities. For example, some Asian-Americans have lived in the United States for four to five generations, although most, over 50 percent, have been here for fewer than 10 years. Among these individuals, then, language, including English fluency, varies significantly. Indeed, according to the 1990 census (U.S. Census Bureau, 1990), from 24 to 76 percent of the various Asian-American groups do not speak English “very well.” A dozen or more distinct groups of Asian-Americans—Japanese, Chinese, Koreans, Vietnamese, Cambodians, and Hmong to name but a few—make up this special population, speaking more than 75 different languages. Similar diversity is evident among African-Americans, Hispanics, Native Americans, and Native Alaskans. Given such variability, interventions cannot be assumed to transfer easily across these lines, any more so than their original extension from the population for whom they were first designed.
Many investigators find the challenge of accommodating diversity of this nature and magnitude almost too daunting to consider. But the quest for cultural competence in this field carries with it an excitement. It promises to enrich the developing science of prevention and yield fulfillment from having successfully bridged differences thought unspannable.
All types of scientific research face challenges in ensuring that their activities adhere to fundamental standards of integrity. Many of these
complex issues are reviewed in Responsible Science: Ensuring the Integrity of the Research Process (NAS, 1992). Policy recommendations were made to help scientists, research institutions, and government agencies protect the scientific process and minimize scientific misconduct, defined as “fabrication, falsification, or plagiarism in proposing, performing, or reporting research” (NAS, 1992, p. 27). This broad range of issues is applicable to prevention research, but the more narrow concern in this section is the ethical issues regarding the participants in preventive intervention research programs.*
Three factors combine to complicate the ethical issues involved in research on the prevention of mental disorders. First, the various disciplines and techniques that are being integrated into prevention research programs each carry their own complex ethical issues. Second, prevention research programs conducted in communities often require commitments, promises, and risks not encountered in basic research. Third, in many cultures mental disorders carry a special stigma.
The development of a specific ethical code for prevention research is premature and perhaps not even desirable. What is needed, however, is a sensitivity on the part of the individual investigator, and of the research community in general, regarding the importance of ethical issues throughout the preventive intervention research cycle. Investigators individually and as a field must recognize these issues in changing circumstances and respond responsibly—with appropriate questions, skills, and decisions. The development of this competence begins with formal training in basic principles but then requires a continuing process of self-education to instill the habit of ethical accountability.
We can begin to think about ethical issues regarding participants in research on the prevention of mental disorders by drawing on considerations from ethics in other areas of clinical research. Several professional associations and the federal government have developed guidelines that apply to research with human subjects. The federal government has a formal policy for the protection of human subjects in all research projects (DHHS, 1991); therefore this policy automatically applies to participants in preventive intervention research projects. Institutional Review Boards (IRBs) at the researcher's home institution must follow certain requirements for approval of research. These requirements include assurances that
*This section is based, in part, on discussions at a one-day meeting, convened by the Institute of Medicine, that focused on ethical considerations in preventive intervention research. (See Appendix C for participants at this meeting.)
risks to subjects are minimized,
risks to subjects are reasonable in relation to anticipated benefits,
selection of subjects is equitable,
informed consent is sought,
informed consent is appropriately documented,
data collection is monitored to ensure the safety of subjects, and
adequate provisions are made to protect the privacy of subjects and to maintain the confidentiality of data.
Additional requirements of the IRBs are made to ensure the rights and welfare of subjects who are likely to “be vulnerable to coercion or undue influence,” such as children, pregnant women, and economically or educationally disadvantaged persons—all potential targets for preventive interventions (DHHS, 1991, p. 6).
In addition to being bound by the values and standards that guard scientific integrity and by the IRB policies, preventive intervention research programs must address numerous specific ethical issues regarding participants. Pope (1990) has suggested several general guidelines for investigators to routinely consider:
Do no harm. It is true that for some individuals, families, and communities, under some circumstances, being involved in prevention programs may have harmful proximal or distal consequences (Lorion, 1987). Pope stressed that focusing on the supposed welfare of the many cannot justify the harm that may come to the few. The importance of this issue is heightened by the fact that preventive interventions, by their very nature, involve people who are deemed “at risk” yet may not ever exhibit the disorder or behavior in question. But what standards of evidence are to be used in determining who is “at risk,” especially in view of the fact that the importance of specific risk factors is not always well known.
Withholding a preventive intervention that is thought to be efficacious, as can happen in a randomized controlled trial design, may also be thought of as constituting inhumane treatment. This is not a problem early in the research cycle, when the investigator has little scientific evidence for believing that a particular prevention program does what it is intended to do, that is, reduce risks and maybe even lower incidence. Also, the preventive intervention is not like treatment, which is an attempt to deal with a current mental disorder that needs attention. However, the ethical dilemma of a randomized controlled trial design becomes more compelling at the stage of large-scale field trials, when the documentation of evidence from confirmatory and replication trials is encouraging (Muñoz and Ying, 1993). One way of dealing with this
dilemma is to have field trials in whole communities or neighborhoods, with control groups being entirely from communities where there is no access to the prevention program (CBASSE, 1993; Seitz, 1987; Cook and Campbell, 1979). Another way of dealing with this problem is to offer the intervention to the waiting list control group after the study has been completed if the results favor the experimental intervention.
Systematic methods must be developed for identifying the possible harm that may occur as a result of a particular intervention or failure to act. Although the methods used to identify possible harm will differ according to the population being targeted, the strategies of intervention being considered, and the problems being addressed, Pope has identified several crucial questions to be considered. First, what are the possibilities for direct harm to people participating in the intervention or to those indirectly affected, such as families, peers, or neighbors? This includes emotional, developmental, physical, interpersonal, educational, economic, and a host of other potential damages. Second, to what degree can delayed damages be anticipated? Because it is hoped that preventive interventions will produce positive long-term effects, it is reasonable to assume that harmful consequences also may be passed along, perhaps even to future generations. Third, if a community is the focus of an intervention, in what way might it be disrupted? Will its natural leadership and social cohesiveness be disturbed? If the group of prevention researchers differs significantly from the community in terms of such factors as social class, race, religion, or politics, what will be the effect on the community? What is informed consent for a community? Fourth, what effects that might be considered harmful could occur in the context of the community 's relationship with the larger society? For example, in a project intended to minimize the incidence of serious depression, anxiety, and impulsive behavior in an economically disadvantaged neighborhood, it could be argued that some of these conditions are natural responses to severe poverty. Helping residents learn to adapt to desperate conditions might merely serve as a subtle, perhaps unintended, form of social control that could hinder the community's ability to identify the true source of its suffering and take effective action (Pope, 1990). And finally, what are the possibilities that effective prevention of one outcome will make another outcome worse?
Practice with competence. In the early stages of a developing field, new approaches and techniques are frequently formulated, applied with enthusiasm, and then discarded when they prove unproductive. However, prevention is moving beyond this point and now possesses a growing body of knowledge and a set of skills that are prerequisites to action. Researchers can accomplish much by carefully analyzing a
proposed intervention to identify the information and areas of expertise likely to be necessary for the competent planning, initiation, management, and evaluation of that project. Because prevention programs focus on individuals, families, and communities—involving economic, political, psychological, social, educational, religious, and other diverse aspects—an interdisciplinary team approach may be a necessity, with each member bringing complementary sets of competences. Without such interweaving of disciplines, important social or scientific facets may go unexamined, possibly leading to unexpected harmful results.
Do not exploit. Researchers are often in a position of power relative to the program's participants, and there are numerous ways this power may be abused. In conducting an intervention effort in an economically disadvantaged neighborhood, for example, researchers may spend many hours establishing relationships with community members, in the process learning “secrets” that would not be apparent to outsiders. If the “life stories” of the residents are used inappropriately—perhaps for professional advancement, with little regard for the impact on those who revealed the information in a bond of trust—both the scientific and the personal damages may be considerable (Pope, 1990). It is critical, then, that researchers be clear about their own contextual constraints before making promises to communities (Trickett and Levin, 1990). Time constraints, economic resources, planning for the research program's termination—all of these issues carry ethical implications for how an intervention program is presented and whether or not it is conducted in good faith.
Treat participants with respect and dignity. At heart, all ethical principles flow from respect for the human dignity of others. Pope maintained that “scrupulous attention must be paid those aspects of our education and training, of our institutional structures, of the language and content of our theories, research, and interventions which tend to diminish the degree to which we can appreciate, hold as fundamentally important, and respond appropriately to the full human dignity of others. When the people who are affected by our primary prevention strategies become primarily known as ‘research subjects,' ‘populations-at-risk,' . . . case studies, etc., we may lose sight of the fact that they are all full human beings struggling with life just as we are” (Pope, 1990, p. 57). In addition, individual characteristics such as race, ethnicity, gender, social class, sexual orientation, and age determine, to a larger extent than is commonly acknowledged, how interventions are conceived and implemented. The ethical ramifications of how this information is used need to be considered for each intervention.
Protect confidentiality. Preventive interventions often involve col-
lection of extremely sensitive and potentially damaging information, such as narcotic use, frequently from large numbers of people. Researchers must identify the sorts of information that are or ought to be treated as confidential. This process includes, but is not limited to, checking carefully any legislation or case law that might govern confidentiality of such things as school records, membership lists of civil or religious organizations, and personal information obtained through interviews. The next step is to ensure that all parties involved or likely to be affected by the intervention clearly understand the boundaries between what is and is not confidential. In addition, the researchers must develop effective mechanisms for handling documents to safeguard their privacy and must develop procedures (including training procedures) to guard against accidental disclosure of confidential information through casual conversation or other means outside the scientific process.
Confidentiality poses especially difficult ethical concerns when researchers discover illegal activity such as evidence of violence, including child maltreatment, which under various state laws may have to be reported to the appropriate social welfare or law enforcement authorities. This combination of circumstances raises serious questions for those who consider as inviolable the trust and confidentiality between researcher and participant (Sieber, 1993). Should the researcher stop and warn a participant who starts to mention abuse? Should the researcher take what is reported and follow the law or seek ethical or legal loopholes to avoid the law? Should the researcher actively seek and report evidence of abuse and neglect? How much professional discretion should the researcher exercise in deciding what degree of abuse requires reporting, in relation to the likely outcome of reporting for the researcher, the project, the institutions involved, the person suspected of committing the abuse, and the person being abused?
Researchers involved in socially sensitive studies can obtain certificates of confidentiality that may preempt state reporting requirements and court subpoenas. They are available from the Department of Health and Human Services and the Department of Justice. Through continued examination of issues of confidentiality, on which opinions now vary considerably, researchers may identify ways to better ensure the efficacy and effectiveness of preventive intervention research programs while exercising responsibility for the welfare of victims and rehabilitation of perpetrators of violence.
Obtain informed consent. An individual's right to understand a proposed intervention and to freely give or refuse consent to participate stands as a cornerstone of research with human subjects. Weithorn
(1987) has reviewed the ethical issues and federal regulations pertaining to informed consent for prevention. Research with children presents special problems. Children may be more susceptible to harmful consequences, such as stigma and labeling, of research participation; they also may be limited in their ability to provide informed consent because of their immature psychological functioning, limited understanding, and their dependent status as minors. Researchers should not confuse the ability of a child to understand procedures when explained with the more complex and mature cognitive task of making the comparative and contingent risk assessment judgments that are required in informed consent.
The ethical dilemmas that arise regarding informed consent often focus on questions of when and from whom to obtain consent, and what types of information to disclose in the consent process. For example, investigators are required to disclose to prospective participants the purposes and procedures involved in the research, possible risks or discomforts, expected benefits to the participants or others, and appropriate alternatives that might be beneficial. Federal laws, however, allow for less than complete disclosure under certain circumstances, although the boundaries are not always clearly defined. Weithorn presented some of the difficulties for a hypothetical study designed to prevent psychopathology in the children of schizophrenics. What should parents and children be told about the reasons for their inclusion in the project? A completely open disclosure would inform them, for example, that some percentage of the children of schizophrenics are likely to develop serious psychological disorders based on their membership in this risk group (Weithorn, 1987). But many investigators fear that such full disclosure would be harmful to the families, causing additional psychological trauma. Moreover, labeling of the group as “at risk” could possibly alter the behavior and expectations of family members and others, placing the child at further risk for the development of problems. A similar, and perhaps even more traumatic, disclosure would be the identification of a person found to be at risk of a mental disorder because of possession of a certain gene, a possibility in the not-too-distant future.
Weithorn (1987) concluded that prevention researchers should regard the federal regulations covering informed consent as an important guide, but view them as the minimum adequate standards and not as a substitute for careful reference to the moral principles that underlie them.
The principles of informed consent are especially difficult to apply when a universal preventive intervention involves an entire community. For example, a minority member in the community may oppose the
project, yet nonetheless may be subjected to its indirect effects. Methods of recognizing and weighing the rights of these individuals are needed.
Promote equity and justice. At a minimum, preventive interventions must not directly affirm or contribute to inequality or injustice. But a truly ethical approach must go beyond the minimum, to active promotion of equity and justice (Pope, 1990). As with the other ethical areas, there are no simple answers concerning how to enhance the general social well-being. Keeping this goal clearly in mind, however, will help ensure that ethical dilemmas are not ignored or discounted, and that researchers and institutions explicitly attend to them.
To identify and attempt to resolve the various ethical issues destined to arise in preventive interventions, researchers will often be best served by joining with the members of the community—the providers of data and the targets of interventions (Trickett and Levin, 1990). As Conner (1990) has pointed out, ethical issues often involve trade-offs among competing sets of values that may or may not be shared between researchers and community members. He concludes that clients should play a central role in planning, implementing, and evaluating prevention programs.
Rather than simply moving into a community and implementing large-scale prevention programs, researchers might announce their plans and then formally listen to the community's response—including the response from broader constituencies than the program's direct participants—before proceeding (Pope, 1990). In a prevention program for children, for example, Trickett and Levin (1990) note that parents might assume a variety of roles that could increase the sensitivity of researchers to potential ethical issues and provide a forum for their resolution. Also, involving administrators in the planning of a school intervention program may lead to a commitment for continuing the program beyond its externally funded demonstration phase—itself an ethical issue.
Although there is consensus about the importance of ethics in prevention research on mental disorders, ethical accountability in this area has not yet received the emphasis it deserves. This remains a developing field, in need of increasing numbers of individuals and organizations possessing heightened sensitivity to ethical concerns and new skills for designing and conducting ethically appropriate intervention programs. Raising these ethical issues is not to suggest that initiation of preventive interventions should await unanimity on the goals and methods (Lorion, 1987). Rather, careful identification and analysis of these issues can be useful in broadening scientific perspective
and in increasing awareness of the potential range and impact of unintended consequences.
The allocation of available resources to activities aimed at reducing the burden of mental problems in our society requires some capacity to estimate the benefits and costs of our efforts. Analysis of costs and benefits can help inform decision makers about which kinds of interventions for mental disorders hold the most promise for yielding net benefits. The basic goal is to reach a decision on whether a particular intervention program is worth undertaking or whether an existing program should be discontinued, expanded, or reduced in scope.
Analysis of benefits and costs seeks systematically to identify and measure all the benefits and costs of a program (Muñoz and Ying, 1993). Obviously, if benefits exceed costs, then society profits from having the program available. On the other hand, if costs exceed benefits, a decision to allocate scarce resources to other purposes would seem warranted. There are two main methods for doing such an analysis. In cost-benefit analysis, costs and benefits are expressed in dollars. This process is adequate and straightforward for some measures, but it means that dollar amounts have to be assigned to all important outcomes. Assigning dollar amounts is difficult or nearly impossible for some measures, such as life and health. Cost-effectiveness analysis, on the other hand, avoids some of the above controversy by using two categories of outcome measures—dollars and health outcomes. Health outcomes can be presented, for example, in “years of healthy life gained.”
The assumption is often made that preventive efforts are cost-effective. Some programs provide evidence that this may be true, but much more confirmation is desirable. The following useful framework is adapted from work by Russell (1986) for analyzing the cost-effectiveness of potential preventive efforts:
Population and risk. The aggregate or net cost of an intervention depends on the size of the targeted population relative to the number of persons in the population who would be likely to develop the negative outcomes without the intervention. Even interventions that are relatively inexpensive per person may be quite costly in the aggregate if the target population is large and the number of persons in the population at risk is small. The more specific the definition of risk groups, the more likely the intervention will be cost-effective, other factors being equal.
Cost and frequency of administration of the intervention. Preven-
tive interventions vary greatly in their cost. School curriculum innovations, for example, may be relatively inexpensive. Schools can replace ongoing programming with much more effective curricula at relatively little new cost in time or effort. On the other hand, intensive personal interventions using expensive professional personnel can be quite costly per person served. The cost of the intervention depends also on start-up costs, the size of the sample, and the required frequency of contacts with participants; as observed in Chapter 7, many preventive interventions require multiple contacts over an extended period of time. Thus the cost of an intervention is a product of its initial cost and the frequency of administration.
Potency of the intervention. Interventions vary in the power of their effects. The efficiency of an intervention—as measured by the proportion of those at risk affected and the size of the effect—should be taken into account in evaluations of cost-effectiveness.
Uncertainty of risk. For many problems and disorders the causal status of associated risk factors, and therefore the benefits from interventions targeted toward reducing those risk factors, remain uncertain, as do possible adverse events. When the risk of developing the disorder or problem is low and uncertain, thought must be given to the costs associated with exposing large populations to interventions that offer no advantages to most and possible adverse effects to some.
Time. Another relevant consideration is the temporal proximity of the result of the intervention to its administration. Benefits are much greater for interventions that bring quick and persisting results than for those with delayed results or results that lessen over time.
Once the preventive intervention moves into the community for the large-scale field trials, as well as on into the service realm, the costs and benefits may fall on different segments of society. The service agency or organization may be willing to share part of the cost. To the extent that a service agency assumes the cost for a preventive intervention, the cost-effectiveness of the intervention should be measured against the cost-effectiveness of other potential services. Alternatively, the preventive intervention may be carried as a public service announcement or administered by a community volunteer agency. Even though there may not be direct costs for the health sector or service agencies, these interventions would still entail costs in that they compete for alternative uses of the resources.
Many potential preventive intervention research programs are consistent with the policy priorities for school districts, and the classroom and teacher resources required to conduct them could be achieved at
relatively low cost. On the other hand, if the program requires the hiring and training of a new cadre of mental health worker—or even the retraining of existing personnel—the costs of the program could be so great as to compromise its viability.
The cost of prevention programs stems not only from the monetary cost of the intervention, but also from the potential costs involved for those receiving the service. This response cost or burden includes the participant's having to travel to a specific location, pay for travel or time lost for other activities, and undergo uncomfortable or time-consuming procedures, as well as the overall amount of effort involved in receiving the intervention. Perceived response cost may vary across sociocultural groups, even if monetary costs remain the same. For example, accepting “free” babysitting services while receiving a preventive intervention may be unacceptable to those cultural groups for whom sharing the childrearing responsibility is considered unwise or dangerous.
Prevention programs should not be funded by withdrawing resources from needed, and usually underfunded, treatment services (see Chapter 2 for definitions). Even though preventive interventions may have a significant impact in terms of increased socialization and reduced psychopathology, cognitive impairment, and psychosocial dysfunction, they are unlikely to result in an immediate reduction in the need for treatment interventions. In part, this is because those currently being treated are a small proportion of those suffering from a disorder.
Cost-benefit and cost-effectiveness analyses appear only infrequently in the treatment literature (Cardin, McGill, and Falloon, 1985; Weisbrod, Test, and Stein, 1980; Paul and Lentz, 1977) and are almost nonexistent in the prevention literature. As requests for these analyses increase over the next decade, there are several important points that policymakers and prevention researchers should consider (Gramlich, 1984). For example, benefits from prevention programs may increase over time. Short-term evaluations may show small or nonexistent benefits, but benefits may accrue as children are engaged over time in less crime, depend less on welfare, or begin to reap the benefits of higher levels of educational achievement. In addition, a well-designed and sensitive benefit-cost analysis can identify gainers and losers in society. Net social benefits may be received by participants in the program themselves, taxpayers, and potential victims of crime. Evaluation can show who gains and who loses as well as how big the overall gain or loss actually is.
One example of a pioneering benefit-cost analysis of a prevention program is that done for the Perry Preschool Program, a selective preventive intervention (Berrueta-Clement, Schweinhart, Barnett, Epstein, and Weikart, 1984). (See Chapter 7 for a description of the
program.) The investigators documented the costs of high-quality preschool education and the benefits resulting from positive program outcomes. Results suggested that the total net benefit to preschool participants themselves was approximately $5,000. On the other hand, the total net benefit to taxpayers and potential crime victims was estimated at around $23,000 for one year of preschool by the time the program recipients reached 19 years of age. As the authors observed, “changes in economic success, self-sufficiency, and social responsibility can be predicted quantitatively from observed effects at age 19” (Berrueta-Clement et al., 1984, p. 89).
Another example of a benefit-cost analysis of a preventive program is that done for the JOBS Project for the Unemployed, a selective preventive intervention aimed at helping the recently unemployed find new employment (Vinokur, van Ryn, Gramlich, and Price, 1991). (See Chapter 7.) Results from a 2½-year follow-up of participants in a randomized field experiment that included a jobs program aimed at increasing reemployment and preventing poor mental health outcomes showed the continued beneficial effects of the intervention on monthly earnings, quality of reemployment, and episodes of employer and job changes. Results of a benefit-cost analysis demonstrated large net benefits of the intervention not only to the participants, but also to federal and state governments, based on increased tax revenues produced by reemployed workers in the randomized trial. Not only did the benefits of the program exceed all costs within less than two years, but, because the wage differences appeared to persist, the benefits were expected to continue to accumulate over many years. According to the researchers' estimates, by the time experimental group participants reach age 60, they can be expected to have accrued $48,151 more benefit per person than their counterparts in the control groups, assuming they continue to be employed.
If the research standards and methodology outlined here are systematically and rigorously applied within the preventive intervention research cycle and the guidelines on cultural, ethical, and economic issues are carefully considered at each step, prevention research will yield progressively more powerful results over the next decade. The ensuing development of prevention into a science will provide a firm base of knowledge for policymakers. This knowledge will inform their decisions on the allocation of available resources toward the ultimate goal of realizing the opportunities presented by the science for the alleviation of
the personal and societal suffering and burdens associated with mental disorders.
Alkin, M. C. ( 1985) A Guide for Evaluation Decision Makers. Beverly Hills, CA: Sage Publications.
Barlow, D. H.; Hersen, M. ( 1973) Single-case experimental designs: Uses in applied clinical research . Archives of General Psychiatry; 29(3): 319–325.
Berrueta-Clement, J. R.; Schweinhart, L. J.; Barnett, W. S.; Epstein, A. S.; Weikart, D. P. ( 1984) Changed Lives: The Effects of the Perry Preschool Program on Youths Through Age 19 (High/Scope Educational Research Foundation, Monograph 8). Ypsilanti, MI: High/Scope Press.
Bestman, E. ( 1986) Cross-cultural approaches to service delivery to ethnic minorities: The Miami Model. In: M. Miranda and H. Kitano, Eds. Mental Health Research and Practice in Minority Communities: Development of Culturally Sensitive Training Programs. Rockville, MD: National Institute of Mental Health; DHHS Pub. No. (ADM) 86–1466: 199–226.
Bliatout, B. T.; Rath, B.; Do, V. T.; Kham One, K.; Bliatout, H. Y.; Lee, D. T. ( 1985) Mental health and prevention activities targeted to Southeast Asian refugees. In: T. C. Owan, Ed. Southeast Asian Mental Health: Treatment, Prevention, Services, Training, and Research. Washington, DC: National Institute of Mental Health; DHHS Pub. No. (ADM) 85–1399: 183–207.
Bloom, J. D.; Kinzie, J. D.; Manson, S. M. ( 1985) Halfway around the world to prison: Vietnamese in Oregon's criminal justice system. International Journal of Medicine and Law; 4: 563–572.
Bolek, C. S.; Debro, J.; Trimble, J. E. ( 1992) Overview of selected federal efforts to encourage minority drug abuse research and researchers. Drugs and Society; 6(3/4): 345–375.
Campbell, D. T. ( 1991) Methods for the experimenting society. Evaluation Practice; 12(3): 223–260.
Cardin, V. A.; McGill, C. W.; Falloon, I. R. H. ( 1985) An economic analysis: Costs, benefits and effectiveness. In: I. R. H. Falloon, Ed. Family Management of Schizophrenia. Baltimore, MD: Johns Hopkins University Press; 115–123.
CBASSE (Commission on Behavioral and Social Sciences and Education) . ( 1993) Understanding Child Abuse and Neglect. Panel on Research on Child Abuse and Neglect, National Research Council . Washington, DC: National Academy Press.
Cohen, J. ( 1988) Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
Conner, R. F. ( 1990) Ethical issues in evaluating the effectiveness of primary prevention programs. In: E. J. Trickett and G. B. Levin, Eds. Ethical Implications of Primary Prevention. New York, NY: The Haworth Press.
Cook, T. D.; Campbell, D. T. ( 1979) Quasi-Experimentation: Design and Analysis Issues for Field Settings . Chicago, IL: Rand McNally.
Cross, T. L.; Bazron, B. J.; Dennis, K. W.; Isaacs, M. R. ( 1989) Toward a Culturally Competent System of Care: Vol. I. Washington, DC: Georgetown University Child Development Center.
DHHS (Department of Health and Human Services). ( 1991) National Institutes of Health.
Office for Protection from Research Risks. Code of Federal Regulations: Part 46—Protection of Human Subjects. Washington, DC: Government Printing Office.
DHHS (Department of Health and Human Services). ( 1990) Breaking new ground for American Indian and Alaska Native youth at risk: Program summaries. Rockville, MD: Office for Substance Abuse Prevention; (OSAP Technical Report 3); DHHS Pub. No. (ADM) 90–1705.
Dinges, N. G. ( 1982) Mental health promotion with Navajo families. In: S. M. Manson, Ed. New Directions in Prevention Among American Indian and Alaska Native Communities. Portland, OR: Oregon Health Sciences University; 119–143.
Dressler, W. ( 1987) The stress process in a Southern black community: Implications for prevention research. Human Organization; 46: 211–220.
Elandt-Johnson, R. C.; Johnson, N. L., Eds. ( 1980) Survival Models and Data Analysis. New York, NY: John Wiley and Sons.
Fergusson, D. M.; Harwood, L. J.; Lloyd, M. ( 1991) Confirmatory factor models of attention deficit and conduct disorder . Journal of Child Psychology and Psychiatry; 32(2): 257–274.
Galanti, G. ( 1991) Caring for Patients from Different Cultures. Philadelphia, PA: University of Pennsylvania Press.
Gallagher, D.; Thompson, L. W. ( 1983) Cognitive therapy for depression in the elderly: A promising model for treatment and research. In: L. D. Breslau and M. R. Haug, Eds. Depression and Aging: Causes, Care, and Consequences. New York, NY: Springer Publishing Company.
Gallimore, R.; Goldenberg, C. N.; Weisner, T. S. (in press) The social construction and subjective reality of activity settings: Implications for community psychology. American Journal of Community Psychology; 21.
Gibbons, R. D.; Hedeker, D.; Elkin, I.; Waternaux, C.; Kraemer, H. C.; Greenhouse, J. B.; Shea, M. T.; Imber, S. D.; Sotosky, S. M.; Watkins, J. T. (in press) Some conceptual and statistical issues in analysis of longitudinal psychiatric data. Archives of General Psychiatry.
Gilbert, J. ( 1990) Ethnographic research strategies in the cross-cultural substance use and abuse field. The International Journal of the Addictions; 25(2A): 123–148.
Gramlich, E. M. ( 1984) Commentary on Changed Lives. In: J. R Barreuta-Clement, L. J. Schweinhart, W. S. Barnett, A. S. Epstein, and D. P. Weikart, Eds. Changed Lives: The Effects of the Perry Preschool Program on Use Through Age 19. Ypsilanti, MI: Monographs of the High Scope Educational Research Foundation; 8: 200–203.
Hasenfeld, Y.; Furman, W. M. (in press) Intervention research as an interorganizational exchange. In: J. Rothman and E. J. Thomas, Eds. Intervention Research: Design and Development for Human Services. Binghamton, NY: The Haworth Press.
Hiltz, S. R. ( 1974) Evaluating a pilot social service project for widows: A chronicle of research problems. Journal of Sociology and Social Welfare; 2(4): 217–224.
Hood, P. D. ( 1990) How can studies of information consumers be used to improve the education communication system? Knowledge in Society; 3(2): 8–25.
The Infant Health and Development Program. ( 1990) Enhancing the outcomes of low birth weight, premature infants: A multisite randomized trial. Journal of the American Medical Association; 263(22): 3035–3042.
Issacs, M. R.; Benjamin, M. P. ( 1991) Toward a Culturally Competent System of Care: Vol. II. Washington, DC: Georgetown University Child Development Center.
Jones, E. ( 1978) Effects of race on psychotherapy process and outcome: An exploratory investigation. Psychotherapy: Theory, Research, and Practice; 15: 226–236.
Jones, E.; Matsumoto, D. ( 1982) Psychotherapy with the underserved: Recent develop
ments. In: L. Snowden, Ed. Reaching the Underserved: Mental Health Needs of Neglected Populations . Beverly Hills, CA: Sage Publications.
Kavanagh, K. H.; Kennedy, P. H. ( 1992) Promoting Cultural Diversity: Strategies for Health Care Professionals . Newbury Park, CA: Sage Publications.
Kelly, J. G.; Dassoff, N.; Levin, I.; Schreckengost, S. P.; Altman, B. E. ( 1988) A Guide to Conducting Prevention Research in a Community: First Steps . New York, NY: The Haworth Press.
Kinzie, J. D.; Manson, S. M.; Do, T. V.; Nguyen, T. T.; Bui, A.; Than, N. P. ( 1982) Development and validation of a Vietnamese-language depression rating scale. American Journal of Psychiatry; 139(10): 1276–1281.
Kleinman, A. ( 1988) The Illness Narratives. New York, NY: Basic Books.
Kleinman, A. ( 1980) Patients and Healers in the Context of Culture. Berkeley, CA: University of California Press.
Kraemer, H. C. ( 1979) Ramifications of a population model for kappa as a coefficient of reliability. Psychometrika; 44: 461–472.
Krueger, R. A. ( 1988) Focus Groups: A Practical Guide for Applied Research. Newbury Park, CA: Sage Publications.
Laird, N. M.; Ware, J. H. ( 1982) Random effects models for longitudinal data. Biometrics; 38: 963–974.
Last, J. M. ( 1988) A Dictionary of Epidemiology. New York, NY: Oxford University Press.
Le Xuan, K.; Bui, D. D. ( 1985) Southeast Asian mutual assistance associations: An approach for community development. In: T. C. Owan, Ed. Southeast Asian Mental Health: Treatment, Prevention, Services, Training, and Research. Rockville, MD: National Institute of Mental Health; DHHS Pub. No. (ADM) 85–1399: 209–224.
Lefley, H. P. ( 1982) Cross-cultural training for mental health personnel. Final Report. Miami, FL: University of Miami School of Medicine; NIMH Training Grant No. 5-T24-MH15249.
Levy, J. E.; Kunitz, S. J. ( 1987) A suicide prevention program for Hopi youth. Social Science and Medicine; 25(8): 931–940.
Lewinsohn, P. M.; Clarke, G. N.; Hoberman, H. H. ( 1989) The Coping with Depression Course: Review and future directions. Canadian Journal of Behavioral Science; 21(4): 470–493.
Liberman, R. P.; Eckman, T. ( 1981) Behavior therapy vs insight-oriented therapy for repeated suicide attempters. Archives of General Psychiatry; 38(10): 1126–1130.
Locke, D. C. ( 1992) Increasing Multicultural Understanding: A Comprehensive Model. Newbury Park, CA: Sage Publications.
Lorion, R. P. ( 1987) The other side of the coin: The potential for negative consequences of preventive interventions. In: Preventing Mental Disorders: A Research Perspective. National Institutes of Health. Washington, DC: DHHS Pub. No. (ADM) 87–1492.
Lum, R. G. ( 1985) A community-based mental health service to Southeast Asian refugees . In: T. C. Owan, Ed. Southeast Asian Mental Health: Treatment, Prevention, Services, Training, and Research. Rockville, MD: National Institute of Mental Health; DHHS Pub. No. (ADM) 85–1399: 283–306.
Maccoby, N.; Alexander, J. ( 1979) Reducing heart disease risk using the mass media: Comparing the effect on three communities. In: R. F. Muñoz, L. R. Snowden, and J. G. Kelly, Eds. Social and Psychological Research in Community Settings. San Francisco, CA: Jossey-Bass Publications.
Manoff, R. K. ( 1985) Social Marketing: New Imperative for Public Health. New York, NY: Random House.
Manson, S. M. ( 1993) Culture and depression: Discovering variations in the experience of
illness. In: W. J. Lonner and R. S. Malpass, Eds. Psychology and Culture. Needham, MA: Allyn and Bacon.
Manson, S. M. ( 1982) New Directions in Prevention Among American Indian and Alaska Native Communities. Portland. OR: Oregon Health Sciences University Foundation.
Manson, S. M.; Brenneman, D. (in press) Chronic disease among older American Indians: Preventing depression and related problems of coping. In: D. Padgett, Ed. Handbook on Ethnicity, Aging, and Mental Health. Westport, CT: Greenwood Press.
Manson, S. M.; Shore, J. H.; Bloom, J. D. ( 1985) The depressive experience in American Indian communities: A challenge for psychiatric theory and diagnosis. In: A. Kleinman and B. Good, Eds. Culture and Depression. Berkeley, CA: University of California Press; 331–368.
Marin, G.; Marin, B. ( 1991) Research with Hispanic Populations. Beverly Hills, CA: Sage Publications.
McCord, J. E. ( 1978) A thirty-year follow-up of treatment effects. American Psychologist; 33(3): 284–289.
Montagne, M. ( 1988) The metaphorical nature of drugs and drug taking. Social Science and Medicine; 26(4): 417–424.
Muñoz, R. F. ( 1993) The prevention of depression: Current research and practice. Applied and Preventive Psychology; 2: 21–33.
Muñoz, R. F.; Chan, F.; Armas, R. ( 1986) Primary prevention: Cross-cultural perspectives. In: J. T. Barter and S. W. Talbott, Eds. Primary Prevention in Psychiatry: State of the Art. Washington, DC: American Psychiatric Press.
Muñoz, R. F.; Ying, Y. W. ( 1993) The Prevention of Depression: Research and Practice. Baltimore, MD: Johns Hopkins University Press.
Murase, K.; Egawa, J.; Tashima, N. ( 1985) Alternative mental health services models in Asian/Pacific communities . In: T. C. Owan, Ed. Southeast Asian Mental Health: Treatment, Prevention, Services, Training, and Research. Rockville, MD: National Institute of Mental Health; DHHS Pub. No. (ADM) 85–1399: 225–227.
NAS (National Academy of Sciences). ( 1992) Responsible Science: Ensuring the Integrity of the Research Process . Washington, DC: National Academy Press.
Neighbors, H. W. ( 1990) The prevention of psychopathology in African Americans: An epidemiologic perspective. Community Mental Health Journal; 26(2): 167–179.
Neighbors, H. W.; Bashshur, R.; Price, R.; Selig, S.; Donabedian, A.; Shannon, G. ( 1992) Ethnic minority health service delivery: A review of the literature . Research in Community and Mental Health; 7: 55–71.
Norton, I. M.; Manson, S. M. ( 1993) An association between domestic violence and depression among Southeast Asian refugee women. Journal of Nervous and Mental Disease; 180(11): 729–730.
O'Donnell, C. R.; Tharp, R. G. ( 1990) Community intervention guided by theoretical development. In: A. S. Bellack, M. Hersen, and A. E. Kazdin, Eds. International Handbook of Behavior Modification and Therapy. 2nd ed. New York, NY: Plenum Press; 251–266.
Olds, D. L.; Henderson, C. R.; Tatelbaum, R.; Chamberlin, R. ( 1988) Improving the life-course development of socially disadvantaged mothers: A randomized trial of nurse home visitation. American Journal of Public Health; 78(11): 1436–1445.
Olds, D. L.; Henderson, C. R.; Tatelbaum, R.; Chamberlin, R. ( 1986) Improving the delivery of prenatal care and outcomes of pregnancy: A randomized trial of nurse home visitation. Pediatrics; 77(1): 16–28.
Olweus, D. ( 1991) Bully/victim problems among schoolchildren: Basic facts and effects of
an intervention program. In: K. Rubin and D. Pepler, Eds. The Development and Treatment of Childhood Aggression. Hillsdale, NJ: Lawrence Erlbaum Associates.
Orlandi, M. A. ( 1992) Defining cultural competence: An organizing framework. In: M. A. Orlandi, Ed. Cultural Competence for Evaluators. Washington, DC: Office of Substance Abuse Prevention; DHHS Pub. No. (ADM) 92–1884.
Page, J. B.; Chitwood, D. D.; Smith, P. C.; Kane, N.; McBride, D. C. ( 1990) Intravenous drug use and HIV infection in Miami. Medical Anthropology Quarterly; 4(1): 56–71.
Paul, B. P. ( 1955) Health, Culture and Community: Case Studies of Public Reactions to Health Programs. New York, NY: Sage Publications.
Paul, G. L.; Lentz, R. ( 1977) Psychosocial Treatment of Chronic Mental Patients. Cambridge, MA: Harvard University Press.
Pope, K. S. ( 1990) Identifying and implementing ethical standards for primary prevention . In: E. J. Trickett and G. B. Levin, Eds. Ethical Issues of Primary Prevention. New York, NY: The Haworth Press.
Rappaport, J.; Seidman, E.; Davidson, W. S. ( 1979) Demonstration research and manifest versus true adoption: The natural history of a research project to divert adolescents from the legal system. In: R. F. Muñoz, L. R. Snowden, and J. G. Kelly, Eds. Social and Psychological Research in Community Settings. San Francisco, CA: Jossey-Bass Publications; 101–144.
Rossi, P. H. ( 1977) Boobytraps and pitfalls in evaluation of social actions programs. In: F. G. Caro, Ed. Readings in Evaluation Research. 2nd ed. New York, NY: Sage Publications.
Russell, L. B. ( 1986) Is Prevention Better than Cure? Washington, DC: The Brookings Institution.
Schilling, R. F.; Schinke, S. P.; Kirkham, M. A.; Meltzer, N. J.; Norelius, K. L. ( 1988) Social work research in social service agencies: Issues and guidelines . Journal of Social Service Research; 11(4): 75–87.
Seitz, V. ( 1987) Outcome evaluation of family support programs: Research design alternatives to true experiments. In: S. L. Kagan, D. Powell, B. Weissbound, and E. Zigler, Eds. America's Family Support Programs: Perspectives and Prospects. New Haven, CT: Yale University Press.
Shadish, W. R., Jr.; Cook, T. D.; Leviton, L. C. ( 1991) Foundation of Program Evaluation. Newbury Park, CA: Sage Publications.
Sieber, J. ( 1993) Issues Presented by Mandatory Reporting Requirements. Paper commissioned by the CBASSE Panel on Child Abuse and Neglect, National Research Council. Washington, DC.
Singer, M.; Flores, C.; Davison, L.; Burke, G.; Castillo, Z.; Scanlon, K.; Rivera, M. ( 1990) SIDA: The economic, social, and cultural context of AIDS among Latinos . Medical Anthropology Quarterly; 4(1): 72–114.
Snowden, L. R.; Muñoz, R. F.; Kelly, J. G. ( 1979) The process of implementing community-based research. In: R. F. Muñoz, L. R. Snowden, and J. G. Kelly, Eds. Social and Psychological Research in Community Settings. San Francisco, CA: Jossey-Bass Publications; 14–29.
Szapocznik, J.; Kurtine, W. M. ( 1993) Family psychology and cultural diversity. American Psychologist; 48(4): 400–407.
Tharp, R. G.; Gallimore, R. ( 1988) Rousing Minds to Life: Teaching, Learning, and Schooling in Social Context. Cambridge, MA: Cambridge University Press.
Thurman, P. J.; Jones-Saumty, D.; Parsons, O. A. ( 1990) Locus of control and drinking behavior in American Indian alcoholics and non-alcoholics. American Indian and Alaska Native Mental Health Research; 4(1): 31–39.
Trickett, E. J.; Levin, G.B. ( 1990) Paradigms for Prevention: Providing a Context for Confronting Ethical Issues. New York, NY: The Haworth Press.
Trotter, R. T., II; Rolf, J.; Quintero, G. A.; Alexander, C.; Baldwin, I. (in press) Cultural models of drug abuse and AIDS on the Navajo Reservation: Navajo youth at risk. Medical Anthropology Quarterly.
True, R. H. ( 1985) An Indochinese mental health service model in San Francisco. In: T. C. Owan, Ed. Southeast Asian Mental Health: Treatment, Prevention, Services, Training, and Research. Rockville, MD: National Institute of Mental Health; DHHS Pub. No. (ADM) 85–1399: 329–342.
U.S. Census Bureau. ( 1990) Census of Population. Unpublished tabulations.
Vega, W. A. ( 1992) Theoretical and pragmatic implications of cultural diversity for community research. American Journal of Community Psychology; 20(3): 375–391.
Vega, W. A.; Murphy, J. ( 1990) Projecto Bienestar: An example of a community-based intervention. In: W. A. Vega and J. W. Murphy, Eds. Culture and the Restructuring of Community Mental Health. Westport, CT: Greenwood Press; 103–122.
Vega, W. A.; Valle, R.; Kolody, B. (submitted for publication) Preventing depression in the Hispanic community: An outcome evaluation of Projecto Bienestar.
Vega, W. A.; Valle, R.; Kolody, B.; Hough, R. ( 1987) The Hispanic social network prevention intervention study: A community-based randomized trial. In: R. Muñoz, Ed. Depression Prevention: Research Directions. Washington, DC: Hemisphere Publishing.
Vinokur, A.D.; van Ryn, M.; Gramlich, E. M.; Price, R. ( 1991) Long-term follow-up and benefit-cost analysis of the Jobs Program: A preventive intervention for the unemployed. Journal of Applied Psychology; 76(2): 213–219.
Walker, P. S.; Walker, R. D.; Kivlahan, D. ( 1988) Alcoholism, alcohol abuse, and health in American Indians and Alaska Natives. In: S. M. Manson and N. G. Dinges, Eds. Behavioral Health Issues Among American Indians and Alaska Natives: Explorations on the Frontiers of the Biobehavioral Sciences. Denver, CO: University of Colorado Health Sciences Center.
Weisbrod, B. A.; Test, M. A.; Stein, L. I. ( 1980) Alternative to mental hospital treatment: III. Economic benefit-cost analysis. Archives of General Psychiatry; 37: 400–405.
Weiss, C. H. ( 1984) Increasing the likelihood of influencing decisions. In: L. Rutman, Ed. Evaluation Research Methods: A Basic Guide. Beverly Hills, CA: Sage Publications: 159–190.
Weithorn, L. A. ( 1987) Informed consent for prevention research involving children: Legal and ethical issues. In: Preventing Mental Disorders: A Research Perspective. Washington, DC: National Institute of Mental Health. DHHS Pub. No. (ADM) 87–1492.
Wellin, E. ( 1955) Water boiling in a Peruvian town. In: B. P. Paul, Ed. Health, Culture and Community. New York, NY: Russell Sage Foundation; 71–103.