Recommendation 10-1: Research funders should invest in studies that (1) aim to replicate findings from earlier trials, (2) evaluate long-term outcomes of preventive interventions across multiple outcomes (e.g., disorders, academic outcomes), and (3) test the extent to which each prevention program is effective in different race, ethnic, gender, and developmental groups.
Being able to obtain replicable results is one of the hallmarks of science, since lack of replicability raises questions about generalizability. Direct replicability corresponds to a test of the same intervention under very similar conditions. Systematic replicability refers to testing of the intervention under conditions that are deliberately modified (e.g., intervention agent, trainer, length of program, target population) in order to examine whether the results change with these modifications (see Chapter 11 for discussion of adaptation to different populations). Given limited funding, lack of interest by review groups in direct replication, and the current state of knowledge about the effects of preventive interventions, we recommend that systematic replications are more appropriate than direct replications.
Funding is often limited for evaluations that assess outcomes beyond the end of an intervention or a short time after the intervention. Yet demonstrating outcomes that endure increases confidence in an intervention and provides a more comprehensive test of the impact of the intervention on children’s lives and its benefit to society. Assessment of long-term outcomes would ideally include consideration of the sustainability of outcomes across developmental periods (Coie, Watt, et al., 1993). Given that most preventive interventions are designed to mitigate developmental processes that can lead to mental, emotional, and behavioral disorders and problems over time, assessment of whether proximal outcomes at one developmental period are sustained in distal outcomes at a later developmental period is needed. Several of the programs discussed in Chapters 6 and 7, including the Nurse-Family Partnership, Life Skills Training, Good Behavior Game, Strengthening Families 10-14, and the Family Check-up, have met this criterion. Although the Society for Prevention Research (Flay, Biglan, et al., 2005) has suggested six months as a minimum follow-up period,2 the committee considers this to be a necessary but insufficient time frame for the majority of outcomes.
As statistical and methodological approaches have been developed in
For “outcomes that may decay over time,” the Society for Prevention Research (Flay, Biglan, et al., 2005, p. 2) recommends that evaluations include “at least one long-term follow-up at an interval beyond the end of the intervention (e.g., at least 6 months after the intervention.” The Society for Prevention Research standards also acknowledge that the interval may need to differ for different types of interventions.