Pre-Post Designs

A pre-post design is another alternative to randomization. Such studies evaluate an intervention on the basis of the changes that occur from a baseline (the “pre” measurement) to after the intervention period (the “post” measurement). This type of design can provide valuable information, particularly when it supports a hypothesized developmental model involving known mediators that lead to expected prevention targets. However, the pre-post design suffers from confounding with developmental changes that are occurring in young people. On one hand, with drug use in adolescents, for example, the sharp increases in drug use with age—as well as seasonal effects—could completely mask the potential benefit of an intervention. On the other hand, lower drug use after the intervention than before would suggest that the intervention has prevention potential. Also, pre-post designs can lead to erroneous conclusions if they involve selecting participants at high risk and assessing whether their risk goes down; improvement might be expected simply because of a regression to the mean effect.

Interrupted Time-Series Designs

An important way to improve pre-post designs is to include multiple measurements of variables of interest. A good example of this is the interrupted time series (or multiple baseline design extended to several groups), in which multiple measurements of the target behavior are made both before and after the intervention. Varying the timing of the intervention across participating individuals or groups, especially if assignment to an intervention time is randomized, can further strengthen the evaluation design. Policy changes, such as wide-scale implementation of a new program, changes in the law or changes in enforcement of existing laws, often provide opportunities to evaluate an intervention in this type of natural experiment. One example is the evaluation of policies that restrict tobacco sales to minors (Stead and Lancaster, 2005). In their examination of the effect of positive reinforcement to tobacco stores and sales clerks to avoid tobacco sales to minors, Biglan, Ary, and colleagues (1996), for example, repeatedly assessed the proportion of stores making underage sales both before and after the intervention, demonstrating that the behavior of clerks is modifiable.

Regression Discontinuity Designs

Another type of natural experiment that provides an opportunity for program evaluation occurs when strict eligibility criteria, such as age or level of risk along a continuum, are imposed for entrance into a program.

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement