Box 8-4

Design Elements Used in Constructing Quasi-Experiments

Assignment (control of assignment strategies to increase group comparability)

  • Cutoff-based assignment: controlled assignment to conditions based solely on one or more fully measured covariates; this yields an unbiased effect estimate.

  • Other nonrandom assignment: various forms of “haphazard” assignment that sometimes approximate randomization (e.g., alternating assignment in a two-condition quasi-experiment whereby every other unit is assigned to one condition).

Measurement (use of measures to learn whether threats to causal inference actually operate)

  • Posttest observations

    • Nonequivalent dependent variables: measures that are not sensitive to the causal forces of the treatment, but are sensitive to all or most of the confounding causal forces that might lead to false conclusions about treatment effects (if such measures show no effect, but the outcome measures do show an effect, the causal inference is bolstered because it is less likely due to the confounds).

    • Multiple substantive posttests: used to assess whether the treatment affects a complex pattern of theoretical predicted outcomes.

  • Pretest observations

    • Single pretest: a pretreatment measure on the outcome variable, useful to help diagnose selection bias.

    • Retrospective pretest: reconstructed pretests when actual pretests are not feasible—by itself, a very weak design feature, but sometimes better than nothing.

    • Proxy pretest: when a true pretest is not feasible, a pretest on a variable correlated with the outcome—also often weak by itself.

    • Multiple pretest time points on the outcome: helps reveal pretreatment trends or regression artifacts that might complicate causal inference.

  • Pretests on independent samples: when a pretest is not feasible on the treated sample, one is obtained from a randomly equivalent sample.

  • Complex predictions such as predicted interaction: successfully predicted interactions lend support to causal inference because alternative explanations become less plausible.

average causal effect that characterizes the difference between the treatment and control groups. The ability to make statements about individual causal effects, important in many clinical and health contexts, is diminished without additional assumptions being made (e.g., the causal effect is constant for all individuals).

A key idea in Rubin’s perspective is that of possible outcomes. The outcome of a single unit (participant) receiving a treatment is compared with the outcome that would have occurred if the same unit had received the alternative treatment. This idea has proven to be a remarkably generative way of thinking about how to obtain precise estimates of causal effects. It focuses the researcher on the precise nature of the comparison that needs to be made and clearly delineates the participants for whom the comparison is appropriate. It provides a basis for elegant solutions to such problems as treatment nonadherence and appropriate matching in nonrandomized studies (West and Thoemmes, 2010).



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement