Still, myriad RCFTs have been successfully conducted to inform social policy. The Digest of Social Experiments (Greenberg and Shroder, 2004) and its successor journal, Randomized Social Experiments, provide many examples.

OBSERVATIONAL STUDIES

Observational studies are nonexperimental research studies in which subjects or outcomes are observed and measured. If two groups are to be compared, the assignment of subjects among the two groups is not under the direct control of the investigator. Two types of observational studies are quasi-experiments (Campbell and Stanley, 1963) and natural experiments (see, e.g., Campbell and Ross, 1968). In the former, the investigator may manipulate the intervention; in the latter, it arises naturally. In neither type of study, however, does the investigator control which subjects receive the treatment. Observational studies can be more than passively observing data and analyzing them: for example, they may involve systematic measurement and aspects of “control,” such as manipulating the timing of an intervention to predefined although nonrandomized groups.

Because they do not involve randomization, however, observational studies may not control for the effects of secondary variables. Without experimental confirmations, the observed outcomes could be the result of any combination of a range of confounding factors. For example, subjects may be self-selected, such as students in a private school who are to be compared with students in a public school, or they may be selected by others but with different characteristics, known or unknown, that may influence the outcome of the intervention. This possible influence is called selection bias. If there is selection bias, how the intervention affects the outcome for the treatment group in comparison with the control group must be described by a model, and that model will always include some assumptions. The model may or may not help with inference for what would have happened in a randomized experiment (see National Research Council, 1998). Moreover, the assumptions underlying the model may not be widely accepted in the scientific community.

Observational studies, however, are important in revealing important associations and in guiding the formulation of theory and models. The observation of a single case can reveal unsuspected patterns and provide explanations for unmotivated forms of behavior. As put by Coburn et al. (2009, p. 1,121): “The in-depth observation made possible by the single case study



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement