The resulting estimate will be an unbiased estimate of the average treatment effect. Every comparison between those in E and those in C involves subsets of children having identical propensities to experience E. Therefore, the potential outcomes of the children compared cannot be associated with their propensities, and the estimates of the treatment effect will be unbiased. This procedure also makes it easy to estimate separate treatment effects for each subgroup.

When children are matched on propensity scores, the validity of the causal estimate depends strongly on the investigator's knowledge of the factors that affect the propensity to experience E versus C. More specifically, if some unknown characteristic of the child predicts the propensity to be in E versus C, and if that characteristic also is associated with the potential outcomes, then the estimate of the treatment effect based on propensity score matching will be biased. The assumption that no such confounding variable exists is a strong assumption. It is the responsibility of the investigator to collect the relevant background data and to provide sound arguments based on theory and data analysis that the relevant predictors of propensity have been controlled. Even then, doubts will remain in the minds of some readers. In contrast, all possible predictors of propensity are controlled in a randomized experiment, including those that would have escaped the attention of the most thoughtful investigator. Rosenbaum (1995) describes procedures for examining the sensitivity of causal inferences to lack of knowledge about propensity when randomization is impossible.

Perhaps the most common strategy for approximating unbiased causal inference in nonexperimental settings is the use of statistical adjustments. In early childhood research, it is very common to use linear models (regression, analysis of variance, structural equation models) to adjust estimates of treatment impact for covariates related to the outcome. These covariates must be pretreatment characteristics of the child or the setting, and the aim is to include all confounders in the set of covariates controlled. By statistically “holding constant ” the confounders in assessing treatment impact, one aims to approximate a randomized experiment. Under some assumptions, this strategy will work. In particular, if the propensity score (the probability of receiving treatment E) is a linear function of the covariates used in the model, then this adjustment strategy will provide an unbiased estimate of the treatment effect. Aside from the possible fragility of this assumption, this strategy is limited, in that only a relatively small set of covariates may be included in the model. In a propensity score matching procedure mentioned earlier, it is possible—and advisable —to use as many possible covariates as one can obtain in the analysis that predicts propensity.

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement