Meta-analysis is a statistical procedure that pools results from a sample of preexisting experimental/quasi-experimental studies to derive a single effect size. Effect size is a quantitative index expressing the difference between the treatment and control group means in standard deviation units. Studies are selected on the basis of some preset criteria, typically a common question about an intervention or therapeutic procedure, a defined target population, and common outcomes of interest.

The chief difficulty with applying meta-analysis is that the number of available studies on a given topic is usually rather small. This difficulty forces researchers to use a backwards logic in justifying the procedure—to start with a selected sample of studies and then imagine a hypothetical population in which the studies belong. Given this scenario, assumptions can never be properly tested; Glass (2000) finds this logic to be untenable today. Further, the studies selected typically vary in the effects they show, from positive to negative. Technical advances now make it possible to improve the analysis, for example, by testing for and statistically ruling out heterogeneity in the sample (Cooper and Hedges, 1994). However, Glass (2000) finds these added procedures to be problematic, as further assumptions need to be made that are both untestable and, often, not defensible.

The most important limitation, according to Glass (2000), is that the meta-analysis is guided by a very limiting question: On average, is the (intervention) effective? The procedure cannot address questions about differential effects of an intervention, reducing findings across studies to an average that frequently removes the most important information about the intervention. It is this loss of information through averaging that Glass now finds regrettable. His current view is that the standard guiding question in meta-analysis needs to be replaced with deeper and broader questions, such as “What type of therapy, with what type of client, produces what kind of