presenting five lessons learned that can lead to better, more useful evaluations.
There have been three large-scale national evaluations of programs for English-language learners. Although these studies provided some information about the education of English-language learners, they were of limited utility for the evaluation of programs. Nonetheless, it is instructive to review them so we can avoid the mistakes of the past, as well as benefit from what was learned. This section also summarizes the findings of a National Research Council report (Meyer and Fienberg, 1992) that reviews two of these studies.
The American Institutes for Research (Dannoff, 1978) conducted the first large-scale national evaluation of programs for English-language learners, commonly referred to as the AIR study. The study compared students enrolled in Title VII Spanish/English bilingual programs and comparable students (in terms of ethnicity, socioeconomic status, and grade level) not enrolled in such programs. In the AIR study, 8,200 children were measured twice during the school year on English oral comprehension and reading, math, and Spanish oral comprehension and reading. Generally, the results from this study showed that students in bilingual education programs did not gain more than students not in such programs.
The study was the subject of a great deal of criticism, the major criticism addressing the strength of the treatment control group comparison (Crawford, 1995). Nearly three-quarters of the experimental group had been in bilingual programs for 2 or more years, and the study measured their gains in the last few months. Additionally, about two-thirds of the children in the control group had previously been in a bilingual program; these children did not represent a control group in the usual sense of the term. Thus, the AIR study did not compare bilingual education with no bilingual education.
In part because of the ambiguity of the conclusions from the AIR study, two major longitudinal studies were commissioned by the U.S. Department of Education in 1983 to look at program effectiveness. The hope was that these studies would provide definitive evidence, one way or the other, about the effectiveness of bilingual education. These two studies are reviewed below.
The National Longitudinal Evaluation of the Effectiveness of Services for Language Minority Limited English Proficient Students (Longitudinal Study)