Assessing Evaluation Studies The Case of Bilingual Education Strategies (1992) / Chapter Skim
Currently Skimming:
4 The Immersion Study
4 The Immersion Study
Pages 67-89
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 67... ...
states (1991a, page 321: "The primary objective of this study is to assess the relative effectiveness of structured English immersion strategy, early-exit, and late-exit transitional bilingual education programs." All three programs were intended for students who speak Spanish, but have limited ability to speak English. All three programs had, as one of He*
|
From page 68... ...
The Longitudinal Study of Structured English Immersion Strategy, Early-Exit and Late-Exit Transitional Bilingual Education Programs This study compared the relative effectiveness of two alternative programs (structured English immersion and late-exit transitional bilingual education) with that of early-exit transitional bilingual education programs.
|
From page 69... ...
The search and classification are described in Chapter 2 of Volume I of He Immersion Report. Immersion and Late-exit Programs are comparatively rare and were He basis for He search.
|
From page 70... ...
Since the programs in the Immersion Study were found and not created to specifications, the programs have aspects that were not part of the definitions of the treatments given earlier. For instance, He Immersion Report notes that more teachers in Late-exit Programs were Hispanic or knew more Spanish than in other programs.
|
From page 71... ...
In addition to unintended aspects of the three different programs, there are also unintended differences in implementation of the "same" program at different sites. Again, the Immersion Report does an admirable job of describing some of these differences.
|
From page 72... ...
Specifically, if an outcome is Heated as a covariate that is, if adjustments are made for an outcome variable-it may distort the actual effects of a treatment, even in randomized experiments. In the Immersion Study, adjustments for outcomes appear in Tree guises.
|
From page 73... ...
As He authors noted (Ramirez et al., 1991a, page 32) , '~e papacy objective of this study is to assess He relative effectiveness of structured English immersion strategy, early-exit, and late-exit transitional bilingual education programs." Lee rd-grade I -exit cohort is of no use for this main objective, Cough it may have other appropnate uses.
|
From page 74... ...
A major focus of the analytical effort in an observational study is distinguishing the actual effects of the treatments from the initial lack of comparability of the treatment groups. This section discusses He comparability of He groups in the Immersion Study, the adjustments made for noncomparability or bias, and He detection and assessment of hidden biases that were not adjusted for.
|
From page 75... ...
Table 4 - compares the three treatment ;,uoups in teens of gross family income. Although family income is not high in any group, children in Late-exit Programs tend to come from families win substantially lower incomes Can children in other programs.
|
From page 76... ...
As the Immersion Report notes (Ramirez et al., l991b, page 93~: When a single school has both programs, the [hrLmersion] and [Early-exit]
|
From page 77... ...
The immersion strategy students tend to score eight to nine points lower than early-exit students on the reading subtest, and the difference is statistically significant at the .05 level (page 108~. The subsequent analyses using the reduced group of students with pretest scores produce qualitatively similar results, whether or not adjusunents are made for pretest scores, though the reading difference is somewhat larger for this group (pages 108-127~.
|
From page 78... ...
Again, Here are simple analyses might provide an answer here. ~ summary, nothing presented in the Immersion Report leads He panel to doubt He report's conclusion that the Early-exit Programs produce greater achievement than Immersion Programs in reading for K-1 students; however, uncertainties attenuate this conclusion, and these uncertainties might have been partially reduced or at least clarified win fairly simple additional analyses.
|
From page 79... ...
These two possible cases are not distinguished in the growth curve analyses. Figure ~1 illustrates this point with two hypothetical growth curves for two students in two hypothetical programs (A and B)
|
From page 80... ...
Given Be large differences between schools win the same type of program, it is not possible to say much about Be effectiveness of programs in different schools. Despite the very large number of statistical tests for program differences Mat were perfonned, very few yielded significant differences (Ramirez et al., l991b, page 286~: "Almost no program differences were found among the one-program schools." Furthermore, the Immersion Report summarizes (page 282~: Perhaps the most important result of the analyses of one-program schools is that school-to-school differences are large even within the same district and program.
|
From page 81... ...
Oslo E~ As we 1~-~ s_ ~ ~ Sony ~ lie pan -~e ~ w~ 1- paws bad Lad no Eve Has. Is mans ~ Abbe ~ Cal so_ ~ ~ _ students iD 0~ paws Mile corolla far bisect distances Bum He dam on He L~e-xh has do not con~bue lo he puma o~ecOve of he saw ~ least lam he active of conve~on~ methods of Sl~iS~C~ Isis.
|
From page 82... ...
~ this traditional statistical literature, which dates back to We creator of randomized experiments, Sir R
|
From page 83... ...
As noted above, Were do appear to have been different versions of different programs. If the Immersion Shady is viewed as several separate horse races between related but not identical treatments (as suggested above)
|
From page 84... ...
We believe that the I~x~nersion Study data are quite valuable and encourage We Department of Education to make the data publicly available in a fond that is easy for researchers to use, while protecting confidentiality. Several further analyses of We data from We Immersion Study would be especially useful in helping one to appraise the findings, particularly the one positive finding of evidence in support of superior reading performance in kindergarten and first-grade for students in Early-exit Programs in comparison to Immersion Programs.
|
From page 85... ...
Suggestions for the Design of Future Studies The panel's analysis of the Immersion Report leads us to propose several features for future studies. First, future studies should compare competing programs within the same schools.
|
From page 86... ...
Third, ideally one would wish to create competing programs for study rather than to find existing programs The panel believes strongly in filming Be ideal into reality. Created programs would have fewer unintended features, so it would be clearer why they succeed or fail.
|
From page 87... ...
In bilingual education such an approach would mean that the competing programs should be seen by all involved as attractive in comparison with what is available outside the experiment and that the study should be designed with sufficient flexibility to allow for changing treatments if one of the programs is clearly superior. Though randomization is highly desirable, it is not always practical or ethical.
|
From page 88... ...
(1991b) Final report: Longitudinal study of structured-english immersion strategy, early~xit and late-exit transitional bilingual education programs for language-minority children, Volume II.
|
From page 89... ...
The Condition of Bilingual Education In the Nation: A Report to the Congress and the President. Office of the Secretary.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.