Page 138

PROGRAM EVALUATION:
SUMMARY OF THE STATE OF KNOWLEDGE

The following key points can be drawn from the literature on program evaluation:

The major national-level program evaluations suffer from design limitations; lack of documentation of study objectives, conceptual details, and procedures followed; poorly articulated goals; lack of fit between goals and research design; and excessive use of elaborate statistical designs to overcome shortcomings in research designs.

In general, more has been learned from reviews of smaller-scale evaluations, although these, too, have suffered from methodological limitations.

It is difficult to synthesize the program evaluations of bilingual education because of the extreme politicization of the process. Most consumers of research are not researchers who want to know the truth, but advocates who are convinced of the absolute correctness of their positions.

The beneficial effects of native-language instruction are clearly evident in programs that are labeled ''bilingual education," but they also appear in some programs that are labeled "immersion." There appear to be benefits of programs that are labeled "structured immersion," although a quantitative analysis of such programs is not yet available.

There is little value in conducting evaluations to determine which type of program is best. The key issue is not finding a program that works for all children and all localities, but rather finding a set of program components that works for the children in the community of interest, given that community's goals, demographics, and resources.

Five general lessons have been learned from the past 25 years of program evaluation:

 

Higher-quality program evaluations are needed.

 

Local evaluations need to be made more informative.

 

Theory-based interventions need to be created and evaluated.

 

We need to think in terms of program components, not politically motivated labels.

 

A developmental model needs to be created for use in predicting the effects of program components on children in different environments.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement