it provides consistent results from one reviewer to another (Kulm and Grier, 1998). The procedure
uses research-based criteria;
requires extensive training (four days are recommended to train evaluators to minimum competency);
demands evidence-based arguments to support all judgments; and
involves two review teams in the examination of each material and subsequent reconciliation of differences.
The procedure can be applied to a variety of K-12 materials, ranging from those that cover a few weeks to several years of classroom programs (Roseman et al., 1997; Roseman, 1997a,b). Although the procedure has been developed for use with the learning goals in Benchmarks (AAAS, 1993) and the Standards (NRC, 1996), it is applicable to state or district curriculum frameworks if the learning goals are clearly articulated.
Project 2061 used a four-step evaluation process: identification of learning goals, content analysis, instructional analysis, and summary report (AAAS, forthcoming c). The process clustered its evaluation questions on content analysis into three groups:
accuracy (examined by scientists and to be published in Science Books and Films);
alignment with standards; and
Material found to be aligned with standards is then subjected to the instructional analysis to determine the likelihood of students learning the specific benchmarks and standards that serve as a basis for the analysis. The seven clusters of evaluation questions (each benchmark-specific) on instructional analysis are:
providing a sense of purpose;
taking account of student ideas;
engaging students with phenomena;
developing and using scientific ideas;
promoting student thinking about phenomena, experiences, and knowledge;
assessing progress; and
enhancing the learning environment.
Also see "Instructional Analysis" in Chapter 5.
Project 2061 has applied its analysis procedure to middle school science textbooks and will publish a report titled Middle Grades Science Textbooks: A Benchmarks-based Evaluation in fall of 1999 (AAAS, forthcoming b). A report on the evaluation of middle grades mathematics programs has also been published and is available on the Project 2061 website (See Contact Information).