current research on learning (Loucks-Horsley et al, 1996). Therefore, the evaluation coordinator should be strongly encouraged to provide references and resources for research on learning, including the Standards (NRC, 1996), Benchmarks (AAAS, 1993), and more recent studies (NRC, 1999a, b).
9. It is more important to evaluate materials in-depth against a few relevant standards than superficially against all standards. The pressures of limited time and funds can drive an evaluation team to inspect instructional materials superficially against all relevant standards. The Committee concluded that if time and funds are limited, it is preferable for the team to select a small number of high-priority standards for an in-depth examination.
10. The review and selection processes should be closely connected even when reviewers are not members of the selection committee. In some school districts, one team evaluates instructional materials and reports to another group that is responsible for final approval and selection. In others, one team is responsible for both evaluating and selecting instructional materials. Considerations such as cost, the local district's ability to refurbish materials, and political acceptability (e.g., attitudes about teaching evolution) may play a role in the final selections. The Committee concluded that in all instances it is important that final selections be based primarily on a standards-based review. It is therefore important that one or more of the members of the evaluation team be on the selection committee.
The Committee's initial prototype tool was designed to include the following characteristics:
reliance on the professional judgment of reviewers;
substantiation of review ratings by cited evidence;
ability to be completed in a reasonable amount of time;
focus on the extent to which the instructional materials matched a standard; and
consideration of scientific inquiry as content and as a way of teaching and learning.
The Committee members tested the prototype themselves. To begin, they participated in a preliminary review of sample materials and compared their results with one another. After scanning materials on middle school environmental science from seven publishers, they chose three that represented various