components, such as materials kits, videotapes, supplementary materials, and student books (if student books are reproduced in the teacher's manual), one set per team may be adequate. When videotapes or other media materials are an integral part of instruction, make the appropriate playback equipment readily available. If materials include software, CD-ROMs, or probeware, it is advisable to have a technical troubleshooter available.

Be sure to communicate to reviewers your arrangements with the publishers, which usually require that the samples be returned promptly in resalable condition.

If you have provided for adequate release time and space, the reviewers will be able to come to a central site to do the reviews. This way, the complete set of materials can be made readily available, but, more importantly, the reviewers will be available to one another. If the reviewers will be working independently off site, plan to facilitate communications with you and with each other. When the reviewers are finished with their individual reviews, you may want to schedule time for a conference among the reviewers of one set of materials. Each reviewer has made independent decisions, but defending those decisions to others and listening to other opinions may strengthen the review process. If there is a broad range of ratings, reviewers should not be pressured to change their original rating unless they find they truly overlooked or misunderstood something. Alternatively, you can convene a conference only when the reviews indicate a need. Looking back on the quality and sources of evidence cited to support an overall judgment should reveal why the reviews differ and will provide discussion points for a conference. In some cases, it may be necessary to carry out another independent review because of disagreements.

Decide who will identify the standards to be used for each unit of instructional materials. This can be done either by you or by the reviewers. If done by you, the review will get off to a faster start. However, this standards selection is a very time-consuming task requiring reading of many materials. Each pair of reviewers assigned a small number of instructional materials can also accomplish the identification and prioritization of standards themselves. Even though it can be a disorienting experience, this approach actually produces more autonomous, flexible reviewers who appear to understand the framework, standards, and their task better. Allow plenty of time and a flexible schedule for this task. (Also see "Directions for Reviewers" below.)

Revisit the purpose and steps of



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement