APPENDIX B:
NSRC Evaluation Criteria for Curriculum Materials
Introduction to Appendix B
Consistent with the National Science Resources Center's (NSRC's) philosophy of science teaching and with the National Science Education Standards of the National Research Council, the materials included in Resources for Teaching Middle School Science are hands-on and inquiry-centered. Briefly described, such materials provide opportunities for students to learn through direct observation and experimentation; they engage students in experiences not simply to confirm the "right" answer but to investigate the nature of things and to arrive at explanations that are scientifically correct and satisfying to young adolescents; and they offer students opportunities to experiment productively, to ask questions and find their own answers, and to develop patience, persistence, and confidence in their ability to tackle and solve real problems.
To produce evaluation criteria for identifying the most effective instructional materials available, the NSRC drew upon three primary sources:
-
the National Science Education Standards (see appendix C in this guide);
-
the experience gained by the NSRC in its ongoing review of science curriculum materials under the auspices of the National Academy of Sciences and the Smithsonian Institution; and
-
the experience of teachers, superintendents, principals, and science curriculum coordinators across the United States.
The NSRC's "Evaluation Criteria for Middle School Science Curriculum Materials" were used by panels of teachers and scientists in the structured review of curriculum materials for this guide. The criteria are organized in the following sections:
-
Pedagogical Appropriateness These criteria elaborate on the following key questions: Do the materials promote effective middle school science teaching and learning? Are inquiry and activity the basis of the learning experiences? Are the topics addressed in the unit and the modes of instruction developmentally appropriate?
-
Science Content and Presentation These criteria address whether the science content is accurate, up to date, and effectively presented. Specific issues addressed include these: Do the suggested investigations lead to an understanding of basic science concepts and principles? Is the writing style interesting and engaging, while respecting scientific language? Which of the subject matter standards from the National Science Education Standards does the material focus on?
-
Organization and Format; Materials, Equipment, and Supplies; and Equity Issues The criteria on organization and format include questions about the presentation of information—for example, whether the suggestions for instructional delivery are adequate and whether the print materials for students are well written, age-appropriate, and compelling in content. The criteria concerning hands-on science materials focus on questions such as the clarity and adequacy of instructions on manipulating laboratory equipment and the inclusion of appropriate safety precautions. Criteria addressing equity issues include the question of whether the material is free of cultural, racial, gender, and age bias.
At the end of each section, reviewers are asked to summarize the strengths and weaknesses of the material being reviewed. Final recommendations are based on a synthesis of the comments in the three sections.
Because instructional materials are designed to be used in different ways, the NSRC has identified three categories for classifying different types of science curriculum materials:
-
Core instructional materials, which are substantial enough to form the foundation of a comprehensive middle school science curriculum.
-
Supplementary units, which often consist of a series of activity-centered lessons. These units can provide enrichment for inquiry-based science teaching but may not have the depth or focus of core curriculum units.
-
Science activity books, which offer a selection of ideas and activities to facilitate science learning. These materials are generally too broad in scope or specific in focus to serve as the foundation of a comprehensive science program.
The evaluation criteria not only apply to materials at these different levels, but they can help identify the most effective use of a particular instructional resource by focusing reviewers' attention on its strengths and weaknesses.
The following considerations should be kept in mind when one is
using the criteria. The expectations for core materials are more comprehensive than for supplementary materials. For example, core materials need to provide assessment strategies, whereas this is not an expectation for science activity books. Likewise, core materials need to allow students to study a concept in depth, while supplementary units may provide only a general introduction to a topic.
The NSRC's "Evaluation Criteria for Middle School Science Curriculum Materials" are reprinted in full in this appendix. Teachers, curriculum specialists, curriculum developers, principals, superintendents, and those involved in various aspects of science education reform may find the criteria not only instructive, but useful as an instrument for reviewing instructional materials for local adoption.
NATIONAL SCIENCE RESOURCES CENTER |
||
SMITHSONIAN INSTITUTION · NATIONAL ACADEMY OF SCIENCES |
||
Resources for Teaching Middle School Science |
||
EVALUATION CRITERIA FOR MIDDLE SCHOOL SCIENCE CURRICULUM MATERIALS |
||
TITLE: or name of resource |
||
SERIES TITLE: if applicable |
||
AUTHOR(S): if applicable |
||
CITY/STATE: where published |
||
PUBLISHER/SOURCE: |
||
COPYRIGHT DATE: |
ISBN NO: |
ADVERTISED GRADE LEVEL(S): grades(s) |
SUPPLIES: availability of materials and kits for core curriculum materials |
||
COST: suggested list price |
||
RESOURCE TYPE: student activity book, teacher's guide, books on teaching science, etc. |
||
SUBJECT: selected from major content categories |
||
Please supply the following information: |
||
REVIEWER: _______________________________________ (reviewer's name) |
DATE: ________________________________ (date of review) |
|
RECOMMENDED USER: |
||
(check each that applies) _____ stu _____ tchr _____ adm _____ other (__________) |
||
GRADE LEVEL(S) RECOMMENDED BY REVIEWER IF DIFFERENT FROM THE ADVERTISED LEVEL(S) STATED ABOVE: |
||
(Please circle the specific grade level(s) for which you believe these materials are most appropriate.) |
||
K 1 2 3 4 5 6 7 8 9 10 11 12 |
||
Reviewer:_____________________________________________________________________ |
||
1 |
||
NATIONAL SCIENCE RESOURCES CENTER Smithsonian Institution · National Academy of Sciences |
PEDAGOGY
Instructions: The following questions are designed to help you identify the important elements of each criterion. Please respond by selecting "Yes" if the material meets this goal and "No" if it does not. If "No" is selected, please explain the reason in the space provided below the question. In some instances, the question may not be applicable; then mark "NA."
CRITERIA |
RATING |
||
I. DOES THE MATERIAL ADDRESS THE IMPORTANT GOALS OF MIDDLE SCHOOL SCIENCE TEACHING AND LEARNING? |
|
||
Does the material focus on engaging students in concrete experiences with science phenomena? |
Yes |
No |
NA |
Reason: |
|||
Does the material enable students to investigate an important science concept(s) in depth over an extended period of time? (Especially necessary for core materials.) |
Yes |
No |
NA |
Reason: |
|||
Does the material contribute to the development of scientific reasoning and problem-solving skills? |
Yes |
No |
NA |
Reason: |
|||
Does the material stimulate student interest and relate to their daily lives? |
Yes |
No |
NA |
Reason: |
|||
Does the material allow for or encourage the development of scientific attitudes and habits of mind, such as curiosity, respect for evidence, flexibility, and sensitivity to living things? |
Yes |
No |
NA |
Reason: |
|||
Are assessment strategies aligned with the goals for instruction? |
Yes |
No |
NA |
Reason: |
|||
Will the suggested assessment strategies provide an effective means of assessing student learning? |
Yes |
No |
NA |
Reason: |
|||
Reviewer:__________________________________________________________________________________ |
|||
2 |
|||
NATIONAL SCIENCE RESOURCES CENTER Smithsonian Institution · National Academy of Sciences |
CRITERIA |
RATING |
||
II. DOES THE MATERIAL FOCUS ON INQUIRY AND ACTIVITY AS THE BASIS OF LEARNING EXPERIENCES? |
|
||
Does the material engage students in the processes of science? |
Yes |
No |
NA |
Reason: |
|||
Does the material engage students in planning and conducting scientific investigations? |
Yes |
No |
NA |
Reason: |
|||
Does the material provide opportunities for students to develop questioning skills related to scientific investigations? |
Yes |
No |
NA |
Reason: |
|||
Does the material provide opportunities for students to make and record their own observations? |
Yes |
No |
NA |
Reason: |
|||
Does the material provide opportunities for students to gather data and defend their own evidence? |
Yes |
No |
NA |
Reason: |
|||
Does the material provide opportunities, where appropriate, for students to use mathematics in the collection and treatment of data? |
Yes |
No |
NA |
Reason: |
|||
Does the material provide opportunities for students to express their results in a variety of ways? |
Yes |
No |
NA |
Reason: |
|||
Does the material encourage students to construct and analyze alternative explanations for science phenomena? |
Yes |
No |
NA |
Reason: |
|||
Does the material provide opportunities for students to work collaboratively with others? |
Yes |
No |
NA |
Reason: |
|||
Reviewer:__________________________________________________________________________________ |
|||
3 |
|||
NATIONAL SCIENCE RESOURCES CENTER Smithsonian Institution · National Academy of Sciences |
CRITERIA |
RATING |
||
III. ARE THE MODES OF INSTRUCTION DEVELOPMENTALLY APPROPRIATE? |
|
||
Does the material present a logical sequence of related activities that will help students build conceptual understanding over several lessons? |
Yes |
No |
NA |
Reason: |
|||
Does the suggested instructional sequence take into account students' prior knowledge and experiences? |
Yes |
No |
NA |
Reason: |
|||
Do the suggested student activities develop critical-thinking and problem-solving skills? |
Yes |
No |
NA |
Reason: |
|||
Are the tools and techniques recommended for gathering, analyzing, and interpreting data appropriate for middle school students? |
Yes |
No |
NA |
Reason: |
|||
Does the material incorporate examples of technological applications of science and the interactions among science, technology, and society? |
Yes |
No |
NA |
Reason: |
|||
Does the material include historical examples to help students understand the nature of scientific inquiry and the interactions between science and society? |
Yes |
No |
NA |
Reason: |
|||
Does the material include suggestions for integrating science with other important areas in the middle school curriculum, such as mathematics, language arts, and social studies? |
Yes |
No |
NA |
Reason: |
|||
Reviewer:__________________________________________________________________________________ |
|||
4 |
|||
NATIONAL SCIENCE RESOURCES CENTER Smithsonian Institution · National Academy of Sciences |
ASSESSMENT AND RECOMMENDATION— PEDAGOGICAL APPROPRIATENESS OF MATERIALS |
Instructions: Provide brief responses to the three requests below. It is not necessary to use complete sentences; words and phrases are sufficient. Then complete the recommendation statement. |
Please provide a brief overview of the concepts taught and the activities suggested in this material. |
With the above criteria in mind, please comment on any particular strengths in this material. |
With the above criteria in mind, please comment on any particular weaknesses in this material. |
After reviewing this material with only the above criteria for pedagogical appropriateness in mind, _______ I recommend this material. _______ I do not recommend this material. |
Reviewer:__________________________________________________________________________________ |
5 |
NATIONAL SCIENCE RESOURCES CENTER Smithsonian Institution · National Academy of Sciences |
SCIENCE CONTENT AND PRESENTATION
Instructions: The following questions are designed to help you identify the important elements of each criterion. Please respond by selecting "Yes" if the material meets this goal and "No" if it does not. If "No" is selected, please explain the reason in the space provided below the question. In some instances, the question may not be applicable; then mark "NA."
CRITERIA |
RATING |
||
I. SCIENCE CONTENT |
|
||
Is the science content in the materials accurately represented? |
Yes |
No |
NA |
Reason: |
|||
Is the science content consistent with current scientific knowledge? |
Yes |
No |
NA |
Reason: |
|||
Are important ideas included? |
Yes |
No |
NA |
Reason: |
|||
Are generalizations adequately supported by facts? |
Yes |
No |
NA |
Reason: |
|||
Are facts clearly distinguished from theories? |
Yes |
No |
NA |
Reason: |
|||
Do the suggested investigations lead to an understanding of basic concepts and principles of science? |
Yes |
No |
NA |
Reason: |
|||
Do experiments and activities promote student understanding of how scientists come to know what they know and how scientists test and revise their thinking? |
Yes |
No |
NA |
Reason: |
|||
Reviewer:__________________________________________________________________________________ |
|||
6 |
|||
NATIONAL SCIENCE RESOURCES CENTER Smithsonian Institution · National Academy of Sciences |
CRITERIA |
RATING |
||
II. SCIENCE PRESENTATION |
|
||
Is science shown to be open to inquiry and free of dogmatism? |
Yes |
No |
NA |
Reason: |
|||
Are different scientific viewpoints presented when appropriate? |
Yes |
No |
NA |
Reason: |
|||
Are personal biases avoided? |
Yes |
No |
NA |
Reason: |
|||
Is the writing style interesting and engaging, while respecting scientific language? |
Yes |
No |
NA |
Reason: |
|||
Is vocabulary used to facilitate understanding rather than as an end in itself? |
Yes |
No |
NA |
Reason: |
|||
Is science represented as an enterprise connected to society? |
Yes |
No |
NA |
Reason: |
|||
Reviewer:__________________________________________________________________________________ |
|||
7 |
|||
NATIONAL SCIENCE RESOURCES CENTER Smithsonian Institution · National Academy of Sciences |
CRITERIA |
|
III. TOPICS ADDRESSED FROM NATIONAL SCIENCE EDUCATION STANDARDS |
|
Instructions: Following is a concise listing of the subject matter standards for physical science, life science, and earth and space science, as well as the science and technology standards, identified for grades 5 to 8 in the National Science Education Standards. (The list is from Tables 6.2 through 6.5, pp. 106–107, in the Standards.) For each standard addressed in the material, place a check mark in the column on the right. |
|
STANDARD |
ADDRESSED |
PHYSICAL SCIENCE |
|
• Properties and changes of properties in matter |
|
• Motions and forces |
|
• Transfer of energy |
|
STANDARD |
ADDRESSED |
LIFE SCIENCE |
|
• Structure and function in living systems |
|
• Reproduction and heredity |
|
• Regulation and behavior |
|
• Populations and ecosystems |
|
• Diversity and adaptations of organisms |
|
STANDARD |
ADDRESSED |
EARTH AND SPACE SCIENCE |
|
• Structure of the earth system |
|
• Earth's history |
|
• Earth in the solar system |
|
STANDARD |
ADDRESSED |
SCIENCE AND TECHNOLOGY |
|
• Abilities of technological design |
|
• Understandings about science and technology |
|
Reviewer:__________________________________________________________________________________ |
|
8 |
|
NATIONAL SCIENCE RESOURCES CENTER Smithsonian Institution · National Academy of Sciences |
ASSESSMENT AND RECOMMENDATION—SCIENCE CONTENT AND PRESENTATION |
Instructions: Provide a brief written response to the first two requests below. It is not necessary to use complete sentences; words and phrases are sufficient. Then respond to the multiple choice question on level of quality. |
With the above criteria in mind, please comment on any particular strengths in this material. |
With the above criteria in mind, please comment on any particular weaknesses in this material. |
What is the overall quality of the science presented in this instructional material? _____ Low _____ Medium _____ High |
After reviewing this material with only the above criteria for science content and presentation in mind, _______ I recommend this material. _______ I do not recommend this material. |
Reviewer:__________________________________________________________________________________ |
9 |
NATIONAL SCIENCE RESOURCES CENTER Smithsonian Institution · National Academy of Sciences |
ORGANIZATION AND FORMAT, MATERIALS, AND EQUITY
Instructions: The following questions are designed to help you identify the important elements of each criterion. Please respond by selecting "Yes" if the material meets this goal and "No" if it does not. If "No" is selected, please explain the reason in the space provided below the question. In some instances, the question may not be applicable; then mark "NA."
CRITERIA |
RATING |
||
I. ORGANIZATION AND FORMAT |
|||
Teacher materials (whether module or textbook): |
|||
Does the background material provide sufficient information for the teacher on the scientific content? |
Yes |
No |
NA |
Reason: |
|||
Does the background material provide sufficient information on common student misconceptions? |
Yes |
No |
NA |
Reason: |
|||
Is the format easy to follow? |
Yes |
No |
NA |
Reason: |
|||
Are the directions for conducting laboratory activities and investigations clear? |
Yes |
No |
NA |
Reason: |
|||
Are the suggestions for instructional delivery adequate? |
Yes |
No |
NA |
Reason: |
|||
Are the suggested times for instructional activities reasonable? |
Yes |
No |
NA |
Reason: |
|||
Does the material include appropriate suggestions for incorporating instructional technology? |
Yes |
No |
NA |
Reason: |
|||
Reviewer:__________________________________________________________________________________ |
|||
10 |
|||
NATIONAL SCIENCE RESOURCES CENTER Smithsonian Institution · National Academy of Sciences |
CRITERIA |
RATING |
||
I. ORGANIZATION AND FORMAT, |
|||
Teacher materials (whether module or textbook): |
|||
Are the print materials for students well-written, age-appropriate, and compelling in content? |
Yes |
No |
NA |
Reason: |
|||
Do the illustrations and photographs reinforce the concepts presented? |
Yes |
No |
NA |
Reason: |
|||
Is the overall readability of the materials appropriate for middle school students? |
Yes |
No |
NA |
Reason: |
|||
Textbooks: |
|||
Are major concepts, principles, and ideas adequately developed? |
Yes |
No |
NA |
Reason: |
|||
Are major concepts, principles, and ideas presented in logical sequence throughout the textbook? |
Yes |
No |
NA |
Reason: |
|||
Is each chapter well-organized? |
Yes |
No |
NA |
Reason: |
|||
Reviewer:__________________________________________________________________________________ |
|||
11 |
|||
NATIONAL SCIENCE RESOURCES CENTER Smithsonian Institution · National Academy of Sciences |
CRITERIA |
RATING |
||
II. HANDS-ON MATERIALS, EQUIPMENT, AND SUPPLIES |
|||
Are the equipment, materials, and supplies recommended for use appropriate for middle school students? |
Yes |
No |
NA |
Reason: |
|||
Are instructions on manipulating laboratory equipment and materials clear and adequate? |
Yes |
No |
NA |
Reason: |
|||
Is a master source list of materials provided? |
Yes |
No |
NA |
Reason: |
|||
Is a list of materials included for each activity? |
Yes |
No |
NA |
Reason: |
|||
Is a complete set of materials readily available at a reasonable cost? |
Yes |
No |
NA |
Reason: |
|||
Are consumable materials easily obtained and affordable? |
Yes |
No |
NA |
Reason: |
|||
Reason: |
Yes |
No |
NA |
Are appropriate safety precautions included? |
Yes |
No |
NA |
Reason: |
|||
III. EQUITY ISSUES |
|||
Is the material free of cultural, racial, ethnic, gender, and age bias? |
|
||
Reason: |
|||
Are appropriate strategies included to address the diversity of middle school students' needs, experiences, and backgrounds? |
Yes |
No |
NA |
Reason: |
|||
Reviewer:__________________________________________________________________________________ |
|||
12 |
|||
NATIONAL SCIENCE RESOURCES CENTER Smithsonian Institution · National Academy of Sciences |
ASSESSMENT AND RECOMMENDATION—ORGANIZATION AND FORMAT, MATERIALS, AND EQUITY |
Instructions: Provide a brief response to the request below. It is not necessary to use complete sentences; words and phrases are sufficient. Then complete the recommendation statement. |
With the above criteria in mind, please comment on any particular strengths in this material. |
After reviewing this material with only the above criteria for organization and format, materials, and equity issues in mind, _______ I recommend this material. _______ I do not recommend this material. |
Reviewer:__________________________________________________________________________________ |
13 |
NATIONAL SCIENCE RESOURCES CENTER Smithsonian Institution · National Academy of Sciences |
RECOMMENDATION BASED ON ALL CRITERIA |
Instructions: Complete the section below. If you "recommend with reservations" or "do not recommend" a material for inclusion, briefly state your primary reason in the space provided. |
Based upon all aspects of my review of this material, _____ I highly recommend this material. _____ I recommend this material. _____ I recommend this material with reservations. |
Primary reason for reservations: _______________________________________________________________ __________________________________________________________________________________________ __________________________________________________________________________________________ |
_____ I do not recommend this material. |
Primary reason for rejection: __________________________________________________________________ __________________________________________________________________________________________ __________________________________________________________________________________________ |
Reviewer:__________________________________________________________________________________ |
14 |
NATIONAL SCIENCE RESOURCES CENTER Smithsonian Institution · National Academy of Sciences |