1 The Relationship of Instructional Materials to Achieving K-12 Science Standards
The goal of the Committee on Developing the Capacity for Selecting Effective Instructional Materials (''the Committee") was to produce a tested standards-based instrument that would be helpful to people who select instructional materials for use in the science classroom. In so doing, the Committee was responding to the request of teachers for instructional materials that would enable them to teach science using a standards-based approach. Without these standards, many teachers will continue to teach science as they have in the past, and the efforts to increase student achievement will falter.
The Committee recognized early on that the selection instrument would have to be flexible in order to accommodate both national and state standards, as well as the diversity of standards and interests involved in decision-making at the local level, including teachers, principals, science supervisors, parents, scientists, and school board members. Consequently, the selection instrument, which begins on page 41 of this report as the Guide to Selecting Instructional Materials, has been designed for use with whatever standards have been adopted by the relevant school district.
The importance of science education has been discussed in-depth in tens if not hundreds of professional and popular articles and books, including the National Science Education Standards (NRC, 1996) and Benchmarks for Science Literacy (AAAS, 1993). These discussions reflect two fundamental conclusions. First, a basic understanding of science is vital for everyone, because science and technology have become relevant to enterprises as varied as business, agriculture, manufacturing, law, and government, and they have a profound impact on many contemporary personal, social, and political issues. Second, the security and economy of the
nation will depend on generating a sufficient number of well-trained scientists and engineers. Thus, science education in U.S. schools must be effective for all students, encouraging talent and interest wherever it is found (NSB, 1998, 1999).
A Nation At Risk (NCEE, 1983) challenged our country to improve science education for all students. While we have made some progress, much work remains. The Third International Mathematics and Science Study (TIMSS) showed clearly that while American elementary school students perform well in comparison to their foreign counterparts, their performance steadily declines in middle and high school (NCES, 1998a; Schmidt, McKnight, and Raizen, 1997; Schmidt and McKnight, 1998; NSB, 1998) Research associated with the TIMSS project found that many science textbooks in use in the United States emphasize breadth of coverage at the expense of deep understanding of fundamental scientific concepts (Schmidt et al., 1997; Schmidt and McKnight 1998). Even if the TIMSS data and interpretation are flawed in some respects — as some have argued (Rotberg, 1998; as referenced in Schmidt and McKnight, 1998) — we should take them as a serious challenge as we continue our efforts to improve instruction and performance.
SCIENCE EDUCATION STANDARDS
The publication of the National Science Education Standards, abbreviated in this report as Standards (NRC, 1996), represents the core initiating element in the National Academies' response to the challenge of changing and improving science education in the United States. Complementary and consistent activities are ongoing at the American Association for the Advancement of Science under the title Project 2061 (AAAS, 1989, 1993).
The Standards (NRC, 1996) and Benchmarks for Science Literacy (AAAS, 1993) were developed to provide goals for the entire nation. They implicitly recognize that U.S. educational policy is made and implemented locally in the states and school districts. It is expected that, depending on local interests and needs, diverse routes will be taken to reach the goals of the standards. Nevertheless, national standards are important if all children are to experience successful science instruction. Currently, there is enormous local variability in the quality and quantity of science programs. In 1996, for example, 41% of eighth grade students in North Dakota met or exceeded the national Goals 2000 proficiency performance standards in science, while only 5% and 20% of eighth grade students in the District of Columbia
Throughout this report the following conventions are used to address the variety of "standards" for science education: The National Science Education Standards are referred to as Standards (with a capital "S") and the Benchmarks for Science Literacy are referred to as Benchmarks. These two documents are referred to as national standards — both being intended to provide guidance nationally and being largely consistent with one another (AAAS, 1997). The standards developed or adopted by states, school districts, or educational enterprises such as ''America's Choice" are referred to as standards (with a lowercase "s") or local standards.
and California, respectively, achieved this level (NCES, 1998b; NEGP, 1998).
Many state governments expressed support for President George Bush's 1989 initiatives to establish national goals for education and responded favorably to the Standards (Stedman, 1993). According to the National Science Board (NSB, 1999) and others (CCSSO, 1997; Celebuski, 1998), all states have adopted or are adopting standards for science education. While these differ extensively in content, breadth, and rigor, the adoption of standards of some kind by all states marks a significant advance. Nevertheless, without a continuing effort to bring state or national standards into the classroom, even those school systems poised to reform can fail to accomplish change. For example, although 66% of public school principals have stated that they require application of standards in science lessons (NEGP, 1998), data indicate that teachers rarely adhere to the standards' recommendations (NCES, 1998a). Data from earlier initiatives to improve science teaching suggest that teachers often do not receive the needed intellectual, financial, and administrative support for new initiatives (Bybee, 1996, 1997; Hutchinson and Huberman, 1993).
The development of the Standards took into account various factors that contribute to the ineffectiveness of current science education. These include excessively broad curricula with no time to cover topics in-depth; absence of hands-on participation in science experiments; the didacticism of much science education; the absence of inquiry-based instruction; poor initial and continuing teacher education in science and science teaching; inadequate provision of necessary materials and equipment; and the poor quality of many available teaching materials, especially textbooks. Hundreds of teachers, scientists, school administrators, educational researchers, and others participated in the development,
drafting, review, and final revisions of the Standards (NRC, 1996).
Since the publication of the Standards, the National Research Council has established the Center for Science, Mathematics, and Engineering Education and has published various reports designed to help school districts and others apply the Standards (NRC, 1997b, 1999a,b, forthcoming). The longterm goal of these activities is achieving quality science education for all K-12 students in the United States.
The Standards encourage teachers to engage students in the process of scientific inquiry by directing them to ask questions about the natural world, design experiments to answer these questions, interpret the experimental results, and discuss the results with their peers. Such inquiry-based teaching enhances student understanding of scientific concepts (NRC, forthcoming), and it is intended to equip all students with the analytical skills they will need in the future to interpret the world around them. Importantly, although the Standards stress inquiry-based teaching, they do not assume that all science can be learned through an inquiry process, given the amount and diversity of scientific concepts that should be learned.
Besides describing scientific content to be learned by grades 4, 8, and 12, and encouraging research-based teaching methods, the Standards present standards for school district administrators, principals, and policy makers, including local school boards (NRC, 1996). The document also contains guidance to help schools develop effective science education programs, specifying a need for:
a curriculum design that presents content at each grade level that is appropriate in-depth and number of topics covered for the age and previous educational experience of the students;
teacher education and continuing professional development that support the curriculum and provide teachers with the skills needed to teach science with an inquiry-based approach;
provision of adequate science materials to all classrooms;
assessment methods that are consistent with the curricula and provide reliable methods for evaluation of student learning and teacher instructional proficiency;
parental involvement in understanding the nature of good science programs and in planning improvements;
commitment of the community, including local business, in ways that demonstrate the relevance of science to adult life and work; and
recognition by local and state school administrators and boards of the vital importance of an understanding of science and technology for the future success of children.
Some of these elements have been further addressed in detailed reports by the National Research Council and others (Bybee, 1996, 1997; NRC, 1997a,b).
Finally, for science teaching programs to achieve the goals of the Standards, teachers and students will require access to instructional materials that are accurate in science content, clear in their presentation of scientific concepts and processes, appropriate for the age of the children who will use them, and suitable for the local community, as well as consistent with the aims of the Standards. This report deals with this issue.
Instructional materials for K-12 school science include textbooks, laboratory manuals, other books about scientific matters, kits, software, CDs, and other multimedia materials, such as videos, that provide equipment and materials for specific inquiry-based lessons. Not only are these materials a primary source of classroom science learning, but because the professional development for teachers is often structured around instructional materials, they also play a profound role in the education of teachers. Thus, to achieve the learning goals of the Standards or Benchmarks, students and teachers must be provided with instructional materials that reflect these standards. Moreover, teachers will be more likely to provide the requisite classroom experiences if professional development programs provided by school systems are grounded in standards-based instructional materials. For these reasons, the selection of instructional materials that reflect the learning goals of the standards is a central issue. This is no simple task, since schools and school districts must select from among the broad array of materials produced by U.S. publishers. As documented in the TIMSS project, many instructional materials used for teaching science in the United States emphasize breadth of coverage at the expense of a deep understanding of fundamental scientific concepts (Schmidt et al., 1997).
Ultimately, teachers decide what to teach in the classroom, and many teachers — especially elementary school teachers — base their lesson plans on the class textbook and on other instructional materials rather than on the "intended" curriculum specified by official policies (Woodward and Elliott, 1990). In 1991, Horizon Research, Inc., surveyed 930 past winners of the Presidential Award for Excellence in Mathematics and Science Teaching (PAEMST), comparing them with a random national sample of 2,065 elementary math and science teachers.
PAEMST teachers rely far less than their peers do on textbooks in their teaching. Only 17% of presidential award-winning science teachers of grades 1-6 said they consider textbooks a "major influence" on what they teach. By contrast, 59% of the national sample of science teachers overall felt that way (Weiss, 1991). Thus, instructional materials play an unexpectedly important role in education: when the materials align with the standards, teachers are more likely to attend to the standard's goals; when they align poorly, teacher goals will diverge from those of the standards.
Another important effect on what teachers teach arises from assessment practices. Statewide assessments can dictate much of what teachers teach. Not surprisingly, teachers want instructional materials that can help them prepare students for mandated assessments. "Assessment of student performance exerts extraordinary influence on the lives of children and their families and on every level of the education system" (Stern, 1999), including the selection of instructional materials. The approaches to science education in the Standards stress classroom assessment as a critical component of instruction. Such assessments are needed by the teacher in order to identify what the students have learned and not learned, thereby informing the subsequent instructional topics and processes. However, statewide assessments generally have a different purpose. They are designed to measure what a student has learned at a given point in time. Moreover, the dependence of the tests on a multiple-choice format tends to put a premium on memorized and isolated facts in comparison to understanding of science concepts. For example, in a 1994 study of assessments in 17 states that test science achievement, only 7 states' assessments were found to include items designed to measure conceptual understanding and application, and 15 of the state tests primarily focused on basic skills measured by multiple-choice items (CPRE, 1996). Teachers, principals, school district administrators, and parents may question whether instructional materials that are aligned with standards will enable students to do well on the statewide assessments.
In addition, instructional materials affect teaching indirectly by influencing the greater community. For instance, parents use the content of the student materials or textbooks to examine what their children are learning. Often the sole link to the classroom, these materials can determine whether parents support or object to the school science programs.
Procedures for Selecting Instructional Materials in Public Schools
There is a great deal of variation from state to state with respect to the statutes, policies, regulations, and resources governing local K-12 education and the selection of instructional materials. Some states mandate that state adoption guides, recommended lists, or state standards be considered; and political issues sometimes affect the development and enforcement of state policies. Ultimately, however, the local level is where the final decisions are made about which science instructional materials will make it into the classroom.
According to information gathered by the Council of Chief State School Officers, 13 states specify that the state will determine which instructional materials may be used or that the state will publish a list of materials from which local school districts may choose. In another 8 states, state authorities recommend materials, but the selection is actually carried out by the local districts. In all of these states but one (Idaho, where districts are restricted by law and must choose only state-approved materials), districts can choose other materials by following a waiver process (CCSSO, 1997). In California, for example, a school district can seek approval from the state board of education to spend state instructional material allocations on materials not on the state adoption list (IMF, 1989).
State adoption lists influence the education of many U.S. students; the adoption list in California alone represents 10% of the textbook market nationwide, or 5.6 million public school students (CBEDS, 1997). Consequently, adoption or recommendation is, for publishers of instructional materials, a high-stakes make-or-break business that provides access to large markets. This is especially true in the largest adoption states — California, Texas, and Florida — which together represent 20% of the national textbook market (Wheeler, 1999b).
Competition for adoption or recommendation causes publishers to adopt cost-saving measures by publishing a single textbook that is acceptable in several states (Tyson, 1997). To do so, textbook publishers often sacrifice quality for quantity by covering multiple curricula (many of which are broad to begin with), thereby sacrificing depth for breadth (Tyson, 1997). As outlined in A Splintered Vision: An Investigation of U.S. Science and Mathematics Education (Schmidt et al., 1997), such materials tend to emphasize scientific vocabulary at the expense of the acquisition of fundamental understanding of scientific concepts.
State and local selection procedures for instructional materials may require vendors to make formal presentations
and provide multiple samples of their wares, as well as professional development for teachers if the materials are chosen. Small vendors often lack the resources to provide such services and are therefore virtually excluded from consideration. However, small suppliers may offer quality science instructional materials. The effect of these practices is to limit the availability of materials that could substantially contribute to attaining the learning goals.
Common Considerations in the Local Selection of Science Instructional Materials
In the 29 states where there are no state-level policies for selection or recommendation of instructional materials, the challenge of finding appropriate instructional materials falls entirely on individual districts or schools. Local school districts may receive some assistance from the state educational authorities. The amount and kind of support, which varies from state to state, may include technical support from state science supervisors or state science consultants, who bring varying degrees of science content expertise to the selection. In comparison to state selection committees, the district or individual school selection committees may be less familiar with standards, and they often lack sufficient human and financial resources for establishing a well-informed and thorough selection procedure.
In these 29 states, publishers play a lesser role. Those charged with making selections can make use of various publications that describe and, in some cases, evaluate instructional materials. Among these are the guides published by the National Science Resources Center (NSRC, 1996, 1998), Project 2061 (Roseman, Kesidou, and Stern, 1997; Roseman, 1997a,b; Kesidou, 1999; AAAS, forthcoming a,b,c); and the National Science Foundation (NSF, 1997).
Just as there is great variation across states regarding the policies and practices for selecting science instructional materials, each local context is different in terms of culture, capacity, and process. Nevertheless, there are several issues that arise repeatedly during local decision-making:
What is the budget for the review and selection process?
From whom can the committee obtain current information about expenditures for such items as instructional materials and professional development?
What student performance and enrollment data are currently available? From whom can the committee get additional data?
Does the district have in place the facilities and systems to support a standards-based science program?
Who will be responsible for facilitating the instructional materials review and selection process?
Who will comprise the review and selection committee(s)? How will they be chosen?
How will the review and selection committee members be prepared for their task?
How will the review and selection committee(s) function? How will decisions be made? How and by whom will the final recommendations be made?
What will be the role of district administrators? What degree of influence will district personnel have on the selection process?
How will a list of vendors be generated?
What materials and information will be solicited from the vendors?
What other sources of information will be provided to the committee(s)?
What are the district's standards or learning goals? Are they widely accepted and in use?
Are the current instructional materials aligned to the standards? Are they being used?
Procedures for Selecting Instructional Materials in Private Schools
In the United States there are a variety of schools other than those administered by local public school systems; these include parochial schools, independent schools, nationally administered public schools run by for-profit organizations (e.g., the Edison Project Schools), and a growing number of public charter schools. Informal inquiries have revealed selection procedures that range from school-wide coherent policies, to departmental committees, to selection by individual teachers.
The Edison Project, for example, selects materials centrally and the same materials are used in all its schools (currently 24,000 students in 50 schools in 12 states). The selection process is initiated by setting curriculum standards and objectives with the advice of consulting groups. Their science standards are described as a synthesis of the Standards and Benchmarks. Instructional materials are then evaluated with reference to the standards. Among the issues considered are (1) how well the materials will support the teachers and (2) evidence that the materials (or program) actually works. The Edison Project reports that there are insufficient studies on science learning to help very much with evaluation of efficacy (Chubb, 1999).
An urban, independent elementary school that emphasizes its science program reported that a science department committee first defines the curriculum and then selects texts or kits
that dovetail with the curriculum. It is generally assumed that a single text or multigrade program will not be adequate; rather, a main text and supplementary materials are chosen. Additional considerations include (James, 1999):
Will the materials support the scope and sequence of the curriculum?
Do the materials consider the history of the scientific discipline?
Are supplementary readings provided at multiple reading levels so that both advanced and learning-disabled students can find appropriate readings?
Are meaningful projects and investigations embedded in the text?
An urban, independent high school reports that departments and teachers have a great deal of independence in instructional material selection. The teacher who will use the materials makes the final decision. The first step is to consider the topics to be covered. This is based on a prior departmental consideration of Standards and Benchmarks and definition of the skills to be acquired before graduation. Then materials — generally textbooks — are inspected for their match to the topics. Materials for inspection are obtained from publishers and also by visiting a local university's education library. Additional considerations include:
Does the text make it possible for the teacher to choose the order in which topics are presented?
Is the material clearly written?
Are laboratory exercises included in the book itself?
How challenging is the material?
Selected materials are presented to other teachers in the department for inspection and comments before the teacher decides on a textbook (McArthur Parker, 1999).
Common Issues Arising During Selection
When selecting science instructional materials, certain problematic situations are common, examples of which are described below.
Publishers typically claim that their science instructional materials are standards-based. Because this may not always be true, evaluators need to establish the reliability of such claims (Kesidou, 1999).
There are vastly different pedagogical approaches in science instructional materials. Some materials are designed around an inquiry-style pedagogy; some emphasize hands-on materials; and others are textbooks that may or may not offer materials for student investigations. Failure to
distinguish among these approaches risks the selection of materials inconsistent with district requirements. Yet, even experienced teachers may not perceive the goals of the standards when they occur in innovative materials (Bush et al., forthcoming).
Selection committees may choose materials without recognizing that their effective classroom use depends on providing teachers with extensive professional development in the pedagogical approach embodied in the materials. Such a situation arises, for example, if the materials represent an activity-based or inquiry-based science program, and the teachers have traditionally depended on textbooks and didactic lessons (Little, 1993).
A related issue is the influence of assessments. Assessment tools used in the school district need to be consistent with the learning goals, pedagogical approach, and assessments built into the materials (Webb, 1997).
Financial resources are almost always an issue. The amounts budgeted for instructional materials may not be sufficient to purchase desirable materials, and tradeoffs may be required. Budget restrictions may also result in the use of dated, even inaccurate, materials long after they should have been set aside.
INSTRUCTIONAL MATERIALS AND TEACHING
Instructional materials are a primary source of science learning in the nation's classrooms. In high schools and middle schools, textbooks are essential supplements to the limited amount of material that can reasonably be presented in the classroom time available to the teacher. Packaged instruments and materials (kits) for laboratory and hands-on experiences are an enormous help to busy teachers at all levels, K-12. The availability of excellent instructional materials is critical for elementary school teachers who, in spite of minimal formal scientific education of their own, are called on to teach a range of scientific concepts from chemistry to natural history, earth science, astronomy, and ecology. The closer instructional materials adhere to the goals of state and national standards, the more likely the teacher is to succeed in achieving those goals.
The Influence of Instructional Materials on Professional Development
Instructional materials influence the continuing professional development of teachers in several ways. For elementary school teachers, the materials often provide basic information on content and pedagogy. Formal professional
development for teaching the curriculum may be provided, but it is often brief and superficial — especially with respect to the content standards (Massell, Kirst, and Hoppe, 1997). Instructional materials are often accompanied by teacher manuals, which are important resources for teachers. If the goal is to teach according to standards, the quality of the instructional materials is as important to teachers as it is to students.
Because instructional materials influence curricula, they also affect the content of professional development workshops covering the adopted curriculum; in particular, inexperienced teachers who are preoccupied with the practicalities of teaching are interested in workshops directly related to their lesson plans (Loucks-Horsley, Stiles, and Hewson, 1996). Thus, the quality of the instructional materials will directly affect the quality of the teaching.
The review of instructional materials during a selection process, if well structured, can serve as an important professional development experience for participants. Review processes that require understanding of the standards and foster rigorous analysis of the materials can be powerful learning experiences (Brearton and Shuttleworth, 1999). Teachers engaged in such reviews can develop a better understanding of the science content, the requirements for inquiry-based teaching, and the resources needed for standards-based science instruction.
THE CRITICAL IMPORTANCE OF INSTRUCTIONAL MATERIALS
In the United States instructional materials direct class curriculum and instruction, define the accuracy of the science knowledge imparted, influence professional development of teachers, and affect the educational roles of parents. From the perspective of promoting standards-based science education, instructional materials are critical tools. Adoption of materials that promote the learning of important ideas and skills is then essential if standards-based education is to become a reality in the nation's classrooms. Such materials would improve curricula and significantly impact daily teaching practices (Tyson, 1997; Tyson-Bernstein, 1988).
Current selection procedures, particularly those at the local level, often lack the capacity to sift systematically through instructional materials and identify those that align with the adopted standards. Evaluation procedures are needed to encourage evaluators to become knowledgeable about the standards and use them when judging instructional materials. Such evaluation procedures would, ideally, also be educational experiences for the
evaluators. Teachers and local school boards can, with the assistance of knowledgeable scientists, ultimately build the capacity to judge the materials themselves. The task becomes more formidable as the number and variety of materials increases in traditional textbook form, in packaged lessons such as kits and videos, and now on the Internet. At present, the conditions surrounding materials selection may lead evaluators to review materials superficially and choose those that look attractive, appear to reduce budget outlays, or simplify teachers' roles. For this reason, building the local capacity to select instructional materials that support the goals of state and national standards is of paramount importance.