Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 39
2 The Influence of the National Science Education Standards on the Science Curriculum James D. Ellis University of Kansas Any attempt to evaluate the influence of national standards on the science curriculum is perplexing. The task illustrates the complexity of the educational system and the lack of clarity in the language used to describe it. In science education, an initial confusion emerges when defining what is meant by national standards in science education. During the past decade, multiple efforts have been undertaken to lead and influence the reform in science education. The American Association for the Advancement of Science (AAAS) established Project 2061—a long-term initiative to improve science literacy—with Science for All Americans (AAAS, 1989) and Benchmarks for Science Literacy (AAAS, 1993) being key early products of this work. The National Science Teachers Association (NSTA) also has been a leader in reform efforts, beginning with its Scope, Sequence, and Coordination project and more recently by disseminating and supporting the use of the National Science Education Standards. The National Research Council (NRC) brought together these reform efforts by producing a unifying document, the National Science Education Standards (NSES), and through its efforts to disseminate and to support states and school districts in translating the NSES into improved science programs. These reform efforts are inseparable because the projects are interrelated. Key leaders have contributed to the work of multiple projects and each organization has built on the work of the other. For this review of the literature, therefore, the author does not claim to separate the influence of one of these reform efforts from another. There are several key ideas from the NSES and Project 2061 that establish the reform agenda for science education: High expectations of science learning are set for all students. When appropriate learning environments are provided, all students can increase their knowledge, understanding, and appreciation of science. Teaching for depth of understanding of important science concepts is preferred, rather than recall of science facts. Teaching less content in depth is better than covering too much content superficially. Science literacy encompasses a wide range of content, including inquiry, history and nature of science, personal and social perspectives of science, science, and technology, in addition to the science domains of life science, physical science, and earth and space science. Science content is organized into a few unifying conceptual themes. Learning is an active process and the program should be developmentally appropriate, interesting, and relevant to students’ lives.
OCR for page 40
Curriculum, instruction, and assessment must be aligned to improve science literacy. Science curriculum should be coordinated with other subjects, especially mathematics. Sufficient resources are required to achieve science literacy, including quality teachers, time, materials, equipment, space, and community. National, state, and local policies must be congruent with and support the science program. Once one accepts the complex nature of national standards in science education, additional issues require clarification. The following two sections will address these issues: What is the science curriculum? What counts as evidence of influence? The third section of the paper will provide the results of the literature review summarizing the evidence of influence of the NSES on the science curriculum. The paper will end with sections on conclusions and recommendations for research. WHAT IS THE SCIENCE CURRICULUM? The simple term “the science curriculum” has many meanings. A common meaning of curriculum is the set of instructional materials used in teaching science, including textbooks, supplementary readings, multimedia materials, and laboratory exercises. For many teachers, the textbook is the curriculum (Schmidt, 2001a; Weiss, Banilower, McMahon, and Smith, 2001). However, as illustrated in Figure 2-1, the curriculum has multiple dimensions: (1) the intended curriculum, (2) the enacted curriculum, and (3) the assessed curriculum (Porter and Smithson, 2001b). For the purposes of this study, the author examined the potential influence of the NSES on each of the three curriculum dimensions illustrated in Figure 2-1. This figure, however, is an incomplete illustration of relationships. Other graphical depictions would better emphasize the relative relationship among these curriculum dimensions. For instance, a Venn diagram would illustrate the overlap among these dimensions (see Figure 2-2). There are goals and outcomes in common among the intended curriculum, enacted curriculum, and assessed curriculum or in common among any two of the three dimensions. Also, there are goals and outcomes that are unique to one dimension, such as being part of the assessed curriculum, but not part of the intended or enacted curriculum. Science literacy is the whole of the Venn diagram. Curriculum alignment is achieved as the circles increase in overlap, and science literacy comes more into focus as alignment is achieved. The concentric circle representation in Figure 2-1, however, is useful in discussing the contents of each of the curriculum dimensions. Science literacy is at the center of Figure 2-1. The purpose of the NSES is to promote science literacy. The NSES document defines science literacy as what all citizens should know and be able to do and provides standards for the educational system to achieve science literacy. The curriculum is a key component in achieving science literacy. Science literacy is a central element of the science curriculum. The morphology of science literacy, however, is transformed from the intended curriculum to the enacted curriculum to the assessed curriculum though the interpretation and actions of educational leaders, parents, teachers, and students. The intended curriculum is a statement of goals and standards that defines the content to be learned and the structure, sequence, and presentation of that content. The intended curriculum is defined by national guidelines, such as the NSES, by state standards and curriculum frameworks, by local standards and curriculum frameworks, and by publishers of instructional materials. The intended curriculum is interpreted by teachers, administrators, parents, and students to create the enacted curriculum. The enacted curriculum is the totality of the opportunities to learn experienced by the students. The enacted curriculum differs from the intended curriculum because it is mediated by the teacher, the students, available instructional materials, and the learning environment.
OCR for page 41
FIGURE 2-1 Three dimensions of science curriculum. SOURCE: Porter and Smithson (2001b).
OCR for page 42
FIGURE 2-2 Venn diagram of overlapping spheres of influence. Instructional materials play a key role in bridging the gap between standards, the intended curriculum, and the enacted curriculum. Instructional materials in themselves, however, are merely a tool for teachers to use as they enact the curriculum in their classrooms. Good teachers can take a traditional textbook, adapt and enrich it with inquiry-investigations, focus on key content rather than coverage of the complete textbook, and enact a high-quality, standards-based curriculum. Instructional materials are not “teacher-proof.” Schools can provide teachers with the most innovative, standards-based materials and find that the materials are not used, are not taught as designed, or are modified so that the curriculum as enacted does not differ significantly from that of teachers using traditional materials. An important question to consider is: What is the role and responsibility of instructional materials in enacting the curriculum? I suggest that while the quality of the tool matters, the more critical question is the quality of the craftsman. I also suggest that a variety of instructional designs and approaches can support teachers in achieving quality science education programs. While the instructional materials ought to support, encourage, enable, and align with best practices outlined in the NSES, no single design or template can meet the diverse needs of students, teachers, and school districts throughout the nation. The assessed curriculum is the narrowest of curriculum dimensions. The assessed curriculum is limited to the knowledge and abilities for which current measurement tools and procedures are available to provide valid and reliable information about student outcomes. There are several layers to the assessed curriculum in science: (1) national assessments, (2) state assessments, (3) classroom summative assessments, and (4) classroom
OCR for page 43
formative assessments. The usefulness of the data from assessments to inform teaching decisions increases the closer the assessment is to student learning (the fourth layer). Figure 2-1 illustrates the potential spheres of influence of the NSES on the science curriculum. The authors of the NSES carefully defined its relationship with the science curriculum. In the NSES, curriculum is defined as “the way the content is delivered: It includes the structure, organization, balance, and presentation of the content in the classroom” (NRC, 1996, p. 22). The NSES are purposely vague. They are not meant to be a national science curriculum. The authors stress, “the content standards are not a science curriculum … are not science lessons, classes, courses of study, or school science programs. The components of the science content described can be organized with a variety of emphases and perspectives into many different curricula” (NRC, 1996, p. 22). The NSES target the intended curriculum as their primary sphere of influence. The NSES represent voluntary, national (not federal) standards for science education. This is an acknowledgment that the Constitution of the United States delegates responsibility for education to the states, and that there is a long tradition of local control of curriculum throughout the nation. States are free to develop their own standards, guidelines, and curriculum frameworks for science education: “Founded in exemplary practice and research, the NSES describe a vision of the scientifically literate person and present criteria for science education that will allow that vision to become a reality” (NRC, 1996, p. 11). Further, “science education standards provide criteria to judge progress toward a national vision of learning and teaching science in a system that promotes excellence, providing a banner around which reformers can rally” (NRC, 1996. p. 12). The NSES provide criteria to help state and local personnel design curriculum, staff development, and assessment programs. While the primary focus of the NSES is on the intended curriculum, they also directly influence the enacted and assessed curriculum. Even though the most manageable system of education might be for the national or state government to establish a singular curriculum framework for science within which teachers and students enact the curriculum and students demonstrate achievement on assessments provided by the state that are fully aligned with the state curriculum standards, the educational system in the United States is much messier. Decisions about what is taught, how it is taught, how it is learned, and how it is assessed are made daily by teachers and students in their classrooms. Therefore, when a teacher interprets the curriculum framework; adapts, modifies, and enriches instructional materials; and accommodates instruction and assessment for the diverse needs and abilities of students, there is an opportunity for the NSES to have influence. If the teacher is aware of and understands the NSES, then there is the potential for the teacher to align the enacted curriculum with them, even where the intended curriculum proscribed by state and local education agencies deviates from the NSES. Similarly, the NSES might directly influence the assessed curriculum through the work of those who develop performance standards and assessment instruments and procedures—assessment specialists for textbook publishers, educational specialists at the state level and local level, and teachers at the classroom level. WHAT COUNTS AS EVIDENCE OF INFLUENCE? The purpose of this investigation is to determine the influence of the NSES on science curriculum. To begin the investigation, however, one must have some idea of the kinds of evidence that might support a claim of the degree of influence. An obvious focus might be to examine changes in instructional materials available in K-12 science education. This derives from the concept of curriculum as synonymous with instructional materials. Using instructional materials, however, as the main source of evidence has serious pitfalls. First and foremost, it takes a decade or more for innovations to appear in mainstream instructional materials, and the NSES were published only six years ago. Another pitfall is that adopting and implementing the materials do not guarantee that the teachers believe in and are practicing the approaches to learning and teaching espoused by the program. The National Science Foundation (NSF) is the primary supporter of projects to develop innovative instructional materials through its Instructional Materials Development (IMD) program. IMD projects typically take at least three years, five for full-year comprehensive projects, to complete the cycle of development, testing,
OCR for page 44
revision, evaluation, and publication. Publishing companies are less likely than NSF to invest heavily in the development of innovative programs until there is evidence of acceptance in the marketplace for the new approaches. So major textbook programs often lag several years behind the introduction of innovative instructional materials. Furthermore, the typical adoption cycle for instructional materials in public schools often stretches for as long as seven years, which means that the infusion of new ideas into the science curriculum might take seven or more years if the sole mechanism of curriculum change were through the adoption of new instructional materials. In addition, it typically takes three or more years for teachers to adopt new approaches to teaching and learning, which are required by the new standards-based programs. Therefore, if the process of curriculum renewal is a linear process beginning with national standards that lead to revisions in state standards that lead to changes in instructional materials, which are adopted and enacted by teachers at the local level, then it would be unreasonable to expect to see substantial evidence of influence of the NSES on the science curriculum in the six years since their publication. Fortunately, curriculum development and national science education standards have co-evolved during the past two decades. The release of A Nation at Risk (National Commission on Excellence in Education, 1983) initiated the process of research and development and of consensus building (a political process) in the scientific and educational communities and the public that culminated in the NSES. More than 300 reports have been published that analyzed and commented on the need for a revised vision of science education. As reported by Cozzens (2000), starting in 1986, NSF began funding major initiatives—known as the Triad Projects—for the development of comprehensive programs in science and mathematics for the elementary grades (K-6), continuing until the present with projects to develop comprehensive materials for all science and mathematics in grades K-12. In addition to funding comprehensive programs, the IMD program has supported the development of a vast array of innovative units of instruction across all areas of science, which serve as models for a variety of approaches to designing high-quality, standards-based materials. By the mid-1990s, multiple national-level projects were undertaken to develop a new vision of science education. AAAS began by producing Science for All Americans (AAAS, 1989), which established a growing consensus of major elements for science literacy and the kind of approaches to curriculum and instruction required to achieve it. NSTA produced The Content Core (1992) and its vision of Scope, Sequence, and Coordination, emphasizing the need for a coordinated coherent curriculum. The Biological Sciences Curriculum Study (BSCS) collaborated with IBM on a design study for elementary school science and health (BSCS and IBM, 1989). The National Center for Improving Science Education (NCISE), in collaboration with BSCS, produced a series of frameworks for curriculum and instruction in science for the elementary years, middle years, and high school (NCISE, 1989, 1990, 1991). AAAS produced Benchmarks for Science Literacy (AAAS, 1993), which provided detailed specifications of science content to be learned at four stages in the K-12 program (K-2, 3-5, 6-8, 9-12). BSCS produced Developing Biological Literacy (1993) and Redesigning the Science Curriculum (Bybee and McInerney, 1995). Therefore, the science education community has been defining science literacy and engaged in curriculum development for at least a decade prior to the release of the NSES. Curriculum developers played a key role in the development of the NSES. A cursory examination of the key leaders in the studies of curriculum reform (including the NSES writing teams) and the leaders of the curriculum development projects finds considerable overlap. This is because the major curriculum development organizations—BSCS, Education Development Center (EDC), Lawrence Hall of Science, Technical Education Research Centers (TERC), and National Science Resources Center (NSRC)—had been working to design and develop curricula that embodied the growing consensus in the science education community. Therefore, it is conceivable and justifiable to analyze instructional materials that have been published by these IMD projects during the past few years for evidence of alignment with the NSES, and to use this evidence to draw conclusions about the potential influence of the NSES on the science curriculum. Instructional materials, however, are only one component of the science curriculum. A thorough search for evidence of the influence on science curriculum would consider evidence of impact on all spheres of potential influence. The first level of influence might be on the outer sphere—the intended curriculum—which includes state and local standards and curriculum frameworks, in addition to science textbooks and instructional materials
OCR for page 45
(previously discussed). What one would hope to find is research literature that investigates the degree of change in state and local curriculum documents toward greater alignment with the NSES. A secondary level of influence of the NSES is on the enacted curriculum. Evidence of the enacted curriculum requires intensive data collection efforts on the materials used by and the beliefs and practices of teachers. Sources of evidence include data on the instructional materials used by teachers, unit plans and lesson plans designed by teachers, surveys, observations, and interviews of teachers and students of their interaction with the curriculum in the learning environment. Research of this type overlaps substantially another area of this overall study—teachers and teaching—that addresses how the curriculum is delivered. The last two spheres of influence for the NSES are the assessed curriculum and science literacy. Evidence of impact of the NSES on assessment and accountability is the focus of another component of this overall study. Evidence of the impact of the NSES on student learning is a tertiary level of influence. Measured student learning, used for accountability, is a result in part of student interaction with the enacted curriculum, and limited to the portion that is defined by the assessed curriculum. However, the enacted curriculum accounts for only a small portion of the variance in student achievement. Other factors that contribute substantially to student achievement include social-economic status, level of education of the student’s family, prior knowledge and experience, student reading ability, and student interest and engagement. Claims about the impact of the NSES directly on science literacy clearly will be tough to substantiate. A separate component of this study will investigate the influence of the NSES on student learning. WHAT IS THE INFLUENCE OF THENSESON SCIENCE CURRICULUM? This study is a literature review of documents related to the NSES and the science curriculum. A total of 245 documents were found related to national science education standards, 128 of which dealt with curriculum issues. The literature primarily addresses the intended curriculum. The majority of the documents dealt with the formation and analysis of curriculum frameworks. A few of the documents focused on instructional materials and national standards. Fewer still provided evidence of the influence of the NSES on the enacted curriculum. Documents addressing teaching, assessment, and learning are included in other components of this research study. After reviewing the core documents related to curriculum, the author selected for the review all documents that were reports based on data and philosophical papers that addressed important issues related to the topic of the study. The author omitted from the review philosophical papers that did not provide additional insight into the issues. The author organized the documents included in the review into four categories. The first three categories address the major levels of the educational system—national level, state level, and local level. The fourth category is for instructional materials, which does not fall neatly into any one of the first three categories. National Level Several authors reported on the context for reform at the national level, which speaks to the potential influence of the NSES on the science curriculum. Johnson and Duffett (1999), in a summary of a national survey conducted by Public Agenda, reported that there is strong support for high standards throughout the United States. The report encouraged educational leaders to prepare the public for the challenges and repercussions of establishing and enforcing high standards. Johnson and Duffett (1999) identified potential pitfalls to be avoided by standards-based reform efforts: (1) standards are not the cure-all and serious social problems in schooling must be addressed, (2) standards and high-stakes accountability must be fairly managed, (3) professional growth of teachers is the key to educational reform, (4) parents are not likely to take an activist role in educational reform, and (5) 100 percent success is not possible. Kirwan (1994) asked educational leaders to recognize that past reform efforts failed to achieve lasting change, in large part, because of a lack of involvement of local people in the reform process. Kirwan emphasizes that people at the local level often do not see the need for local change. He points out conflicting findings in
OCR for page 46
national surveys: people recognize that the nation needs to improve science education; however, when parents and administrators were asked how local schools were doing, they gave high ratings. Kirwan cites two other cautions for educational reform: (1) do not seek universal solutions for local problems (national-level instructional materials or universal instructional strategies) and (2) ensure that teachers have the support, knowledge, and skills necessary to make reforms work. Wright and Wright (1998) pointed out the wide gap between science education as it is and as described in the NSES and the work required of teachers and students to enact that vision in classrooms. They explained that while the NSES are a brilliant definition of what success is, they do too little to address the issues of implementation of the change required to achieve that vision. Wright and Wright fear that science teachers will see different messages about the goals and changes underlying the NSES, based upon their own perceptions of science literacy. The authors call for small-scale, authentic, inquiry-based projects to investigate strategies for implementing reform as a better approach than large-scale systemic reform efforts. In a similar vein, in a policy blueprint on leadership for implementation of Project 2061, Porter (1993) described four models of K-12 science programs developed by six school districts throughout the nation. He identified four major challenges to achieving the vision of Project 2061: (1) acceptance by the public and educational community of the reform objectives of making the content challenging and useful and accessible to all students, (2) understanding the changes needed in instruction, (3) believing that change is possible, and (4) removing obstacles to change that come from the educational hierarchy. In addition to studies of the context of reform, other major national and international studies have investigated the status of science and mathematics in the United States. These include the reports from the Third International Mathematics and Science Study and from the National Survey of Science and Mathematics Education. A series of reports have emerged from the Third International Mathematics and Science Study (TIMSS) that pertain to the status of the science curriculum in the United States. Reports have reviewed the science achievement testing results from TIMSS in the context of the curriculum and instruction provided in 41 countries (Schmidt, 2001a; Valverde and Schmidt, 1997). The achievement results in science ranged from being tied for second among TIMSS countries at the fourth-grade level, to being just slightly above the international average at the eighth grade, to being at the bottom of the countries at the twelfth grade. When one is looking at specific topic areas of the science tests, a picture emerges in which, on some topics (e.g., organs and tissues), no countries outperformed U.S. students. U.S. students did best in life science and earth science on the grade 4 and grade 8 tests and they performed worst in physical science. This pattern is consistent with the emphasis on life science and earth science in the seventh- and eighth-grade curriculum in the United States. The authors concluded that curriculum makes a difference, and that the United States does not have a coherent, coordinated view of what children are to know in science. The U.S. curriculum lacks focus and covers many more topics each year, compared to the rest of the TIMSS countries. This is true of state frameworks that define what children should learn, of textbooks, and of what is actually taught by teachers. Grade 8 textbooks in the United States cover 65 science topics as compared to around 25 typical of other TIMSS countries. The authors note that “U.S. eighth-grade science textbooks were 700 or more pages long, hardbound, and resembled encyclopedia volumes. By contrast, many other countries’ textbooks were paperbacks with less than 200 pages” (Valverde and Schmidt, 1997, p. 3). U.S. frameworks and textbooks lack coherence, failing to connect ideas to larger and more coherent wholes. The U.S. curriculum lacked intellectual rigor at the eighth grade and covered many of the same topics that were done in earlier grades. In another report on the TIMSS results, Stevenson (1998) summarized the results of the three TIMSS case studies of mathematics and science teaching in the United States, Germany, and Japan. Major findings included the following. The amount of national control of the science curriculum varied among the three nations. In the United States, there is no mechanism at the federal level for controlling the curriculum. Even though state and voluntary national standards do influence school curricula, there is a strong drive for local decision making in what is taught. In the United States, the content of textbooks may impart a “de facto curriculum” when teachers do not have other resources or enough depth of understanding of subject matter to utilize additional approaches to teaching and learning of science. Publishers in the United States also develop products that conform to the
OCR for page 47
requirements of the largest purchasers of their books, thereby hoping to maximize sales. In Germany, the Conference of Ministers of Education, with representatives from each state, oversees the educational polices and coordinates the structure, institutions, and graduation requirements. This national-level effort forms a basis for a degree of comparability across the German states. In Germany, the textbooks must conform to state guidelines and be approved by a state committee. Textbooks establish the content and organization of the courses, but the German teacher is able to develop his or her own course material. In Japan, the Ministry of Education develops national curricular guidelines and standards, but flexibility is given to schools to decide exactly what is to be taught at each grade level. The Ministry of Education approved the textbooks to ensure their adherence to the curriculum guidelines and quality of presentation. There has been a follow-up study to TIMSS called TIMSS Repeat (TIMMS-R). In a recent report, Schmidt (2001a) summarized findings from TIMSS-R with implications for science curriculum. TIMSS-R assessed student learning at the eighth-grade level in 13 states and 14 school districts. Schmidt indicates that “the states are remarkably similar to each other and do not differ appreciably from the United States as a whole in either mathematics or science…. The relatively poor comparative performance of U.S. eighth graders is the story for participating states. Nationally, this is related to a middle-school curriculum that is not coherent, and is not as demanding as that found in other countries we studied…. We have learned from TIMSS that what is in the curriculum is what children learn” (p. 1). Another large-scale study—the National Survey of the Status of Science and Mathematics Education—was conducted by Weiss, Banilower, McMahon, and Smith (2001). This was a continuation of three previous national surveys of science and mathematics education conducted by Weiss et al. The survey provided information and identified trends in the areas of teacher characteristics, curriculum, instruction, and instructional materials in science and mathematics. Most of the curriculum-related information in the report addressed general issues of time devoted to the science curriculum and the titles of courses taught. However, some of the data addressed specific evidence of elements of the NSES reform recommendations being implemented in schools. In the area of curriculum, the survey collected data on the nature of science and mathematics courses offered and the instructional materials used. As recommended in the NSES, science concepts were a major focus in science classes at all grade levels (two-thirds or more science classes giving concepts heavy emphasis). In addition, as recommended in the NSES, two-thirds of teachers in grades 5-12 gave heavy emphasis to science inquiry, and almost half (46 percent) of grades K-4 teachers gave heavy emphasis in this area. The NSES content standards with the least emphasis were the history and nature of science and learning about applications of science in business and industry. The most common activities in science classes at all grade levels (occurring at least once a week) were working in groups, doing hands-on/laboratory science activities or investigation, and following specific instruction in an activity or investigation. In grades 9-12, other common activities included students listening and taking notes and answering textbook or worksheet questions. Least frequent activities were working on extended science investigations or projects, designing their own investigations, using computers as a tool, participating in field work, taking field trips, and making formal presentations to the rest of the class. The survey points out the significant influence that textbook publishers have on the enacted curriculum. Commercially published textbooks are the predominant instructional material used in science: in grades K-4 (65 percent), grades 5-8 (85 percent) and grades 9-12 (96 percent) a high percentage of teachers use commercial textbooks. Many teachers report that they use one textbook or program most of the time in science (37 percent for K-4; 48 percent for 5-8; 63 percent for 9-12). The science textbook market was controlled at each level by three publishers holding approximately 70 percent of the market. Efforts at educational reform that ignore textbook publishers are missing a key defining component of the science curriculum. The national survey by Weiss et al. (2001) also included questions related to implementation of the NSES. The results suggest that the NSES are beginning to have an influence on science education at the local level. The report indicated that roughly one-third of schools were engaged in school-wide efforts to make changes aligned with national science standards. Only 23-30 percent of the designated science program representatives, however, reported that they were prepared to explain the science standards to their colleagues.
OCR for page 48
In another national survey, Blank, Porter, and Smithson (2001) studied the enacted curriculum in mathematics and science. The study used self-reporting from schools and teachers (more than 600) in 11 states to collect the data. Concerning the impact of science standards, the study found that science teachers reported that some policies have a positive influence on instruction, including the following listed from most to least influence—district curriculum framework, state curriculum framework, preparation of students for the next grade or level, and state tests. The textbook, district test, and national standards were viewed as less influential. State Level Several large-scale national surveys have investigated the progress in state-level reform of science and mathematics education. Other studies investigated the impact of large federal funding initiatives at the state level: (1) Eisenhower Mathematics and Science State Curriculum Frameworks Projects and (2) National Science Foundation State Systemic Initiatives (SSI). Surveys of State Reform Several studies reported on national surveys to determine the status of states in developing and implementing standards. The Council of Chief State School Officers (CCSSO, 1996) reported on its survey of states. The study concluded that the standards movement was well under way in 1996. The report found that Nevada was the only state listed as at the beginning of the standards process. Thirty states were in the process of developing standards, and 26 states were in the process of implementing standards as tools of systemic reform. In a 1997 report, the Council of Chief State School Officers, in collaboration with Policy Studies Associates and a panel of experts in mathematics and science education, investigated the status of state standards development since 1994. The report was based on three kinds of data: (1) a concept mapping analysis of all state curriculum frameworks and standards documents in science and mathematics, (2) interviews with state mathematics and science specialists to identify all current state documents, works in progress, and dissemination and implementation activities, and (3) an in-depth, qualitative review of new state standards from 16 states, conducted by a panel of experts. The major findings of the study were: How standards were developed Forty-six states completed mathematics and science standards. Three approaches were used in standards development: (a) state framework, (b) content standards, (c) content standards plus supplementary documents for educators. Standards were shaped by educators, officials, and the public. Consistent, ongoing process is needed. State standards links to national professional standards Main categories of state standards are similar to national. State standards include subject content and expectations for students; expectations differ markedly by state. Standards have potential to focus curriculum and reduce breadth. State mathematics standards give a strong, consistent push for greater emphasis on higher-level mathematics for all students, and less differentiation of curriculum for different groups of students. State science standards emphasize active hands-on student learning and doing of science. Key contributors to quality of state standards Statements of content are rigorous and challenging; expectations are clear and specific.
OCR for page 49
How standards link to education improvement must be communicated. Strategies toward equity are needed. Teaching, assessment, and program standards are part of only 10 states’ standards. Implementation of state standards and frameworks Strategies and quality examples can help demonstrate curriculum change. Extended state support is needed for standards implementation. Assessments should align with standards. Performance standards and levels are still in development. Professional development plans are needed in many states. The Council of Chief State School Officers (2000a) also produced a study concerning state policies on K-12 education. For this study, the researchers collected information from state education staff via a survey and also used information from reports prepared by the National Association of State Directors of Teacher Certification. The following information in the report addresses issues related to the curriculum: Forty-six states had content standards in science. Twenty-one states had a state policy for textbook and curriculum materials for classrooms. Eleven had a state policy defining state selection of textbooks and materials to be used, and 10 recommended texts or materials to the local school districts. Twenty-three states required two science credits for graduation, 16 required 1.5-3.5 credits, and four required four credits. From 1987 to 2000, 14 states raised their requirements one or more credits in science. Thirty-four states required 180 or more days of school in a year. The most recent report by CCSSO (Blank and Langesen, 2001) presented the following trends in science and mathematics education. The researchers selected the trend indicators using the following criteria: (1) policy issues reflecting state needs, (2) quality data based on reliability, validity, and comparability, and (3) research-based model. The report included the following summary findings: The amount of time in instruction and the number and level of secondary courses students take are strongly related to achievement. (p. 27) More than 95 percent of students nationally completed a first-year course in biology. Nationally, 54 percent of students took chemistry by graduation in 2000, as compared to 45 percent in 1990, an increase of 9 percentage points in 10 years. The national average for physics enrollment increased three points over the decade to 23 percent in 2000. (p. 35) There is a general trend of increased percentage of students taking earth science, physical science, general science, and integrated science in grade 9. There is a split among states for biology, with most states having the majority of students taking it at the tenth-grade level and a few having greater numbers of students taking it at the ninth-grade level. (p. 40) Sixteen states required 2.5 to 3.0 credits of science, four required four credits, and 18 states required two science credits for graduation. The number of states requiring at least two credits in science and mathematics for graduation has increased from nine states in 1990 to 42 states in 2000. (p. 41) Twenty-four of 33 states reporting on trend data on course enrollments since 1990 showed an increase of three percentage points or more in the proportion of high school students taking higher-level science courses, and 10 states increased enrollments by 10 points or more. Nationally, 28 percent of high school students took higher-level science courses in 2000, an increase from 21 percent in 1990. A total of 80 percent of high school students were taking a science course during the 1999-2000 school year. (p. 42)
OCR for page 53
Difficulties in achieving professional and/or public consensus about the nature and design of standards slowed the pace of reform. Newer practices such as including affective outcomes, constructivist practices, and performance-based assessment were criticized by religious and conservative groups and also by the general public and educators. State and district policy makers have responded by seeking balance between new and older approaches, rather than calling for wholesale return to conventional practices. State standards were intentionally broad for both political and pedagogical reasons, but district administrators and teachers often wanted more guidance and support. More than half of the districts located in states with standards in place reported that the standards initiatives had influenced their own instructional guidance efforts. National-level projects, including national standards documents, influenced local standards. There was a concern about the lack of coherence of messages about good practices that local officials received from the variety of state and local groups promoting standard-based reform. In another study, Zucker, Shields, Adelman, and Humphrey (1997) investigated the connection between general findings from the Third International Mathematics and Science Study and data sets collected by SRI from prior investigations of State Systemic Initiatives and from evaluations of the Dwight D. Eisenhower Mathematics and Science Education Curriculum Framework Projects. Zucker et al. found from the TIMSS studies that the science curriculum in the United States tried to cover a great many topics but sacrificed intensity of coverage, and deeper understanding, by doing so. SRI studies of state initiatives found that instructional materials were the weak link, especially in high school science. Only six State Systemic Initiatives focused on instructional materials as a major part of their change strategy. The SRI report recommended that schools identify and adopt high-quality curriculum materials and link professional development to those materials. It discouraged districts and schools from developing their own instructional materials. The report called for publicly available reviews of textbooks in mathematics and science as an important step toward educational reform. A study by the Council of Chief State School Officers (CCSSO, 2000c) reported on a survey of science and mathematics teachers in 11 states to characterize the enacted curriculum in science and mathematics. The findings of the study included: State frameworks/standards and national standards are reported by most teachers as strong positive influences on their curriculum. In middle school math and science, most recommended standards are covered, but the level of expectation and depth of coverage varied widely among schools and classes. Data revealed differences in extent of teaching science content across the standards and the extent of articulation between grades. Teachers reported spending 20 to 30 percent of teaching time on life science, physical science, and earth science; 20 percent on the nature of science; and 12 percent on measurement and calculation. There was wide variation of time spent in each category among schools. Teachers reported spending slightly more time on understanding concepts than on memorization. Schools that were involved in state initiatives for the reform of science education reported slightly more time on nature of science than schools not involved in state reform efforts. Initiative classes had higher expectations for analyzing information about the nature of science and understanding concepts, and slightly higher expectations for conducting experiments. One-fourth of science class time was spent on hands-on science or laboratory activities, but there was a wide variation among schools. Elementary classes spent more time on active learning in science than middle-grades classes. The most common activity was “use science equipment,” “follow step-by-step directions,” and “make tables, graphs, or charts,” while students spent less time “changing something in an experiment to see what happens.” Less than half (.33 alignment) of the items on the state science test were in common with content topic expectations reported by teachers.
OCR for page 54
Local Level A series of large-scale evaluation studies have been conducted on NSF-supported Urban Systemic Initiatives (USI). A study by Blank, Kim, and Smithson (2000) investigated the impact of the USI program on four urban school districts. The project collected data using the Survey of Enacted Curriculum, focusing on enacted curriculum contents and teaching practices. For the study, data were collected from 80 teachers from 20 elementary and middle schools for each site. The survey addressed the six drivers of educational system reform identified by the National Science Foundation: (1) implementation of comprehensive, standards-based curricula, (2) development of a coherent, consistent set of polices, (3) convergence of the usage of all resources that are designed for, or that reasonably could be used to support, science and mathematics education, (4) broad-based support from parents, policy makers, institutions of higher education, business and industry, foundations, and other segments of the community, (5) accumulation of a broad and deep array of evidence that the program is enhancing student achievement, and (6) improvement in the achievement of all students, including those historically underserved. The results of the study relevant to the science curriculum are as follows: Hands-on or laboratory materials was the largest activity (25 percent of the time). Schools involved in the USI program had elementary students who were less likely to “follow step-by-step instructions” and more likely to “change something in an experiment to see what will happen.” Students in USI middle schools spent more time “using science equipment and tools in experiments or investigations” and in “collecting data” and “designing ways to solve a problem,” but spent less time to “make predictions, guesses, or hypotheses” or to “draw conclusions from science data.” When working in small groups, the highest use of class time was to “write results or conclusions of a laboratory activity” (about 22 percent of time). High-implementation USI schools spent less time on “review assignments and problems.” Teachers in USI implementation schools spent more time on life science and chemistry, and less on physical science. Classes in comparison schools emphasized “memorize” and “analyze information” more than USI implementation schools. At the elementary level, USI implementation schools taught “nature of science” 25 percent of the time and “life science” an average of 32 percent of the time versus comparison teachers’ average times of 10 percent and just over 20 percent, respectively. Another USI evaluation investigated Children Achieving (1998)— a single, massive systemic reform initiative ($150 million in support) undertaken by Philadelphia public schools. The Consortium for Policy Research in Education evaluated the project between 1995 and 2001, interviewing hundreds of teachers, principals, parents, students, district officials, and civic leaders; observing in classrooms; surveying teachers; and analyzing the District’s test results. A report by Foley (2001) focused on the role of the central office in curriculum reform. One of the first major activities of the central office was to create “world-class” content standards. This was a move away from what had been a standardized curriculum for each subject area and grade level toward a more decentralized curriculum based on core standards. Concerns developed that some school-based purchases were not standards-based and that increased school authority created extra burdens for teachers. Forming local school councils and serving on small learning communities demanded much time and energy. Efforts of the central office staff were focused on capacity building rather than on control, but much confusion resulted in how to build local capacity for change. To further clarify its role, the central office developed detailed curriculum frameworks that defined grade-specific skills and content and offered suggestions for units and activities that addressed the content standards. The frameworks identified constructivism as the underlying pedagogical philosophy. The frameworks, which helped fill the gap between the current curriculum and where the reform was to be, were well received by school personnel. CPRE (1997) found that with the publication of the curriculum frameworks more teachers were moving toward standards-based instruction. An important finding of the study was that the focus on “doing it all at once” created reform overload throughout the District and was a strong contributor to the inability of school staff to focus their efforts around clearly defined and manageable
OCR for page 55
instructional priorities. Another key issue was underestimation of the time and support required to transform instruction to a constructivist approach, which requires new curriculum and deep changes in teaching that occur only over extended periods of time with intensive support. Huinker, Coan, and Mueller (1999) reported on the evaluation of the Milwaukee USI. The project focused on collaborative vision setting; high standards and performance assessments; narrowing achievement gaps; developing high-content; inquiry-based, technology-rich curriculum; and breaking down boundaries between community and classrooms. The report presented results of formative surveys (prior to project and two years after participation) of teachers in schools that participated in the initial phase of the project. Science and mathematics teachers at the elementary, middle, and high school levels responded to the survey. For science teachers who participated in the project, the results related to curriculum enactment included the following highlights: There was a substantial increase of teachers at all levels in their familiarity with the NSES. Middle- and elementary-level teachers indicated a decrease in belief that it is important to emphasize broad coverage of many scientific concepts and principles, while high school teachers increased in this belief. Approximately two-thirds of the elementary teachers reported using the science kits and guides developed by the District. Teachers reported increased use of student-generated experiments for elementary, middle, and high school levels. Teacher satisfaction with time available for science increased at all levels. Science teachers at all levels indicated some increasing confidence that all students would be able to meet the new school board graduation policy for science. Teachers expressed less confidence that, as students get older, an inadequate science background can be overcome by good science teaching. Singer, Marx, Krajcik, and Chambers (2000) reported on an evaluation report of the Detroit USI. The project evaluation of student learning, using a pre-post test of content and processes, yielded significant positive effect sizes for four different curriculum units. The authors noted that the evaluation was not a controlled experiment and that there were large differences in effects among teachers for each unit. The authors proposed several variables that might affect the results: the teacher, instruction, social-economic context, instructional resources, and administrative support. In addition, the authors found that it takes several iterations of curriculum revision to produce effective materials. The report identified the following areas needing additional research and development: supports to promote discourse among students, supports to help students learn from inquiries, and the role that instructional materials play in teacher learning. This report provided a good model for designing standards-based curriculum materials. It begins with identifying key principles of the NSES (goals, learning, teaching, assessment), then collaboratively designing instructional materials, piloting the materials with multiple teachers, undertaking one or more cycles of revision and testing, and evaluating the effectiveness of the materials by examining student learning of science content and science inquiry. Instructional Materials Another area of the literature emphasizes the potential impact of instructional materials on the science curriculum. Instructional materials cross the boundaries of the intended and enacted curriculum and are designed and developed at the national, state, and local levels and implemented by teachers in individual classrooms. In recognition of the implications of the NSES for science curriculum, the Biological Sciences Curriculum Study (BSCS), with NSF support, held a conference to address this issue (Bybee and McInerney, 1995). The report provided concerns and recommendations from a range of constituent groups. Elementary school teachers indicated that the NSES and AAAS Benchmarks were a positive force to improve effectiveness of elementary school science programs but were concerned that elementary school teachers will not see the standards as their
OCR for page 56
issue and that the emphasis given to science in the student’s day does not lend itself to promoting the goals of the standards. Middle school teachers were encouraged that the NSES and Benchmarks specifically identified standards and benchmarks at the middle grades, but were concerned that the NSES and Benchmarks should reflect the special needs of early adolescents, that the NSES and Benchmarks represent the floor rather than the ceiling of expectations, and that the NSES and Benchmarks might not be useable by middle-level teachers. High school teachers indicated that the NSES and Benchmarks are just a fad, require considerable energy, and will not result in much change. Science supervisors were concerned about the lack of coordination among national, state, and local projects to develop standards and that there are no resources to support staff development aligned with implementation of the standards. Curriculum developers indicated that the NSES and Benchmarks have the potential to stimulate the reform of science education and that they see curriculum developers as having a central role in the reform of science education, but they were concerned that standards might be too prescriptive and that the standards, models, and strategies for broad implementation and teacher development must be developed. College and university faculty were concerned that college and university personnel have little knowledge of the NSES and Benchmarks, will be late in recognizing the implications of the standards, and will focus on critiquing rather than implementing the national standards. The National Science Foundation has had a significant influence on the science curriculum. The Instructional Materials Development (IMD) program of the National Science Foundation has invested heavily in the development of high-quality, standards-based materials. According to Cozzens (2000), reform in mathematics and science education requires an innovative, comprehensive, and diverse portfolio of instructional materials that implement standards-based reform. The goal of the IMD program is to develop instructional materials, aligned with standards for content, teaching, and assessment that: (1) enhance the knowledge, thinking skills, and problem-solving abilities of all students, (2) apply the latest research on teaching and learning, (3) are content accurate and age appropriate, (4) incorporate the recent advances in disciplinary content and educational technologies, (5) assist teachers in changing practices, and (6) ensure implementation in broadly diverse settings. The IMD program guidelines require that successful proposals must have a design and process for developing high-quality materials that are standards-based and that are consistent with research and best practices. NSF uses expert panels of scientists, science educators, and science teachers to review IMD proposals. This peer review process helps ensure that the materials proposed are aligned with the national vision for science education, which is embodied in the NSES. In addition, IMD projects are required to provide evidence throughout the project and at the end of the project, through internal and external evaluations, that the materials are of high quality, standards-based, and effective at improving student learning. Periodically, NSF has reviewed its portfolio of IMD projects and evaluated the success of the IMD program. NSF evaluations of the IMD program have found that its products are making progress toward providing models of instructional materials that align with the vision outlined in the NSES (Cozzens, 2000). Cozzens (2000), in her report on the IMD program, identified serious issues that must be addressed to implement standards-based instructional materials. Standards-based instructional materials require a significant amount of professional development for teachers in both content and pedagogy. Publishers are not prepared to provide the needed teacher support activities and often do not realize teachers need more than they did with traditional texts. The textbook adoption process is an expensive process that some smaller publishers of innovative materials are not prepared to undertake. Implementation requires support and buy-in from administrators, parents, and the community; when support is missing from one group, the whole reform movement can be in jeopardy. Assessment of student learning must be linked to the instructional materials. Articulation across grade levels and disciplines is essential. Teacher preparation in colleges and universities must be linked with the new materials to facilitate implementation.
OCR for page 57
Other studies have undertaken the task of evaluating the quality of instructional materials to serve as a guide to states and local school districts when making adoption decisions. A report by Muscara (1998) investigated the process of the evaluation of science and mathematics programs and instructional resources to determine if they are of high quality and standards-based. The study summarized processes developed by 12 science and mathematics organizations to review preK-12 mathematics and science products. The report listed five components common to all program and resource evaluation efforts: (1) a focus or purpose of the evaluations, (2) an identified audience for the evaluation effort, (3) criteria used to evaluate, (4) the process employed during each evaluation, and (5) evaluation results. Several evaluation criteria were common across organizations: quality of program, accuracy/currency of content, pedagogical effectiveness, correlation with state/national standards, attention to equity and lack of bias, multiple content connection, and developmentally appropriate. The Office of Educational Research and Improvement (1994) reported an early review of instructional materials. It reviewed the extent to which 66 projects by the 10 regional education laboratories (funded by the U.S. Department of Education) were aligned with national curriculum standards, had evidence of effectiveness, and were transferable to other settings. The collection of programs was identified through a thorough search and review process involving educators throughout the nation. The promising programs spanned elementary, middle, and secondary levels in science, mathematics, and technology or were interdisciplinary. Each program description included a general description and a description of teaching and assessment strategies and of the alignment of the program with the framework developed by the National Center for Improving Science Education (because the National Science Education Standards were not yet released). No program was listed as not being of sufficient quality. The National Science Foundation conducted a review of comprehensive instructional materials in middle school science (NSF, 1997). NSF limited its review to products produced with funding from the Instructional Materials Development (IMD) program. The purpose of the study was to provide feedback on the status of the IMD portfolio of middle school science projects. The central criteria for the review were: (1) Is the science content correct? (2) How well do the materials provide for conceptual growth in science? and (3) How well do the materials align with the NSES? NSF convened an expert panel of 20 scientists, science/technology educators, and science teachers for the review process. Each set of materials was reviewed by a team of a scientist, science/technology educator, and science teacher. The team met and exchanged results and prepared written summaries. A second panel of experts reviewed the process and findings of the teams and recommended future directions for the IMD program. The panel judged that there are some high-quality, standard-based materials for middle-school science. The study pointed out the strengths of particular programs in addressing core content for the middle level, in providing good models for pedagogical practices, in effective use of assessment approaches, in the treatment of equity issues, and in the support provided for implementing the materials. General findings included: (1) most of the 13 sets of materials were rated three or higher on the five-point scale and are generally consistent with the NSES; (2) most materials do not explicitly address strategies for improving the performance of students with diverse abilities, backgrounds, and needs; (3) earth science was the content area least frequently included in the materials; (4) connections between science and mathematics were not well developed in most of the materials; (5) the history and nature of science received the weakest treatment of any of the NSES; and (6) too few materials incorporated significant and appropriate usage of instructional technologies. The American Institute of Biological Sciences (AIBS) produced a review of instructional materials for high-school biology (Morse and AIBS Review Team, 2001). The purpose of the project was to evaluate instructional materials in biology education to inform school-based adoption decisions. A nine-person team of scientists, teachers, and science educators developed an instrument and procedures based on the NSES to evaluate 10 biology programs with publication dates from 1997-2000. The choice of the 10 textbooks did not represent all of the materials that were available on the market, but were limited to those that the principal investigator was able to obtain from publishers. All textbooks received were included in the study. No attempt was made to omit “traditional” textbooks from the study. The evaluation criteria were based on the life science standards, other content standards (other than physical science and earth/space science), pedagogical standards, and program/ system standards, and the materials were examined for content accuracy and currency. Six separate reviews
OCR for page 58
were conducted for each program. During the review process, the team met to compare results and to calibrate the rating system. The AIBS review grouped the instructional materials into three categories: (1) traditional instructional materials that do not particularly respond to the standards (three programs), (2) innovative instructional materials that are specifically designed to meet all of the NSES (three programs), and (3) mixed instructional materials that come from the traditional background, but have responded to some or all of the pedagogy and other standards in presentation (three programs). The study found that: (1) there was great variability in how well different programs addressed standards-based science content, (2) most textbooks simply added more content to address new standards, covering too much content with too little focus, (3) nine out of 10 programs adequately represented important topics in biology, but more attention was needed in creating environments that foster learning and in meeting the other content standards and the pedagogy standards, and (4) no programs were considered overall to be exemplary, but nine of the 10 programs ranged between adequate and excellent. The reviewers found that while the life science content was present, accurate, and up-to-date in these programs, there was vast room for improvement in the treatment of other content standards and the use of standards-based pedagogy. The report indicated “most books are just too large, still too encyclopedic, and leave too much responsibility on the teachers to figure out how to use them” (p. 1). This study raised the issue of what is required for a program to be considered adequately standards-based. None of the biology programs were considered to be exemplary (i.e., fully aligned with all standards, including pedagogy). All programs but one were considered to adequately address important life science content as designated in the NSES. However, there was significant variability in the degree to which the programs met the “less traditional” content standards (inquiry, history and nature of science, science and technology, personal and social dimensions). There also was considerable variability in addressing the teaching standards (approach to learning, learning environment, and instruction). The AIBS study briefly refers to the AAAS study (discussed in the next paragraph) that also evaluated biology textbook programs, which did not find any biology programs to be of high quality, based upon standards. To judge a program as “standards-based,” therefore, significant questions remain: (1) To what extent must a program address all content standards (beyond traditional disciplinary content)? (2) To what extent must instructional materials explicitly espouse and provide concrete support for a particular approach to teaching? As mentioned in the AIBS study, Project 2061 also has undertaken a review of instructional materials. During the past year, AAAS released reports on the quality of middle-school science programs and high-school biology programs. One study evaluated the quality of high-school biology texts (AAAS, 2001c). AAAS has developed a rigorous and thorough approach to evaluating the degree of alignment of science textbooks with Benchmarks for Science Literacy and with the NSES. The materials were evaluated by content specialists, biology teachers, and university biology faculty. Each textbook was examined by four two-member teams for a total of 1,000 person hours per book. Prior to reviewing the materials, each member of the review team participated in several days of intensive training in the use of the Project 2061 curriculum analysis tool. The evaluation was conducted in two stages: (1) content specialists evaluated the textbooks for the quality of content, and (2) teams of biology teachers and university faculty applied a set of research-based instructional criteria to judge the textbooks’ treatment of four core biology topics. The evaluators were required to provide specific evidence from the materials to justify their ratings. The study found that the molecular basis of heredity is not covered in a coherent manner in the textbooks, providing needless details and missing the overall story. Overall the study found that “today’s high-school biology textbooks fail to make biology ideas comprehensible and meaningful to students” (AAAS, 2001c, p. 1). In its evaluation of science texts for the middle grades, AAAS (2001d) examined the texts’ quality of instruction aimed at key ideas and used criteria drawn from the best available research about how students learn. The study followed the same rigorous process used in the evaluation of the high school biology textbooks described above. The reviewers received several days of training on the use of the Project 2061 curriculum analysis instrument. For the study, each text was evaluated by two independent teams of teachers, curriculum specialists, and science educators. The study reported that “not one of the widely used science textbooks for middle school was rated satisfactory … and the new crop of texts that have entered the market fared no better in the evalua
OCR for page 59
tion” (AAAS, 2001d, p. 1). The study found that most textbooks cover too many topics in too little depth. The study also found that many of the learning activities were irrelevant or disconnected from underlying ideas. In two articles, Bybee (2001, 2002), executive director of the Biological Sciences Curriculum Study (which developed two of the instructional programs included in the AAAS and AIBS reviews), continued the discussion about what constitutes a quality review of instructional materials, which was addressed earlier in the AIBS review.1 Bybee (2001) expressed concern that curriculum evaluations, no matter how positive the intentions, can result in significant unintended negative consequences. He challenged the findings of the Project 2061 review of high school biology programs. Bybee stated that the AAAS “was an unacceptable evaluation…. I simply must question a judgment that all biology textbooks are woefully inadequate, represent the central barrier to student learning, and are ultimately unacceptable. Yet, this is the judgment of Project 2061” (2001, p. 2). According to Bybee, the result of this evaluation puts an enormous burden on teachers. Biology teachers can either ignore the evaluation and adopt what Project 2061 views as an unacceptable textbook or form a district committee to develop its own life science program. The result of the second choice likely would be a biology curriculum that lacks scientific accuracy, educational consistency, and pedagogical quality. Bybee (2001, p. 2) illustrates his point by indicating “I recently heard of a school district where a superintendent decided to adopt a creationist book because the major texts were unacceptable. This is clearly an unacceptable consequence of the Project 2061 evaluation.” In his second article, Bybee (2002) commented on the AIBS review of high-school biology programs. Bybee pointed out that biology teachers need evaluations that are neither uncritically positive (such as the Office of Educational Research and Improvement report) nor categorically negative (such as the Project 2061 evaluation). According to Bybee, the AIBS review meets his criterion. He praised the approach of the AIBS study: “The consumer report approach of numerical ratings, graphical comparisons, and general discussions of all textbooks gives adoption committees the opportunity to review potential programs with an eye toward local criteria and constraints” (2002, p. 7). Bybee emphasized that an approach that highlights both the strengths and weaknesses of a program encourages variations in programs. As Bybee pointed out, “the evolution of better textbooks, the programs biology teachers deserve, is the consequence of the variation among those textbooks” (2002, p. 8). George Nelson (2001), director of Project 2061, provided a counterpoint to Bybee’s critique of the Project 2061 analysis of high school biology programs: “Project 2061 disagreed with the statement by Rodger Bybee— because the study finds all the textbooks to be unsatisfactory, the analysis itself is unacceptable” (Nelson, 2001, p. 146). Nelson disagreed that the Project 2061 review limits textbook adoption choices. He noted, “To the contrary, Project 2061’s evaluation adds information into the system that educators can use to make more sophisticated decisions, based on the specific strengths and weaknesses of the texts. Once a textbook adoption decision is made the Project 2061 data can help define the kinds of supplementary materials and instruction that may be needed to make up for any shortcomings. For example, none of the textbooks adequately accounts for students’ prior knowledge or for their preconceptions or misconceptions, although these are known to be major factors in student learning…. We recommend, for example, that educators use some of the excellent trade books on the market that have been published on science topics to compensate for unsatisfactory textbooks” (p. 146). He also wrote, “A concern we share with Dr. Bybee is that our reviews will encourage teachers and schools to develop their own biology materials…. We agree that ‘home-built’ curricula would be unlikely to fair well on our analysis” (p. 147). The National Research Council (NRC, 1999c) responded to the need expressed by school district administrators, science teachers, scientists, and parents for a tested procedure for evaluating and selecting K-12 science instructional materials that is consistent with state and/or national standards. The NRC recognized that the instrument would need to be flexible to accommodate the diversity of state standards and interests at the local level and should accommodate the time constraints faced by evaluators of instructional materials. In the process of developing its evaluation tool, the NRC reviewed several national efforts to evaluate instructional materials, 1 The author of this paper (Ellis) has been a senior staff associate at BSCS and a program officer at the National Science Foundation in the Instructional Materials Development program.
OCR for page 60
including those produced by Project 2061, the National Science Resources Center, the National Science Foundation, the U.S. Department of Education, and the Center for Science, Mathematics, and Engineering Education. The NRC report identified general principles of an effective tool for evaluation of instructional materials: The evaluation tools should fulfill needs not met by other instruments. The evaluation tool should assume that a set of standards and a curriculum framework will inform the work of evaluators. The evaluation process should require reviewers to provide evidence to support their judgments. The usefulness of the information will be enhanced when evaluators provide a narrative response rather than make selections on a checklist. Effective evaluations include one or more scientists on the review teams. An evaluation instrument needs to serve diverse communities, each one of which has its own needs. Tension exists between the need for well-informed, in-depth analyses of instructional materials and the real limitations of time and other resources. Many evaluators using the tool will be unfamiliar with current research on learning. It is more important to evaluate materials in depth against a few relevant standards than superficially against all standards. The review and selection processes should be closely connected. CONCLUSIONS What does an analysis of the literature yield about the influence of the NSES on science curriculum? The results of the analysis fall within the potential spheres of influence illustrated in Figure 2-1—the intended curriculum, the enacted curriculum, and the assessed curriculum. The Intended Curriculum Much has happened in the reform of science education since the release of A Nation at Risk. The NSES have had an influence on multiple layers that delineate the intended curriculum for schools—the national level, instructional materials, state level, and local level. National Level The National Science Education Standards document represents the national consensus of scientists, science educators, and the public about the vision for the science education program needed to achieve science literacy for all students. The NSES are supported by all major professional societies relevant to science and science education, including the American Association for the Advancement of Science, National Science Teachers Association, National Association of Biology Teachers, American Chemical Society, American Institute of Physics, American Institute of Biological Sciences, and Council of Science Society Presidents. Major funding agencies, including the National Science Foundation, U.S. Department of Education, and National Aeronautics and Space Administration, use it as a guide to make decisions about proposed educational reform projects. The influence of the NSES on the meaning of a quality education in science at the national level has been extraordinary. Decisions about the science curriculum, however, are not made, for the most part, at the national level. Decisions about what students are to know and be able to do, and about the sequence, organization, and delivery of the content are made at the state, local, and teacher levels. It is at these other levels one must look to determine the impact of the NSES on the science curriculum in the nation’s schools.
OCR for page 61
Instructional Materials As found in numerous national surveys reviewed in this paper, instructional materials influence the curriculum. In most cases the textbook is the de facto curriculum. There is evidence of influence of the NSES on instructional materials. The Instructional Materials Development (IMD) program of the National Science Foundation has invested approximately 1 billion dollars in IMD projects since A Nation at Risk was published in 1983 (NSF, 1994a; Cozzens, 2000). Through the IMD program, curriculum developers have produced multiple comprehensive programs (complete materials for a set of grade levels or a course) at all levels K-12—elementary-school science, middle-level science, and all areas of high school and have produced a myriad of innovative modules in nearly every imaginable area of science. The reviews of IMD-produced materials by NSF (1997), Cozzens (2000), and AIBS (Morse, 2001) provide evidence of the quality of these materials. One might think of the reform of instructional materials as a journey toward the NSES without a road map, rather than as a construction project where they are the blueprint. The NSES define science literacy and some elements of the educational system required to achieve it. The NSES, however, are not a curriculum framework. At this point, there is no clear consensus of the design for “standards-based instructional materials.” Curriculum developers are producing a variety of designs based on the NSES, educational research, and wisdom of best practices. Little evidence based on student learning, however, is available that any one approach is better than another. So, while we do have examples of instructional materials that are moving toward standards-based practices, we do not have “exemplars” of standards-based curriculum. At this point, the educational community does not know what is exemplary, because it has not seen it yet. Textbook publishers provide the vast majority of science instructional materials adopted and used in K-12 schools (Weiss et al., 2001). Textbook publishers are aware of the national dialogue about the needed reform in science education, which is represented in the NSES. Even a cursory look at textbooks published in the past five years provides evidence that textbook publishers are acknowledging the influence of the NSES. Most provide a matrix of alignment of the content in their text with the NSES. Recommendations to textbook publishers in national reports, however, will not influence textbook publishers, who are accountable to their shareholders. Textbook publishers respond to market forces. If we want textbook publishers to produce and sell standards-based materials to schools, then teachers, school districts, and states must establish the demand by purchasing only standards-based materials. Textbook publishers likely will be quick to respond to such demand. The research literature reviewed for this study, with the exception of the AIBS report (Morse, 2001), however, provided little evidence about the degree of influence of the NSES on textbook programs. The NSF study of the middle-level science materials limited its scope to NSF-supported materials (NSF, 1997), the OERI study of promising practices did not include textbook programs in its review (OERI, 1994), and major textbook programs failed to pass through the initial screening of instructional materials for the AAAS reviews (AAAS, 2001c, 2001d). Only the AIBS study (Morse, 2001) included any major textbooks in its review. The influence of the NSES on instructional materials, therefore, is difficult to determine without solid evidence from the literature. However, it is reasonable to say that the NSES have stimulated thinking about curriculum development and design, which is supported by the studies of the IMD program and by examinations of textbooks. The analysis of the reviews of instructional materials, however, provides complex, and perhaps conflicting, findings. All of the studies yield evidence of major features in the most recent innovative materials that are consistent with the NSES ideals. There is considerable disagreement among reviewers, however, as to where one sets the bar to determine whether a set of materials is considered to be standards-based. Overall, the research supports the following findings: (1) progress is being made toward providing models of standards-based instructional materials; (2) the vast majority of materials being used by teachers, however, fall short of these models and have not been brought in line with the NSES; and (3) the difficulty of adoption and use of high-quality, standards-based instructional materials is a significant barrier to realization of the science education envisioned in the NSES.
OCR for page 62
State Level As seen from the several national surveys of states and evaluations of state systemic initiatives and state curriculum framework projects summarized in this report, considerable evidence is available about the influence of the NSES on state frameworks and curriculum frameworks. Overall, the evidence clearly supports the claim that states are moving toward the science education envisioned in the NSES. All states have developed or are in the process of developing standards (AFT, 2001) and at least 47 of these states have established standards for science education (Blank, Manise, and Brathwaite, 1999). The NSES and Benchmarks for Science Literacy have been key documents guiding the development of state standards (Humphrey, 1996; AAAS, 1997a; CPRE, 1995; VDE, 1996; Massel, et al., 1997; Adelman, 1998a, 1998b). However, states have not progressed as far with translating standards into science curriculum. States vary in how they exert control over the science curriculum. Twenty-one states have a state policy for the selection of instructional materials for the classroom (CCSSO, 2000a). The summary study by Clune (1998) of case studies of nine states involved in NSF SSI projects found that curriculum had the lowest rating of change when compared to reform and policy initiatives. Therefore, the evidence indicates that while change is taking place at the state level, state policies overall are slow to influence change in the curriculum. Local Level Several studies investigated the impact of the NSES on the science curriculum used in districts and schools at the local level. The TIMSS reports (Schmidt, 2001a; Valverde and Schmidt, 1997; Stevenson, 1998; Zucker et al., 1997) and the national survey by Weiss et al. (2001) provide substantial evidence on what is taught in U.S. schools. The overall picture is of a lack of focus, coherence, and coordination in the science curriculum (Schmidt, 2001a) and for the vast majority of schools, commercial textbooks are the curriculum at the local level (Weiss et al., 2001; Zucker et al., 1997). Because there is a lack of studies of the degree to which commercial textbooks align with the NSES, it is difficult to judge the degree of their influence on the local science curricula. However, evidence from the studies by AAAS (2001c, 2001d) and AIBS (Morse, 2001) indicated, either by omission (in the case of AAAS) or by the lower ranking assigned to the textbooks included in the review (as in the AIBS review), that commercial textbooks overall are not considered to be fully standards-based. Other studies have investigated reform at the local level. NSF has funded several projects to stimulate reform at the local level, including the Urban Systemic Initiatives (USI) and Local Systemic Change (LSC). The NSF program guidelines and solicitation for the LSC projects (NSF, 1999) required that the project be based upon the implementation of high-quality, standards-based materials. This emphasis on standards-based practices was to guide the expert panels and program officers to recommend proposals for LSC awards. However, no studies have investigated the degree to which districts involved in these LSC projects ultimately limited their adoption process for K-12 science to standards-based materials, nor are there data to determine the degree to which these materials were in use by teachers in the schools. The NSF USI projects, however, have been studied (Blank et al., 2000; Foley, 2001; CPRE, 1996; Huinker et al., 1999; Singer et al., 2000), but there is conflicting evidence concerning curriculum implementation from which to judge the influence of the NSES on the science curriculum at the local level. The National Science Foundation required the USI projects to implement standards-based reform, including standards-based curriculum. Overall, the studies of the USI projects indicated mixed results in progress toward standards-based reform. The overall study by Blank et al. (2000) provides evidence of classroom practices that align with standards-based reforms in the science curriculum. Singer et al. (2000) report success at designing, developing, and implementing standards-based instructional materials in Detroit Public Schools, and Huinker et al. (1999) provide evidence that two-thirds of the elementary teachers in Milwaukee Public Schools were using kit-based materials (which arguably is a move toward standards-based curriculum). Other studies found that districts were making slow progress towards adoption and implementation of high-quality, standards-based materials (Foley, 2001; CPRE, 1996). Additional studies of changes in the science program and teaching practices are summarized in other papers in this overall study, which address teaching, learning, and assessment.
OCR for page 63
RECOMMENDATIONS FOR RESEARCH Upon completion of a review of the literature related to the influence of the NSES on the science curriculum, one is left with many unanswered or partially answered questions. There are many gaps in the research literature. The following recommendations are offered to researchers and funding agencies to consider as a research agenda for the next decade: Innovative designs are needed to learn more about the nature of standards-based instructional materials in K-12 science. Consumer report studies are needed to characterize the degree to which available instructional materials in science at all levels and in all subjects are standards-based. These studies should be repeated at least once every three years, because instructional materials are continuously changing. The results of these studies should be disseminated widely. States and school districts need assistance and support in identifying and selecting high-quality, standards-based materials. Studies are needed at regular intervals to determine the degree to which local school districts are adopting high-quality, standards-based materials and to determine the factors that influence successful use. For reform to proceed, intensive and extended professional development and substantial resources are required to support teachers in enacting standards-based curriculum, instruction, and assessment practices. Studies are needed to investigate the nature of the enacted curriculum in classrooms throughout the nation to determine the quality of the program and the alignment with best practices. Large-scale studies are needed to investigate the impact of standards-based science programs (where curriculum, instruction, and assessment are well aligned) on student achievement. Failure to conduct these studies will ultimately cast doubt on the value of the massive expenditures on standards-based reform. The public and educators alike will demand a continuous chain of evidence that strongly supports the claim that standards-based reform has improved the quality of science education in our nation’s schools. Without establishing alignment of all aspects of the system, however, it will be impossible to draw valid conclusions about the value of national standards in science.
Representative terms from entire chapter: