8
Annotated Bibliography

Karen S. Hollweg

Abrams, L., Clarke, M., Pedulla, J., Ramos, M., Rhodes, K., and Shore, A. (2002, April). Accountability and the Classroom: A Multi-State Analysis of the Effects of State-Mandated Testing Programs on Teaching and Learning. National Board on Testing and Public Policy, Boston College. Paper presented at the American Educational Research Association Annual Meeting, New Orleans, LA.

ACCESS ERIC. K-8 Science and Mathematics Education. The ERIC Review.6(2), Fall1999.

Adams, P.E. and Krockover, G.H. (1999). Stimulating Constructivist Teaching Styles Through Use of an Observation Rubric. Journal of Research in Science Teaching.36(8), 955-971.

This study sought to relate a science teacher’s use of the Secondary Science Teaching Analysis Matrix (STAM), which is consistent with the style of teaching advocated by the NSES, with his development over time from a didactic to a more constructivist teacher. Citing others, the authors argue that, despite their pre-service experiences, beginning teachers often adopt “survival strategies” rather than those advocated by the NSES. Using a mechanism like STAM, they argue, teachers can conduct self-assessment and have a heuristic to guide them toward more student-centered styles of teaching. The study was of one teacher who was purposefully selected because his teaching had changed, as measured by the STAM instrument. The authors conducted extensive formal and informal interviews with the teacher, as well as direct classroom observations and videotaped observations, and collected classroom handouts.

The authors analyzed their data with several qualitative analytical techniques, including analytic induction, extensive use of memos, and synthesis of the various data sources. The analysis done in this study seems quite appropriate, but the study is a classic outlier study where the authors chose a case that demonstrated their conclusion and sought to verify it, rather than choose a teacher before they knew the impact of the STAM instrument and seek to see if their hypotheses would hold. The authors inferred that, since both the subject of the case (“Bill”) and their own data pointed to the influence of the STAM as a roadmap for Bill’s progression from a didactic to a constructivist teacher, the use of such an instrument can help novice teachers reflect on and change their teaching practices.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 127
8 Annotated Bibliography Karen S. Hollweg Abrams, L., Clarke, M., Pedulla, J., Ramos, M., Rhodes, K., and Shore, A. (2002, April). Accountability and the Classroom: A Multi-State Analysis of the Effects of State-Mandated Testing Programs on Teaching and Learning. National Board on Testing and Public Policy, Boston College. Paper presented at the American Educational Research Association Annual Meeting, New Orleans, LA. ACCESS ERIC. K-8 Science and Mathematics Education. The ERIC Review.6(2), Fall1999. Adams, P.E. and Krockover, G.H. (1999). Stimulating Constructivist Teaching Styles Through Use of an Observation Rubric. Journal of Research in Science Teaching.36(8), 955-971. This study sought to relate a science teacher’s use of the Secondary Science Teaching Analysis Matrix (STAM), which is consistent with the style of teaching advocated by the NSES, with his development over time from a didactic to a more constructivist teacher. Citing others, the authors argue that, despite their pre-service experiences, beginning teachers often adopt “survival strategies” rather than those advocated by the NSES. Using a mechanism like STAM, they argue, teachers can conduct self-assessment and have a heuristic to guide them toward more student-centered styles of teaching. The study was of one teacher who was purposefully selected because his teaching had changed, as measured by the STAM instrument. The authors conducted extensive formal and informal interviews with the teacher, as well as direct classroom observations and videotaped observations, and collected classroom handouts. The authors analyzed their data with several qualitative analytical techniques, including analytic induction, extensive use of memos, and synthesis of the various data sources. The analysis done in this study seems quite appropriate, but the study is a classic outlier study where the authors chose a case that demonstrated their conclusion and sought to verify it, rather than choose a teacher before they knew the impact of the STAM instrument and seek to see if their hypotheses would hold. The authors inferred that, since both the subject of the case (“Bill”) and their own data pointed to the influence of the STAM as a roadmap for Bill’s progression from a didactic to a constructivist teacher, the use of such an instrument can help novice teachers reflect on and change their teaching practices.

OCR for page 127
Adelman, N. (1998a). A Case Study of Delaware’s SSI (Project 21), 1991-1997. In P.M. Shields and A.A. Zucker (Eds.), SSI Case Studies, Cohort 1: Connecticut, Delaware, Louisiana, and Montana. Menlo Park, CA: SRI International. This is a report of a case study of the Delaware State Systemic Initiative, which was supported by the National Science Foundation. The Delaware SSI focused on professional development and curriculum improvement in 34 schools. By the end of the project, 30 percent of the state’s schools and 25 percent of its mathematics and science teachers had been involved. However, only a few of the schools had made whole-school progress toward school change and reform of instruction. The lack of district support, administrative leadership, and technical assistance for overall school change contributed to the disappointing results of the model schools strategy. During the last year of the project, the SSI mathematics and science specialists produced a database of more than 200 standards-based curriculum materials in mathematics and science for consideration for use by school districts. Adelman, N. (1998b). A Case Study of Maine’s SSI (Maine: A Community of Discovery), 1992-1997. In P.M. Shields and A.A. Zucker (Eds.), SSI Case Studies, Cohort 2: California, Kentucky, Maine, Michigan, Vermont, and Virginia. Menlo Park, CA: SRI International. This is a report of a case study of the Maine State Systemic Initiative, which was supported by the National Science Foundation. The goal of the Maine SSI was to improve science and mathematics outcomes in grades K-12 throughout the state. The SSI strongly influenced state policy-making activities, supported seven local demonstrations of systemic reform, provided technical assistance to local school districts on request, and developed statewide and regional leadership. The SSI played a key role in development of a state curriculum framework for science and mathematics and in the development of legislative policy on performance standards aligned with the curriculum framework. Over a five-year period leaders of the SSI estimated that they had introduced approximately 60 percent of the state’s science and mathematics teachers to standards-based educational reform and had worked intensively with about 20 percent of them. A key to the success of the Maine SSI was that it was established as a not-for-profit organization that was independent of governmental agencies. The project had less of an impact on reform in high schools and in the state’s largest cities. Albert, L.R. and Jones, D.L. (1997). Implementing the Science Teaching Standards through Complex Instruction: A Case Study of Two Teacher-Researchers. School Science & Mathematics.97(6), 283-291. Alberts, B. (1994, April). Science Education Standards. In Scientists, Educators, and National Standards: Action at the Local Level, Sigma Xi Forum Proceedings, Sigma XI, The Scientific Research Society, Research Triangle Park, NC, April 14-15, 1994. American Association for the Advancement of Science. (1989). Science for All Americans: A Project 2061 Report on Literacy Goals in Science, Mathematics, and Technology.Washington, DC: Author. American Association for the Advancement of Science. (1993). Benchmarks for Science Literacy.New York: Oxford University Press. American Association for the Advancement of Science. (1997a). Project 2061: Science Literacy for a Changing Future. Update 1997.Washington, DC: Author. This is a report of a yearlong evaluation by SRI International of the impact of Science for All Americans and Benchmarks for Science Literacy. The researchers collected data through expert interviews, reviews of state science curriculum frameworks and textbooks, telephone and mail surveys, and case studies of reform activities in six states. The report claims, “Project 2061 has been a major influence on the development of national science education standards and on reform initiatives sponsored by the National Science Foundation, the U.S. Department of Education, and a number of other national education and science organizations” (p. 2). The report also found that the reform ideas promoted by Project 2061 have not been widely adopted by textbook publishers. The

OCR for page 127
study found that 90 percent of educational leaders from 27 states refer to Benchmarks in their day-to-day work. The study found that Project 2061 has had an impact on state curriculum frameworks. American Association for the Advancement of Science. (1997b). Resources for Science Literacy: Professional Development.New York: Oxford University Press. American Association for the Advancement of Science. (1998). Blueprints for Reform: Science, Mathematics, and Technology Education.New York: Oxford University Press. American Association for the Advancement of Science. (2001a). Atlas of Science Literacy.Washington, DC: Author. American Association for the Advancement of Science. (2001b). Designs for Science Literacy.New York: Oxford University Press. American Association for the Advancement of Science. (2001c). High School Biology Textbooks Evaluation. Washington, DC: Author. This study reports on an evaluation of high school biology texts by AAAS. The materials were evaluated by content specialists, biology teachers, and university biology faculty. Each textbook was examined by four two-member teams for a total of 1,000 person hours per book. The evaluators were required to provide specific evidence from the materials to justify their ratings. The study found that the molecular basis of heredity is not covered in a coherent manner in the textbooks, providing needless details and missing the overall story. Overall, the study found that “today’s high-school biology textbooks fail to make biology ideas comprehensible and meaningful to students.” American Association for the Advancement of Science. (2001d). Middle Grades Science Textbooks Evaluation. Washington, DC: Author. This is an AAAS report of its evaluation of science texts for the middle grades. The study “examined the text’s quality of instruction aimed specifically at the key ideas, using criteria drawn from the best available research about how students learn.” For the study, each text was evaluated by two independent teams of teachers, curriculum specialists, and science educators. The study reported that “not one of the widely used science textbooks for middle school was rated satisfactory … and the new crop of texts that have entered the market fared no better in the evaluation.” The study found that most textbooks cover too many topics in too little depth. The study also found that many of the learning activities were irrelevant or disconnected from underlying ideas. American Association for the Advancement of Science. (2001e). Project 2061: Science Literacy for a Changing Future. Update 2001-2002.Washington, DC: Author. American Federation of Teachers. (1994). What College-Bound Students Abroad Are Expected to Know About Biology. Exams from England and Wales, France, Germany and Japan. In M. Gandal, Defining World Class Standards.Volume 1. Washington, DC: Author. American Federation of Teachers. (1999). Making Standards Matter 1999.Washington, DC: Author. This is an annual report that analyzes the quality of the academic standards in 50 states, the District of Columbia, and Puerto Rico. For this study, the authors reviewed state standards, curriculum documents, and other supplemental material and interviewed state officials to obtain information about state standards and their implementation. The study examined two major issues: (1) Does the state have, or is it in the process of developing, standards in the four core academic subjects—English, math, science, and social studies and (2) are the standards clear and specific enough to provide the basis for a common core curriculum from elementary through high school? The authors looked for the following qualities in the standards: (1) standards must define

OCR for page 127
in every grade, or for selected clusters of grades, the common content and skills students should learn in each of the core subjects; (2) standards must be detailed, explicit, and firmly rooted in the content of the subject area to lead to a common core curriculum; (3) for each of the four core curriculum areas, particular content must be present (for science, that was life, earth, and physical sciences); and (4) standards must provide attention to both content and skills. For the purpose of analysis, the standards were divided into 12 large categories using a three-by-four matrix (three levels of elementary, middle, and high school by four core subject areas). For a state to be judged as having quality standards overall, at least nine of the 12 categories must be clear and specific and include the necessary content. The major findings of the study are as follows: States’ commitment to standards reform remains strong. The District of Columbia, Puerto Rico, and every state except Iowa have set or are setting common academic standards for students. The overall quality of the state standards continues to improve. Twenty-two states—up three from 1998— have standards that are generally clear and specific and grounded in particular content to meet AFT’s common core criterion. Although standards have improved in many states, most states have more difficulty setting clear and specific standards in English and social studies than in math and science. In science, 30 states meet the AFT criteria for all three levels. Thirty-four states have clear and specific standards at the elementary level, 39 at the middle level, and 36 at the high school level. The NSES are widely accepted in the field and cited often in state standards documents. Every state but Iowa, Montana, and North Dakota is committed to measuring student achievement toward the standards. Through test items, scoring rubrics, and/or student work samples, many states (26) describe the level that master students must demonstrate to meet the state standards. Fourteen states have policies for ending social promotion—the practice of passing students from grade to grade regardless of whether they have mastered the standards. Twenty-eight states have or will have high school exit exams based on the standards. Twenty-three states have or are developing incentives (advanced diplomas, free college tuition) to motivate students to achieve a higher standard than that required for all students. Although 40 states require districts to provide intervention to students who are struggling to meet standards, only 29 states fund such programs. American Federation of Teachers. (2001). Making Standards Matter 2001.Washington, DC: Author. This is a report of the status of the development and implementation of academic standards in states. For the study, the project analyzed state standards and supplemental documents to determine the quality of the academic standards. The project used the following criteria: (1) standards must define the common content and skills students should learn in each of the core subjects for every grade level or for selected grade spans in elementary, middle, and high school; (2) standards must be detailed, explicit, and firmly rooted in the content of the subject area to lead to a common core curriculum; (3) for each of the four core curriculum areas, particular content must be present (e.g., earth, physical, and life sciences); and (4) standards must provide attention to both content and skills. Each state was rated on the extent to which the standards in each of the four curriculum areas for each of the levels (elementary, middle, and high school) were clear and specific and include the necessary content (a total of 12 categories of standards). For a state to be judged as having quality standards overall, 75 percent of the categories of standards (nine out of 12) had to meet the criteria of quality. The report also included an analysis of the state curriculum, assessments, accountability, and the overall standards-based system. For the analysis of curriculum work in the states, to be complete, a curriculum must be grade by grade and contain the following five components: a learning continuum, instructional resources, instructional strategies, performance indicators, and lesson plans. For a state to be judged as having a well-developed curriculum, it had to have at least three of the five curriculum components at each of the three levels in each subject area. For the assessment analysis, the project looked for: (1) the state tests students at each

OCR for page 127
educational level in all four core subjects; (2) the state reports information on alignment of the standards and the assessments; and (3) the state indicates the standards to be assessed. To meet the criteria on alignment, a state must: (1) use a test that it developed and specify the standards to be measured, or (2) use an off-the-shelf test, release information about the percentage of test items that are aligned with the state standards, and indicate the standards that are assessed. The project also analyzed the accountability measures in each state. For accountability, the project looked for: (1) the state requires and funds extra help for students having difficulty meeting the standards, and (2) the state developed policies to encourage students to take learning more seriously by providing rewards and consequences based, in part, on state assessment results. To judge state efforts to build a coherent standards-based system, the project looked for: (1) are the tests aligned to the standards? (2) are all of the aligned tests based on strong standards? (3) are curricula developed in all of the aligned test areas? (4) are all promotion and graduation polices based on aligned tests? and (5) do promotion or graduation policies include intervention? The results of the study are as follows: States’ commitment to standards-based reform remains strong. Every state and the District of Columbia have set or are setting common academic standards for students. The overall quality of the state standards continues to improve. Thirty states—up from 22 in 1999—have standards that meet the AFT’s common core criterion. Most states have more difficulty setting clear and specific standards in English and social studies than in math and science. Thirty-nine states meet the AFT criteria in science at all three levels, and 43 states meet the criteria at the elementary level, 46 at the middle level, and 42 at the high-school level. State efforts in curriculum have just begun. No state has a fully developed curriculum. Only nine states have 50 percent or more of the components of a fully developed curriculum. States are more likely to have curriculum materials for English than for the other areas. Nine states have at least three of the curriculum components in science at all three levels. Thirty-two states assess science at the elementary level, 35 at the middle level, and 40 at the high-school level. Only nine states have aligned tests in the four core subject areas at all three educational levels. States use a mixture of commercially developed, off-the-shelf standardized tests and their own “home-grown” assessments to measure and report on student achievement. During the past two years, there is a decrease in the number of states (28 to 25) that require and fund academic intervention programs for students at risk. Seventeen states have policies for ending social promotion. Twenty-seven states have or will have high-school exit exams based on the standards. Thirty states, up from 23 in 1999, have or are developing incentives (e.g., advanced diplomas, free college tuition) to motivate students to achieve a higher standard than required of all students. Many state assessment programs are based on weak standards. Many state assessment programs use tests unaligned to their standards. A number of states use results of nonaligned tests to hold back students or to deny them a diploma. Many states impose sanctions on students but fail to mandate intervention and to provide the resources to help them. The report makes the following recommendations regarding the curriculum: Involve teachers in the development of grade-by-grade curriculum aligned to the standards in the core subjects. Specify the learning continuum in the core subjects to show the progression and development of critical knowledge and skills from grade to grade. Identify instructional resources that are aligned to the standards. Provide information on instructional strategies.

OCR for page 127
Provide performance indicators to clarify the quality of student work required. Develop lesson plan data banks that include exemplary lessons and student work. Provide guidance and incentives to schools so that they attend to important areas of the curriculum that are not addressed—e.g., art, music, foreign languages. Andersen, H.O. (2000). Emerging Certifications and Teacher Preparation. School Science & Mathematics.100(6), 298-303. In this paper, the author reports on a state’s transition from certification based upon inputs to a performance-based teacher certification program. The paper describes changes in both Indiana University’s and the state of Indiana’s teacher preparation program. Up until the date of the article (2000), the state had a certification program that required students to complete coursework in order to receive their teaching certification. The author explains that the state is planning (but has not yet instituted) a performance-based certification process. Teachers who complete their pre-service programs and pass certification exams will receive initial licensure for two years. At that point they will have to submit a portfolio of evidence that they have successfully taught a variety of students and have a personal plan for continued professional development. Teachers’ “evidence competence” comes from standards developed by the Interstate New Teacher Assessment and Support Consortium (INTASC). INTASC’s standards, the author explains, are based upon the standards of other organizations, including the National Science Education Standards. The portfolio should include a series of instructional plans, and the identification of a variety of strategies to ensure that every student in the class becomes engaged in learning. The sequence of instruction is to cover materials described by local and national standards. The author’s biggest concern with this system is the quality of the mentors that will support teachers through this process. The author also argues that while the performance assessment is being constructed to evaluate the teaching performance of individual teachers, it could also be used to evaluate institutions that prepare teachers. Anderson, R.D. and Helms, J.V. (2001). The Ideal of Standards and the Reality of Schools: Needed Research. Journal of Research in Science Teaching.38(1), 3-16. Anderson and Helms note that a variety of research perspectives can inform our understanding of science education reform, and argue for research that gives simultaneous attention to all of the relevant elements of the system as well as the interactions among them. The authors summarize what existing research tells us about the challenges involved in putting the National Science Education Standards into widespread practice, and suggest some areas where additional research “has the greatest potential for furthering the reform of science education.” Most of the research cited in this article is socio-cultural in perspective and qualitative in nature; the authors do not describe the process they used in selecting these particular studies for review. Conclusions drawn from existing research include: (1) the changes called for in the NSES require significant changes in teachers’ values and beliefs about science education, and in any event are difficult to put into full practice; (2) teachers face multiple dilemmas in the process, such as the extent to which to focus on standards-based content and pedagogy versus traditional instruction that is presumed necessary to prepare students for the next level of schooling; (3) substantial teacher collaboration in the work context can be a powerful influence on teachers and teaching; and (4) parental support for reform ideas and practices is essential. The authors suggest a need for further research that is approached from multiple perspectives and conducted in the “real world,” focusing on conventional school practices and without the assumption that change can be driven solely from the top down. One area recommended for research is identifying the most productive roles for students, the desired nature of student work, and how to engage students in that work “in ordinary classroom contexts.” Other areas highlighted for further research include how teachers can best be engaged over time in taking responsibility for their own professional growth, and how to involve parents most effectively in the science education reform process. Armstrong, J., Davis, A., Odden, A., and Gallagher, J. (1988). The Impact of State Policies on Improving Science Curriculum.Denver, CO: Education Commission of the States.

OCR for page 127
Atkin, J.M. and Black, P. (1997). Policy Perils of International Comparisons: The TIMSS Case. Phi Delta Kappan. 79(1), 22-28. Austin, J.D., Hirstein, J., and Walen, S. (1997). Integrated Mathematics Interfaced with Science. School Science & Mathematics.97(1), 45-49. Banilower, E. (2000). Local Systemic Change through Teacher Enhancement.Chapel Hill, NC: Horizon Research. Banilower reported on the data available from the evaluations of the Local Systemic Change (LSC) projects. The LSC projects were surveyed to ascertain whether they had undertaken any studies examining the impact of the LSC on student achievement. As examining student data was not a requirement of the evaluation, few projects had examined their impact on student achievement. Although 47 of the 68 projects responded, 38 projects indicated that they had no student achievement data available. Thus, data were available only from nine of 68 projects. Eight of the nine projects showed a positive relationship between teacher participation in the LSC and student achievement in mathematics and science, though only half of these constructed a convincing case that the impact could be attributed to the LSC. However, results need to be interpreted with caution, since in more cases, it is difficult to make the case that the impact is due primarily to the LSC and not to other, unmeasured interventions or policies. Many of the studies do not present enough information to build a convincing case that the LSC was responsible for improved student achievement. Given the small number of compelling studies, the data are insufficient to support claims about the impacts of the LSCs in general. It is also important to note that many of these studies reported only group means and did not statistically test group differences. Finally, Banilower points out that the remaining studies were flawed by (1) a lack of control groups (i.e., the study reported gain scores for schools in the LSC, but not for schools outside of the LSC); (2) failure to account for initial differences between control and experimental groups (i.e., while the study may have reported that LSC students scored higher than non-LSC students, it was unclear as to whether the two groups at the same achievement level); or (3) sample selection bias in the choice of participating schools or students (i.e., the study did not address how teachers were selected for participation in LSC training and whether this may have affected the study’s results). Banilower, E.R., Smith, P.S., and Weiss, I.R. (2002). Examining the Influence of National Standards: Data from the 2000 National Survey of Mathematics and Science Education.Chapel Hill, NC: Horizon Research. Barnhardt, R., Kawagley, A.O., and Hill, F. (2000). Cultural Standards and Test Scores, Sharing Our Pathways. Fairbanks: University of Alaska. Barnhardt, Kawagley, and Hill report that eighth-grade students in schools participating in the Alaska Rural Systemic Initiatives (AKRSI) scored significantly higher than students in nonparticipating schools on the CAT-5 mathematics achievement test. With regard to student achievement, there was a differential gain of 5.9 percent in the number of students who are performing in the top quartile for AKRSI partner schools over non-AKRI rural schools. The AKRSI districts have 24.3 percent of their students testing in the upper quartile, and they are only 0.7 percent below the national average. Based on these results, the authors conclude that using Cultural Standards designed by the AKRSI has positive impacts on standardized test scores. For several years, the AKRSI had been working intensively with 20 of 48 rural school districts in the state to implement the Cultural Standards that are intended to systematically document the indigenous knowledge systems of Alaska Native people and develop educational policies and practices that effectively integrate indigenous and Western knowledge through a renewed educational system. Two outcomes of this work are worthy of consideration. First, building an education system with a strong foundation in the local culture appears to produce positive effects in all indicators of school success, including dropout rates, college attendance, parent involvement, grade-point averages, and standardized achievement test scores. Second, the Cultural Standards were compiled by educators from throughout the state as an outgrowth of the work that was initiated though the AKRSI and implemented in varying degrees by the participating schools. The authors also argue that when a persistent effort is made to

OCR for page 127
forge a strong “cultural fit” between what we teach, how we teach, and the context in which we teach, we can produce successful, well-rounded graduates who are also capable of producing satisfactory test scores. Baron, J.B. (1991). Strategies for the Development of Effective Performance Exercises. Applied Measurement in Education.4(4), 305-318. Bay, J.M., Reys, B.J., and Reys, R.E. (1999). The Top 10 Elements That Must Be in Place to Implement Standards-Based Mathematics Curricula. Phi Delta Kappan.80(7), 503-506. Berggoetz, B. (2001, November). Indiana Chosen to Be in School Standards Study. Indianapolis Star.November 27, 2001. Berkheimer, G.D., Anderson, C.W., and Blakeslee, T.D. (1988). Matter and molecules teacher’s guide: Activity book. Occasional paper number 122. East Lansing, MI: Michigan State University, Institute for Research on Teaching. Berkheimer, G.D., Anderson, C.W., Lee, O., and Blakeslee, T.D. (1988). Matter and molecules teacher’s guide: Science book. Occasional paper number 121. East Lansing, MI: Michigan State University, Institute for Research on Teaching. Berns, B.B. and Swanson, J. (2000). Middle School Science: Working in a Confused Context, April 28, 2000. Paper presented at the American Educational Research Association Annual Meeting, New Orleans, LA. Biddle, B.J. (1997). Foolishness, Dangerous Nonsense, and Real Correlates of State Differences in Achievement. Phi Delta Kappan.79(1), 8-13. Biddle questions the fundamental premise that standards have an influence on student achievement. He argues that improving achievement is about making resources available to children and to their teachers, not about setting standards. Biddle backs up his argument with analyses of three data sets from the Second International Mathematics Study (SIMS), the Third International Mathematics and Science Study (TIMSS), and the National Assessment of Educational Progress (NAEP). This report presents evidence that (1) the United States has greater disparities in school funding and higher levels of child poverty than other developed countries participating in the study and (2) these differences are strongly correlated with the differences in achievement among school districts and among states. Factors such as school funding and child poverty do affect student learning, and they will continue to do so whether we have national standards or not. For example, Biddle explored predictors of eighth-grade achievement scores for public schools. Results revealed statistically significant, net effects for both school funding (β = +.296, p<.01) and child poverty (β = –.358, p<.01). These effects persisted even when controls were entered for such potent variables as race and level of curriculum to which students had been exposed. Moreover, district-level differences in school funding and child poverty explained more than 25 percent of the variance of differences in mathematics achievement. Biddle also discovers that state differences in school funding are correlated with mathematics achievement at r = +.433 (p<.01), whereas the child poverty/ achievement correlation is a mammoth r = –.700 (p<.001). When funding and poverty are considered as joint predictors of achievement in a regression analysis, the net effects of both factors remain statistically significant, with β = +.262 (p<.03) for school funding and β = –.629 (p<.001) for child poverty, and that these two factors predict an astounding 55 percent of the variance of state differences in average achievement. In other words, not only do differences in school funding and child poverty matter at the state level, they are major predictor of state-level averages in mathematics achievement. Indeed, the impact of child poverty seems to be stronger at the state level than at the district level.

OCR for page 127
Biological Sciences Curriculum Study. (1993). Developing Biological Literacy.Colorado Springs, CO: Author. BSCS, with support from the National Science Foundation, developed a curriculum framework for high school biology. For this project, BSCS commissioned papers, reviewed the literature, and held a conference to develop its recommendations. The three major recommendations were: (1) the content of biology must be unified by the theory of evolution, (2) biology classes must provide opportunities for students to experience science as a process and to understand science as a way of knowing, and (3) programs should help students develop biological literacy. The report identifies four levels of biological literacy: nominal, functional, structural, and multidimensional. According to the report, “education in biology should sustain students’ interest in the natural world, help students explore new areas of interest, improve their explanations of biological concepts, help them develop an understanding and use of inquiry and technology, and contribute to their making informed personal and social decisions.” The report recommends that assessment instruments be closely linked with instructional strategies. The report recommends the 5-E instructional model for biology programs and that the curriculum should be organized around major conceptual themes of biology, such as evolution. The major themes are: evolution, interaction and interdependence, genetic continuity and reproduction, growth, development, and differentiation, energy, matter, and organization, and maintenance of dynamic equilibrium. Biological Sciences Curriculum Study and International Business Machines. (1989). New Designs for Elementary School Science and Health.Colorado Springs, CO: Biological Sciences Curriculum Study. This was a design study for elementary school science and health, supported by the National Science Foundation and IBM. The project had three major goals: (1) to design a framework for an elementary school science and health program consistent with current trends and needs as identified by the education and science communities, (2) to determine the appropriate uses of microcomputer technology in elementary science and health programs, and (3) to produce a plan for implementing educational computing consistent with an exemplary science and health program for elementary schools. The report presents a rationale for a new approach to elementary school science and health; a curriculum framework with scope and sequence for a proposed elementary school science and health program; an instructional model (5-E) for elementary school science and health; recommendations for the integration of technology and elementary-school science and health; a description of a technology-oriented learning environment; a description of educational courseware for a technology-oriented elementary school science and health program; and recommendations for implementation of a technology-oriented curriculum. Birman, B.F., Reeve, A.L., and Sattler, C.L. (1998). The Eisenhower Professional Development Program: Emerging Themes from Six Districts.Washington, DC: U.S. Department of Education; The American Institute for Research. This study reports on an evaluation of the Eisenhower professional development program in six districts. The evaluation report, the first in a series of reports on different aspects of the Eisenhower program, focuses on six exploratory district case studies conducted in the spring of 1997. The six sites were chosen for geographic and programmatic diversity. Data for the case studies included document review, site visits, administrative interviews in each site, focus groups with teachers and professional development providers in each site, and follow-up phone interviews with Eisenhower coordinators in the states of each site. The analysis methodologies are not reported. The authors viewed these exploratory case studies primarily as a way to familiarize themselves with some of the sites and to identify themes for more in-depth exploration. The findings of the report are organized around 10 emerging themes. The themes, or findings, are quite broad. For example, the authors report that the program supported a wide variety of activities; that most efforts went toward mathematics and science professional development; that most of the professional development that the funding supported was consistent with standards for high-quality professional development; and that the reliability of the Eisenhower funding allowed districts to engage in long-term planning and to leverage other funds. Overall, the authors conclude that the Eisenhower-funded activities emphasized several elements of high-quality professional development, including sustained and intensive professional development, the use of teachers as leaders, and promoting alignment with high standards. They found that the Eisenhower coordinators were able to identify some components of high-quality professional development.

OCR for page 127
Bischoff, P.J., Watford, L.J., and Hatch, D.D. (1999). The State of Readiness of Initial Level Preservice Middle Grades Science and Mathematics Teachers and Its Implications on Teacher Education Programs. School Science & Mathematics.99(7), 394-399. Bishop, J. (1998). Do Curriculum-Based External Exit Exam Systems Enhance Student Achievement?Philadelphia, PA: Consortium for Policy Research in Education. This investigation used four existing data sets to test the hypothesis that curriculum-based external exit examination systems (CBEEES) improve achievement. The four data sets included science and mathematics achievement of seventh and eighth graders in the 40-nation Third International Mathematics and Science Study (TIMSS); science and mathematics scores of 13-year-olds on the International Assessment of Educational Progress (IAEP) for 16 nations and nine Canadian provinces; and SAT and NAEP mathematics scores for New York State versus the rest of the United States. Of the 40 countries that participated in TIMSS, 22 national school systems were classified as having CBEEES. Regression analyses produced results that show a substantial relationship between countries with CBEEES and achievement in science and mathematics. Bishop studied assessment results for New York State because of its use of the Regents Examinations in the early 1990s, which, for the purpose of this study, the author identified as a CBEEES. New York students were found to do significantly better on the SAT than students of the same race and social backgrounds in other states. NAEP mathematics scores for New York supported these findings. Data used in this study were all collected prior to the release of the National Science Education Standards and cannot be used to support the impact of these standards on student achievement. The general findings do produce evidence of the relationship between high accountability systems and achievement by comparing nations and states. However, this study only considers relational data and does not provide any evidence of how improved content standards may have an impact on student learning. The improved learning could be for other reasons, such as increased study time or reduced class size, rather than being curriculum-associated. If the external exit examinations are standards-based, then the findings from this study suggest that student learning would be improved in the directions advocated by the standards. Black, P. and Wiliam, D. (1998). Inside the Black Box: Raising Standards Through Classroom Assessment. Phi Delta Kappan.80(2), 139-144. This article reports the results of a meta-analysis of over 40 studies showing increased formative assessment produces substantial learning gains. A review of the results from 23 studies on classroom assessment of children with mild handicaps was published in 1986. Black and Wiliam reviewed more than 20 additional studies that showed innovations, including strengthening the practice of formative assessment, that produced significant and often substantial learning gains. In addition to the importance of formative assessment to learning in general, the researchers found that formative assessment helped low achievers more than other students. They suggested that this would lead to reducing the range in achievement, while raising achievement overall. The researchers then went on to cite literature that identified the shortcomings in the everyday practice of classroom assessment, including some articles that addressed assessment in science. After identifying deficiencies in formative assessment practices, the researchers offer ways that formative assessment practices can be improved. Some of these included giving students feedback on the quality of their work and avoiding comparisons with other students, students having a clear understanding of learning targets, and the value of self-assessment. The meta-analysis foundation for this article was very thoroughly done and located findings that supported the value of formative assessment, including some experimental studies. The researchers then expanded this finding to describe how formative assessment and teaching can be improved, building some on the literature, but mainly depending on experience and logic. Blank, R.K. (2000). Summary of Findings from SSI and Recommendations for NSF’s Role with States: How NSF Can Encourage State Leadership in Improvement of Science and Mathematics Education.Washington, DC: Council of Chief State School Officers. This paper is designed to inform policy makers and the National Science Foundation about the lessons of systemic reform in science and mathematics. It is a review of studies and evaluations of NSF’s Statewide Sys-

OCR for page 127
temic Initiatives (SSI). The review clearly states its data sources, which include a review of existing studies, the results of a conference of findings of the SSI programs, and discussions with state leaders. A planning committee developed a framework for analysis and reporting the findings in six areas: support for systemic reform, leadership, resources/partnerships, policy/infrastructure, strategic decisions/interventions, sustainability, and outcomes/evaluations. The paper contains three major sections. In the first, it highlights the findings in the six areas. The second section contains recommendations on each of these findings from state leaders on how to more effectively implement standards-based mathematics and science education statewide. The final section discusses the implications for new NSF programs. In terms of the influence of the standards on the systems of professional development, several findings are pertinent. First, successful SSIs developed and effectively promulgated a vision for reform in their state based on the standards. Second, effective SSIs included leadership for local leaders in their training. Third, successful states aligned policies that supported changes in the state infrastructures related to teacher quality such as licensure and teacher preparation. Fourth, effective states focused their professional development on standards-based curriculum and materials, content knowledge, and active learning. Blank, R.K., Bush, M.H., Pechman, E.M., Goldstein, D., and Sardina, S.L. (1997). A State-by-State Look at Content Standards and Benchmarks: Examples of Mathematics and Science Standards.Washington, DC: Council of Chief State School Officers. Blank, R.K., Kim, J.J., and Smithson, J. (2000). Survey Results of Urban School Classroom Practices in Mathematics and Science: 1999 Report. Using the Survey of Enacted Curriculum Conducted During Four USI Site Visits. How Reform Works: An Evaluative Study of the National Science Foundation’s Urban System Initiatives. Study Monograph No. 2.Washington, DC: Council of Chief State School Officers. This report investigated the impact of the Urban Systemic Initiative (USI) program on four urban school districts. The project collected data using the Survey of Enacted Curriculum, focusing on enacted curriculum contents and teaching practices. For the study, data were collected from 80 teachers from 20 elementary and middle schools for each site. The survey addressed the six drivers of educational system reform identified by the National Science Foundation: (1) implementation of comprehensive, standards-based curricula, (2) development of a coherent, consistent set of polices, (3) convergence of the usage of all resources that are designed for or that reasonably could be used to support science and mathematics education, (4) broad-based support from parents, policy makers, institutions of higher education, business and industry, foundations, and other segments of the community, (5) accumulation of a broad and deep array of evidence that the program is enhancing student achievement, and (6) improvement in the achievement of all students, including those historically underserved. The results of the study relevant to the science curriculum are as follows: Hands-on or laboratory materials was the largest activity (25 percent of the time). Teachers reported students were engaged more often in “use science experiment,” “follow step-by-step directions,” and “make tables, graphs or charts” and less often in “changing something in an experiment to see what happens” or “designing an experiment.” However, in schools involved in the USI program, elementary students were less likely to “follow step-by-step instructions” and more likely to “change something in an experiment to see what will happen.” Students in USI middle schools spent more time “using science equipment and tools in experiments or investigations and in “collecting data” and “designing ways to solve a problem,” but spent less time to “make predictions, guesses, or hypotheses” or to “draw conclusions from science data.” When working in small groups, the highest use of class time was to “write results or conclusions of a laboratory activity” (about 22 percent of the time). High-implementation USI schools spent less time on “review assignments and problems.” Teachers in USI implementation schools spent more time on life science and chemistry, and less on physical science.

OCR for page 127
work and external pressures that contribute to district leaders’ perceptions about teaching and learning and consequently about the types of learning opportunities that they provide for teachers in their districts. Spillane, J.P. (2001). Challenging Instruction for “All Students”: Policy, Practitioners, and Practice. In S.H. Fuhrman (Ed.), From the Capitol to the Classroom: Standards-Based Reform in the States, One Hundredth Yearbook of the National Society for the Study of Education (Chapter 11, pp. 217-241). Chicago: University of Chicago Press. Spillane, J.P., and Zeuli, J.S. (1999). Reform and Teaching: Exploring Patters of Practice in the Context of National and State Mathematics Reforms. Educational Evaluation and Policy Analysis.21(1), 1-27. This article investigated 25 classroom teachers’ patterns of mathematics instructional practice in the context of national, state, and local efforts to reform mathematics education. The goal of the study was to look carefully within practice to understand progress of reform, identifying efforts that are in the direction of reform and those that remained unchanged. Both quantitative and qualitative methods were used to collect the data. The TIMSS questionnaire, with a set of items related to the reforms identified, was administered to 640 third-, fourth-, seventh- and eighth-grade teachers from nine Michigan school districts in mid-size city, suburban, and rural areas; 283 teachers responded (44 percent). A subsample of 25 teachers (18 third/fourth-grade and 7 seventh/ eighth-grade mathematics teachers) who reported practice that was fairly well aligned with the reform vision were interviewed and observed. The analysis focused on the intersection of classroom tasks and discourse patterns with principled and procedural mathematics knowledge; three distinctively different patterns of instruction were identified, with some dimensions of practice found to be more responsive to reform than others. Pattern one, found in four of the 25 classrooms, was the closest to reform practices. It involved principled knowledge tasks and principled knowledge discourse. Pattern two, observed in 10 classrooms, was not as closely aligned with reform. While it highlighted principled knowledge tasks, the discourse focused more on procedural knowledge. Pattern three, evident in 11 classrooms, included aspects of reform such as group work and use of manipulatives; however instruction was primarily grounded in procedural knowledge tasks and discourse. This study highlights the need for caution in interpreting self-report data on standards-based practice; the authors noted that even when teachers report teaching in ways consistent with mathematics reforms, they create diverse responses to the reforms because of their beliefs, knowledge, and experiences. Spiri, M.H. (2001). Children Achieving: School Leadership and Reform: Case Studies of Philadelphia Principals. The Evaluation of the Annenberg Challenge in Philadelphia. Philadelphia, PA: Consortium for Policy Research in Education. SRI International (1998). “Appendix” Evaluations of Student Outcomes in Seven SSIs. In K.G. LaGuarda, Assessing the SSI’s Impact on Student Achievement: An Imperfect Science. Menlo Park, CA: Author. Stecher, B.M., Barron, S., Kaganoff, T., and Goodwin, J. (1998). The Effects of Standards-Based Assessment on Classroom Practices: Results of the 1996-97 RAND Survey of Kentucky Teachers of Mathematics and Writing.CSE Technical Report 482. Los Angeles: CRESST. This is the first report of a multiyear research project in Kentucky investigating the consequences of standards-based assessment reform at school and classroom levels. The influence of the Kentucky standards-based reform, driven by the Kentucky Education Reform Act (KERA), on teachers’ classroom practices in mathematics and writing was studied. A random sample of about 400 teachers from Kentucky responded to a written questionnaire on their classroom practices. Researchers selected a stratified random sample of 280 schools, grouped by gain in mathematics or writing biennial scores (1992-1994 vs. 1994-1996) (low, medium, and high) and by size (small and large). Four samples of 70 schools were selected, one each for grade 4 writing, grade 5 mathematics, grade 7 writing, and grade 8 mathematics. Seventy percent of the teachers sampled responded to the written survey. A closed-form question was used for most questions. Teachers were asked

OCR for page 127
about current practices and change in practices over the past three years. Statistical differences between responses for teachers in low- and high-gain schools were computed using chi-square and t-tests. Over one-third of the elementary teachers reported increasing the amount of time spent on science to four hours a week. Over half of the elementary teachers said they increased the frequency of the times when they had integrated mathematics with science. These are the only two findings related to science. Most teachers of mathematics felt that the changes in the school mathematics program did not have a large impact on state assessment scores; rather, improved performance was more related to greater familiarity with the test format. However, a greater number of teachers from schools with high gains than from those with low gains attributed higher student scores to improved practices associated with the state reform. Two-thirds of the grade 8 mathematics teachers from the high-gain schools reported that the NCTM Standards had a great deal of influence over content and teaching strategies compared to 37 percent of grade 8 mathematics teachers from low-gain schools. Teachers reported that the state assessments and the curriculum materials provided by the state had a strong influence on mathematics instruction. This is a comprehensive study based on teacher self-report information. Findings contrasting high- and low-gain schools are subjects for review and can be biased due to selection on the dependent variable. Stefanich, G.P., and Egelston-Dodd, J. (Eds.). (1994). A Futures Agenda: Proceedings of a Working Conference on Science for Persons with Disabilities.Missoula, MT: Montana University Affiliated Rural Institute. St. John, M., Carroll, B., Century, J., Eggers-Pierola, C., Hirabayashi, J., Houghton, N., Jennings, S., Tibbitts, F., and Von Blum, R. (1999, April). The Quality of the Teaching of Mathematics, Science and Technology in K-12 Classrooms in New York State. A Summary of Findings.Inverness, CA: Inverness Research Associates. Available at: http://www.inverness-research.org [September 3, 2002]. This report summarizes the findings of The New York State Landscape Study, a component of the New York Statewide Initiative (NYSSI) funded by the National Science Foundation (NSF) and evaluated by Inverness Research Associates. The purpose of the study was to determine the current status and quality of mathematics, science, and technology instruction in K-12 classrooms. The evaluation sample included seven randomly selected districts of varying types; a total of 156 K-12 classroom observations of mathematics, science, and technology (MST) lessons were conducted using an observation protocol developed by Horizon Research, Inc. In addition to summarizing the quality of MST teaching, this report provides data summaries that describe differences between MST lessons, and differences in quality between grade levels and different district types. The findings from the classroom observation data indicate that only a small fraction of MST lessons reflected the vision for classrooms as stated in the national standards documents. The underlying culture of the classrooms interfered with student learning, and the lessons were not likely to enhance student ability and interest in the discipline. In comparing subject-specific lessons, the researchers found that technology lessons were rated favorably overall, with only minor differences between mathematics and science lessons. The variation in quality of lessons was found to be greater within each district than across districts, however significant differences were seen between urban and non-urban districts. Concluding comments indicate that MST instruction in New York K-12 classrooms is merely in the beginning stages of effective implementation. The authors argue for ongoing examinations of the quality of teaching in real classrooms, in hopes that they can provide incentives and guidance for improvements in instruction. Stepanek, J. (1997, June). School Improvement Program, Science and Mathematics Standards in the Classroom: It’s Just Good Teaching.Portland, OR: Northwest Regional Educational Lab. Stevens, F.I. (1996). Opportunity to Learn Science: Connecting Research Knowledge to Classroom Practice. Mid-Atlantic Laboratory for Student Success.Philadelphia, PA: National Research Center on Education in the Inner Cities.

OCR for page 127
Stevenson, H.W. (1998, March). A Study of Three Cultures: Germany, Japan and the United States—An Overview of the TIMSS Case Study Project. Phi Delta Kappan.79(7), 524-29. This article summarizes the results of the three case studies of mathematics and science teaching in the United States, Germany, and Japan. The studies used a quasi-ethnographic methodology that involved observations and interviews with families and teachers and information obtained from school authorities and government policy experts. The study focused on: national standards, teacher training and teachers’ working conditions, attitudes toward dealing with differences in ability, and the place of school in adolescents’ lives. Careful attention was given to the selection of research sites, hiring of researchers, and devising research procedures. Major findings included the following. The amount of national control of the science curriculum varies among the three nations. In the United States, there is no mechanism at the federal level for controlling the curriculum. Even though state and voluntary national standards do influence school curricula, there is a strong drive for local decision making in what is taught. In the United States, textbooks are the de facto curriculum, with publishers producing books that maximize sales. In Germany, the Conference of Ministers of Education, with representatives from each state, oversees the educational polices and coordinates the structure, institutions, and graduation requirements. This national-level effort forms a basis for a degree of comparability across the states. In Germany, the textbooks must conform to state guidelines and be approved by a state committee. Textbooks establish the content and organization of the courses, but the German teacher is able to develop his or her own course material. In Japan, the Ministry of Education develops national curricular guidelines and standards, but flexibility is given to schools to decide exactly what is to be taught at each grade level. The Ministry of Education approves the textbooks to ensure their adherence to the curriculum guidelines and quality of presentation. Supovitz, J.A. (2001). Translating Teaching Practice into Improved Student Achievement. In S.H. Fuhrman (Ed.), From the Capitol to the Classroom: Standards-Based Reform in the States, The One Hundredth Yearbook of the National Society for the Study of Education , pp. 81-98. Chicago: University of Chicago Press. Supovitz, J.A., Mayer, D.P., and Kahle, J.B. (2000). Promoting Inquiry-Based Instructional Practice: The Longitudinal Impact of Professional Development in the Context of Systemic Reform. Educational Policy.14(3), 331–356. Supovitz, J.A., and Turner, H.M. (2000). The effects of professional development on science teaching practices and classroom culture. Journal of Research in Science Teaching.37(9), 963-80. This study reports a strong and significant relationship between professional development and a teacher’s practice and classroom cultures. Both teaching practices and classroom cultures were affected most deeply after intensive and sustained staff development activities. Supovitz and Turner found that teachers’ self-reports of inquiry teaching practices and investigative classroom cultures depended on the quantity of professional development in Local Systemic Change projects. It was only teachers with more than two weeks of professional development who reported teaching practices and classroom cultures above average. It appears that it was somewhat more difficult to change classroom culture than teaching practices. The positive results came for teachers who had spent 80 hours in focused professional development. The best change in investigative culture came only after 160 hours of in-service education. Supovitz and Turner argue that standards-based classroom practices require substantial investments in standards-based curricula or professional development. All the LSC projects have a heavy standards emphasis and are required to use NSF-approved curriculum materials in support of their initiatives. Teachers in this study were provided with curriculum materials of grade-level appropriate and content-rich activities linked to larger science concepts as well as sequenced to meet national standards. The authors also argue that the most powerful predictors of reform teaching are (1) content preparation as an individual teacher factor and (2) school factors such as differences in class size, discipline, and time allocations. Thiessen, D. (2000). Developing Knowledge for Preparing Teachers: Redefining the Role of Schools of Education. In K.S. Gallagher and J.D. Bailey (Eds.), The Politics of Education Reform, pp. 129-144. The National Commission on Teaching and America’s Future. Thousand Oaks, CA: Corwin Press.

OCR for page 127
Thompson, B. (2002). What Future Quantitative Social Science Research Could Look Like: Confidence Intervals for Effect Sizes. Educational Researcher.31(3), 25–32. Thompson, D.L., Spillane, J., and Cohen D.K. (1994). The State Policy System Affecting Science and Mathematics Education in Michigan.East Lansing, MI: MSSI Policy and Program Review Component, Michigan Partnership for a New Education. Thorson, A. (Ed.). (2000). Assessment That Informs Practice. Eisenhower National Clearinghouse for Mathematics and Science Education. Enc Focus.7 (2). Available at: http://enc.org/focus/assessment [August 8, 2002]. Tuomi, J. (1994, April). Teachers: The Vision Supported. In Scientists, Educators, and National Standards: Action at the Local Level, Sigma Xi Forum Proceedings. Sigma XI, The Scientific Research Society, Research Triangle Park, NC, April 14-15. Underhill, R.G., Abdi, S.W., and Peters, P.F. (1994, January). The Virginia State Systemic Initiative: A Brief Overview of the Lead Teacher Component and a Description of the Evolving Mathematics and Science Integration Outcomes. School Science & Mathematics.94 (1), 26-29. This article describes the Lead Teacher Component of an NSF-funded State Systemic Initiative, called Virginia’s Quality Education in Science and Technology (V-QUEST). Noting that both AAAS’ Project 2061: Science for All Americans and NCTM’s Curriculum and Evaluation Standards for School Mathematics urge schools to prepare mathematically and scientifically literate students, the authors argue that the traditional practice of teaching mathematics and science separately hinders students’ ability to develop into citizens who are literate in mathematics and science. After briefly describing the beliefs of the project’s planning team, the article explains how the lead teacher component of V-QUEST includes classroom activities that are designed to help teachers integrate the two subjects. The article goes on to share more details about the V-QUEST project as a whole, including its guiding principles, objectives, and strategies. The article also shares some insights gained from the project’s pilot year and first summer institutes efforts; for example, they found that “our approach of focusing on conceptions and projects has been beneficial but inadequate.” It does not describe the evidence upon which these statements are based. While many of the project beliefs are consistent with national standards, integration of mathematics and science is the centerpiece of this reform initiative, but not central to the national standards documents. Valverde, G.A., and Schmidt, W.H. (1997). Refocusing U.S. Math and Science Education. Issues in Science and Technology Online.Winter 1997. Available at: http://ustimss.msu.edu [August 8, 2002]. This is a report summarizing results from the Third International Mathematics and Science Study (TIMSS) that pertain to the status of the science curriculum in the United States. The achievement results in science ranged from being tied for second among TIMSS countries at the fourth-grade level, to being just slightly above the international average at the eighth grade, to being at the bottom of the countries at the twelfth grade. When looking at specific topic areas of the science tests, a picture emerges where on some topics (e.g., organs and tissues), no countries outperformed U.S. students. U.S. students did best in life science and earth science on the grade 4 and grade 8 tests and they performed worst in physical science. This pattern is consistent with the emphasis on life science and earth science in the seventh- and eighth-grade curriculum in the United States. The authors concluded that curriculum makes a difference, and that the United States does not have a coherent, coordinated view of what children are to know in science. The U.S. curriculum lacks focus and covers many more topics each year, compared to the rest of the TIMSS countries. This is true of state frameworks that define what children should learn, of textbooks, and of what is actually taught by teachers. Grade 8 textbooks in the United States cover 65 science topics as compared to around 25 typical of other TIMSS countries. The authors note that “U.S. eighth-grade science textbooks were 700 or more pages long, hardbound, and resembled encyclopedia volumes. By contrast, many other countries’ textbooks were paperbacks with less than 200 pages”

OCR for page 127
(p. 3). U.S. frameworks and textbooks lack coherence, failing to connect ideas to larger and more coherent wholes. The U.S. curriculum lacked intellectual rigor at the eighth grade and covered many of the same topics that were done in earlier grades. Van Zee, E.H., Iwasyk, M., Kurose, A., Simpson, D., and Wild, J. (2001). Student and Teacher Questioning During Conversations About Science. Journal of Research in Science Teaching.38(2), 159-190. Vermont State Department of Education. (1996). Vermont’s Framework of Standards and Learning Opportunities. Montpelier, VT: Author. This report describes Vermont’s framework of standards and learning opportunities. The document is to be used to provide structure for the development, organization, implementation, and assessment of curricula; to provide the basis for the development of a state, local, and classroom comprehensive assessment system; and to specify what may be included in statewide assessments of student learning. The framework has four main parts: vital results standards, field of knowledge standards, learning opportunities, and appendices that describe how the framework was developed and is to be used. Vital Results Standards include communication, reasoning and problem-solving, personal development, and civic/social responsibility. Fields of Knowledge Standards are provided in the following areas: (1) arts/language and literature, (2) history and social sciences, and (3) science, mathematics, and technology. Learning opportunities refer to issues of access, instruction, assessment and reporting, connections among subjects, and best practices in the fields of knowledge. The development of the framework began in 1993 and was completed in 1996, concurrent with the development of the NSES. Teachers, school administrators, school board members, parents and community members, health and human services staff, business and higher education representatives, consultants, staff of the Vermont Institute for Science, Mathematics, and Technology, and school improvement teams at the Vermont Department of Education were involved in the development of the framework. An effort also was made to reflect the work of the New Standards project in the Vermont Standards. Von Driel, J.H., Beijaard, D., and Verloop, N. (2001). Professional Development and Reform in Science Education: The Role of Teachers’ Practical Knowledge. Journal of Research in Science Teaching.38(2), 137-158. In this article, professional development focused on developing teachers’ practical knowledge is discussed in light of the current education reforms in science, including the NSES in the United States and reform documents in other western countries. Teachers’ practical knowledge is defined as the combination of experiential knowledge, formal knowledge, and personal beliefs held in the context of the teachers’ work. On the basis of a literature review, the authors argue that many reform efforts have been unsuccessful because teachers’ practical knowledge was rarely taken into account. The authors provide only skeletal detail about the studies they used. Based on their review, the authors suggest that future studies with multi-method designs are needed to understand this complex type of knowledge. It is recommended that reform efforts take into account teachers’ practical knowledge from the start, and that changes in this knowledge be monitored throughout reform projects. The authors also conclude that long-term professional development programs are the best option for lasting change in teaching practices, with the following strategies showing the most potential: (1) learning in networks, (2) peer coaching, (3) collaborative action research, and (4) the use of cases. Von Secker, C.E., and Lissitz, R.W. (1999). Estimating the Impact of Instructional Practices on Student Achievement in Science. Journal of Research in Science Teaching.36(10), 1110-1126. Von Secker and Lissitz report on analyses of data on science achievement from the 1990 High School Effectiveness Study. They found that traditional teacher-centered instruction was related to lower average science achievement. There was a positive correlation between tenth-grade science achievement, as measured by science tests constructed by the Educational Testing Service, and laboratory-centered instruction. There is a positive relationship with individual environment and differences such as SES, gender, and minority. This study uses a hierarchical linear model (HLM) to estimate direct and indirect effects of instructional practices recommended by the NSES on individual achievement. It applied unconditional HLM and unconditional Within-School

OCR for page 127
HLM, as well as conditional Between-School HLM. These results suggest that the NSES are more likely to promote equity if they are supported by national, state, and local efforts to provide equal opportunities for access to laboratory facilities, equipment, and supplies. De-emphasizing traditional teacher-centered instruction is expected to increase average science achievement and minimize gaps in achievement between individuals of different socioeconomic status. However, from the HLM results, teacher-centered instruction does not cause inequity in achievement associated with SES, and multiple explanations for this association are reasonable. The findings suggest that instruction matters. School excellence and equity can be positively or negatively affected by the way science is taught. Ware, M., Richardson, L., and Kim, J.J. (2000, March). What Matters in Urban School Reform. How Reform Works: An Evaluative Study of National Science Foundation’s Urban Systemic Initiatives. Study Monograph No. 1. Available at: http://www.systemic.com/publication.cfm#usi [August 8, 2002]. Warren, B., Ballenger, C., Ogonowski, M., Rosebery, A.S., and Hudicourt-Barnes, J. (2001, May). Rethinking Diversity in Learning Science: The Logic of Everyday Sense-Making. Journal of Research in Science Teaching. 38(5), 529-552. Warren, Ballenger, Ogonowski, Rosebery, and Hudicourt-Barnes argue that it is crucial to understand children’s diverse sense-making practices as intellectual resources in science learning and teaching. The authors discuss how the relationship between everyday and scientific knowledge and ways of knowing has been conceptualized in the field of science education research. It is important to take seriously the ideas and ways of talking and knowing that children from diverse communities bring to science. Science learning is not simply the accumulation of different ways with words and ways of seeing. Rather, it is from different perspectives as a creative critical process, in which diverse ways with words and ways of seeing are probed, challenged, and perhaps even transformed to the benefit of all students. The authors suggest that the diverse ideas and ways of talking and knowing of all children be brought into contact with each other as well as with recognized canonical views and modes of organizing explanations and arguments. Too little attention has been paid by researchers and teachers alike to the potentially profound continuities between everyday and scientific ways of knowing and talking, and thus to the pedagogical possibilities that may be derived from such an analysis, especially for typically marginalized children. It is necessary to have a framework for understanding the everyday sense-making practices of students from diverse communities as an intellectual resource in science learning and teaching. Two case studies illustrate this point of view. Through analysis of Haitian American and Latino students’ talk and activity, the authors show how the students work to understand metamorphosis and experimentation with diverse sense-making practice. Watson, S., Foley, E., Tighe, E., and Wang, A. (2001). Children Achieving: Recruiting and Retaining Teachers: Keys to Improving the Philadelphia Public Schools.Philadelphia, PA: Consortium for Policy Research in Education. Webb, N.L. (1992). Assessment of Students’ Knowledge of Mathematics: Steps Toward a Theory. In D.A. Grouws (Ed.), Handbook of Research on Mathematics Teaching and Learning, pp. 334-368. New York: Macmillan. Webb, N.L. (1997, April). Criteria for Alignment of Expectations and Assessments in Mathematics and Science Education. Research Monograph No. 8. Madison, WI, and Washington, DC: National Insitute for Science Education and Council of Chief State School Officers. This monograph presents a conceptual framework for thinking about and analyzing the alignment among expectations and assessments. Alignment is defined as “the degree to which expectations and assessments are in agreement and serve in conjunction with one another to guide the system toward students learning what they are expected to know and do” (p. 3). Alignment is distinguished from validity because it is an attribute of the relationship between expectations and assessments rather than an attribute of an assessment only. Twelve criteria for judging alignment grouped into five general categories are specified: content focus, articulation across grades and ages, equity and fairness, pedagogical implications, and system applicability. Most commonly,

OCR for page 127
alignment has been thought of only as content focus, with the other categories being ignored. Explanations and illustrative examples of the 12 different criteria are drawn from research and literature in science and mathematics education. A content analysis of the NSES and the Benchmarks for Science Literacy is used to illustrate an expert review approach to studying alignment—in this case, alignment between two documents. The conceptual framework draws upon research and was developed with the input of an expert panel formed as a cooperative effort between the Council of Chief State School Officers (CCSSO) and the National Institute for Science Education (NISE) funded by the National Science Foundation. Webb, N.L. (1999, August). Alignment of Science and Mathematics Standards and Assessments in Four States. Research Monograph No. 18. Madison, WI, and Washington, DC: National Institute for Science Education and Council of Chief State School Officers. Reviewers analyzed the alignment of assessments and standards in mathematics and science from four states at a four-day institute. Six reviewers compared the match between assessment items and state standards in mathematics, and seven compared the match in science. Data from these analyses were processed and used to judge the degree of alignment on the basis of four criteria: categorical concurrence, depth-of-knowledge consistency, range-of-knowledge correspondence, and balance of representation. In science, seven analyses were performed—at two grade levels for two states and three grade levels for one state. The three states varied in the proportion of the standards found to be aligned with the assessments, but within each state there were only small differences among the grade levels. In general, the science standards and assessments were found to be aligned on three of the four criteria—categorical concurrence (number of items per standard), range-of-knowledge correspondence (proportion of objectives of standard assessed), and balance of representation (emphasis given to specific objectives on the assessment). The standards and assessment were less aligned on the depth-of-knowledge consistency criterion. A major goal of the study was to develop a valid and reliable process for analyzing the alignment among standards and assessments. The process did produce credible results that distinguished among the different attributes of alignment and detected specific ways in which alignment could be improved. The states that participated volunteered to be a part of the study and wanted the information in order to achieve better alignment of their assessments and standards. The study employed content analysis to derive the results and the researcher acknowledged that full alignment is determined by the degree to which standards and assessments work together to improve student learning. Weiss, I.R. (1994) A Profile of Science and Mathematics Education in the United States: 1993.Chapel Hill, NC: Horizon Research. This report presents results of the 1993 National Survey of Science and Mathematics Teaching conducted by Horizon Research, Inc. Six thousand teachers in grades 1 through 12 at 1,250 schools completed the survey after a process of sampling was used to select teachers who would accurately estimate the national population. An 88 percent response rate was obtained for school program representatives and 84 percent for science and mathematics teachers. Teachers gave information about their teaching practices, beliefs, and background. School representatives answered questions about the types of courses offered, money spent for different types of educational materials, and problems/obstacles that faced the school. The findings of this study include the movement of science and mathematics education toward current reform ideas. Specifically, hands-on activities have increased, especially in elementary mathematics. However, the goal of quality education for “all students” is still not in sight as inadequate facilities, equipment, and the lack of money to purchase consumable supplies are still formidable barriers. Lack of content preparedness is another obstacle for elementary teachers, although most high school teachers have more extensive backgrounds than their counterparts at lower grades. There is evidence that more teachers are participating in science and mathematics in-service activities, but the small amount of time spent on these activities apparently did not address teachers’ expressed needs for content preparedness and preparedness to teach a diverse student population (e.g., students of different ethnic groups, English Language Learners, and learning disabled).

OCR for page 127
Weiss, I.R. (1997, June). The Status of Science and Mathematics Teaching in the United States: Comparing Teacher Views and Classroom Practice to National Standards. NISE Brief.1(3). The brief addresses teacher attitudes about and classroom implementation of the NCTM Standards and the NSES, using data from the 1993 National Survey of Science and Mathematics Education conducted by Horizon Research, Inc. The 1993 National Survey involved a probability sample of 1,250 schools and approximately 6,000 teachers in grades 1-12 throughout the United States. Teachers were asked to provide information about their qualifications and preparedness, participation in professional activities, and beliefs about math and science instruction. Department heads or teacher-leaders were also asked to report about their school’s science and mathematics programs. The author focuses on the findings that although teachers typically report instructional objectives in line with the vision of the standards, classroom activities are often not well aligned with the recommendations of NCTM and NRC standards, and students do not have equal access to quality education as envisioned by the reform agenda. Support for these findings include the high proportion of classroom time spent learning basic facts and terminology and preparing for standardized tests, and evidence that classes with high percentages of minority students do not have access to the same resources as other classes. Based on the survey data, the author concludes that many teachers do not feel well prepared to teach various content areas or to use the recommended instructional strategies, nor do they feel they get the support they need to implement the recommendations. While many teachers reported support for pedagogical reform, the instructional strategies they reported using leave classroom practice far behind the vision described in the NSES, and the goal of “quality education for all” has not been reached. Implications of these findings and recommendations of the research for the education system include improving teacher preparation such that teachers are grounded in the content they are expected to teach; provided with models of effective standards-based instruction; and given the materials, facilities, and support they need to implement such instruction. Weiss, I.R., Banilower, E.R., McMahon, K.C., and Smith, P.S. (2001). Report of the 2000 National Survey of Science and Mathematics Education.Chapel Hill, NC: Horizon Research. This report summarizes data collected as part of two national surveys—one in 1993, another in 2000—of science and mathematics teachers in grades K-12 public and private schools. Both studies involved national probability samples. The 1993 study sampled 6,000 teachers, and the 2000 study sampled 9,000. Both samples allowed calculations of national estimates. In addition to the questionnaires completed by teachers, science and mathematics program representatives at each study school (approximately 1,000 in each study) completed a questionnaire. Weiss, I. R., Banilower, E. R., Overstreet, C. M., and Soar, E. H. (2002). Local Systemic Change Through Teacher Enhancement: Year Seven Cross-Site Report.Chapel Hill, NC: Horizon Research. Weiss, I.R., Matti, M.C., and Smith, P.S. (1994). Report of the 1993 National Survey of Science and Mathematics Education.Chapel Hill, NC: Horizon Research. Weiss, I.R., and Raphael, J.B. (1996). Characteristics of Presidential Awardees: How Do They Compare with Science and Mathematics Teachers Nationally?Chapel Hill, NC: Horizon Research. Wiggins, G. (1989, May). A True Test: Toward More Authentic and Equitable Assessment. Phi Delta Kappan.70 (9) , 703-713. Wilcox, J., Hoover, J., and Burthwick, P. (1999, March). Disability Research Encompassing Native Americans in Math and Science: A Demonstration Inclusion Project. In Rural Special Education for the New Millennium, Conference Proceeding of the American Council on Rural Special Education (ACRES), pp. 185-190. Albuquerque, NM: ACRES.

OCR for page 127
Wilcoxson, C. (1997, October). Achieving the Vision of the National Standards in Nebraska: A Framework as a First Step to Classroom Implementation. School Science & Mathematics.97(6), 311-315. Wilson, S.M., and Floden, R.E. (2001). Hedging Bets: Standards-Based Reform in Classroom. In S.H. Fuhrman (Ed.), From the Capitol to the Classroom: Standards-Based Reform in the States, The One Hundredth Yearbook of the National Society for the Study of Education, Part 2, pp. 193-216. Chicago: University of Chicago Press. This paper provides a preliminary analysis of a three-year study conducted by the Consortium for Policy Research in Education (CPRE), in which researchers tracked curriculum and assessment reforms in 23 school districts in eight states. Interviews were conducted with teachers, principals, and district staff from these 23 school districts “as they responded to local, state, and national pressures to reform teaching and learning.” In addition, four states were chosen for more intensive interviewing and observations, and all teachers were surveyed in the study’s third year. The goal of the study was to determine the impact of standards-based reform by looking at two questions: (1) What varieties of standards-based reform do teachers encounter in schools? and (2) What is the impact of those reforms? In addressing these questions, the paper first describes the experiences of four schools that are representative of the view of standards-based reform. Then it examines three critical issues—teaching and learning, accountability, and communication—concerning standards-based reform and its impact. The analysis reveals two findings. First, the concept of standards-based reform is interpreted in a wide variety of ways, with perceptions differing even within schools. For some educators, it is hardly noticeable among the other reforms, but for others it has provided a clarity and language for thinking about instruction. Second, teacher interviews, classroom observation, and teacher survey data indicate that classroom practice reflects a balance between traditional and standards-based practices. Instruction still looks traditional, with a mix of reform-oriented practices. Based on these findings, the authors highlight the hopes and concerns for standards-based reform, suggesting that while the rhetoric would make people believe it has the potential for transforming teaching and learning, the evidence is showing otherwise. Elements of reform may be evident, but traditional teaching is prevalent. Wolf, R.M. (1998, May). National Standards: Do We Need Them?Educational Researcher.27(4), 22-25. Wright, J.C., and Wright, C.S. (1998). A Commentary on the Profound Changes Envisioned by the National Science Education Standards. Teachers College Record.100(1), 122-143. In this conceptual paper, the authors, from the perspective of a university faculty member who teaches physical sciences, voice their opinions about the nature of science literacy and how to attain it. The authors point out the difficult challenge of educating our students to achieve science literacy while simultaneously developing the capacity of science teachers to change the nature of the teaching and learning experience. They stress that the standards fail to define the problem they are trying to solve and do not define scientific literacy with sufficient precision required to guide classroom practice. They call for more specific, detailed descriptions of goals of science literacy and of the nature of teaching and learning than are found in the NSES. The authors explain that while the NSES are a brilliant definition of what success is, they do too little to address the issue of implementation of the change required to achieve that vision. The authors believe that science faculty will see different messages about the goals and attitudes underlying the NSES based on their own perceptions of science literacy. The authors call for small-scale, authentic, inquiry-based projects to investigate strategies for implementing reform as a better approach than large-scale systemic reform efforts. They find that teachers and administrators need data, teaching toolkits, menus of approaches, good assessment tools, and clear examples of how changes are implemented and how they work before they will be prepared to tackle wholesale reform. The authors propose that active learning is the lever for moving along reform and that reform should shift from a focus on issues of control to the new paradigm of ownership. The paper questions that the potential impact of standards on science curricula will be constrained unless: (1) science literacy is clearly defined and understood by all stakeholders, (2) reformed curricula develop higher-level conceptual understanding and problem-solving skills, (3) the student is given ownership and responsibility

OCR for page 127
for learning, (4) stakeholders change their attitude and understanding as to the nature of science literacy and how to achieve it, and (5) approaches to teaching, learning, and assessment must change. Yager, R.E., Lutz, M.V., and Craven III, J.A. (1996, June). Do National Standards Indicate the Need for Reform in Science Teacher Education?Journal of Science Teacher Education.7(2), 85-94. Yin, R.K., Noboa-Rios, A., Davis, D., Castillo, I., and MacTurk, R. (2001). Update and Ongoing Work: Cross-Site Evaluation of the Urban Systemic Program.Bethesda, MD: Cosmos. This report describes the cross-site evaluation of the National Science Foundation’s Urban Systemic Program (USP). The USP is currently in 18 sites in two cohorts. The report describes both the formative and the summative components of the cross-site evaluation, including the research design, logic model, and research questions. The report describes a logic model that would explain different stages of systemic reform and proposes an evaluation design that would capture the “systemicness” of each site and the program as a whole. After discussing various traditional evaluation designs, the authors propose a replication design in which each site is considered to be a naturally occurring experiment and cross-site patterns are seen as evidence of replication. The evaluation design focuses on the components in each site that make them systemic. Proposed data collection includes interviews with key officials, document analysis, and direct field observations. The authors also report on their first year of field work with the five first cohort sites. They report early signs of “systemicness” around strategic vision, assessment, professional development, parent and community roles, pre-service education, resource convergence, and partnering. They also discuss the threat of external events to continued progress. Yinger, R.J., and Hendricks-Lee, M.S. (2000). The Language of Standards and Teacher Education Reform. In K.S. Gallagher and J.D. Bailey (Eds.), The Politics of Education Reform, pp. 94-106. The National Commission on Teaching and America’s Future . Thousand Oaks, CA: Corwin Press. Yoon, B., and Young, M.J. (2000, October). Validating Standards-Referenced Science Assessments.CSE Technical Report No. 529. Los Angeles: California University, Center for the Study of Evaluation. Center for Research on Evaluation, Standards, and Student Testing. Zucker, A.A., Shields, P.M., Adelman, N.E., Corcoran, T.B., and Goertz, M.E. (1998, June). A Report on the Evaluation of the National Science Foundation’s Statewide Systemic Initiative (SSI) Program.Menlo Park, CA: SRI International. This report is intended primarily for individuals with an interest in federal education policy. The final report in a series of more than 15 reports, this report summarizes and synthesizes findings from all other reports on a national evaluation of NSF’s Statewide Systemic Initiative (SSI). Through SSI, the National Science Foundation provided funding for five years to selected states undertaking ambitious system-wide reforms in science, mathematics, and technology education. Each state adopted different reform strategies for improving instruction in mathematics and science for all students. The appendices in this report summarize the implementation strategies and impact of the SSI for each state. The authors developed a conceptual model of systemic reform, both to incorporate all the elements that would play a role in achieving SSI’s objectives and to frame their evaluative process. To complete their final assessment, the authors pooled data from a variety of sources: quantitative data gathered annually from the principal investigators in each SSI, repeated site visits in every SSI and subsequent phone interviews, and secondary data analysis of data sets gathered by many SSIs to evaluate their own efforts. The analytical methodologies were not reported. The authors examined the accomplishments and lessons learned by the SSI program and their application to standards-based reform efforts. The following accomplishments were observed: increases in inquiry-based instruction, development and use of high-quality instructional materials, improved professional development, standards-based state curriculum policies, assessments aligned with curriculum, improved student achievement, additional funding sources and mobilized stakeholders, and more highly developed leadership pools. The

OCR for page 127
authors point out that these accomplishments only affected a small fraction of teachers and students within the states and more time is required to see reform efforts reach a larger population. The lessons learned from the SSI program and described in detail in this report will aid reform efforts in the future. As confirmed by the authors, the SSI program created a partnership between federal and state agencies and helped jump-start the movement toward standards-based reform in mathematics and science education. Zucker, A.A., Shields, P.M., Adelman, N.E., and Humphrey, D. (1997). Reflections on State Efforts to Improve Mathematics and Science Education in Light of Findings from TIMSS.Menlo Park, CA: SRI International. The purpose of this study was to investigate how states are implementing their standards. The data for this study came from data sets collected for prior investigations of State Systemic Initiatives and evaluations of the Dwight D. Eisenhower Mathematics and Science Education Curriculum Framework Projects. This report by SRI International summarizes the general findings from TIMSS and found similarities with SRI studies: The science curriculum tries to cover a great many topics but sacrifices intensity of coverage, and deeper understanding, by doing so. SRI studies have found that instructional materials are the weak link, especially in high school science.