National Academies Press: OpenBook
« Previous: 7. Background and Methodology
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

8
Annotated Bibliography

Karen S. Hollweg

Abrams, L., Clarke, M., Pedulla, J., Ramos, M., Rhodes, K., and Shore, A. (2002, April). Accountability and the Classroom: A Multi-State Analysis of the Effects of State-Mandated Testing Programs on Teaching and Learning. National Board on Testing and Public Policy, Boston College. Paper presented at the American Educational Research Association Annual Meeting, New Orleans, LA.

ACCESS ERIC. K-8 Science and Mathematics Education. The ERIC Review.6(2), Fall1999.

Adams, P.E. and Krockover, G.H. (1999). Stimulating Constructivist Teaching Styles Through Use of an Observation Rubric. Journal of Research in Science Teaching.36(8), 955-971.

This study sought to relate a science teacher’s use of the Secondary Science Teaching Analysis Matrix (STAM), which is consistent with the style of teaching advocated by the NSES, with his development over time from a didactic to a more constructivist teacher. Citing others, the authors argue that, despite their pre-service experiences, beginning teachers often adopt “survival strategies” rather than those advocated by the NSES. Using a mechanism like STAM, they argue, teachers can conduct self-assessment and have a heuristic to guide them toward more student-centered styles of teaching. The study was of one teacher who was purposefully selected because his teaching had changed, as measured by the STAM instrument. The authors conducted extensive formal and informal interviews with the teacher, as well as direct classroom observations and videotaped observations, and collected classroom handouts.

The authors analyzed their data with several qualitative analytical techniques, including analytic induction, extensive use of memos, and synthesis of the various data sources. The analysis done in this study seems quite appropriate, but the study is a classic outlier study where the authors chose a case that demonstrated their conclusion and sought to verify it, rather than choose a teacher before they knew the impact of the STAM instrument and seek to see if their hypotheses would hold. The authors inferred that, since both the subject of the case (“Bill”) and their own data pointed to the influence of the STAM as a roadmap for Bill’s progression from a didactic to a constructivist teacher, the use of such an instrument can help novice teachers reflect on and change their teaching practices.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Adelman, N. (1998a). A Case Study of Delaware’s SSI (Project 21), 1991-1997. In P.M. Shields and A.A. Zucker (Eds.), SSI Case Studies, Cohort 1: Connecticut, Delaware, Louisiana, and Montana. Menlo Park, CA: SRI International.

This is a report of a case study of the Delaware State Systemic Initiative, which was supported by the National Science Foundation. The Delaware SSI focused on professional development and curriculum improvement in 34 schools. By the end of the project, 30 percent of the state’s schools and 25 percent of its mathematics and science teachers had been involved. However, only a few of the schools had made whole-school progress toward school change and reform of instruction. The lack of district support, administrative leadership, and technical assistance for overall school change contributed to the disappointing results of the model schools strategy. During the last year of the project, the SSI mathematics and science specialists produced a database of more than 200 standards-based curriculum materials in mathematics and science for consideration for use by school districts.

Adelman, N. (1998b). A Case Study of Maine’s SSI (Maine: A Community of Discovery), 1992-1997. In P.M. Shields and A.A. Zucker (Eds.), SSI Case Studies, Cohort 2: California, Kentucky, Maine, Michigan, Vermont, and Virginia. Menlo Park, CA: SRI International.

This is a report of a case study of the Maine State Systemic Initiative, which was supported by the National Science Foundation. The goal of the Maine SSI was to improve science and mathematics outcomes in grades K-12 throughout the state. The SSI strongly influenced state policy-making activities, supported seven local demonstrations of systemic reform, provided technical assistance to local school districts on request, and developed statewide and regional leadership. The SSI played a key role in development of a state curriculum framework for science and mathematics and in the development of legislative policy on performance standards aligned with the curriculum framework. Over a five-year period leaders of the SSI estimated that they had introduced approximately 60 percent of the state’s science and mathematics teachers to standards-based educational reform and had worked intensively with about 20 percent of them. A key to the success of the Maine SSI was that it was established as a not-for-profit organization that was independent of governmental agencies. The project had less of an impact on reform in high schools and in the state’s largest cities.

Albert, L.R. and Jones, D.L. (1997). Implementing the Science Teaching Standards through Complex Instruction: A Case Study of Two Teacher-Researchers. School Science & Mathematics.97(6), 283-291.

Alberts, B. (1994, April). Science Education Standards. In Scientists, Educators, and National Standards: Action at the Local Level, Sigma Xi Forum Proceedings, Sigma XI, The Scientific Research Society, Research Triangle Park, NC, April 14-15, 1994.

American Association for the Advancement of Science. (1989). Science for All Americans: A Project 2061 Report on Literacy Goals in Science, Mathematics, and Technology.Washington, DC: Author.

American Association for the Advancement of Science. (1993). Benchmarks for Science Literacy.New York: Oxford University Press.

American Association for the Advancement of Science. (1997a). Project 2061: Science Literacy for a Changing Future. Update 1997.Washington, DC: Author.

This is a report of a yearlong evaluation by SRI International of the impact of Science for All Americans and Benchmarks for Science Literacy. The researchers collected data through expert interviews, reviews of state science curriculum frameworks and textbooks, telephone and mail surveys, and case studies of reform activities in six states. The report claims, “Project 2061 has been a major influence on the development of national science education standards and on reform initiatives sponsored by the National Science Foundation, the U.S. Department of Education, and a number of other national education and science organizations” (p. 2). The report also found that the reform ideas promoted by Project 2061 have not been widely adopted by textbook publishers. The

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

study found that 90 percent of educational leaders from 27 states refer to Benchmarks in their day-to-day work. The study found that Project 2061 has had an impact on state curriculum frameworks.

American Association for the Advancement of Science. (1997b). Resources for Science Literacy: Professional Development.New York: Oxford University Press.

American Association for the Advancement of Science. (1998). Blueprints for Reform: Science, Mathematics, and Technology Education.New York: Oxford University Press.

American Association for the Advancement of Science. (2001a). Atlas of Science Literacy.Washington, DC: Author.

American Association for the Advancement of Science. (2001b). Designs for Science Literacy.New York: Oxford University Press.

American Association for the Advancement of Science. (2001c). High School Biology Textbooks Evaluation. Washington, DC: Author.

This study reports on an evaluation of high school biology texts by AAAS. The materials were evaluated by content specialists, biology teachers, and university biology faculty. Each textbook was examined by four two-member teams for a total of 1,000 person hours per book. The evaluators were required to provide specific evidence from the materials to justify their ratings. The study found that the molecular basis of heredity is not covered in a coherent manner in the textbooks, providing needless details and missing the overall story. Overall, the study found that “today’s high-school biology textbooks fail to make biology ideas comprehensible and meaningful to students.”

American Association for the Advancement of Science. (2001d). Middle Grades Science Textbooks Evaluation. Washington, DC: Author.

This is an AAAS report of its evaluation of science texts for the middle grades. The study “examined the text’s quality of instruction aimed specifically at the key ideas, using criteria drawn from the best available research about how students learn.” For the study, each text was evaluated by two independent teams of teachers, curriculum specialists, and science educators. The study reported that “not one of the widely used science textbooks for middle school was rated satisfactory … and the new crop of texts that have entered the market fared no better in the evaluation.” The study found that most textbooks cover too many topics in too little depth. The study also found that many of the learning activities were irrelevant or disconnected from underlying ideas.

American Association for the Advancement of Science. (2001e). Project 2061: Science Literacy for a Changing Future. Update 2001-2002.Washington, DC: Author.

American Federation of Teachers. (1994). What College-Bound Students Abroad Are Expected to Know About Biology. Exams from England and Wales, France, Germany and Japan. In M. Gandal, Defining World Class Standards.Volume 1. Washington, DC: Author.

American Federation of Teachers. (1999). Making Standards Matter 1999.Washington, DC: Author.

This is an annual report that analyzes the quality of the academic standards in 50 states, the District of Columbia, and Puerto Rico. For this study, the authors reviewed state standards, curriculum documents, and other supplemental material and interviewed state officials to obtain information about state standards and their implementation. The study examined two major issues: (1) Does the state have, or is it in the process of developing, standards in the four core academic subjects—English, math, science, and social studies and (2) are the standards clear and specific enough to provide the basis for a common core curriculum from elementary through high school? The authors looked for the following qualities in the standards: (1) standards must define

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

in every grade, or for selected clusters of grades, the common content and skills students should learn in each of the core subjects; (2) standards must be detailed, explicit, and firmly rooted in the content of the subject area to lead to a common core curriculum; (3) for each of the four core curriculum areas, particular content must be present (for science, that was life, earth, and physical sciences); and (4) standards must provide attention to both content and skills. For the purpose of analysis, the standards were divided into 12 large categories using a three-by-four matrix (three levels of elementary, middle, and high school by four core subject areas). For a state to be judged as having quality standards overall, at least nine of the 12 categories must be clear and specific and include the necessary content.

The major findings of the study are as follows:

  1. States’ commitment to standards reform remains strong. The District of Columbia, Puerto Rico, and every state except Iowa have set or are setting common academic standards for students.

  2. The overall quality of the state standards continues to improve. Twenty-two states—up three from 1998— have standards that are generally clear and specific and grounded in particular content to meet AFT’s common core criterion.

  3. Although standards have improved in many states, most states have more difficulty setting clear and specific standards in English and social studies than in math and science. In science, 30 states meet the AFT criteria for all three levels. Thirty-four states have clear and specific standards at the elementary level, 39 at the middle level, and 36 at the high school level. The NSES are widely accepted in the field and cited often in state standards documents.

  4. Every state but Iowa, Montana, and North Dakota is committed to measuring student achievement toward the standards.

  5. Through test items, scoring rubrics, and/or student work samples, many states (26) describe the level that master students must demonstrate to meet the state standards.

  6. Fourteen states have policies for ending social promotion—the practice of passing students from grade to grade regardless of whether they have mastered the standards.

  7. Twenty-eight states have or will have high school exit exams based on the standards.

  8. Twenty-three states have or are developing incentives (advanced diplomas, free college tuition) to motivate students to achieve a higher standard than that required for all students.

  9. Although 40 states require districts to provide intervention to students who are struggling to meet standards, only 29 states fund such programs.

American Federation of Teachers. (2001). Making Standards Matter 2001.Washington, DC: Author.

This is a report of the status of the development and implementation of academic standards in states. For the study, the project analyzed state standards and supplemental documents to determine the quality of the academic standards. The project used the following criteria: (1) standards must define the common content and skills students should learn in each of the core subjects for every grade level or for selected grade spans in elementary, middle, and high school; (2) standards must be detailed, explicit, and firmly rooted in the content of the subject area to lead to a common core curriculum; (3) for each of the four core curriculum areas, particular content must be present (e.g., earth, physical, and life sciences); and (4) standards must provide attention to both content and skills. Each state was rated on the extent to which the standards in each of the four curriculum areas for each of the levels (elementary, middle, and high school) were clear and specific and include the necessary content (a total of 12 categories of standards). For a state to be judged as having quality standards overall, 75 percent of the categories of standards (nine out of 12) had to meet the criteria of quality.

The report also included an analysis of the state curriculum, assessments, accountability, and the overall standards-based system. For the analysis of curriculum work in the states, to be complete, a curriculum must be grade by grade and contain the following five components: a learning continuum, instructional resources, instructional strategies, performance indicators, and lesson plans. For a state to be judged as having a well-developed curriculum, it had to have at least three of the five curriculum components at each of the three levels in each subject area. For the assessment analysis, the project looked for: (1) the state tests students at each

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

educational level in all four core subjects; (2) the state reports information on alignment of the standards and the assessments; and (3) the state indicates the standards to be assessed. To meet the criteria on alignment, a state must: (1) use a test that it developed and specify the standards to be measured, or (2) use an off-the-shelf test, release information about the percentage of test items that are aligned with the state standards, and indicate the standards that are assessed. The project also analyzed the accountability measures in each state. For accountability, the project looked for: (1) the state requires and funds extra help for students having difficulty meeting the standards, and (2) the state developed policies to encourage students to take learning more seriously by providing rewards and consequences based, in part, on state assessment results. To judge state efforts to build a coherent standards-based system, the project looked for: (1) are the tests aligned to the standards? (2) are all of the aligned tests based on strong standards? (3) are curricula developed in all of the aligned test areas? (4) are all promotion and graduation polices based on aligned tests? and (5) do promotion or graduation policies include intervention?

The results of the study are as follows:

  1. States’ commitment to standards-based reform remains strong. Every state and the District of Columbia have set or are setting common academic standards for students.

  2. The overall quality of the state standards continues to improve. Thirty states—up from 22 in 1999—have standards that meet the AFT’s common core criterion.

  3. Most states have more difficulty setting clear and specific standards in English and social studies than in math and science. Thirty-nine states meet the AFT criteria in science at all three levels, and 43 states meet the criteria at the elementary level, 46 at the middle level, and 42 at the high-school level.

  4. State efforts in curriculum have just begun. No state has a fully developed curriculum. Only nine states have 50 percent or more of the components of a fully developed curriculum.

  5. States are more likely to have curriculum materials for English than for the other areas. Nine states have at least three of the curriculum components in science at all three levels.

  6. Thirty-two states assess science at the elementary level, 35 at the middle level, and 40 at the high-school level.

  7. Only nine states have aligned tests in the four core subject areas at all three educational levels. States use a mixture of commercially developed, off-the-shelf standardized tests and their own “home-grown” assessments to measure and report on student achievement.

  8. During the past two years, there is a decrease in the number of states (28 to 25) that require and fund academic intervention programs for students at risk.

  9. Seventeen states have policies for ending social promotion.

  10. Twenty-seven states have or will have high-school exit exams based on the standards.

  11. Thirty states, up from 23 in 1999, have or are developing incentives (e.g., advanced diplomas, free college tuition) to motivate students to achieve a higher standard than required of all students.

  12. Many state assessment programs are based on weak standards.

  13. Many state assessment programs use tests unaligned to their standards.

  14. A number of states use results of nonaligned tests to hold back students or to deny them a diploma.

  15. Many states impose sanctions on students but fail to mandate intervention and to provide the resources to help them.

The report makes the following recommendations regarding the curriculum:

  • Involve teachers in the development of grade-by-grade curriculum aligned to the standards in the core subjects.

  • Specify the learning continuum in the core subjects to show the progression and development of critical knowledge and skills from grade to grade.

  • Identify instructional resources that are aligned to the standards.

  • Provide information on instructional strategies.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
  • Provide performance indicators to clarify the quality of student work required.

  • Develop lesson plan data banks that include exemplary lessons and student work.

  • Provide guidance and incentives to schools so that they attend to important areas of the curriculum that are not addressed—e.g., art, music, foreign languages.

Andersen, H.O. (2000). Emerging Certifications and Teacher Preparation. School Science & Mathematics.100(6), 298-303.

In this paper, the author reports on a state’s transition from certification based upon inputs to a performance-based teacher certification program. The paper describes changes in both Indiana University’s and the state of Indiana’s teacher preparation program. Up until the date of the article (2000), the state had a certification program that required students to complete coursework in order to receive their teaching certification. The author explains that the state is planning (but has not yet instituted) a performance-based certification process. Teachers who complete their pre-service programs and pass certification exams will receive initial licensure for two years. At that point they will have to submit a portfolio of evidence that they have successfully taught a variety of students and have a personal plan for continued professional development. Teachers’ “evidence competence” comes from standards developed by the Interstate New Teacher Assessment and Support Consortium (INTASC). INTASC’s standards, the author explains, are based upon the standards of other organizations, including the National Science Education Standards. The portfolio should include a series of instructional plans, and the identification of a variety of strategies to ensure that every student in the class becomes engaged in learning. The sequence of instruction is to cover materials described by local and national standards. The author’s biggest concern with this system is the quality of the mentors that will support teachers through this process. The author also argues that while the performance assessment is being constructed to evaluate the teaching performance of individual teachers, it could also be used to evaluate institutions that prepare teachers.

Anderson, R.D. and Helms, J.V. (2001). The Ideal of Standards and the Reality of Schools: Needed Research. Journal of Research in Science Teaching.38(1), 3-16.

Anderson and Helms note that a variety of research perspectives can inform our understanding of science education reform, and argue for research that gives simultaneous attention to all of the relevant elements of the system as well as the interactions among them. The authors summarize what existing research tells us about the challenges involved in putting the National Science Education Standards into widespread practice, and suggest some areas where additional research “has the greatest potential for furthering the reform of science education.” Most of the research cited in this article is socio-cultural in perspective and qualitative in nature; the authors do not describe the process they used in selecting these particular studies for review. Conclusions drawn from existing research include: (1) the changes called for in the NSES require significant changes in teachers’ values and beliefs about science education, and in any event are difficult to put into full practice; (2) teachers face multiple dilemmas in the process, such as the extent to which to focus on standards-based content and pedagogy versus traditional instruction that is presumed necessary to prepare students for the next level of schooling; (3) substantial teacher collaboration in the work context can be a powerful influence on teachers and teaching; and (4) parental support for reform ideas and practices is essential. The authors suggest a need for further research that is approached from multiple perspectives and conducted in the “real world,” focusing on conventional school practices and without the assumption that change can be driven solely from the top down. One area recommended for research is identifying the most productive roles for students, the desired nature of student work, and how to engage students in that work “in ordinary classroom contexts.” Other areas highlighted for further research include how teachers can best be engaged over time in taking responsibility for their own professional growth, and how to involve parents most effectively in the science education reform process.

Armstrong, J., Davis, A., Odden, A., and Gallagher, J. (1988). The Impact of State Policies on Improving Science Curriculum.Denver, CO: Education Commission of the States.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Atkin, J.M. and Black, P. (1997). Policy Perils of International Comparisons: The TIMSS Case. Phi Delta Kappan. 79(1), 22-28.

Austin, J.D., Hirstein, J., and Walen, S. (1997). Integrated Mathematics Interfaced with Science. School Science & Mathematics.97(1), 45-49.

Banilower, E. (2000). Local Systemic Change through Teacher Enhancement.Chapel Hill, NC: Horizon Research.

Banilower reported on the data available from the evaluations of the Local Systemic Change (LSC) projects. The LSC projects were surveyed to ascertain whether they had undertaken any studies examining the impact of the LSC on student achievement.

As examining student data was not a requirement of the evaluation, few projects had examined their impact on student achievement. Although 47 of the 68 projects responded, 38 projects indicated that they had no student achievement data available. Thus, data were available only from nine of 68 projects. Eight of the nine projects showed a positive relationship between teacher participation in the LSC and student achievement in mathematics and science, though only half of these constructed a convincing case that the impact could be attributed to the LSC. However, results need to be interpreted with caution, since in more cases, it is difficult to make the case that the impact is due primarily to the LSC and not to other, unmeasured interventions or policies. Many of the studies do not present enough information to build a convincing case that the LSC was responsible for improved student achievement. Given the small number of compelling studies, the data are insufficient to support claims about the impacts of the LSCs in general. It is also important to note that many of these studies reported only group means and did not statistically test group differences. Finally, Banilower points out that the remaining studies were flawed by (1) a lack of control groups (i.e., the study reported gain scores for schools in the LSC, but not for schools outside of the LSC); (2) failure to account for initial differences between control and experimental groups (i.e., while the study may have reported that LSC students scored higher than non-LSC students, it was unclear as to whether the two groups at the same achievement level); or (3) sample selection bias in the choice of participating schools or students (i.e., the study did not address how teachers were selected for participation in LSC training and whether this may have affected the study’s results).

Banilower, E.R., Smith, P.S., and Weiss, I.R. (2002). Examining the Influence of National Standards: Data from the 2000 National Survey of Mathematics and Science Education.Chapel Hill, NC: Horizon Research.

Barnhardt, R., Kawagley, A.O., and Hill, F. (2000). Cultural Standards and Test Scores, Sharing Our Pathways. Fairbanks: University of Alaska.

Barnhardt, Kawagley, and Hill report that eighth-grade students in schools participating in the Alaska Rural Systemic Initiatives (AKRSI) scored significantly higher than students in nonparticipating schools on the CAT-5 mathematics achievement test. With regard to student achievement, there was a differential gain of 5.9 percent in the number of students who are performing in the top quartile for AKRSI partner schools over non-AKRI rural schools. The AKRSI districts have 24.3 percent of their students testing in the upper quartile, and they are only 0.7 percent below the national average. Based on these results, the authors conclude that using Cultural Standards designed by the AKRSI has positive impacts on standardized test scores. For several years, the AKRSI had been working intensively with 20 of 48 rural school districts in the state to implement the Cultural Standards that are intended to systematically document the indigenous knowledge systems of Alaska Native people and develop educational policies and practices that effectively integrate indigenous and Western knowledge through a renewed educational system. Two outcomes of this work are worthy of consideration. First, building an education system with a strong foundation in the local culture appears to produce positive effects in all indicators of school success, including dropout rates, college attendance, parent involvement, grade-point averages, and standardized achievement test scores. Second, the Cultural Standards were compiled by educators from throughout the state as an outgrowth of the work that was initiated though the AKRSI and implemented in varying degrees by the participating schools. The authors also argue that when a persistent effort is made to

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

forge a strong “cultural fit” between what we teach, how we teach, and the context in which we teach, we can produce successful, well-rounded graduates who are also capable of producing satisfactory test scores.

Baron, J.B. (1991). Strategies for the Development of Effective Performance Exercises. Applied Measurement in Education.4(4), 305-318.

Bay, J.M., Reys, B.J., and Reys, R.E. (1999). The Top 10 Elements That Must Be in Place to Implement Standards-Based Mathematics Curricula. Phi Delta Kappan.80(7), 503-506.

Berggoetz, B. (2001, November). Indiana Chosen to Be in School Standards Study. Indianapolis Star.November 27, 2001.

Berkheimer, G.D., Anderson, C.W., and Blakeslee, T.D. (1988). Matter and molecules teacher’s guide: Activity book. Occasional paper number 122. East Lansing, MI: Michigan State University, Institute for Research on Teaching.

Berkheimer, G.D., Anderson, C.W., Lee, O., and Blakeslee, T.D. (1988). Matter and molecules teacher’s guide: Science book. Occasional paper number 121. East Lansing, MI: Michigan State University, Institute for Research on Teaching.

Berns, B.B. and Swanson, J. (2000). Middle School Science: Working in a Confused Context, April 28, 2000. Paper presented at the American Educational Research Association Annual Meeting, New Orleans, LA.

Biddle, B.J. (1997). Foolishness, Dangerous Nonsense, and Real Correlates of State Differences in Achievement. Phi Delta Kappan.79(1), 8-13.

Biddle questions the fundamental premise that standards have an influence on student achievement. He argues that improving achievement is about making resources available to children and to their teachers, not about setting standards. Biddle backs up his argument with analyses of three data sets from the Second International Mathematics Study (SIMS), the Third International Mathematics and Science Study (TIMSS), and the National Assessment of Educational Progress (NAEP). This report presents evidence that (1) the United States has greater disparities in school funding and higher levels of child poverty than other developed countries participating in the study and (2) these differences are strongly correlated with the differences in achievement among school districts and among states. Factors such as school funding and child poverty do affect student learning, and they will continue to do so whether we have national standards or not. For example, Biddle explored predictors of eighth-grade achievement scores for public schools. Results revealed statistically significant, net effects for both school funding (β = +.296, p<.01) and child poverty (β = –.358, p<.01). These effects persisted even when controls were entered for such potent variables as race and level of curriculum to which students had been exposed. Moreover, district-level differences in school funding and child poverty explained more than 25 percent of the variance of differences in mathematics achievement. Biddle also discovers that state differences in school funding are correlated with mathematics achievement at r = +.433 (p<.01), whereas the child poverty/ achievement correlation is a mammoth r = –.700 (p<.001). When funding and poverty are considered as joint predictors of achievement in a regression analysis, the net effects of both factors remain statistically significant, with β = +.262 (p<.03) for school funding and β = –.629 (p<.001) for child poverty, and that these two factors predict an astounding 55 percent of the variance of state differences in average achievement. In other words, not only do differences in school funding and child poverty matter at the state level, they are major predictor of state-level averages in mathematics achievement. Indeed, the impact of child poverty seems to be stronger at the state level than at the district level.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Biological Sciences Curriculum Study. (1993). Developing Biological Literacy.Colorado Springs, CO: Author.

BSCS, with support from the National Science Foundation, developed a curriculum framework for high school biology. For this project, BSCS commissioned papers, reviewed the literature, and held a conference to develop its recommendations. The three major recommendations were: (1) the content of biology must be unified by the theory of evolution, (2) biology classes must provide opportunities for students to experience science as a process and to understand science as a way of knowing, and (3) programs should help students develop biological literacy. The report identifies four levels of biological literacy: nominal, functional, structural, and multidimensional. According to the report, “education in biology should sustain students’ interest in the natural world, help students explore new areas of interest, improve their explanations of biological concepts, help them develop an understanding and use of inquiry and technology, and contribute to their making informed personal and social decisions.” The report recommends that assessment instruments be closely linked with instructional strategies. The report recommends the 5-E instructional model for biology programs and that the curriculum should be organized around major conceptual themes of biology, such as evolution. The major themes are: evolution, interaction and interdependence, genetic continuity and reproduction, growth, development, and differentiation, energy, matter, and organization, and maintenance of dynamic equilibrium.

Biological Sciences Curriculum Study and International Business Machines. (1989). New Designs for Elementary School Science and Health.Colorado Springs, CO: Biological Sciences Curriculum Study.

This was a design study for elementary school science and health, supported by the National Science Foundation and IBM. The project had three major goals: (1) to design a framework for an elementary school science and health program consistent with current trends and needs as identified by the education and science communities, (2) to determine the appropriate uses of microcomputer technology in elementary science and health programs, and (3) to produce a plan for implementing educational computing consistent with an exemplary science and health program for elementary schools. The report presents a rationale for a new approach to elementary school science and health; a curriculum framework with scope and sequence for a proposed elementary school science and health program; an instructional model (5-E) for elementary school science and health; recommendations for the integration of technology and elementary-school science and health; a description of a technology-oriented learning environment; a description of educational courseware for a technology-oriented elementary school science and health program; and recommendations for implementation of a technology-oriented curriculum.

Birman, B.F., Reeve, A.L., and Sattler, C.L. (1998). The Eisenhower Professional Development Program: Emerging Themes from Six Districts.Washington, DC: U.S. Department of Education; The American Institute for Research.

This study reports on an evaluation of the Eisenhower professional development program in six districts. The evaluation report, the first in a series of reports on different aspects of the Eisenhower program, focuses on six exploratory district case studies conducted in the spring of 1997. The six sites were chosen for geographic and programmatic diversity. Data for the case studies included document review, site visits, administrative interviews in each site, focus groups with teachers and professional development providers in each site, and follow-up phone interviews with Eisenhower coordinators in the states of each site. The analysis methodologies are not reported. The authors viewed these exploratory case studies primarily as a way to familiarize themselves with some of the sites and to identify themes for more in-depth exploration. The findings of the report are organized around 10 emerging themes. The themes, or findings, are quite broad. For example, the authors report that the program supported a wide variety of activities; that most efforts went toward mathematics and science professional development; that most of the professional development that the funding supported was consistent with standards for high-quality professional development; and that the reliability of the Eisenhower funding allowed districts to engage in long-term planning and to leverage other funds. Overall, the authors conclude that the Eisenhower-funded activities emphasized several elements of high-quality professional development, including sustained and intensive professional development, the use of teachers as leaders, and promoting alignment with high standards. They found that the Eisenhower coordinators were able to identify some components of high-quality professional development.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Bischoff, P.J., Watford, L.J., and Hatch, D.D. (1999). The State of Readiness of Initial Level Preservice Middle Grades Science and Mathematics Teachers and Its Implications on Teacher Education Programs. School Science & Mathematics.99(7), 394-399.

Bishop, J. (1998). Do Curriculum-Based External Exit Exam Systems Enhance Student Achievement?Philadelphia, PA: Consortium for Policy Research in Education.

This investigation used four existing data sets to test the hypothesis that curriculum-based external exit examination systems (CBEEES) improve achievement. The four data sets included science and mathematics achievement of seventh and eighth graders in the 40-nation Third International Mathematics and Science Study (TIMSS); science and mathematics scores of 13-year-olds on the International Assessment of Educational Progress (IAEP) for 16 nations and nine Canadian provinces; and SAT and NAEP mathematics scores for New York State versus the rest of the United States. Of the 40 countries that participated in TIMSS, 22 national school systems were classified as having CBEEES. Regression analyses produced results that show a substantial relationship between countries with CBEEES and achievement in science and mathematics. Bishop studied assessment results for New York State because of its use of the Regents Examinations in the early 1990s, which, for the purpose of this study, the author identified as a CBEEES. New York students were found to do significantly better on the SAT than students of the same race and social backgrounds in other states. NAEP mathematics scores for New York supported these findings. Data used in this study were all collected prior to the release of the National Science Education Standards and cannot be used to support the impact of these standards on student achievement. The general findings do produce evidence of the relationship between high accountability systems and achievement by comparing nations and states. However, this study only considers relational data and does not provide any evidence of how improved content standards may have an impact on student learning. The improved learning could be for other reasons, such as increased study time or reduced class size, rather than being curriculum-associated. If the external exit examinations are standards-based, then the findings from this study suggest that student learning would be improved in the directions advocated by the standards.

Black, P. and Wiliam, D. (1998). Inside the Black Box: Raising Standards Through Classroom Assessment. Phi Delta Kappan.80(2), 139-144.

This article reports the results of a meta-analysis of over 40 studies showing increased formative assessment produces substantial learning gains. A review of the results from 23 studies on classroom assessment of children with mild handicaps was published in 1986. Black and Wiliam reviewed more than 20 additional studies that showed innovations, including strengthening the practice of formative assessment, that produced significant and often substantial learning gains. In addition to the importance of formative assessment to learning in general, the researchers found that formative assessment helped low achievers more than other students. They suggested that this would lead to reducing the range in achievement, while raising achievement overall. The researchers then went on to cite literature that identified the shortcomings in the everyday practice of classroom assessment, including some articles that addressed assessment in science. After identifying deficiencies in formative assessment practices, the researchers offer ways that formative assessment practices can be improved. Some of these included giving students feedback on the quality of their work and avoiding comparisons with other students, students having a clear understanding of learning targets, and the value of self-assessment. The meta-analysis foundation for this article was very thoroughly done and located findings that supported the value of formative assessment, including some experimental studies. The researchers then expanded this finding to describe how formative assessment and teaching can be improved, building some on the literature, but mainly depending on experience and logic.

Blank, R.K. (2000). Summary of Findings from SSI and Recommendations for NSF’s Role with States: How NSF Can Encourage State Leadership in Improvement of Science and Mathematics Education.Washington, DC: Council of Chief State School Officers.

This paper is designed to inform policy makers and the National Science Foundation about the lessons of systemic reform in science and mathematics. It is a review of studies and evaluations of NSF’s Statewide Sys-

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

temic Initiatives (SSI). The review clearly states its data sources, which include a review of existing studies, the results of a conference of findings of the SSI programs, and discussions with state leaders. A planning committee developed a framework for analysis and reporting the findings in six areas: support for systemic reform, leadership, resources/partnerships, policy/infrastructure, strategic decisions/interventions, sustainability, and outcomes/evaluations. The paper contains three major sections. In the first, it highlights the findings in the six areas. The second section contains recommendations on each of these findings from state leaders on how to more effectively implement standards-based mathematics and science education statewide. The final section discusses the implications for new NSF programs. In terms of the influence of the standards on the systems of professional development, several findings are pertinent. First, successful SSIs developed and effectively promulgated a vision for reform in their state based on the standards. Second, effective SSIs included leadership for local leaders in their training. Third, successful states aligned policies that supported changes in the state infrastructures related to teacher quality such as licensure and teacher preparation. Fourth, effective states focused their professional development on standards-based curriculum and materials, content knowledge, and active learning.

Blank, R.K., Bush, M.H., Pechman, E.M., Goldstein, D., and Sardina, S.L. (1997). A State-by-State Look at Content Standards and Benchmarks: Examples of Mathematics and Science Standards.Washington, DC: Council of Chief State School Officers.

Blank, R.K., Kim, J.J., and Smithson, J. (2000). Survey Results of Urban School Classroom Practices in Mathematics and Science: 1999 Report. Using the Survey of Enacted Curriculum Conducted During Four USI Site Visits. How Reform Works: An Evaluative Study of the National Science Foundation’s Urban System Initiatives. Study Monograph No. 2.Washington, DC: Council of Chief State School Officers.

This report investigated the impact of the Urban Systemic Initiative (USI) program on four urban school districts. The project collected data using the Survey of Enacted Curriculum, focusing on enacted curriculum contents and teaching practices. For the study, data were collected from 80 teachers from 20 elementary and middle schools for each site. The survey addressed the six drivers of educational system reform identified by the National Science Foundation: (1) implementation of comprehensive, standards-based curricula, (2) development of a coherent, consistent set of polices, (3) convergence of the usage of all resources that are designed for or that reasonably could be used to support science and mathematics education, (4) broad-based support from parents, policy makers, institutions of higher education, business and industry, foundations, and other segments of the community, (5) accumulation of a broad and deep array of evidence that the program is enhancing student achievement, and (6) improvement in the achievement of all students, including those historically underserved.

The results of the study relevant to the science curriculum are as follows:

  • Hands-on or laboratory materials was the largest activity (25 percent of the time).

  • Teachers reported students were engaged more often in “use science experiment,” “follow step-by-step directions,” and “make tables, graphs or charts” and less often in “changing something in an experiment to see what happens” or “designing an experiment.” However, in schools involved in the USI program, elementary students were less likely to “follow step-by-step instructions” and more likely to “change something in an experiment to see what will happen.” Students in USI middle schools spent more time “using science equipment and tools in experiments or investigations and in “collecting data” and “designing ways to solve a problem,” but spent less time to “make predictions, guesses, or hypotheses” or to “draw conclusions from science data.”

  • When working in small groups, the highest use of class time was to “write results or conclusions of a laboratory activity” (about 22 percent of the time).

  • High-implementation USI schools spent less time on “review assignments and problems.”

  • Teachers in USI implementation schools spent more time on life science and chemistry, and less on physical science.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
  • Classes in comparison schools emphasized “memorize” and “analyze information” more than USI implementation schools. At the elementary level, USI implementation schools taught “nature of science” 25 percent of time and “life science” an average of 32 percent of the time vs. comparison teachers’ average times of 10 percent and just over 20 percent, respectively.

Blank, R.K. and Langesen, D. (1999). State-by-State Trends and New Indicators from the 1997-1998 School Year. Washington, DC: Council of Chief State School Officers.

Blank, R.K. and Langesen, D. (2001). State Indicators of Science and Mathematics Education 2001: State-by-State Trends and New Indicators from the 1999–2000 School Year.Washington, DC: Council of Chief State School Officers.

Blank and Langesen report data on progress of student achievement on a national scale to look for the general influence of standards and on achievement of different ethnic groups from the National Assessment of Educational Progress (NAEP). For example, in mathematics, the number of eighth-grade students achieving proficiency on the exam increased from 15 percent in 1990, before the NCTM standards could have had a substantial impact, to 26 percent in 2000. Similar gains were recorded at the fourth-grade levels as well, where 25 percent of fourth-grade students scored at/above the Proficient level, an 8 percent improvement from 1992 to 2000. In science, the achievement showed much more modest gains during the shorter period since the introduction of the National Science Education Standards—nationally, 30 percent of grade 8 students scored at/above the Proficient level, or a 3 percent improvement in eighth-grade proficiency levels between 1996 and 2000. The authors note that only nine states made significant improvement in the percentage of grade 8 students reaching the Proficient level on the NAEP science assessment. Thirteen states had more than 35 percent of students score at/above the Proficient level in 2000. Blank and Langesen also report data on achievement of different ethnic groups from the NAEP. All states have a significant disparity in achievement levels between the percentage of European American students at or above the Basic level and the percentage for the largest minority group in the eighth-grade mathematics and science test in 1996. The changes in disparity in achievement levels remain disturbingly high from 1992 to 2000. For example, in 2000, 77 percent of European American students scored at the basic level or above as compared to 32 percent of African American students, and 40 percent of Hispanic students in the eighth-grade mathematics test. The difference between white and Hispanic students scoring at/ above the Basic level was reduced by 11 percentage points over the eight-year period since 1990. The white– African American disparity was reduced by 2 percent.

Blank, R., Manise, J., and Brathwaite, B.C. (1999). State Education Indicators with a Focus on Title I 1999. Washington DC: Council of Chief State School Officers.

The study reports state-by-state indicators of education organized into four categories: school and teacher demographics, student demographics, statewide accountability information, and student achievement. The goal of the report is to chart the progress of states in developing Title I accountability systems. The overall summary results of the study of relevance for science education include:

  • Forty-seven states have completed and implemented content standards for science.

  • While 25 states have developed performance standards in language arts/reading and mathematics, no such data are available for science, which was not part of the Title I mandate.

  • Thirty-three states reported state assessment results using three or more proficiency levels that were defined by the state.

  • Thirty-five states reported that assessment results could be disaggregated by characteristics of schools and students.

  • Nineteen states reported two years of assessment results using consistent assessments and 11 states reported three years of results that could be analyzed as trends.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Blank, R.K. and Pechman, E.M. (1995). State Curriculum Frameworks in Mathematics and Science: How Are They Changing Across the States?Washington, DC: Council of Chief State School Officers.

Blank, R.K., Porter, A., and Smithson, J. (2001). New Tools for Analyzing Teaching, Curriculum and Standards in Mathematics & Science: Results from Survey of Enacted Curriculum Project.Washington, DC: Council of Chief State School Officers.

For this project, the researchers developed and administered surveys of enacted curriculum in mathematics and science. The study used self-reporting from schools and teachers (more than 600) in 11 states to collect the data. The data were collected for a two-dimensional matrix—content topic by expectations for learning. The authors emphasize that “K-12 education presents an exceptionally complex system with numerous steps in the causal change between goals and initiatives for reform and student achievement. One way to simplify the causal change is to divide the system into three components: the intended curriculum, the enacted curriculum, and the learned curriculum (i.e., student outcomes)…. In this project, we have been able to show that the Survey of Enacted Curriculum (SEC) and related data analysis provide the necessary sets of data to trace a causal chain for K-12 education from policy initiatives to achievement” (p. 3). The SEC addresses concepts such as: active learning in science, mathematics and science content, multiple assessment strategies, use of educational technology, and alignment of content taught with state assessments.

The study tested the theory that the more curriculum policies reflect four characteristics— prescriptiveness, consistency, authority, and power—the stronger the influence that policies will have on instructional practice. In addition, the study analyzed gains in student achievement to examine the contributions of classroom experience to student achievement over specified periods of time. Student achievement was controlled for prior achievement and socioeconomic status.

Results from the study related to science curriculum issues were as follows:

  • Science teachers reported that some polices have a positive influence on instruction, including the following listed from most to least influence: district curriculum framework, state curriculum framework, preparation of students for the next grade or level, and state tests. The textbook, district test, and national standards were viewed as less influential.

  • Seventy-five percent of science teachers reported attending professional development activities related to implementing state or national standards, while only 25 percent reported attending an extended institute (40 contact hours or more).

  • Professional development in science education is supporting the goals of standards-based initiatives.

  • There is some significant variation in science instruction among the 11 states. For example, Massachusetts had higher means for teacher readiness for equity, student reflection on science, and multiple uses of assessment; Louisiana and West Virginia reported more use of educational technology; Minnesota stood out in communicating scientific understanding; and Kentucky was higher in professional collegiality.

  • State science instruction aligns more closely with the state science assessment than with tests in other states, suggesting that standards-based reform is bringing instruction into alignment with state tests.

  • Teachers indicate that they would benefit from more opportunities to work with other teachers. Teachers reported that much of the time in professional development did not focus on the curriculum or subject they are expected to teach.

Bond, L., Roeber, E., and Braskamp, D. (1997). Trends in State Student Assessment Programs Fall 1996.Washington, DC: Council of Chief State School Officers.

This document describes the trends in statewide assessment programs as reported in fall of 1996. The Council of Chief State School Officers (CCSSO) mailed a survey to state assessment directors for them to describe the assessment program they operated during the 1995-96 school year. Data are reported for this year and for the four prior years. The report includes information on assessments by grade, content areas, and type of assessments. One chapter is a report on non-traditional assessment. Another chapter describes assessment of

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

students with disabilities and limited English. The report concludes with a discussion of the statewide assessment history and trends. In 1995-96, 30 states reported assessing students’ knowledge of science. At least eight states used non-traditional items, including those requiring students to produce short answers or extended response. Most states reported using a blend of assessment approaches. At least four states that were actively pursuing the use of alternative forms of assessment discontinued them for a number of reasons and turned toward more traditional approaches that were more cost-effective and technically sound. The longevity of implementing performance assessment was related to low visibility and how the results were used. People were more accepting of using performance assessments as end-of-year examinations rather than higher-stakes assessments. CCSSO’s report of its annual assessment survey is the main source of information on the state assessment programs. Some information is reported by content areas, including science, but the major focus of the report is on types of assessments, assessment policies, and the use of assessment results. This was an interpretative report that gave major attention to the use of alternative forms of assessment.

Boone, W.J. and Kahle, J.B. (1997). Implementation of the Standards: Lessons from a Systemic Initiative. School Science & Mathematics.97(6), 292-300.

This study presents attitudinal data gathered via questionnaire from 90+ principals and 450 science teachers at 126 randomly selected middle schools in Ohio. Teachers sampled were evenly distributed across grades 6 through 9. The demographic percentages of schools were reflective of Ohio in terms of urban, suburban, rural, etc. Design, collection and analysis of data was rigorous. A response rate of 86 percent for principals and 82 percent for teachers was obtained via follow-up phone calls and on-site visits. Data in this report represent two of seven subscales within the questionnaire, namely, “What students do” and “Principals’ support.” Items for principals and teachers were essentially identical; principals ranked items in terms of importance whereas teachers ranked items in terms of frequency. A stochastic Rasch model was used for analysis to convert ordinal scales to interval data. This model allows for measurement errors to be calculated for all respondents and items. Inferences drawn from this implementation/process evaluation were as follows: Teachers made frequent use of NSES-based practices not highly ranked by principals; both groups infrequently used or supported activities that would promote the understanding of the nature of science; and support for implementation of the NSES varied. Thus, it was recommended that professional development assistance is needed for both teachers and principals in terms of understanding (1) the nature of science, (2) how children learn scientific thinking, and (3) a process of inquiry that emphasizes the duplication of experiments as well as time to discuss/debate results. Finally, the authors recommend that any NSES implementation should incorporate assessment of progress and problems.

Brearton, M.A. and Shuttleworth, S. (1999). Racing a Comet. Journal of Staff Development.20(1), 30-33.

Breckenridge, J.S. and Goldstein, D. (1998). A Case Study of Louisiana’s SSI (LaSIP), 1991-1996. In A.A. Zucker and P.M. Shields (Eds.), SSI Case Studies, Cohort 1: Connecticut, Delaware, Louisiana, and Montana. Menlo Park, CA: SRI International.

This case study looks at Louisiana’s Statewide Systemic Initiative (LaSIP) aimed at reforming science and mathematics education within the state during the funded years of 1991-96. LaSIP’s primary strategy for reform was to provide professional development in the form of intense summer institutes with school year follow-up for classroom teachers of mathematics and science, concentrating on those who teach in grades 4-8. This professional development, which reached more than 4,100 teachers, aimed to prepare teachers to practice high-quality mathematics and science instruction as described by NCTM and AAAS standards documents. The case study also analyzes the progress of the other LaSIP components of teacher preparation; teacher certification; curricula and assessment; evaluation; education technology; information and dissemination; equity and diversity; and community partnerships.

External evaluators conducted interviews, site visits, classroom observations, focus groups, and analyzed state education policies and test scores for this case study. The following impacts of LaSIP on Louisiana’s K-16 education system have been cited. Participation in the more than 125 mathematics or science professional development institutes resulted in teachers having more positive attitudes and increased involvement in profes-

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

sional organizations. However, the degree to which LaSIP-trained teachers were able to integrate the principles of reform into their classroom practice varied widely, with many teachers understanding the changes conceptually but appearing uncomfortable or unable to apply them in their classroom. The degree of support from fellow teachers and administrators varied greatly as well.

The study reports that LaSIP had a positive impact on student achievement, as students in LaSIP teachers’ classrooms scored slightly higher on the statewide mathematics test than did non-LaSIP students. (Since science is not tested in this state, evidence of student science achievement was not available.) LaSIP also made strides forward in reform by creating standards-like mathematics and science curriculum frameworks and by revising teacher certification requirements so that teachers of grades 1-8 will need a minimum of 15 semester hours in science and 12 semester hours in mathematics.

Bredekamp, S. and Rosegrant, T. (Eds). (1995). Reaching Potentials: Transforming Early Childhood Curriculum and Assessment.Volume 2.Washington, DC: National Association for the Education of Young Children.

Briars, D.J. and Resnick, L.B. (2000). Standards, Assessments—and What Else? The Essential Elements of Standards-Based School Improvement. CSE Technical Report 528. Los Angeles, CA: University of California, National Center for Research on Evaluation, Standards, and Student Testing, Center for the Study of Evaluation.

Briars and Resnick report on standards-based reform efforts in the Pittsburgh Public Schools (PPS). The authors argue that adoption of a standards-based mathematics educational program supported by a systematic program substantially increases fourth-grade students’ achievement in mathematics skills, conceptual understanding, and problem-solving. These increases occurred during the year that the cohort of students who had been using Everyday Mathematics reached the fourth grade, and they occurred primarily in strong implementation schools. As a whole, the district showed respectable gains in achievement on a performance assessment aligned to the official program, and even some improvement on a norm-referenced mathematics test not specifically aligned to the curriculum. These measured gains appeared when all of the three elements had been in place for at least two years prior to testing: The adoption of an NSF-supported elementary mathematics curriculum (Everyday Mathematics), professional development supported by a Local Systemic Change grant, and an assessment system using tests developed by the New Standards program. Using an aligned system of standards, assessments, curriculum, and professional development, the PPS showed that it is possible to produce very large gains in elementary school students’ mathematics learning. The claim that systemic rather than piecemeal innovation is needed is, thus, well supported by elementary mathematics experience. The authors also suggest the components of a standards-based system in PPS: (1) content and performance standards, (2) standards-based assessments, (3) standards-based instructional materials, (4) standards-based professional development for teachers and administrators, and (5) accountability.

Buccino, A. (1994). State Infrastructure Support for Science Education Reform. In Scientists, Educators, and National Standards: Action at the Local Level, Sigma Xi Forum Proceedings, Sigma XI, The Scientific Research Society, Research Triangle Park, NC, April 14-15, 1994.

Bybee, R.W. (2001). Guest Editorial: Unintentional Consequences of an Unacceptable Evaluation. American Biology Teacher.63(1), 2-3.

In this editorial for the American Biology Teacher, Bybee, executive director of the Biological Sciences Curriculum Study, discussed what constitutes a quality review of instructional materials. Bybee expressed concern that curriculum evaluations, no matter how positive the intentions, can result in significant unintended negative consequences. He challenged the findings of the Project 2061 review of high-school biology programs. Bybee stated that the AAAS “was an unacceptable evaluation…. I simply must question a judgment that all biology textbooks are woefully inadequate, represent the central barrier to student learning, and are ultimately unacceptable. Yet, this is the judgment of Project 2061” (p. 2). According to Bybee, the result of this evaluation puts an enormous burden on teachers. Biology teachers can either ignore the evaluation and adopt what Project 2061 views as an unacceptable textbook or form a district committee to develop its own life science program. The

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

result of the second choice likely would be a biology curriculum that lacks scientific accuracy, educational consistency, and pedagogical quality. Bybee (p. 2) illustrates his point by indicating that “I recently heard of a school district where a superintendent decided to adopt a creationist book because the major texts were unacceptable. This is clearly an unacceptable consequence of the Project 2061 evaluation.”

Bybee, R.W. (2002). Guest Editorial: The Benefits of a Review That Is Neither Categorically Negative nor Uncritically Positive. American Biology Teacher.64(1), 7-8.

In this article, Bybee commented on the AIBS review of high school biology programs. Bybee pointed out that biology teachers need evaluations that are neither uncritically positive (such as the OERI report) nor categorically negative (such as the Project 2061 evaluation). According to Bybee, the AIBS review meets his criterion. He praised the approach of the AIBS study. “The consumer report approach of numerical ratings, graphical comparisons, and general discussions of all textbooks gives adoption committees the opportunity to review potential programs with an eye toward local criteria and constraints” (p. 7). Bybee emphasized that an approach that highlights both the strengths and weaknesses of a program encourages variations in programs. As Bybee pointed out, “the evolution of better textbooks, the programs biology teachers deserve, is the consequence of the variation among those textbooks” (p. 8).

Bybee, R.W. and McInerney, J.D. (1995). Redesigning the Science Curriculum: A Report on the Implications of Standards and Benchmarks for Science Education.Colorado Springs, CO: Biological Sciences Curriculum Study.

This report is the result of a project conducted by the Biological Sciences Curriculum Study (BSCS) on the implications of the National Science Education Standards for the science curriculum. The project had the following goals: (1) review science curriculum development 1958 to 1993, (2) review the National Science Education Standards from a curriculum development perspective, (3) propose designs for science curriculum in the context of standards-based reform, (4) consider the contributions and conflicts of different curriculum frameworks, benchmarks, and standards in the reform of science education, (5) address basic questions of curriculum reform from local, regional, and national perspectives, and (6) outline recommendations for public and private funding agencies involved with transforming the NSES into science programs and practices. The project involved three phases: preparing commissioned papers on curriculum reform; holding a conference to review the papers and presentations on the NSES, Project 2061, and the Scope, Sequence, and Coordination project; and publishing and disseminating the recommendations from the conference. The report ended with a listing of concerns and recommendations from a range of constituent groups. Elementary school teachers indicated that the NSES were a positive force to improve effectiveness of elementary school science programs but were concerned that elementary-school teachers will not see the NSES as their issue and that the emphasis given to science in the student’s day does not lend itself to promoting the goals of the NSES. Middle-school teachers were encouraged that the NSES specifically identified standards and benchmarks at the middle grades, but were concerned that the NSES should reflect the special needs of early adolescents, that the NSES represent the floor rather than the ceiling of expectations, and that the NSES might not be useable by middle-level teachers. High-school teachers indicated that NSES are just a fad, require considerable energy, and will not result in much change. Science supervisors were concerned about the lack of coordination among national, state, and local projects to develop standards and that there are no resources to support staff development aligned with implementation of the NSES. Curriculum developers indicated that the NSES have the potential to stimulate the reform of science education and that they see curriculum developers as having a central role in the reform of science education, but they were concerned that the NSES might be too prescriptive and that the NSES’ models and strategies for broad implementation and teacher development must be developed. College and university faculty were concerned that college and university personnel have little knowledge of the NSES, will be late in recognizing the implications of the NSES, and will focus on critiquing rather than implementing the national standards.

Carnegie Corporation of New York. (1995). Your Body, Your Life: Human Biology for the Middle Grades. Carnegie Quarterly.Summer/Fall 1995.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Center for Applied Linguistics. (1993). The Issues of Language and Culture. Proceedings of a Symposium Convened by the Center for Applied Linguistics.Washington, DC: Author.

Champagne, A.B. and Kouba, V.L. (2000). Writing to Inquire: Written Products as Performance Measures. In J.J. Mintzes, J.H. Wandersee et al. (Eds.), Assessing Science Understanding: A Human Constructivist View, pp. 223-248. San Diego, CA: Educational Psychology Press.

Champagne and Kouba argue that writing is a more effective strategy for keeping students’ minds on science than having students engage in science activities. A major purpose of their chapter, based on their research, is to persuade science educators that writing as a performance measure can be effectively used to articulate the general guidelines expressed in reform documents (e. g., AAAS Benchmarks and the NRC NSES) and to inform the development of local norms for science literacy. They build their argument on social constructivism, making the point that science is humanistic and that inquiry is a social process. As a foundation for their argument, they define assessment as data-gathering with a purpose. Performance assessment is an alternative assessment that incorporates student writing analyzed for scientific accuracy and quality of reasoning. Reform documents in science education advocate inquiry by students and teaching through inquiry, but do not explicitly state what constitutes inquiry in the classroom. Champagne and Kouba believe the authors of these documents recognized that inquiry in the classroom can take many forms. Champagne and Kouba used social constructivist theory to describe an environment that affords students an opportunity to learn how to inquire. Such environments have social, intellectual, and physical features. A teacher facilitates the development of the social and intellectual characteristics. Discourse serves to develop the science literacy of students and provides evidence of students’ learning. Writing facilitates the process of learning to inquire by engaging in introspection and communication, both important to inquiry. To draw the full meaning of inquiry from the AAAS Benchmarks and the NRC NSES requires developing performance expectations—the ideal performance of students upon completion of the program, course, or lesson. Teams that prepare performance expectations need to consider the standards, student work, and information from experts. Student writing then serves a dual role: enhancing student learning, and assessment of the attainment of the performance expectations.

Christman, J.B. (2001). Children Achieving: Powerful Ideas, Modest Gains: Five Years of Systemic Reform in Philadelphia Middle Schools, The Evaluation of the Annenberg Challenge in Philadelphia. Philadelphia, PA: Consortium for Policy Research in Education. Available at: http://www.cpre.org/Publications/children05.pdf [September 3, 2002].

Over a five-year period, from 1996-2000, evaluators investigated the impact of the $50 million, five-year Annenberg grant to improve education in Philadelphia public schools. This report presents findings of middle schools in the district during this time, along with findings on other levels. Evaluators collected longitudinal data on the district’s Performance Responsibility Index (PRI); two census surveys of teachers; school indicators collected at two points in time; qualitative data from 11 middle schools; and interviews of a number of school, district, and civic leaders. Along with reading and mathematics, the percentage of students scoring at or above basic as measured by the SAT-9 are reported for 1996 and then again for 2000. The gain in percentage was positive for all three content areas at all three levels (elementary, middle, and high school). The highest gains were in elementary science. The report presents general findings that are not broken down by content area. Slightly more than half of the middle school teachers reported that the SAT-9 had had a positive effect on their schools. In 1999, grade 8 students were required to pass all major subjects, including science (along with other criteria), to be promoted. To improve student test performance, schools reorganized staff and schedules, purchased new test-preparation materials, and increased instructional time on test-taking skills. Evaluators found that the new accountability system and the assessment did drive classroom instruction. However, classroom-based assessments never became a priority, and very few teachers routinely reviewed student work against the standards. A general conclusion was that reform leaders need to craft strategies for improvement that are well suited to the different levels of schooling and to the varying capacities of teachers. This is a comprehensive report of a very complex task, evaluating change in a large urban district. Assessment data are reported for science, but the general conclusions and inferences are not associated with a particular content area. However,

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

there is no reason why the findings on assessment and accountability are not applicable to science as distinguished from the other content areas.

Clewell, B.C., Hannaway, J., Cosentino de Cohen, C., Merryman, A., Mitchell, A., and O’Brian, J. (1995). Systemic Reform in Mathematics and Science Education: An Urban Perspective.Washington, DC: The Urban Institute.

Close, D., Miller, J., Titterington, L., and Westwood, D. (1996, September). National Standards and Benchmarks in Science Education: A Primer.ERIC Clearinghouse for Science, Mathematics, and Environmental Education, Columbus, OH. ERIC Digest. September 1996.

Clune, W. (1998, December). Toward a Theory of Systemic Reform: The Case of Nine NSF Statewide Systemic Initiatives.Research Monograph No. 16. Madison, WI: National Institute for Science Education, Wisconsin Center for Education Research.

This is a report of a secondary analysis of case studies of nine Statewide Systemic Initiatives funded by the National Science Foundation. The goals of the study were to test the central thesis of systemic reform and to derive lessons about strengths and weaknesses of reform strategies used in policy and practice. The report describes student assessments, teacher networks, missing pieces in the reform system, and the forces that influence curriculum content. The central thesis of systemic reform is that greater coherence or alignment of instructional policies is necessary to attain higher levels of student achievement. Components of systemic reform include: curriculum frameworks, instructional materials and curricula, in-service professional development, pre-service professional development, student assessments and accountability, school site autonomy and restructuring, and supportive services from districts and the state. Standards-based curricula are seen as a key element of systemic reform. The study identified a theory of systemic reform that included four basic elements: systemic reform, through its purposeful activities, leads to systemic policy, which leads to a rigorous implemented curriculum for all students, which leads to measured high student achievement in the curriculum as taught. The study describe systemic curriculum as being made up of content and pedagogy, the material actually conveyed to students in classrooms, and the instructional methods by which it is taught. The curriculum was rated on breadth (the number of schools, teachers, grades, subjects that demonstrated change) and depth (the extent of the change in substantially upgrading content and pedagogy). The study found that systematic, observable data on the implemented curriculum, however, were rare. The study collected data on the four elements of systemic reform in case studies of nine states. The study found that higher achievement ratings were associated with higher ratings in reform, policy, and curriculum. Across all states, however, curriculum had the lowest rating of change when compared to reform and policy initiatives. One design problem identified among the systemic initiatives was a lack of emphasis on curriculum content and whole-school restructuring and the focus on pedagogy rather than content. The authors reported a constant source of frustration was the absence of assessments that are aligned, or fully aligned with the reform objectives.

Cohen, D.K. and Hill, H.G. (2000). Instructional Policy and Classroom Performance: The Mathematics Reform in California. Teachers College Record.102(2), 294-343.

Cohen and Hill examine the mathematics reform efforts in California, based on data from a 1994 survey of California elementary school teachers and 1994 student California Learning Assessment system (CLAS) scores. The data in this study were randomly selected within the 250 schools and one teacher from each of grades 2 through 5 was selected at random. They found evidence that teachers’ learning experience about the CLAS affected teachers’ practices under certain conditions, and that learning then translates into changed practice and ultimately improved student achievement. They also showed that both teachers’ practice and policy measures positively relate to student achievement. Schools in which teachers report classroom practice that is more oriented to the math frameworks have higher average student scores in the fourth grade, controlling for the demographic characteristics of schools. Cohen and Hill argue that teachers’ classroom practices and student achievement in mathematics were affected by the influence of assessment, curriculum, and professional development. The overall picture was complex, but in general, student achievement on the CLAS mathematics tests was

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

higher when (1) teachers used materials aligned with the California mathematics framework, (2) teachers participated in professional development programs aligned with the framework, (3) teachers were knowledgeable participants in the CLAS system, and (4) teachers reported that they engaged in teaching practices consistent with the framework. The authors also argue that policy can affect practice, and both can affect student performance. Finally, they propose a rudimentary instructional model, in which students’ achievement was the ultimate dependent measure of the effects of instructional policy, and in which teacher’ practice was both an intermediate dependent measure of policy enactment and a direct influence on students’ performance.

Colorado State Department of Education. (1999). The Teachers’ Guide to the Colorado Student Assessment Program for Eighth Grade Science: An Assessment of Fifth Through Eighth Grade Benchmarks.Denver: Author.

Consortium for Policy Research in Education. (1994, September). Reform of High School Mathematics and Science and Opportunity to Learn. CPRE Policy Briefs. New Brunswick, NJ: Author.

Consortium for Policy Research in Education. (1995, May). Reforming Science, Mathematics, and Technology Education: NSF’s State Systemic Initiatives. CPRE Policy Briefs. New Brunswick, NJ: Author.

This report describes 26 State Systemic Initiatives and summarizes the results of a national evaluation study of these projects. Systemic reform initiatives generally include: (1) efforts to develop professional and public support for higher standards, (2) adoption of ambitious common goals for student learning, (3) setting challenging academic standards for all students, (4) aligning state and local polices in support of goals and standards, (5) increased collaboration and resource-sharing, (6) expanded opportunities for teachers to enhance their knowledge of subject-matter content and to acquire, practice, and critique new approaches to curriculum, pedagogy, and assessment. The states’ visions of science education have been significantly influenced by the National Science Education Standards, which were concurrently under development. The researchers found that reform is under way in the states participating in the Systemic Initiative Program. However, they found that more work is needed to develop public understanding and support needed to sustain these initiatives.

Consortium for Policy Research in Education. (1995, July). Tracking Student Achievement in Science and Math: The Promise of State Assessment Programs. CPRE Policy Briefs. New Brunswick, NJ: Author.

The policy brief tracks the effects of NSF-funded Statewide Systemic Initiatives (SSIs) on student performance in science and math. In order to evaluate the success of the SSIs, Policy Studies Associates (an NSF cooperating organization) conducted a survey in the spring of 1994 to examine the capacity of states to adequately assess student performance in science and math. They surveyed state-level assessment staff in 25 states; states selected were those that received multiple years of SSI statewide funding. The policy brief does not contain a copy of the survey or details of the methodology used for participant selection, survey administration, or analysis. The survey data were used to predict the likelihood that science and math assessment would produce sufficient evidence of SSI influence. Major issues in developing state assessment systems for state policy makers were also highlighted. They found that more states assess students in mathematics than in science. State assessments systems that had their origins in the basic skills movement of the 1970s do not consider science to be a “basic” subject. State assessments results are limited in the information they convey, particularly if they are not aligned with standards. The study describes the various types of assessment used by states to assess science and math. The majority of tests given to students are still using traditional multiple-choice items; however, many states were in the process of developing performance-based assessment systems, or were revising existing systems. Both criterion-referenced and norm-referenced tests are used by states. States with systemic connections between their SSI goals, curriculum, and assessment were able to better demonstrate impact of the SSI initiatives than states that had no alignment. The low-alignment states lack the ability to measure either SSI intervention strategies or the types of higher-order thinking in math and science that the SSIs are trying to promote. Not all states test at all three K-12 levels of elementary, middle, and high school schools. Obtaining data for evaluation is difficult in states that do not publicly release test results. The variety of state objectives and testing programs across states limits the use of tests for comparison. Of the 25 states

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

surveyed, only four met the criteria in science. The study concluded that state-testing systems produced inadequate data for evaluating student performance in science and math.

Consortium for Policy Research in Education. (1997). A First-Year Evaluation Report of Children Achieving: Philadelphia’s Education Reform, Executive Summary, 1995-1996. Available at: http://www.cpre.org/Publications/Publications_Research.htm [September 3, 2002].

This is an interim report of the first-year evaluation of Children Achieving. It focuses on the first six of the projected 22 school clusters to be served by the project. The critical drivers of reform in this project are the standards and incentives to be embedded in the yet-to-be defined accountability system. The district’s plan will provide standards but no specific prescription for how they are to improve teaching and learning. By the end of the first year, the first six clusters were up and running, content standards were drafted, and critical pieces of the support infrastructure were operating. Overall the researchers found that (1) the project was on schedule and gaining momentum, (2) despite fiscal and political challenges, the reform moved forward, (3) the vision underlying the reform was understood and generally accepted among central office and cluster staff members, but less well understood in the schools, (4) key organizational components of reform were gaining acceptance, but understanding and support varied across schools, (5) supports for reform were inadequately coordinated and sometimes lacked focus, (6) standards and accountability topped educators’ priority issues, (7) educators questioned decentralization, (8) schools’ response to reform priorities was uneven, and (9) schools that made the most progress in implementing reforms shared a handful of key characteristics.

Consortium for Policy Research in Education. (1998). Children Achieving: Philadelphia’s Education Reform, a Second-Year Evaluation, Executive Summary.CPRE: Progress Report Series 1996-1997. Available at: http://www.cpre.org/Publications/execsumm.pdf [August 8, 2002].

This is a summary report of the second year of an evaluation of Philadelphia’s education reform. The report presents a snapshot of Philadelphia’s standards and accountability systems. The 1995 Philadelphia Standards Writing Teams, including one for science, drafted academic content standards based on those developed by national organizations. Concurrently, the district adopted benchmarks on the SAT-9 assessment as interim performance standards. The district chose the SAT-9 because it was believed that the test was based on national standards, as were the district standards. The district developed an accountability system for schools based on several performance indicators that were combined into a Performance Responsibility Index (PRI). In spring of 1997, a district-wide survey of teachers indicated that they had a high awareness of the standards, but only about one-third of them believed that the content standards had had an effect on their school. Teachers felt (1) the Philadelphia standards were implemented in too short a time and that (2) they lacked understanding about what a standards-based classroom should look like. Among other things, teachers cited a misalignment between standards and the SAT-9. This was counter to the reason given for choosing the SAT-9. Even though teachers reported a misalignment between the content standards and the assessment, student performance improved in 1996-97 compared to the previous year. This study depended heavily on teacher report data that were collected both through a district-wide survey and interviews of more than 300 people, including 116 teachers. Data also were gathered by observations and an analysis of documents. The large number of respondents to the survey, over 7,000 teachers, adds to the credibility of the information reported. There is substantial evidence that the findings reported are valid and represent this large school district under transition toward standards-based reform. Science is one content area with standards, but the results are not disaggregated by content areas. It can only be implied that the results reported are relevant to science.

Consortium for Policy Research in Education. (2000). Deepening the Work: A Report on the Sixth Year of the Merck Institute for Science Education, 1998-1999.Philadelphia: Author.

In 1993, Merck & Co., Inc. committed to a ten-year partnership with four public school districts in New Jersey and Pennsylvania in an effort to reach their vision of high-quality math and science education where guided inquiry is an integral and regular part of classroom experiences. From the beginning, the Merck Institute recognized that training for teachers would be insufficient and they would need to employ a systemic strategy.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Seeking to develop districts’ support of its vision, the Merck Institute’s influential role targeted classrooms, administration, state assessments, as well as public outreach.

This report, the sixth of similar annual reports, details the assessment of progress and impact by the Merck Institute for Science Education during the school year 1998-1999. This report opens with an executive summary of the 1998-1999 evaluation and continues with a brief history of the Merck Institute and summaries of report findings for the five years prior. Appendices to the report outline the guiding questions that were employed for the evaluation; data sources which included interviews, observations, document reviews, and results from achievement tests; and the multivariate regression model used to compare student performances. The authors note that all observers had been trained by the national evaluator of NSF’s Local Systemic Change (LSC) initiative and used both the framework developed by Horizon Research, Inc. for the LSC Initiative and an “authentic pedagogy” framework during observations. The observations yielded both quantitative and qualitative data.

During 1998-1999, the Merck Institute increased the number of and access to Peer Teacher Workshops, and the authors reported that this effort had an impact on teachers and their teaching practices. Nearly three-quarters of the teachers in the districts participated in these workshops and they began integrating learned practices into their classrooms. Use of multivariate regression models predicted higher fifth and seventh grade NCE scores for students of those teachers who participated in the workshops than students whose teachers did not participate. However, the authors caution that scientific literacy is also dependent on high school instruction, and the Partnership has yet to have an affect on high school curriculum. Though successful, efforts to improve the workshops diverted resources from the Institute’s original intention of aligning district policies with its vision. The authors recommend that the Institute move from managing the professional development of individual teachers to mentoring school officials. In such a role, the Institution would assist officials with policy reform, teacher recruitment, and systemwide professional development. The authors also propose that the Institute become an advocate for statewide access to high-quality professional development and experiment with science instruction by science specialists in grades 2-4. The authors noted that, during 1998-1999, the partnership composed an assessment plan to supply meaningful measures of student learning and meet the needs of multiple audiences. This plan will be put into effect in the coming year.

Over the years, the Merck Institute’s systemic approach has been successful. Partner districts have placed a priority on science and have integrated inquiry-centered curriculum in grades K-6. The Peer Teacher Workshops, which model standards-based pedagogy, have improved teachers’ knowledge and skills in inquiry-centered instruction. As evidence of active support from district leaders, changes in policy, organization, and assignments reflected the Partnership’s vision of science education. Progress and evaluation in the future years will determine whether the standards have a significant influence on assessment and student learning in these districts.

Consuegra, G. (1994). Helping Teachers Change Science Instruction. In Scientists, Educators, and National Standards: Action at the Local Level., Sigma Xi Forum Proceedings, Sigma XI, The Scientific Research Society, Research Triangle Park, NC, April 14-15, 1994.

Corcoran, T.B. and Matson, B.S. (1998). A Case Study of Kentucky’s SSI (PRISM), 1992–1997. In P.M. Shields and A.A. Zucker (Eds.), SSI Case Studies, Cohort 2: California, Kentucky, Maine, Michigan, Vermont, and Virginia. Menlo Park, CA: SRI International.

This case study of Kentucky’s Statewide Systemic Initiative, the Partnership for Reform Initiatives in Science and Mathematics (PRISM), describes the context within which the reform was launched; its strategy for reforming science and mathematics education in the state; and its impacts on policy, practice, and student learning. The main strategy employed by PRISM was to develop regional cadres of specialists in mathematics, science, and technology who would model and spread the new approaches to teaching and learning aligned with the standards. The case study draws on extensive visits to the state, interviews with state and PRISM leaders, school administrators and teachers, and review of state and PRISM documents. The methods and analytical strategies that produced the study are not described. The authors find that the designers of PRISM made flawed

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

assumptions that impeded the implementation of their strategy. They assumed that the specialists would be willing and able to provide professional development to their peers. They also assumed that local administrators would value the specialists and provide opportunities for them to work with their peers and play leadership roles in their schools. Once these problematic assumptions were revealed, PRISM shifted to a regional, school-oriented approach late in its five-year cycle. This study contributes to the evidence of the influence of the standards on the professional development strategies employed by major reform efforts. The fact that PRISM essentially set up a professional development system outside of existing professional development providers in the state raises questions about how deeply the standards influenced the already standing professional develop-ment apparatus in the state.

Corcoran, T.B., Shields, P.M., and Zucker, A.A. (1998, March). The SSIs and Professional Development for Teachers.Menlo Park, CA: SRI International.

This research sought to take a cumulative look at the extent of the professional development provided by the Statewide Systemic Initiatives in mathematics and science. The report was based upon data provided by each of the 25 SSI states, including evaluation reports and internal documentation describing professional development strategies, reach, and impact. The researchers conducted case studies of 12 of the 25 SSI states. Abt Associates, which monitors and reports on the SSIs for NSF, also provided independent data on the 25 states. The researchers provided a meta-analysis of available data. The report is largely descriptive, and the validity of its conclusions is dependent on the accuracy and quality of local data. The researchers found that professional development was a main strategy of almost all SSI states. They found the quality of the professional development to be generally high and consistent with state and national standards. However, the professional development in almost all cases was not integrated into the states’ professional development infrastructure that provided most of the learning opportunities for teachers. Consequently, while the professional development reached tens of thousands of teachers, they only touched a small proportion of the teaching population. With one exception (Puerto Rico), none of the SSIs had feasible plans to scale up their efforts to reach most or all teachers.

The study demonstrates that the standards had a substantial influence on the SSIs’ conceptions of quality professional development, which were largely consistent. However, since the SSIs were largely independent of the dominant infrastructures of learning opportunities in the states, their reach was limited.

Council for Basic Education. (2000, February). Closing the Gap: A Report on the Wingspread Conference, “Beyond the Standards Horserace: Implementation, Assessment, and Accountability—The Keys to Improving Student Achievement.” Available at: http://www.c-b-e.org/siteref/reports.htm [August 9, 2002].

This report features a collection of 1999 Wingspread conference papers written by Tom Welch, Deborah Loewenberg Ball and David K. Cohen, Vicki L. Phillips, Nancy S. Grasmick, and Margaret E. Goertz. Conference attendees included educators, policy makers, principals, and teachers, who spent three days reflecting on the challenges the standards movement faces at all levels of the education enterprise. In his paper, Welch provides an account of the principal’s role in transforming the traditional concept of “school” by implementing a system focused on student-centered learning and standards-based education. Drawing on the body of research in mathematics reform, Ball and Cohen write about the challenges of improving instructional practice, including experiences in using knowledge in instruction, managing coordination of instruction, creating incentives for high-quality instruction, and learning from practice. Phillips, superintendent of the school district of Lancaster, gives her perspectives and recommendations on standards-based reform based on her experiences in implementing reform in Kentucky, the city of Philadelphia, and Pennsylvania. Grasmick, Maryland state superintendent of schools, recounts the history of Maryland’s reform efforts, highlighting the development of the state assessment and accountability systems, and the safety nets, interventions, and incentives used to strengthen reforms in minority performance, reading, middle school learning, teacher quality, and K-12 and business partnerships. Goertz uses data from eight states and 23 districts to describe the status of state-level policies for implementing standards-based reform and their impact on local policies and practice. Four of the five papers are of note in that they provide essential background on the implementation of standards at various system levels

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

across the nation. The information contained in these four papers is largely experiential and anecdotal, and the stories are context-specific. Goertz’s paper is more research-oriented; she conducts a comparative analysis of state and district systems using data collected in a recent CPRE study, and her conclusions and recommendations are well substantiated. The conference findings, synthesized from discussion sections, expand upon and support the ideas set forth in the papers. Conference attendees support the belief that standards are a prominent force for reform at every level, but that many challenges still remain, including: (1) improvements in high-stakes, state-level standardized test alignment and opportunities for student to learn what is tested, (2) lack of coherent professional development for teaching to the new high standards, (3) a paucity of strong leadership for reform, (4) ensuring equity and providing all students the chance to meet high standards, and (5) maintaining the public’s trust. Participants discussed the conference papers and came up with the following categories for improving standards-based reform: (1) helping every student reach high standards, (2) improving educator capacity, (3) aligning accountability and assessment systems with standards, and (4) working to improve public will and community engagement.

Council of Chief State School Officers. (1996). States’ Status on Standards: 1996 Update.Washington, DC: Author.

This is a report of a survey by the Council of Chief State School Officers (CCSSO) to determine each state’s current status in the development and implementation of standards for systemic improvement of education. For the study, representatives of each state answered a set of questions based on whether they have developed standards, are in the process of developing standards, or are just beginning the standards development process. The results of the survey clearly indicate that the standards movement was well under way in 1996. The report found that Nevada was the only state listed as at the beginning of the standards process. Thirty states were in the process of developing standards, and 26 states were in the process of implementing standards as tools of systemic reform.

While this report addresses education standards without regard to specific content, it does find some common patterns among states’ treatment of standards that are informative to science educators: (1) standards are not just a measure of quality, but a definition of essential skills, (2) most states are developing standards by grade-level clusters, (3) states are developing curriculum frameworks, assessment frameworks, and instructional guides in addition to the standards, (4) reform efforts address teachers and teaching, curriculum, and assessment as a system, (5) science is included as one of the first subject areas in which standards are being developed, (6) public input and understanding are key elements of standards development, and (7) budget and staffing needs are seen as major challenges in standards-based reform. In states that were further in the process of reform (implementation phase), curriculum/content standards were being linked with assessments and/or performance standards and many of these states were including graduation requirements/exams as part of the initiative. In implementing the standards, most states put a strong emphasis on local districts retaining control over their curriculum with guidance from the standards. A few states have done extensive work on educator/ professional training linked to state standards and assessments.

Council of Chief State School Officers. (1997). Mathematics and Science Content Standards and Curriculum Frameworks: States Progress on Development and Implementation.Washington, DC: Author.

CCSSO, in collaboration with Policy Studies Associates and a panel of experts in mathematics and science education, conducted a study of standards development since 1994. These findings extended those from the 1996 report: (1) 46 states had completed mathematics and science standards, (2) main categories of state standards are similar to national standards, (3) state standards include subject content and expectations for students, although expectations differ markedly by state, (4) state science standards emphasize active hands-on student learning and doing of science, (5) quality standards provide rigorous, challenging statements of content and clear, specific expectations, (6) strategies toward equity are needed, (7) teaching, assessment, and program standards are part of only 10 states’ standards, (8) extended state support is needed for standards implementation, (9) assessments should align with standards, (10) performance standards and levels are still under development, and (11) professional development plans are needed in many states.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Council of Chief State School Officers. (1998). Comprehensive School Reform Demonstration Program: Enhancing the Role of State Leadership in Implementation and Evaluation.Washington, DC: Author.

The focus of this report is on Title I programs in mathematics and reading. The report includes papers describing implementation sites in three large school districts. These papers shared their experiences with implementing school reform.

Council of Chief State School Officers. (1999). Status Report: State Systemic Education Improvements, September. Washington, DC: Author.

This is a report of states’ efforts on components of systemic education improvement included in Title III of the Goals 2000 program. These components include: content standards, performance standards, student assessments, opportunity-to-learn standards, role of the teacher, professional preparation, learning technology, governance and management, community involvement, and education reform. The report is intended as a resource for researchers and policy makers. The information in the report was self-reported by each state department of education. Summary findings include:

  • The majority of the states have composed content and performance standards in the core disciplines and are currently implementing the standards in their local school districts.

  • A major struggle for states has been the issues related to the alignment of state content standards to local curricula, pedagogy, and assessments.

  • Technology is playing a major role in states’ efforts for school improvement and education reform.

  • States report revising state policy for professional preparation, continuing education, and licensure of teachers to a performance-based model.

Council of Chief State School Officers. (2000a). Key State Education Policies on K-12 Education: 2000, Standards, Graduation, Assessment, Teacher Licensure, Time and Attendance.Washington DC: Author.

Designed as a status report to policy makers and educators, this CCSSO report presents results for the 2000 Policies and Practices Survey of the State Departments of Education. The report summarizes current information on six key policy areas: (1) time and attendance policies, (2) graduation requirements, (3) content standards, (4) teacher preparation and licensure, (5) school leader and administrator licensure, and (6) student assessment. The report is the sixth in a series of reports based on surveys that have been administered to all 50 states’ departments of education since 1987. State education staff information acquired in the survey was supplemented with information from other CCSSO surveys and a certification report published by the National Association of State Directors of Teacher Certification. The report presents current findings and trends since 1987 in summary form and provides detailed state-by-state descriptive data in tables. For example, the report notes that 14 states have raised their graduation requirements by one or more credits in science since 1987, and 20 states now require specific science courses required for graduation. By the year 2000, 46 states had established content standards in science. Between 1984 and 1999, the number of states requiring statewide testing in science more than doubled, increasing from 13 to 33. Most states assess students’ science performance using multiple choice tests in grades 4, 8, and 11, but 12 states are now using more nontraditional extended response and short answers to assess students. Reporting of state performance levels ranges from a pass/fail designation to a proficiency rank based on up to five levels of performance, with three and four levels of performance most commonly reported. The report is a compilation of descriptive information and indicators on a selected set of state educational policies as self-reported to CCSSO over a period of years. The report is not evaluative in nature and does not interpret state educational policy changes; however, the report does provide some simple longitudinal data and points out state trends over time.

Council of Chief State School Officers. (2000b). State Policies to Support Middle School Reform: A Guide for Policymakers.Washington DC: Author.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Council of Chief State School Officers. (2000c). Using Data on Enacted Curriculum in Mathematics & Science, May 2000.Washington DC: Author.

The report is a summary of the Survey of Enacted Curriculum project conducted by the Council of Chief State School Officers and the Wisconsin Center for Educational Research. The document provides an overview of some of the findings of the study, gives examples of how data on enacted curriculum might be analyzed and reported, and identifies possible uses of the data by schools, districts, and states. This study was not designed to provide evidence of the impact of standards. Rather, the authors intended to offer a research tool by which educators could objectively analyze current classroom practice in relation to the goals of systemic initiatives and content standards, and to fill the gap in availability of reliable data on curriculum and teaching as they are actually presented in classrooms. The Survey of Enacted Curriculum was originated by CCSSO under a grant from the National Science Foundation to develop, demonstrate, and test survey instruments for classroom curriculum. The study involved schools and teachers from over 600 schools across 11 states that volunteered to participate; state leaders were asked to select schools and teachers based on their particular state initiative, including schools of varying urbanicity and student composition. Teachers responded to survey items about their instructional practices, preparation, and professional development. They also reported on the subject areas taught in their classes using a “subject content matrix.” The major concepts underlying the design of the survey were derived from content standards and prior studies and initiatives, and included the following main topics: active learning in science, problem-solving in mathematics, mathematics and science content, multiple assessment strategies, use of technology and equipment, influences on curriculum and teaching practice, alignment of content with state assessments, and teacher preparation. Survey results are reported using a variety of complex formats including item profiles, summary scales, and content maps.

Council of Chief State School Officers. (2001). Annual Survey of State Student Assessment Programs, Summary Report and Vol. 1 and 2 (1998-1999 Data).Washington DC: Author.

Cozzens, M.B. (2000). Instructional Materials Development (IMD): A Review of the IMD Program, Past, Present, and Future.Arlington, VA: National Science Foundation.

This report describes the history, status, and future of the Instructional Materials Development (IMD) program of the National Science Foundation. Reform in mathematics and science education requires an innovative, comprehensive, and diverse portfolio of instructional materials that implement standards-based reform. The goal of the IMD program is to develop instructional materials, aligned with standards for content, teaching, and assessment that enhance the knowledge, thinking skills, and problem-solving abilities of all students; apply the latest research on teaching and learning; are content-accurate and age-appropriate; incorporate the recent advances in disciplinary content and educational technologies; assist teachers in changing practices; and ensure implementation in broadly diverse settings. Instructional materials developed through funding from IMD are developed by a collaborative of scientists, mathematicians, teachers, and educators; are based on research in teaching and learning; align with standards; contain appropriate student assessment; are field-tested in diverse settings; and have undergone formative and summative evaluation, which include impact data from field test sites. Starting in 1986, IMD supported a series of TRIAD projects—first at the elementary level, then at the middle school level. The TRIAD projects were required to be a partnership of a curriculum developer, partner schools, and a publisher. These projects, however, were mostly completed prior to the release of the National Science Education Standards. Since early 1992, however, the projects were advised to keep close track of the development of Project 2061 and the NSES. The TRIAD experiment did give rise to a number of exemplary programs, such as the Full Option Science Series. Starting in 1986, IMD also supported the development of instructional materials at the high school level, including programs such as ChemCom, Active Physics, EarthComm, Biology, A Community Context, and BSCS Biology: A Human Approach. More recently, IMD has funded programs that integrate science, mathematics, and technology, such as the Integrated Mathematics, Science, and Technology Project. IMD is refocusing its effort on issues related to dissemination, implementation, and evaluation of standards-based materials. The report identifies serious issues that must be addressed to implement standards-based instructional materials:

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
  • Standards-based instructional materials require a significant amount of professional development for teachers in both content and pedagogy.

  • Publishers are not prepared to provide the needed teacher support activities and often do not realize teachers need more than they did with traditional texts.

  • The textbook adoption process is an expensive process that some smaller publishers of innovative materials are not prepared to undertake.

  • Implementation requires support and buy-in from administrators, parents, and the community; when support is missing from one group, the whole reform movement can be in jeopardy.

  • Assessment of student learning must be linked to the instructional materials.

  • Articulation across grade levels and disciplines is essential.

  • Teacher preparation in colleges and universities must be linked with the new materials to facilitate implementation.

Darling-Hammond, L. (2000). Afterword: Teaching for America’s Future: National Commission and Vested Interests in an Almost Profession. In K.S. Gallagher and J.D. Bailey (Eds.), The Politics of Teacher Education Reform, pp.162-183. Thousand Oaks, CA: Corwin Press.

Deal, D., and Sterling, D. (1997, March). Kids Ask the Best Questions. Educational Leadership.54(6), 61-63.

DeBray, E., Parson, G., and Woodworth, K. (2001). Patterns and Response in Four High Schools Under State Accountability Policies in Vermont and New York. In S.H. Fuhrman (Ed.), From the Capitol to the Classroom: Standards-Based Reform in the States, The One Hundredth Yearbook of the National Society for the Study of Education, Part 2., pp.170-192. Chicago: University of Chicago Press.

In this chapter, DeBray, Parson, and Woodworth point out that school-level responses to new accountability systems tend to vary not as much by differences in state policy, as by differences in school structures, norms, and existing internal accountability mechanisms. The authors gathered data from four high schools in two states, Vermont and New York, each of which had recently adopted new accountability policies. In each state, one “high-performing” school and one “low-performing” school (assumed to be the target of the new policies) were selected for study. The authors found that high-performing and low-performing schools often responded differently depending on their capacity to respond to new policies and structures, and how they filtered these new policies through their own internal theory of action regarding accountability. High-performing schools were found to have the capacity, structure, and norms necessary to translate student performance results into school improvements. Low-performing schools struggled to reconcile new policies and regulations with their current beliefs and practice, they lacked the skills to use data in planning, and they needed assistance to execute continuous improvement and action planning in order to effectively influence changes in curriculum and instruction. The authors admitted their sample was limited in size (it was a slice of a larger study and sample from a five-year CPRE project), acknowledged that the schools were not representative of the general high school population in each state, and acknowledged that the results were not likely to be replicable given that the new state policies had yet to fully implement any sanctions or rewards. While the authors’ findings may lack substantiation, the research is of interest in that it serves to raise some interesting questions about the strength and weaknesses of how accountably systems play out at the school level. The authors challenge states to rethink their assumptions of how accountability policies will be interpreted and implemented at the school level. In particular they challenge the assumption that low-performing schools will respond adequately to public pressure to improve poor performance. Low-performing schools may need assistance to align their internal accountability with the new external accountability mechanisms, such as assistance with school improvement planning, use of data, incentives for motivating instructional change, and addressing feasible short-term improvement goals.

Donmoyer, R. (1995). Rhetoric and Reality of Systemic Reform: A Critique of the Proposed National Science Education Standards.Columbus, OH: National Center for Science Teaching and Learning.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Doran, R. L., Reynolds, D., Camplin, J., and Hejaily, N. (1992). Evaluating Elementary Science. Science and Children.November/December, 33-35, 63-64.

Doyle, L.H., Huinker, D., and Posnanski, T. (1997, July). Analysis of Initial Interviews with First Cohort Mathematics/Science Resource Teachers: A Study of the Milwaukee Urban Systemic Initiative.Milwaukee, WI: University of Wisconsin-Milwaukee, Center for Mathematics and Science Education Research.

Doyle, L.H. and Huinker, D. (1999, August). Lessons Learned: Implementation of the Milwaukee Urban Systemic Initiative in Years One and Two. Report for the Milwaukee Public Schools.Milwaukee, WI: University of Wisconsin-Milwaukee, Center for Mathematics and Science Education Research.

Education Commission of the States. (2001, January). Building on Progress: How Ready Are States to Implement President Bush’s Education Plan? A Status Report by the Education Commission of the States.Denver, CO: Author.

This policy brief summarizes the main features of President Bush’s “No Child Left Behind” education plan as proposed in January 2001, and provides a status report on the states’ progress and readiness in regard to implementing the plan. Bush’s plan proposes major initiatives and improvements in (1) student achievement, (2) standards and accountability, (3) literacy, (4) teacher quality school safety, (5) math and science instruction, (6) English language fluency, and (7) parental options and innovative programs. The Bush plan calls for developing Math and Science Partnerships. The majority of the information for the policy brief comes from ECS surveys and reports. The brief also draws on information attained from status and evaluation reports on state-level educational systems conducted by secondary sources such as the American Federation of Teachers, National Assessment of Educational Progress, National Center for Education Information, and the Fordham Foundation. A review of these data showed a great deal of variability in the states’ progress to date and readiness to implement the initiatives. While most states had established mathematics, reading, science, and social studies standards, less than half of the states had established science and social studies standards at all three K-12 educational levels (elementary, middle, and high school). More than half of the states test students in reading and mathematics, but only 15 test students annually in these subjects from grades 3-8. The Bush plan calls for annual testing of students in mathematics and reading using the NAEP, yet only 41 states currently participate in the NAEP testing.

Education Development Center. (1997, November). Proficiency Score Standards for the Wisconsin Student Assessment System (WSAS) Knowledge and Concepts Examinations for Elementary, Middle, and High School at Grades 4, 8, and 10. Final Summary Report.Madison, WI: Author.

This report details the process the state of Wisconsin used for setting proficiency cut scores for its statewide testing in grades 4, 8, and 10 in April 1997. Based on the test contractor CTB/McGraw-Hill standard-setting procedures, 185 panelists from 100 Wisconsin schools districts met to set proficiency score standards in math, reading and language arts/writing, science, and social studies. The proficiency cut scores are stated in terms of the state assessment scale scores and are expressed in four categories: advanced, proficient, basic, and minimal performance. The report provides details of the proficiency descriptors in each content area and by grade. The standard-setting activities and process are also described. The standards-setting process required panelists to: (1) study individual test items, (2) determine their difficulty, (3) determine which items represent appropriate content and expected student performance in each proficiency category, (4) “bookmark” items at the proficiency dividing points, and (5) write descriptions of expected student performance at each level after the cut scores had been determined. Panelists referred to test booklets provided by the test contractor and relied on the broad expertise of participating panelists to determine the cut scores. According to the report, national and state standards for the various subject areas were not directly incorporated into the process. The state of Wisconsin uses the proficiency score standards as the primary way to report statewide test results. Of interest in the report is the story of how one state went about setting proficiency benchmarks for its state assessment program.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Education Trust. (1999). Not Good Enough: A Content Analysis of Teacher Licensing Examinations. Thinking K-16.3(1).

This study by the Education Trust, a Washington-based education program developer and advocacy group, examines the content of teacher-licensing exams in English language arts, mathematics, and science. The goal of the study is to analyze the licensing exams in contrast to the expectations of state and national standards. If teachers are expected to help students meet standards, the authors argue, then licensing exams should test teacher preparation to teach to the standards. The study focused on the two major examinations, the Praxis series by the Educational Testing Service and state-specific exams designed by National Evaluation Systems. The instruments were Education Trust staff and outside consultants using a methodology developed by a national review panel (although not described in the document). The results of the review were not encouraging. The majority of the tests, the authors reported, were multiple-choice assessments dominated by high-school level material. In a few cases, essay examinations required candidates to demonstrate their depth of knowledge. But the essays were used by far fewer states than the lower-level multiple-choice tests. Further, the reviewers found, knowledge for teaching was a gaping hole in the licensing exams. Despite the fact that the tests were mostly low-level, the data on passing rates are fairly low, with between 10 and 40 percent of takers failing the tests. The authors conclude their paper by arguing that the licensing exams are not intended to set high expectations, but rather to establish a floor. The reason for this is due to the potential for litigation.

Education Week. (2001). Seeking Stability for Standards-Based Education. In Special Report: Quality Counts 2001: A Better Balance: Standards, Tests, and the Tools to Succeed.20(17), January 11.

Education Week. (2002). The State of the States. In Special Report: Quality Counts 2002: Building Blocks for Success. 21(17), January 10.

Eisenhower National Clearing House. (2001). ENC Focus. New Horizons in Mathematics and Science Education, A Magazine for Classroom Innovators.8(4).

Elmore, R.F., Abelmann, C.H., and Fuhrman, S.H. (1996). The New Accountability in State Education Reform: From Process to Performance. In H. Ladd (Ed.), Holding Schools Accountable, pp. 65-125. Washington, DC: The Brookings Institution.

As early as 1993, CRPE research began detecting a shift in state accountability systems from regulating and ensuring compliance based on district and school inputs, to accountability systems focused on student performance. Elmore, Abelmann, and Fuhrman propose the emergence of a “new model of state and local school governance,” based on measures of student performance, linked to standards for comparability, and focused on school improvement through systems of rewards and sanctions. Drawing on their experience with extended studies of state accountability systems conducted by CPRE in the 1990s, the authors profiled the emerging state accountability systems of Kentucky and Mississippi to illustrate their model. Design elements of these new state accountability systems vary by goals, level, and standard of accountability; types of assessments, subject areas, and grades tested; indexes and rankings, as well as by rewards and sanctions. In transforming an accountability system from compliance to performance orientation, states must address the following questions: What is proficient? What progress is realistic and sufficient? How can a complex system be made transparent to the public and parents? What are the appropriate incentives for districts, schools, and teachers? Issues of fairness, technical assistance, and professional development also influence design greatly. States must also consider the alignment and balance of their assessment system with state standards, and accountability mechanisms. Public pressure, resource constraints, political stability, public understanding, and lingering input and process standards must also factor into the new design. The authors contend that these new accountability systems are at a critical stage of development. New systems will need to be: (1) understandable and defensible, (2) fairly designed and implemented, (3) focused on improvement, (4) supported and maintained by states, and (5) connected to stable political environments. The paper is a formative assessment of the design, development, and early implementation of what the authors refer to as “the new educational accountability.” Their conclusions are

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

broadly drawn and rely heavily on the collective experience of the authors’ own research and experience. The paper provides a useful model and formative evaluation framework for analysis of other state accountability systems.

EPPI-Centre Review Group Manual, Version 1.1 (2001). Available at: http://eppi.ioe.ac.uk/EPPIWebContent/downloads/RG_manual_version_1_1.pdf [August 22, 2002].

Fairman, J.C. and Firestone, W.A. (2001). The District Role in State Assessment Policy: An Exploratory Study. In S.H. Fuhrman (Ed.), From the Capitol to the Classroom: Standards-Based Reform in the States, The One Hundredth Yearbook of the National Society for the Study of Education, Part 2, pp. 124-147. Chicago: University of Chicago Press.

Fairman and Firestone conducted a qualitative study of administrative and teacher responses to testing policies in states that had recently adopted performance-based middle school assessments. They studied the ways in which state policies were locally interpreted in Maryland and Maine, using an embedded case study design, by looking at teachers within districts within states. The sample included two middle schools from each of two Maryland school districts and six middle schools or junior high schools from three Maine school districts. The researchers collected data using interviews and classroom observations that focused on mathematics. They studied districts’ will (motivation) and capacity (knowledge, personnel, money, and resources) at both the organizational and the individual levels. They found that state standards could influence districts to attend to certain aspects of content and pedagogy when supported by other policies. In addition, when districts did attend to state standards, these policy documents could influence the instructional content. These findings were qualified somewhat by the increased attention to test-related activity in the higher-capacity Maryland districts, which produced instructional practices that were only partially consistent with state or national mathematics standards. The study was competently done and reported with findings that served more as hypotheses than as variable findings, thus the clarification in the title as an exploratory study.

Foley, E. (2001, August). Contradictions and Control in Systemic Reform: The Ascendancy of the Central Office in Philadelphia Schools.Philadelphia: Consortium for Policy Research in Education. Available at: http://www.cpre.org/Publications/children03.pdf [August 8, 2002].

This report discusses Children Achieving—a massive systemic reform initiative ($150 million in support) undertaken by Philadelphia public schools. This report focuses on the role of the central office in the reform effort. The Consortium for Policy Research in Education (CPRE) evaluated the project between 1995 and 2001, interviewing hundreds of teachers, principals, parents, students, district officials, and civic leaders; observing in classrooms; surveying teachers; and analyzing the District’s test results. One of the first major activities of the central office was to create “world-class” content standards. This was a move away from what was a standardized curriculum for each subject area and grade level toward a more decentralized curriculum based on core standards. Concerns developed that some school-based purchases were not standards-based and that increased school authority creates extra burdens for teachers. Forming local school councils and serving on small learning communities demanded much time and energy. Efforts of the central office staff were focused on capacity building rather than on control, but much confusion resulted in how to build local capacity for change. To further clarify its role, the central office developed detailed curriculum frameworks that defined grade-specific skills and content and offered suggestions for units and activities that addressed the content standards. The frameworks identified constructivism as the underlying pedagogical philosophy. The frameworks, which helped fill the gap between the current curriculum and where the reform was to be, were well received by school personnel. CPRE found that with the publication of the curriculum frameworks, more teachers were moving toward standards-based instruction. An important finding of the study was that the focus on “doing it all at once” created reform overload throughout the District and was a strong contributor to the inability of school staff to focus their efforts around clearly defined and manageable instructional priorities. Another key issue was underestimation of the time and support required to transform instruction to a constructivist approach, which requires new curriculum and deep changes in teaching that occur only over extended periods and with intensive support.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Forseth, C. (1992). Portfolio Assessment in the Hands of Teachers. The School Administrator.December, 24-28.

Francis, R.W. (1996, March). Connecting the Curriculum Through the National Mathematics and Science Standards. Journal of Science Teacher Education.7(1), 75-81.

This article describes the use of a matrix to establish connections between the content standards in national standards for science and mathematics. The report argues that the matrix analysis meets a need for teachers to understand the standards, to create connections across standards, and to become self-directed curriculum developers. The author suggests that teachers identify the key standards in science and mathematics for their curriculum and then identify learning opportunities that would enable students to achieve both sets of standards. This is accomplished by listing standards and sub-standards for mathematics on one dimension of the matrix and for science on the other dimension. The cells represent curriculum intersects where the subjects can be connected. The author concludes with the recommendation that the curriculum matrix process be a regular part of the planning process and will help guide educators in implementing effective activities that embed the standards and connections within the curriculum.

Fuhrman, S.H. (2001). Introduction. In S.H. Furhman (Ed.), From the Capitol to the Classroom: Standards-Based Reform in the States, The One Hundredth Yearbook of the National Society for the Study of Education, Part 2, pp. 1-12. Chicago: University of Chicago Press.

Fuhrman, S.H. (2001). Conclusion. In From the Capitol to the Classroom: Standards-Based Reform in the States, The One Hundredth Yearbook of the National Society for the Study of Education, Part 2, pp. 263-278. Chicago: University of Chicago Press.

Gallagher, J.J. (2001, February). Preface: Furthering the Contemporary Reform Agenda. Journal of Research in Science Teaching.38(2), iii-iv.

Garet, M.S., Birman, B.F., Porter, A.C., Desimone, L., Herman, R., and Yoon, K.S. (1999). Designing Effective Professional Development: Lessons from the Eisenhower Program.Washington, DC: U.S. Department of Education.

This report synthesizes the lessons from the Eisenhower mathematics and science professional development program, Title II of the Elementary and Secondary Education Act (ESEA), which is the federal government’s largest investment in developing teachers’ knowledge and skills. It is based upon a sophisticated sample and analysis of the survey results of a nationally representative probability sample of teachers in districts and 10 in-depth case studies in five states. This is a rich report and findings are numerous. On the survey, about 70 percent of teachers who participated in the programs reported effects on their knowledge of mathematics and science, but only roughly half of the teachers in the sampled districts reported influence, suggesting that the reach of the programs were not uniform. The authors compare the survey results to those of other NSF professional development programs and find them roughly comparable and thus conclude the quality is similar. The quality of the Eisenhower activities were examined on six dimensions: organization, duration, collective participation, content focus, active learning, and coherence. The findings relative to quality suggest that most Eisenhower-assisted activities are traditional workshops rather than study groups, networks, or mentorships. The workshops lasted an average of 25 hours. Relatively few of the activities emphasize collective participation of teachers in schools or districts, but mostly focused on individual teachers. Finally, content emphasis, active learning, and coherence were evident in about 60 percent of activities observed. The authors were able to link these features of high quality to teacher self-reported instructional outcomes. The report also discusses district and higher-education institution management of Eisenhower-assisted activities and finds that co-funding, alignment, continuous improvement, and teacher involvement in planning lead to higher-quality professional development.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Gess-Newsome, J. (2001). The Professional Development of Science Teachers for Science Education Reform: A Review of the Research. In J. Rhoton and P. Bowers (Eds.), Professional Development Planning and Design, pp. 91-100. Issues in Science Education. Arlington, VA: National Science Teachers Association.

Gibbons, S., Kimmel, H., and O’Shea, M. (1997, October). Changing Teacher Behavior through Staff Development: Implementing the Teaching and Content Standards in Science. School Science & Mathematics.97(6), 302-309.

Goertz, M.E. (2001). Standards-Based Accountability: Horse Trade or Horse Whip? In S.H. Fuhrman (Ed.), From the Capitol to the Classroom: Standards-Based Reform in the States, The One Hundredth Yearbook of the National Society for the Study of Education, Part 2, pp. 39-59. Chicago: University of Chicago Press.

Goertz presents this chapter from a historical perspective, highlighting shifts in the focus of state accountability systems from the 1970s to the present. Goertz writes of the changes over time in accountability orientation from inputs to outcomes, from minimal competency to performance standards, and from district- to schoollevel accountability for student performance. Her chapter describes the current status of performance-based accountability systems and how states, districts, and schools function within those systems. The main purpose of the three-year study was to study standards-based reform and its influence on state accountability systems in regard to progress, changes in policy, coherence across educational units, and effects on policy, practice, and capacity. Site visits and interviews were conducted in 23 districts (selected for their diversity and activism in school improvement and standards-based reform) and 57 schools (mostly elementary) in 10 states. Goertz found a great deal of variation between states and within states’ accountability systems; and that state and district contexts make a difference in how accountability systems are developed and implemented. State accountability systems examined in the study in 1998-99 held schools accountable for student performance, yet lacked incentives, motivation, and consequences for students to take testing seriously. Few states had resolved the controversial issue of teacher accountability. States also vary by types of accountability system: (1) public reporting systems are the most basic, (2) locally defined systems allow schools to define standards, planning, and performance criteria, and (3) state-defined systems set goals for districts, schools, and students, and are the most common type of accountability system. The more autonomy a state allows local districts, the more variation occurs in the accountability system; locally defined districts tend to use multiple measures of student performance and set their own goals and performance measures. Goertz concludes that, increasingly, many state accountability systems hold students alone to high stakes accountability; such performance-based accountability systems are becoming the norm in standards-based reform. Goertz recommends that more needs to be done to diversify responsibility and to hold adults and schools accountable. She also recommends the need for realignment of state accountability policies with Title I requirements, state standards, and state assessment. She concludes that work remains in ensuring that standards-based reform is equitable, that efforts to close the achievement gap are successful, and that valid and reliable assessments are available to include all students in assessment and accountability systems. In addition, Goertz argues that performance-based accountability systems have yet to adequately address the capacity needs (knowledge, human, and financial resources) of districts and low-performing schools. This chapter draws upon the same research as that presented in Goertz’s paper for the 1999 Wingspread Conference. (See annotation for: Council for Basic Education. (2000). Closing the Gap. A Report on the Wingspread Conference. Beyond the Standards Horserace: Implementation, Assessment, and Accountability–The Keys to Improving Student Achievement.

Goertz, M., and Carver, R. (1998). A Case Study of Michigan’s SSI (MSSI), 1992-1997. In P.M. Shields and A. A. Zucker (Eds.), SSI Case Studies, Cohort 2: California, Kentucky, Maine, Michigan, Vermont, and Virginia. Menlo Park, CA: SRI International.

This report provides a case study of the Michigan Statewide Systemic Initiative (MSSI) from 1992-97. The report contains analyses of the context for educational reform in Michigan, the structure and strategies of the MSSI, and analysis of the impacts of the initiative. The authors do not describe the methodology for their data collection and analysis, but it is apparent from reading the report that they used a variety of data sources in

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

compiling their report, including state and MSSI documents, interviews with a variety of sources both inside and outside the MSSI, and descriptive analysis of state test data. As the authors describe it, the MSSI strategy for systemic reform in the state focused on policy and program review, support and technical assistance to a cadre of 24 diverse urban and rural districts, the redesign of teacher preparation, professional development, and communication. The authors conclude that the MSSI adopted a more systemic strategy than most, but that the time to make deep reforms and their complexity hampered their ability to demonstrate measurable impacts on a wide scale. In terms of professional development, the MSSI took a broad view of its task. Rather than provide direct service to teachers, the MSSI emphasized communicating a standards-aligned paradigm of professional development to those who provided it, supplying professional development to the main providers in the state, cataloging and disseminating information about the sources of professional development in the state to consumers, and working with policy makers to incorporate the principles of high-quality professional development into state policy. Higher education pre-service providers reported being influenced by the MSSI’s vision of professional development for teachers.

Goertz, M., Duffy, M., and LeFloch, K.C. (2001, March). Assessment and Accountability in the 50 States: 1999-2000. CPRE Research Report Series: RR-046. Available at: http://www.cpre.org/Publications/rr46.pdf [August 8, 2002].

Goertz and Duffy offer a comprehensive review of state assessment and accountability systems and the extent to which state policies address federal policy objectives such as those set forth in IASA Title I. Goertz and Duffy focus their analysis on states’ use of assessments to measure student performance, standards-based reform that includes all students, and a review of district, school, and student accountability policies. The authors used a 50-state survey conducted by CPRE in the spring of 2000 to gather information on state assessment and accountability systems that were “in place” during the 1999-2000 school year. Data from Education Week’s Quality Counts 1999 and 2000; reports from the Council of Chief State School Officers (CCSSO) and the American Federation of Teachers (AFT); interviews of state directors of assessment; and reviews of state department of education Web sites were used to verify the accuracy of the information and to triangulate the analysis. Verified data were used to write state profiles of each state’s assessment, inclusion, reporting, accountability assistance, and Title I policies and practice, and to identify proposed changes in these state policies. Goertz and Duffy acknowledge the “transitory” nature of assessment and accountability systems, noting that these systems respond to a variety of forces resulting in continuous redesign and modifications. The report presents a vast array of findings regarding state policies and practice in (1) measuring student performance, (2) including all students in assessment, (3) types of state, school and district accountability systems, (4) reporting practices, (5) setting goals and targets, (6) identifying low-performing schools, (7) establishing consequences; and (8) aligning with Title I and other federal policies. The authors conclude by summarizing their concerns about the challenges that remain for states as they continue to develop systems of educational accountability. Rigorous attention to the substantiation of data and information allows the authors to offer a highly detailed and accurate analysis of state assessment and accountability systems. This report goes beyond the usual reports that summarize descriptive statistics of state assessment systems. It also offers the reader an in-depth analysis of current state policies and practice, and provides insights into future directions, developments, and changes proposed for school reform at the state level.

Goertz, M.E., Massell, D., and Corcoran, T.B. (1998). A Case Study of Connecticut’s SSI (CONNSTRUCT), 1991-1996. In A.A. Zucker and P.M. Shields (Eds.), SSI Case Studies, Cohort 1: Connecticut, Delaware, Louisiana, and Montana. Menlo Park, CA: SRI International.

This report provides a case study of the Connecticut Statewide Systemic Initiative (called CONNSTRUCT) from 1991-96. This represents the first phase of the SSI’s efforts, as the SSI also received a second five-year funding award from the National Science Foundation. The report contains analyses of the context for educational reform in Connecticut, the structure and strategies of CONNSTRUCT, and analysis of the impacts of the initiative. The authors do not describe the methodology for their data collection and analysis, but it is apparent from reading the report that they used a variety of data sources in compiling their report, including state and SSI

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

documents, surveys, interviews with a variety of sources both inside and outside the SSI, and they conducted descriptive analyses using state test data. The authors report that the SSI focused on five major strategies. First, the SSI developed an independent Academy to serve as a catalyst, advocate, and broker for reform. Second, the SSI focused assistance on 19 urban and rural disadvantaged districts. Third, the SSI provided grants to higher education institutions to foster change in teacher education and undergraduate math and science programs. Fourth, the SSI sought to create partnerships with a variety of community organizations. Finally, the SSI intended to build public understanding for the need for reform. The report describes progress and difficulties in each of these areas. Overall, the authors concluded that the variation in impacts were due to their dependence on the willingness and capacity of districts and schools to identify their needs, tap the resource networks, and use resources to institute curricular and instructional changes. Although the SSI lacked leverage with higher education institutions, they instigated conversations about the preparation of teachers and the pre-service structures in the state, and several IHE’s altered courses and institutionalized co-teaching.

Gold, E., Rhodes, A., Brown, S., Lytle, S., and Waff, D. (2001). Children Achieving: Clients, Consumers, or Collaborators? Parents and Their Roles in School Reform During Children Achieving, 1995-2000.Philadelphia, PA: Consortium for Policy Research in Education.

Greeno, J.G., Pearson, P.D., and Schoenfeld, A.H. (1996). Implications for NAEP of Research on Learning and Cognition. Report of a Study Commissioned by the National Academy of Education.Panel on the NAEP Trial State Assessment, conducted by the Institute for Research on Learning. Stanford, CA: National Academy of Education.

Hammrich, P.L. (1997, March). Teaching for Excellence in K-8 Science Education: Using Project 2061 Benchmarks for More Effective Science Instruction.Presented at the 70th Annual Meeting of the National Association for Research in Science Teaching, Oakbrook, IL, March 23, 1997.

The author of this study reports on her experience as the instructor of a K-8 science methods course for teacher candidates. The author argues that teachers’ conceptions of science teaching are guided by their conceptions of science. Therefore, in order for teachers to model practices of teaching and learning outlined by the standards, they need to participate in activities that will cause them to reflect and have practice applying the standards to lessons. The purpose of the study was to explore teacher-candidates’ conceptions of science, knowledge construction, and the principles implied in the national reform initiatives. The methodology for the qualitative study is clearly described by the author: she randomly sampled approximately half of the students in her class, and conducted pre- and post-experience interviews with them. Grounded theory was used for analysis. The author finds that teacher-candidates’ conceptions of effective science instruction were directly influenced by their conception of science, that they had differing views on the teachers’ role in students’ construction of knowledge, and that the principles reflected in the national reform initiatives were viewed as beneficial, but time-consuming, and may not be worth the time investment. The author concludes that pre-service experiences of teachers must be dramatically changed in order for teachers to apply the principles of the standards in the classroom. This study surfaces some of the implications that the standards have in pre-service courses for teachers and provides a model for aligning the standards and pre-service experiences for teachers.

Hannaway, J., and Kimball, K. (1998) Big Isn’t Always Bad: School, District Size, Poverty, and Standards-Based Reform . In S.H. Fuhrman (Ed.), From the Capitol to the Classroom: Standards-Based Reform in the States, The One Hundredth Yearbook of the National Society for the Study of Education, Part 2. Chicago: University of Chicago Press.

Harris, J. (Ed.). (1997). SSRP: Software for Problem Solving and Inquiry in Grades K-4. Columbus, OH: Eisenhower National Clearinghouse for Mathematics and Science Education.

Hawkes, M., Kimmelman, P., and Kroeze, D. (1997, September). Becoming “First in the World” in Math and Science. Phi Delta Kappan.79(1), 30-33.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Hein, G. (1991). Active assessment for active science. In V. Perrone (Ed.), Expanding Student Assessmentpp. 106-129. Alexandria, VA: Association for Supervision and Curriculum Development.

Herman, J. (2000). Performance Assessment Links in Science (PALS) Final Evaluation Report.Los Angeles: Center for Research on Evaluation, Standards and Student Testing (CRESST). Available at: http://www.pals.sri.com [August 8, 2002].

Performance Assessment Links in Science (PALS) is a project funded by the National Science Foundation to obtain science performance assessments from a range of resources and make these generally available on the Web, CD, or in print. The assessments are indexed to the National Science Education Standards. Users are able to search online for assessments that correspond to specific standards. The external evaluation, for each of three years, appraised the assessment-collection efforts, reviewed data-collection instruments and analyses, and specified additional analyses as appropriate. Important information used by the external evaluator was collected through the project’s evaluation that included documenting the use of PALS products on the Web, user feedback, and educators’ judgment on the quality and utility of the materials. PALS had difficulty obtaining technical information on the performance assessment activities from those who provided the activities. Such information was deemed as important by the external evaluator if the assessment activities were used for high-stakes purposes, but less important if used by teachers to learn more about implementing performance assessments. PALS produces a guide to inform teachers and other users on how they can adapt or develop performance assessments to meet their needs. The external evaluator concluded that PALS had surpassed it goals in developing an online resource of performance assessments in science. Users were very positive about the materials provided. This report is very general in nature and provides some information about PALS, but does not go into great detail about the evaluation of the program. What is significant about PALS is that it directly links assessments with the NSES. As such, the resource is a specific example of the NSES application for cataloguing performance assessment items so teachers and others are better able to determine if students are learning what is required by the NSES.

Hill, F., Kawagley, O., and Barnhardt, R. (2000). AKRSI Final Report: Phase I, 1995-2000.Fairbanks, AK: University of Alaska.

Hoffman, K.M. and Stage, E.K. (1993). Science for All: Getting It Right for the 21st Century. Educational Leadership.February 1993, 27-31.

Hollweg, K.S., Kubota, C., and Ferrell, P. (1998). Changing What We Do: Constructing a Team-Based, Problem-Centered Professional Development Experience.Troy, OH: North American Association for Environmental Education.

This publication is both a description and an outcome evaluation of a problem-centered, team-based professional development innovation that had the goal of integrating community-based science programs into classrooms and curricula. The community-based science program, VINE (Volunteer-led Investigations of Neighborhood Ecology), is designed for third- through fifth-graders who work with trained community volunteers in inquiry-based ecology projects within their communities. The professional development was designed to address the “problem” of establishing previously missing links between this community-based program and the ongoing school curriculum. The goal was to enable students to actually do more science themselves and consequently construct meaning from their experiences. Although the Follow-Through project was planned prior to the publication of the National Science Education Standards, its program and design are aligned with their professional development standards.

The Follow-Through project was evaluated by external evaluators using data collected from site visits during the VINE Summer Institutes, interviews with team members, document reviews, and pre-coded teacher logs. To assess classroom effects, the teacher-participants were asked to complete 20 logs documenting VINE-related science activities over the course of the year. These were then compared with logs completed by a

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

matched sample of non-participant teachers. The pre-coded logs had been validated in prior studies and were adapted for use in this evaluation. Teachers’ classroom strategies were coded as traditional, progressive (i.e., constructivist), or both. While differences between the treatment and control groups were noted, there was no corroborating observation or information to triangulate with the teachers’ self-reports.

Outcomes noted by the external evaluators related to successful team-building strategies, changes in teacher practice, as well as overall impressions of this professional development innovation. Specifically, the evaluation revealed that in virtually every measure the teacher-participants used more “best practices” as promoted by the NSES than did their non-participant colleagues. Finally, illustrative vignettes were presented from qualitative data gathered at the three sites that demonstrated alignment with the NSES Content Standards.

Horizon Research, Inc. (2000). Validity and Reliability Information for the LSC Classroom Observation Protocol. Chapel Hill, NC: Author.

Horizon Research, Inc. (2002). Special Tabulations on Data from the 2000 National Survey of Science and Mathematics Education. Unpublished.

Huinker, D., and Coan, C. (1999, May). Second Year Site Visits to Milwaukee Urban System Initiative Schools. Report for the Milwaukee Public Schools. Milwaukee: WI: University of Wisconsin-Milwaukee, Center for Mathematics and Science Education Research.

Huinker, D., Coan, C., and Mueller, L. (1999, August). Survey Results for First Wave Schools of the Milwaukee Urban System Initiative . Report on Milwaukee Public Schools. Milwaukee Urban System Initiative. Milwaukee, WI: University of Wisconsin-Milwaukee, Center for Mathematics and Science Education Research.

This paper reports on the evaluation of the Milwaukee Urban Systemic Initiative, which was supported by the National Science Foundation. The project focused on collaborative vision-setting, high standards and performance assessments, narrowing achievement gaps, developing high-content, inquiry-based technology rich curriculum, and breaking down boundaries between community and classrooms. This paper presents the results of formative surveys (prior to project and two years after participation) of teachers in schools that participated in the initial phase of the project. Science and mathematics teachers at the elementary, middle, and high school levels responded to the survey. For science teachers who participated in the project, the results included the following highlights:

  • They increased the use of student-generated experiments for elementary, middle, and high-school levels.

  • Approximately two-thirds of the elementary teachers reported using the science kits and guides developed by the District.

  • Teacher satisfaction with time available for science increased at all levels.

  • Teachers at all levels indicated a slight increase in the use of open-ended questions and performance-based assessment.

  • Teachers at all levels indicated a slight increase in the usage of computers for science.

  • There was a substantial increase of teachers at all levels in their familiarity with the NSES.

  • Middle- and elementary-level teachers indicated a decrease in belief that it is important to emphasize broad coverage of many scientific concepts and principles, while high-school teachers increased in this belief.

  • Science teachers at all levels indicated some increasing confidence that all students would be able to meet the new School Board graduation policy for science.

  • As students get older, teachers expressed less confidence that an inadequate science background can be overcome by good science teaching.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Huinker, D., and Pearson, G. (1997, October). The Journey Begins: First Year Activities of the MUSI Mathematics/Science Resource Teachers. A Report on the Milwaukee Public Schools. Milwaukee, WI: University of Wisconsin-Milwaukee, Center for Mathematics and Science Education Research.

This report contributes data to the formative evaluation of the National Science Foundation’s Milwaukee Urban Systemic Initiative (MUSI) concerning its first year of implementation. The main strategy of the MUSI was to develop a cadre of mathematics/science resource teachers who each served two schools in order to build capacity for change at the classroom, school, and district levels. The report does not describe much about the structure of the MUSI, nor the way that the resource teachers were selected and trained. The report primarily consists of summaries, compilations, and reflections about the activities that the resource teachers engaged in during the first year of the MUSI (1996-1997). The data sources for the report were three qualitative reports that were submitted by the resource teachers about their strategies and activities. The researchers took the resource teachers’ reports and organized the data into themes, which included how the resource teachers assessed the needs of their schools, developed strategies to meet the needs of their schools, provided professional development in their sites, contributed to a district community of learners, and worked with principals. The authors conclude that, through their self-reports, the resource teachers demonstrate that they have been actively involved in improving mathematics and science teaching and learning in a variety of communities, including the classroom, school, and district. The variety of professional development activities offered by the resource teachers reflected many aspects of what the standards call for. They included formal staff in-service, grade-level mentoring, facilitating the development of school action plans, assisting teachers to prepare students for high-stakes testing, participating with teachers in other professional development activities and then helping them reflect and discuss implications for instructional practice, and arranging teachers to visit and observe each others’ practice.

Huinker, D., Pearson, G., Posnanski, T., Coan, C., and Porter, C. (1998, August). First Year Site Visits to Milwaukee Urban System Initiative Schools. A Report on the Milwaukee Public Schools. Milwaukee Urban Systemic Initiative. Milwaukee, WI: University of Wisconsin-Milwaukee, Center for Mathematics and Science Education Research .

Humphrey, D.C., Anderson, L., Marsh, J., Marder, C., and Shields, P.M. (1997). Eisenhower Mathematics and Science State Curriculum Frameworks Projects: Final Evaluation Report.Washington, DC: U.S. Department of Education.

The purpose of this study was to summarize findings from the evaluation of 16 projects funded by the U.S. Department of Education to develop curriculum frameworks in mathematics and science for grades K-12. This report provides useful information for evaluating the impact of the National Science Education Standards on state curriculum frameworks. The methodology of the study included:

  • Review of state curriculum frameworks project documents, each year during the four years of the study.

  • Review of state data from a variety of secondary sources.

  • Telephone interviews with project directors, state officials, SSI directors, Eisenhower state coordinators, and key participants.

  • Use of a panel of educational experts to evaluate the quality of the framework documents.

  • Site visits to a sample of eight of the 16 states, including interviews with state officials, teachers, and district officials in a sample of two to three districts in each state.

  • Use of data from other related studies conducted by others, including the evaluation of NSF’s Statewide Systemic Initiatives, AAAS Project 2061, and the Pew Network for Standards-Based Reform and the analysis of curriculum frameworks by CCSSO.

The findings of the project include:

  • Fifteen of the 16 states completed curriculum frameworks as a result of their grants. (However, because 48 states have developed or are developing standards documents, it seems likely that the 16 states that received Eisenhower grants would have developed curriculum frameworks without the grants.)

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
  • Four states designed, piloted, and evaluated model professional development programs.

  • Nine of the states were developing new certification and/or new recertification requirements.

  • Six states used the frameworks in the development of new teacher licensure programs.

  • The states followed similar processes in developing the frameworks.

  • The projects used a variety of strategies in development of model professional development programs, model guidelines for teacher education and certification, and criteria for teacher recertification.

  • The state frameworks expanded beyond a basic-skills emphasis to focus more on higher-order skills.

  • Some state frameworks omitted some of the major categories of the national standards, suffered from a lack of usability, or failed to convey adequately how equity can be achieved.

  • Most frameworks presented sample activities or vignettes that often were either inconsistent with national standards or inadequately annotated and explained

  • Frameworks tended to address classroom assessment, but not large-scale assessment.

  • Fifteen of the 16 states were planning, developing, piloting, or implementing new statewide assessment systems. In 10 of the states, the project’s framework played a central role in the assessment development process.

  • For effective use of frameworks and standards, districts engaged the standards documents from a foundation of previous reform activity and as part of a whole-school change strategy that promoted collegial and professional school culture and provided extensive and intensive professional development opportunities that focused on standards.

  • At the district level, schools and teachers adapt the standards rather than adopt them. Districts tend to emphasis content over pedagogy. Teachers were struggling with the sometimes conflicting purposes of assessment. Districts were only beginning to explore ways to build professional development into the structure and organization of the school day.

  • Much more work is needed before curriculum frameworks will be well used in a majority of districts and schools. Districts and individual schools need more time and resources to translate the state frameworks into local curriculum guidance.

Humphrey, D.C., and Carver, R. (1998). A Case Study of New York’s SSI (NYSSI), 1993-1997. In A.A. Zucker and P. M. Shields (Eds.), SSI Case Studies, Cohort 3: Arkansas and New York. Menlo Park, CA: SRI International.

This case study of the New York SSI examines the funded years from 1993 through 1997. The goal of this SSI was to change entire schools and the teaching practice of every educator therein. Twelve Research and Demonstration (R&D) schools in New York’s six largest urban districts were chosen for concentrated reform effort. Strategies for reform targeted two levels. First, there was a state-level focus on policy alignment, including development of high standards, new assessments to measure student progress toward meeting those standards, and an incentive system. The second level involved schools as the unit of change, driven by improvements in mathematics, science, and technology education.

The impact of these many reforms on students was examined primarily through the results of the statewide testing system. This limited assessment reveals that students in R&D schools made larger gains on test scores compared to the rest of the state during the same time period, although the differences were only modest in favor of the R&D schools. Likewise, modest progress toward change in teaching practice is found when measuring the amount of change in teacher practice, though teachers varied in their understanding and implementation of the inquiry-based strategies. The goal of transforming whole schools proved more challenging and the 12 R&D schools varied greatly in their progress toward reform. There was also not much success at influencing the other educational institutions in the R&D schools’ districts, which has been attributed primarily to the frequent changes in leadership in those districts.

Though critical of the New York SSI for the lack of completeness and rigor in their “research and demonstration” in the R&D schools, the authors do point out that had these schools focused more on rigorous research and development than on demonstration, more significant results would likely have emerged from this state’s unique SSI reform strategy.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Given the apparently low levels of implementation of standards-based policy and practice in the New York SSI, it would be difficult to attribute either gains or lack of gains in student achievement to the influence of standards.

Humphrey, D.C., Shields, P.M., and Anderson, L. (1996). Evaluation of the Dwight D. Eisenhower Mathematics and Science State Curriculum Frameworks Projects: First Interim Report, 1996.Menlo Park, CA, and Washington, DC: SRI International and Policy Studies Associates.

This interim report (Part I) summarizes progress of 16 states (including the District of Columbia) that received funding from the U.S. Department of Education to develop curriculum frameworks in mathematics and science and to develop new approaches to teacher education, certification, recertification, and professional development. Phase I of the research study, included in this report, examined the organization and development of the projects. Researchers reviewed original proposals, continuation proposals, draft and completed framework documents, and available evaluation documents; reviewed state data from a variety of secondary sources; and conducted telephone interviews with project directors, state officials, and other key individuals. The researchers also examined data collected by a national evaluation of NSF’s Statewide Systemic Initiatives and an analysis of curriculum frameworks by the Council of Chief State School Officers.

The report includes the following findings: (1) there is a similar vision across frameworks and an apparent consensus that national standards should form the basis for high-quality mathematics and science education, (2) teachers are a key audience for all frameworks, (3) twenty-two drafts or final versions of curriculum frameworks have been completed out of the 28 proposed by the 16 states, (4) it takes more than three years to develop a curriculum framework, (5) states varied in the development of secondary products such as model guidelines for teacher education and certification, criteria for teacher recertification, and model professional development programs, (6) all projects involved college and university faculty and teachers and administrators from public and private schools in designing the frameworks, and (7) states differed in approval requirements (i.e., formal approval by state boards of education). Three issues emerged in the states as they developed their frameworks: (1) the new curriculum frameworks generally avoid long lists of discrete skills and tend to give more general guidance on content, pedagogy, and school and classroom environment, (2) technology is treated in varied ways in the state frameworks—both as a tool for learning (i.e., a computer) and as a subject (like engineering) to learn, and (3) most frameworks encourage teachers to integrate the disciplines in their lessons, perhaps because integration fits well with the thematic approaches and constructivist learning often advocated by the frameworks.

Humphrey, D.C. and Wilson, C.L. (1998). A Case Study of Arkansas’ SSI (AR SSI), 1993-1997. In A. A. Zucker and P. M. Shields (Eds.), SSI Case Studies, Cohort 3: Arkansas and New York. Menlo Park, CA: SRI International.

This report describes the Arkansas State Systemic Initiative (SSI), which was supported in part by the National Science Foundation. The Arkansas SSI focused its efforts on intensive professional development through a K-4 Integrated Math/Science Crusade and a 6-12 Science Crusade. The project also addressed leadership development and policy revision in teacher preparation and certification. By the last year of the project, 35 percent of the K-4 teachers had participated in the Integrated Crusade and 22 percent of 5-12 science teachers had participated in the Science Crusade and more than 4,000 administrators had participated in leadership training. However, no achievement data were available from the state to evaluate the impact of the project on student learning. Statewide test results, course-taking patterns, and other indicators documented strong gains in the reform of science education during the project. The achievement gap between whites and minority students, however, remained high.

Johnson, J., and Duffett, A. (1999, September). Standards and Accountability: Where the Public Stands. A report from Public Agenda for the 1999 National Education Summit, September 30, 1999. New York: Public Agenda.

Kahle, J.B. and Kelly, M.K. (2001a). Equity in reform: Case studies of five middle schools involved in systemic reform. Journal of Women and Minorities in Science and Engineering.7, 79-96.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Kahle, J.B. and Kelly, M.K. (2001b). Science Teacher Professional Development: A Researcher’s Perspective. In J. Rhoton and P. Bowers (Eds.), Professional Development Planning and Design, pp. 101-113. Issues in Science Education. Arlington, VA: National Science Teachers Association.

This article describes the professional development program of the Ohio Statewide Systemic Initiative, including some research findings of impacts on classroom practice and student achievement. The program focused mainly on middle school teachers.

The professional development program was originally designed and delivered as a six-week institute on a university campus. Two-week to four-week programs, delivery spread throughout one or more summers and academic years, and delivery at local school sites emerged in later years in order to reach more teachers.

Findings on teaching practice included that teachers who participated in the SSI professional development reported increases in reform-related teaching practices in the first year following the professional development (effect size approximately 0.8 in mathematics, and approximately 0.4 in science). These reported practices were sustained in the second and third years following the professional development. Items reflected a range of teaching practices, such as having students work in small groups, doing inquiry activities, making conjectures, and exploring possible methods to solve a problem.

One study of student achievement controlled for student demographics by using matched comparison classrooms within the same school. Disaggregated data showed white and African American males and females of SSI teachers scoring higher on the SSI science achievement test than similar students in the matched classrooms taught by non-SSI teachers. A second study controlled for teacher volunteer effect by using a comparison group of teacher applicants to the SSI professional development program who were not treated due to limited admission. This study compared predicted scores on the SSI mathematics and science achievement tests. For all subgroups of white or African American males or females, students of the SSI teachers had higher predicted achievement scores than comparable students of non-SSI teachers.

Kahle, J.B., Meece, J., and Scantlebury, K. (2000, November). Urban African-American Middle School Science Students: Does Standards-Based Teaching Make a Difference?Journal of Research in Science Teaching.37(9), 1019-1041.

Kahle, Meece, and Scantlebury examine the influence of standards-based teaching practices on the achievement of middle-school students. Students whose teachers participated in the professional development component of Ohio’s Systemic Statewide Initiative (SSI) were matched with classes of teachers who had not participated. Analyses indicate that teachers who frequently used standards-based teaching practices positively influenced urban African American students’ science achievement and attitudes. The findings support the efficacy of high-quality professional development to change teaching practices and to enhance student learning. Ohio’s SSI used professional development to address teachers’ lack of content knowledge and use of standards-based teaching practices in science and mathematics. The goals of Ohio’s SSI professional development programs were to provide content information taught by inquiry, and to develop a network of support for the sustained professional development of teachers. These programs were clearly focused on enhancing the achievement of all students through changed teaching practice. This study also showed that 15 percent of the variation in students’ science achievement scores was due to teacher differences. This between-teacher variation was largely explained by two factors: (1) teacher gender, and (2) the use of standards-based teaching practices. There was the effect of teacher gender on science achievement. There was a higher level of science achievement in female teachers’ classes, compared to that in male teachers’ classes. Especially, teachers’ involvement in the SSI’s professional development was positively related to the reported use of standards-based teaching practices in the classroom. However, teacher participation in the SSI’s professional development was not as strong a predictor of achievement as was the frequency of use standards-based teaching practices.

Kahle, J.B. and Rogg, S.R. (1998). A Pocket Panorama of the Landscape Study, 1997. Oxford, OH: Miami University.

Kahle, J.B., Tobin, K.G., and Rogg, S.R. (1997). Impressions of Reform in Ohio Schools. Source unknown.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Kannapel, P.J., Aagaard, L., Coe, P., and Reeves, C.A. (2001). The Impact of Standards and Accountability on Teaching and Learning in Kentucky. In S.H. Fuhrman (Ed.), From the Capitol to the Classroom: Standards-Based Reform in the States, The One Hundredth Yearbook of the National Society for the Study of Education, Part 2, pp. 242-262. Chicago: University of Chicago Press.

This chapter highlights the key findings from an extensive study that examined the effects on classroom teaching and learning in four rural Kentucky school districts that resulted from the implementation of the Kentucky Education Reform Act (KERA). The chapter opens with a brief history of KERA (a standards-based approach to learning) and a description of the research study. The ten-year (1990-2000) study was qualitative in nature, collecting and analyzing data from more than 1,200 interviews with state policy makers, school administrators, teachers, school board members, parents, students, and community members; more than 500 hours of observations in classrooms, professional development activities, school district meetings and parent events, and Kentucky Board of Education meetings; and regular review of documents such as local newspapers, school improvement plans, assessment results, lesson plans, and school board and school council minutes. Analysis of the data obtained from the six schools that were studied was supplemented with findings from broader studies of KERA. The analytical methodologies were not reported.

The authors report that the KERA standards-based reform efforts have been difficult for Kentucky teachers but some changes have occurred. In the schools analyzed for this study, the authors observed an increased emphasis on writing in all subjects and attempts at other instructional practices such as group work, hands-on experiences, and analysis of real life problems. In addition to parents reporting improved learning, student scores on KIRIS (the Kentucky Instructional Results Information System) increased over the decade. KIRIS, implemented from 1991 to 1998, was produced to steer instruction, assess progress on KERA goals, and hold schools accountable. Student scores also increased to a lesser extent on other achievement measures such as the National Assessment of Educational Progress (NAEP) and the Comprehensive Test of Basic Skills (CTBS). The authors identified one elementary school that promoted and embraced the attitude that all students could achieve and was the only study school to meet its KIRIS goal every biennium. Besides mentioning a formal system for regularly assessing individual student progress, the authors did not report on specific instructional and assessment practices implemented in this school. In contrast, other study schools made curriculum changes based on the Core Content for Assessment in an effort to raise their overall KIRIS test score. The Core Content for Assessment is a document released in 1996 identifying content assessed under KERA goals regarding basic subject matter. The authors discuss focusing future resources on the development of professional support and high-quality measures for classroom assessment. Upon implementation of KERA, classroom assessments changed as a result of instructional practices geared toward preparing students for KIRIS. Classroom assessments did not change to reflect evaluating individual student performances, as was the original intent of KERA. Teachers in the study indicated that they had insufficient time to cover all subjects. They also expressed insufficient time and resources to develop instructional practices needed to reach diverse learners. Instead, teachers reported that they focused on subjects emphasized on KIRIS.

The description of KERA in this chapter illustrates the influence that standards have had in Kentucky on assessment, accountability, and goals for learning at the state level. Available evidence indicates that the implementation of KERA influenced and altered teaching practices and improved student achievement during the first decade but has yet to attain high achievement for all students.

Keating, P. (2000, June). Education Standards for Teaching and Learning: A Bibliography.Washington, DC: Office of Educational Research and Improvement.

Keiffer-Barone, S., McCollum, T., Rowe, J., and Blackwell, B. (1999, March). Science Curriculum Development as Teacher Development: A Descriptive Study of Urban School Change. Paper presented at the Annual Meeting of the National Association for Research in Science Teaching. Available at: http://www.narst.org. [August 8, 2002].

This article investigates the process of standards-based curriculum development by a group of teachers in a high-minority urban district involved in an NSF-supported Urban Systemic Initiative. Participant observation,

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

semi-structured interviews, document analysis, and written surveys were used to evaluate the use of curriculum development as professional development. Interviews were conducted with 10 science teachers involved in the project, and also with the lead science teacher responsible for K-12 curriculum revision. Draft documents of the curricula, in-service plans, and curriculum writing meeting notes from a four-year period were reviewed. Likert scale surveys were sent to science teachers involved in the development, asking whether they felt their involvement in this initiative increased their knowledge in five areas: professional knowledge, collegiality, instruction, curriculum, and professional development. Surveys also included open response items seeking to capture teachers’ perceptions of their learning, their views of the successes and failures of the initiatives, and their conceptions of the curriculum. Of the 36 surveys sent out, 67 percent were returned. Chi-square tests were run on survey results to determine in which areas participants felt that professional development had occurred, and whether there was an interaction between the depth of involvement in the curriculum initiative and teacher learning. Three patterns emerged: (1) in writing the curriculum, teachers came to conceptualize curricula as including pedagogy as well as content; (2) teachers viewed the process of curriculum development as ongoing; and (3) teachers considered the process to be good professional development due to its collaborative and reflective nature. Teachers’ understanding of district, state, and national reform initiatives increased, indicating that involving teachers in curriculum development can be an effective vehicle for professional development. While teachers claimed that the process of curriculum development changed the way they thought about teaching, there seemed to be no effect on instruction, judging from teachers’ responses to a survey administered at the end of the study. The authors note that they have little evidence suggesting that inquiry- or laboratory-based instruction increased as a result of the initiative.

Keys, C.W., and Bryan, L.A. (2001, August). Co-Constructing Inquiry-Based Science with Teachers: Essential Research for Lasting Reform. Journal of Research in Science Teaching.38(6), 631-645.

Kim, J.J., Crasco, L.M., Blank, R.K, and Smithson, J. (2001, April). Survey Results of Urban School Classroom Practices in Mathematics and Science: 2000 Report. An Evaluative Study of National Science Foundation’s Urban Systemic Initiatives.Study Monograph No. 3. Washington, DC: Council of Chief State School Officers.

This report describes the results of surveys completed by elementary and middle school teachers in eight Urban Systemic Initiative (USI) sites in 1999 and 2000. The survey instrument used, called the Survey of Enacted Curriculum, is a sophisticated self-report survey instrument developed at the University of Madison-Wisconsin by Andrew Porter and John Smithson. The survey asked teachers about their curriculum coverage, classroom practices, and professional development experiences. The response rate reported in 1999 was 61 percent. The authors do not report the response rate for 2000, although they do say it was better than in 1999. The report presents simple descriptive statistics (mean and standard deviation) of both survey scales and the individual items that make up each scale for both elementary and middle school teachers. Many of the results compare the reports of teachers with low and high levels of professional development (as defined as greater or less than 16 hours). The methodology used by the authors seems thorough and appropriate. Among the findings that the authors highlight are that 80 to 90 percent of the USI teachers were actively involved in professional development, which they reported was focused on content standards, in-depth study of content, curriculum implementation, multiple strategies for assessment, and new methods of teaching. Teachers also reported that the professional development they received was being used and applied in the classroom. In science, elementary teachers with high professional development report greater use of multiple assessments than do teachers with low professional development. Finally, science teachers reported that state and district standards and frameworks influenced their curriculum.

Kim, J.J., Crasco, L.M., Smith, R.B., Johnson, G., Karantonis, A., and Leavitt, D.J. (2001, April). Academic Excellence for All Urban Students: Their Accomplishments in Science and Mathematics. Urban Systemic Initiatives. Systemic Research, Norwood, MA.

Kim, Crasco, Smith, Johnson, Karantonis, and Leavitt compared achievement of minority and European American students in science and mathematics. Kim et al. present preliminary findings from an evaluative study

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

of NSF’s Urban Systemic Initiative (USI) program among 22 large urban school districts. This report presents evidence of noteworthy reduction of students’ achievement gaps among racial/ethnic groups, with the greatest gains seen in school districts that have participated in the USI program for the longest period of time. For example, at 14 urban sites the investigators compared the achievement scores of European Americans and the largest ethnic group over two successive years. In five predominantly Hispanic sites, there was a reduction in the average achievement gap of 8 percent in mathematics and 5.6 percent in science. In nine predominantly African American sites there was an increase in the achievement gap of 1 percent in math and 0.3 percent in science. These study findings indicate that implementation of the drivers of systemic reform has an important influence in successfully reforming and restructuring school district infrastructure within each city. NSF’s six drivers of systemic reform are: (1) standards-based curriculum, instruction, and assessment, (2) policy support for high-quality learning and teaching, (3) convergence of educational resources, (4) partnerships and leadership: broad-based support, (5) measures of effectiveness focused on student outcomes, and (6) achievement of all students, including racially and ethnically minority students. The authors argue that there is evidence that urban districts are developing the infrastructure to sustain achievement gains for all ethnically diverse students—policies that encourage enrollment in gate-keeping and higher-level mathematics and science courses, strengthened professional development programs, new ways of managing partnerships and resources, and data-driven accountability systems.

Kirst, M.W., Anhalt, B., and Marine, R. (1997). Politics of Science Education Standards. The Elementary School Journal.97(4), 315-328.

Kirst, M.W. and Bird, R.L. (1997). The Politics of Developing and Sustaining Mathematics and Science Curriculum Content Standards. Advances in Educational Administration.5, 107-132.

Kirwan, W.E. (1994, April). Reform and National Standards: Implications for the Undergraduate Education and Professional Development of Science and Mathematics Teachers. In Scientists, Educators, and National Standards: Action at the Local Level, Sigma Xi Forum Proceedings, pp. 51-63Sigma XI, The Scientific Research Society, Research Triangle Park, NC, April 14-15.

In this article, Kirwan comments on the impact of national standards on the reform of science and mathematics education. He points out that early reform efforts failed to achieve lasting change, in large part because of the lack of involvement of people at the local level in the reform process. The article cites research that indicates that science and mathematics literacy is on the decline or at best is not changing. A major concern is that while national surveys show that people recognize that our nation needs to improve science and mathematics education, when parents and administrators were asked how local schools were doing they gave high ratings. His point is that people do not see the need for local change. Another reason for the failure of reform efforts is that they seek universal solutions (instructional materials, teaching materials, teaching techniques) for complex, local problems. A third reason for the failure of the reform movement is the lack of attention given to ensuring that teachers have the support, knowledge, and skills necessary to make the reforms work.

Klein, S., Hamilton, L., McCaffrey, D., Stecher, B., Robyn, A., and Burroughs, D. (2000). Teaching Practices and Student Achievement: Report of the First-Year Findings from the “Mosaic” Study of Systemic Initiatives in Mathematics and Science.Santa Monica, CA: Rand Education.

Klein, Hamilton, McCaffrey, Stecher, Robyn, and Burroughs reported on the first-year results of the Mosaic study, which looked for relationships between student achievement measures and teachers’ responses to questionnaires concerning their teaching practices. The authors found that after controlling for student background variables, the reform practices are associated with improved student achievement in both mathematics and science. The teachers’ use of reform practices appeared to be positively related to student achievement at most sites, but the effects were quite small (about 0.1 SD effect size) and rarely reached statistical significance. This relationship was somewhat stronger when achievement was measured with open-response tests than with multiple-choice tests. By contrast, the use of traditional practices was generally negatively related to student

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

achievement, particularly in mathematics, but again the relationships were weak. The foregoing trends held for both mathematics and science and they were generally consistent across the six sites, i.e., in most cases, the pooled results across sites were not driven by the data at one or two sites. However, as with most large-scale field studies, there are many factors that may have artificially increased or decreased the observed effect sizes. Teachers may not always have provided accurate reports of the extent to which they used various instructional practices, and some may not have become proficient in the use of the reform practices at the time the data were collected. The tests used to measure student achievement may not have been aligned especially well with the reform curriculum. Students whose teachers use the reform practices relatively frequently may differ from other students for reasons that are unrelated to the use of the reform practices per se. Finally, student may not have to experience the reform practices for more than one year in order for these practices to have a significant impact on student achievement. Nevertheless, the consistency of results across sites, despite the differences among sites, is encouraging.

Klentschy, M., Garrison, L., and Maia-Amaral, O. (1999). Valle Imperial Project in Science (VIPS) Four-Year Comparison of Student Achievement Data 1995-1999. Journal of Research in Science Teaching.

Klentschy, Garrison, and Maia-Amaral examined the relationship of inquiry-based materials on standardized student achievement scores compared to a more traditional textbook approach from the Valle Imperial Project in Science (VIPS), which provided teachers in California’s Imperial Valley with professional development and inquiry-based instructional units in science. The authors argue that a hands-on science program positively affected student science achievement scores. This study applies a one-way and two-way analysis of variance, post hoc test by Tukey’s pairwise comparisons, and a linear regression analysis. The results are: (1) there are distinct differences between students who participated in the district science program during the 1998-99 school year and had been in attendance in the El Centro School district continuously for the prior four years; (2) there is a strong correlation between achievement scores on the science section of the Stanford Achievement Test and the number of years of participation in the inquiry-based science program, the VIPS. In each grade level, fourth and sixth, there are significant differences from year 0 to year 4. There is a positive correlation between the two variables, science achievement and grade level, with r = .9909 for grade 4 and r = .9934 for grade 6; and (3) the longer they were exposed to the inquiry-based science program, the higher their achievement scores were in science. Thus, the authors suggest that teacher professional development, the efforts of implementation, and inquiry-based science have the potential to get the success in student achievement. There is a correlation between the number of years of participation in a kit-based program of science education and the strength of their scores on a norm-referenced test.

Kouba, V.L., Champagne, A.B. et al. (1998). Literacy in the National Science and Mathematics Standards: Communication and Reasoning.Albany, NY: National Research Center on English Learning and Achievement.

Kumar, D. and Berlin, D. (1998, June). A Study of STS Themes in State Science Curriculum Frameworks in the United States. Journal of Science Education & Technology.7(2), 181-197.

Kwartler, T.J. (1993). PMEEP: Does It Creep Into the Worldview of Participants?Microethnography Inquiry in Progress.February 19.

Laguarda, K., Goldstein, D.S., Adelman, N.E., and Zucker, A.A. (1998, March). Assessing the SSIs’ Impacts on Student Achievement: An Imperfect Science.Menlo Park, CA: SRI International.

Laguarda, Goldstein, Adelman, and Zucker argue that systemic reform such as Statewide Systemic Initiatives (SSI) can be a feasible strategy for raising student achievement and help to close the gap in performance for historically underserved populations. They found seven SSIs for which some student achievement data were available. In general, these data showed small advantages for students whose teachers were participating in SSI-sponsored programs. Laguarda et al. caution, however, that in the space of only a few years, the number of students affected and the size of the gains are not likely to be large. There is limited evidence of SSI impact on

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

student achievement. The authors found the following: (1) The amount of data is limited in most states. The evidence that student achievement has risen across all SSI schools in limited in most states. The data reported here were gathered in only one round of testing. In some cases, the SSI collected student achievement data over only one year; in other cases, the SSI chose to test different grade levels or carry our different analyses each year. (2) Evidence of gains in student achievement is uneven or contradictory. One interpretation of results is that they are an effect of self-selection bias, rather than any intervention by the SSI, because those high schools willing to seek out SSI services might be those more likely to score well on Kentucky’s assessment in the first place. (3) Effect sizes are small. Because the SSI did not assess effect size and because they did not publish information about the variance of individual scores, it is difficult to be sure. The reason that the evidence of SSI impact on student outcomes is so limited and so uneven lies in the fact that gathering such evidence is extremely difficult and expensive to do. (4) There is limited choice of assessment instruments. Evidence of student achievement that can be linked in a credible way to SSI activities is not generated automatically by established assessment systems. Finally, the authors argue that it is important to have multiple indicators that could include attitudes toward the material, and students’ use of more sophisticated problem-solving techniques.

La Marca, P.M., Redfield, D., and Winter, P.C. (2000). State Standards and State Assessment Systems: A Guide to Alignment.Washington, DC: Council of Chief State School Officers.

The State Collaborative on Assessment and Student Standards, Comprehensive Assessment Systems for IASA Title I Alignment Study Group, one of the Council of Chief State School Officers’ collaboratives, developed this guide to assist states and districts in aligning their assessment systems with their content and performance standards. The group drew upon existing research and literature to produce this primer on alignment. Beginning with definitions of standards, assessments, and alignment, the guide lays out working assumptions on alignment, including the need to incorporate curriculum, instructional practices, the connection between the state and local agencies, and the high visibility of standards and assessments. It cited also the importance of alignment being an ongoing process. Finally, the sixth assumption states that valid and meaningful data-based decision making is dependent on the degree to which standards and assessments are aligned. The guide identifies a number of factors that need to be considered to determine the extent to which the educational system components are aligned. These factors include content match, depth match, emphasis, performance match, and accessibility. Five approaches are identified to the study of alignment. These include coding assessment activities and standards using common criteria; coding these documents independently using a common framework; classifying items by content areas; and expert examination of critical features of the assessment and standards. Inclusion of the study guide recognizes that system alignment is a dynamic process and that the degree of alignment will depend in part on the purposes of an assessment system. The appendix includes a standards-assessment alignment checklist. For the time at which the guide was written, it provides much of the current thinking on alignment. It clearly recognizes that very little research on alignment had been done. The guide does not address any specific content area and is generally applicable to thinking about the coherence of assessments and standards in any content area.

Larson, K., Guidera, A.R., and Smith, N. (1998, May). The Formula for Success: A Business Leader’s Guide to Supporting Math and Science Achievement.Washington, DC: Business Coalition for Education Reform, National Alliance of Business.

Lederman, N.G., and Niess, M.L. (2000, March). Problem Solving and Solving Problems: Inquiry About Inquiry. School Science & Mathematics.100(3), 113-116.

Lee, O. (2001, May). Preface: Culture and Language in Science Education: What Do We Know and What Do We Need to Know?Journal of Research in Science Teaching.38(5), 499-501.

Lee discusses why the practices encouraged by the Standards are not likely to reduce the achievement gap between students of different races, cultures, or social classes. There are potential difficulties and conflicts when culturally based approaches to instruction and assessment are put into practice in the context of high-stakes

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

testing and accountability. In order to overcome these difficulties, Lee argues that the research effort of science content, learning, teaching, and assessment needs to be actively developed in the consideration of diversity and equity at the same time. As diverse students in languages and cultures bring to the science classroom various ways of knowing, talking, and interacting that are sometimes different from those in the mainstream, it is a big challenge for teachers to understand all students’ diversity in language and cultures for successful science learning and teaching. Thus, we need to give serious consideration to the research on culturally and linguistically diverse students in science education. To achieve equitable outcomes with diverse students, for practice, teachers need to have both knowledge of science and understanding of the students’ language and cultures. It is not easy for teachers to integrate content-specific science teaching and students’ language and cultures in ways that are meaningful and relevant for their students. Equitable instruction and assessment practices for diverse students involve consideration of their cultural and linguistic experiences in preparing them to function competently in the institutions of power as well as in their homes and communities. Finally, the author notes that research on language and culture in science education could inform practitioners and policy makers who seek to provide equitable educational opportunities for all students, including those from diverse languages and cultures.

Lee, O., Eichinger, D., Anderson, C.W., Berkheimer, G.D., and Blakeslee, T.D. (1993). Changing Middle School Students’ Conceptions of Matter and Molecules. Journal of Research in Science Teaching.30(3), 249-270.

Lee, Eichinger, Anderson, Berkheimer, and Blakeslee found that teachers using the Matter and Molecules curriculum were able to increase their students’ understanding of physical changes in matter and of molecular explanations for those changes. The study involved 15 sixth-grade science classes taught by 12 teachers in each of two successive years. Every sixth-grade teacher in an urban school district participated in the study (16 teachers in Year 1, and 14 teachers in Year 2). The teachers received only one day of in-service training before teaching the revised unit. The students acquired molecular conceptions concerning the nature, arrangement, and motion of molecules as well as macroscopic conceptions concerning the nature of matter and its physical changes. Even under these less than ideal conditions, about 50 percent of the students achieved understanding of the scientific conceptions of physical changes in matter of molecular explanations. Lee et al. argue that teaching materials based on conceptual change research can greatly enhance teachers’ effectiveness even under the less than ideal conditions referred to above. Conversely, even the best-prepared teachers face a long and difficult struggle if they wish to teach for meaningful understanding using currently available commercial materials. The results also showed that urban sixth-grade students taught by the revised unit in Year 2 performed significantly better than the students taught by the original commercial curriculum unit in Year 1 in nine of the 10 conceptual categories. The actual percentage of the students understanding key concepts approximately doubled (from 25 percent to 49 percent) when performance of students using Matter and Molecules was compared with the performance taught by the same teachers using a commercial unit that taught the same concepts.

Lee, V.E., Smith, J.B., and Croninger, R.G. (1995). Another Look at High School Restructuring. More Evidence That It Improves Student Achievement and More Insight into Why. Center on Organization and Restructuring of Schools, Madison, WI. Issues in Restructuring Schools.9, 1-10.

This study reports that the achievement gains are positively associated with School Restructuring efforts. Lee, Smith, and Croninger found evidence that students in restructuring schools continue to show significantly larger academic gains in most mathematics and science than students in other types of schools. The authors point out that restructured schools based on the Organic model, in which teachers have greater authority over instruction and curriculum, affect student learning. The Organic model is characterized by having (1) a common academic curriculum, (2) academic press, (3) authentic instruction, and (4) a collective sense of responsibility. The organic model views teaching and learning as processes that cannot really be controlled through standardized procedures directed from central authorities. Findings indicate that the presence of organic school-organization characteristics explained much of the improvement in student learning and that the restructuring effects on student learning increased during the later years of high school.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Lehrer, R. and Schauble, L. (2002). Investigating Real Data in the Classroom: Expanding Children’s Understanding of Math and Science.New York: Teachers College Press.

Lehrer and Schauble report the five years of significant efforts of a working relationship between researchers and elementary-school teacher partners to foster and study the development of students’ model-based reasoning in mathematics and science. The elementary teachers have inquired into their students’ classroom inquiry, investigating how children transform their experiences into data, develop techniques for representing and displaying their data, and search for patterns and explanations in their data and how teachers can work with children to improve their knowledge and practice. In this study, professional development was a major, ongoing part of their effort, as they think that improving students’ learning is only possible by improving teaching. The professional development program included the development of interrelated forms of knowledge, including knowledge of the domain, student thinking, and appropriate pedagogical strategies. A fundamental aspect of the professional development was the development and influence of teacher community. Over the five years of this program, this study conducted many classroom investigations of student thinking in the context of instruction in mathematics (e.g., data modeling, classification, distribution, similarity) and science (e.g., growth, diversity, motion, density). Lehrer and Schauble document that using the first year as a “control,” average student achievement increased substantially at every grade level. There are impressive gains in achievement among their participating students, both cross-sectionally (i.e., grade 1 students achieve more and more each year) and longitudinally (i.e., students who remain in the program across years make significant gains in traditional and nontraditional forms of mathematics).

Linn, R.L., and Herman, J.L. (1997). Standards-Led Assessment: Technical and Policy Issues in Measuring School and Student Progress.CSE Technical Report 426. Los Angeles: CRESST.

This technical report discusses in some detail what is meant by standards-led assessments and how this form of assessment differs from the more traditional norm-referenced tests. The authors recognize the importance of standards-led assessments because of the increasing adoption of tough new standards by states across the country. This new form of assessment typically engages students in problem-solving and complex tasks. As with other forms of assessment, standards-led assessments need to be valid and reliable, but they also need to be aligned with existing standards. There are a number of challenges facing standards-led assessment systems. Not the least of these are building state and local consensus, providing strong standards, achieving alignment, assuring accurate measures, setting the stakes, building local capacity, and others. This report focuses on standards-led assessment in general and does not directly reference science or the NSES. To the extent that state and district standards are influenced by the NSES, the report is applicable to science and the relation of the NSES to assessment. Many of the references used in this report pertain to technical and practical assessment issues. The report clearly discusses in some detail the critical issues underpinning the implementation of standards-led assessment and, as such, is relevant to assessment in science.

Llamas, V.J. (1999, January). UCAN: A Four-State Rural Systemic Initiative. Year Three Performance Effectiveness Review.Las Vegas, NM: New Mexico Highlands University.

Llamas reports data indicating that the Rural Systemic Initiative (RSI) in Utah, Colorado, Arizona, and New Mexico (UCAN) has marginal positive effects on student performance and achievement. The author argues that standards-based teaching is an ultimately explicit part of the package to enhance student achievement and performance. Overall, the number of students taking exams increased for most math/science exams, with increases ranging from +4 tests per 1000 in Calculus AB to +1 tests taken in Chemistry, Computer Science A, and Calculus BC. Arizona Stanford 9 standardized test results by grade also tell that there are positive gains for UCAN schools. Participating schools demonstrated a greater gain in the percentage of students scoring at or above the 50th percentile rank in all but fifth grade. UCAN-eligible (non-targeted) schools also increased their percentage (except for grade 8) but at a lesser rate than UCAN-targeted schools. However, the percentage of students scoring at or above the 50th percentile in UCAN-targeted schools is still below the national norm of 50 percent. This study reports that UCAN supports systemic reform of mathematics, technology, and science education for rural students, focusing on schools with high enrollments of American Indian and Hispanic

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

students. During Year 3 (September 1997–August 1998), UCAN worked with 124 focal schools enrolling 36,656 students with 43 percent American Indian and 41 percent Hispanic. RSI defines six dimensions of full implementation: (1) curriculum and assessment, (2) policy, (3) resource convergence, (4) community support, (5) student attainment, and (6) underrepresented student attainment.

Llamas, V.J. (1999, September). UCAN: A Four-State Rural Systemic Initiative Year Four Annual Report.Las Vegas, NM: New Mexico Highlands University.

Llamas reports that students in schools that have actively participated in Utah, Colorado, Arizona, and New Mexico–Rural Systemic Initiative (UCAN-RSI) for four years are more likely to have increased their mathematics mean Normal Curve Equivalent (NCE) scores than other UCAN schools or non-eligible schools in the rest of the state. On the other hand, there are no statistically significant effects on student science achievement for Year 4, as data indicate that students in UCAN schools show no significant change in science in fourth-, sixth-, and eighth-grade students. The author interprets this result as a consequence of the fact that there has been an emphasis on mathematics; there is tentative evidence of student achievement gains in UCAN schools in this area. Although the evidence for higher student achievement in UCAN schools where more than 75 percent of the teachers have implemented a standards-based curriculum is clear and dramatic, it is also obvious that science achievement lags behind mathematics not only in UCAN schools but also in statewide average compared to national norms. Llamas argues that as UCAN entered its fourth year of operation, its efforts were focused on the support needed to accelerate the process of implementation of a standards-based curriculum in its focal schools. However, the data offered in this report, in the main, are only based on data from New Mexico and Arizona. Furthermore, in Arizona, only math data are available. Thus, it would be difficult to make a firm judgment whether UCAN-RSI actually affected students’ science achievement with the data presented in this study.

Loucks-Horsley, S. and Matsumoto, C. (1999). Research on professional development for teachers of mathematics and science: The state of the scene. School Science and Mathematics.99(5), 258-271.

Loucks-Horsley, S., Styles, K., and Hewson, P. (1996, May). Principles of Effective Professional Development for Mathematics and Science Education: A Synthesis of Standards. NISE Brief, Volume 1, Number 1. Madison, WI: National Institute for Science Education. Available at: http://www.wcer.wisc.edu/nise [September 3, 2002].

Loveless, T. (1998). The Use and Misuse of Research in Educational Reform. In D. Ravitch, (Ed.), Brookings Papers on Education Policy, pp. 279-317. Washington, DC: Brookings Institution.

Loveless claims that the evidence shows that “constructivist” standards impede student learning. He argues that educational reform has been undermined by the fundamental limitations of both research and policy. The author uses examples from California and Massachusetts to illuminate structural problems in the relationships among educational research, policy, and practice. In the case of California’s instructional reforms, Loveless argues that there is an unprecedented level of prescriptiveness for the documents in which the reforms were presented. For example, California’s curriculum frameworks in language arts and mathematics embrace constructivism. Both are based on the latest research, whereas both ignore the limitations of the research they cite. Thus, California’s failed instructional reforms illustrate the difficulty of converting educational research into educational reform. Loveless claims that these are failures of governance, not of teaching; the failure of state officials to supply teachers with the whole, unvarnished research on recommended instructional practice; and the failure of state curricular documents, by focusing on methods instead of content, to present a model curriculum for children to learn. In the case of Massachusetts’ tracking reforms, local policy makers have found the political advocacy on the issue persuasive, specifically, the assertion that detracking will help students of low socioeconomic status and students of color—despite the lack of research verifying this claim. Finally, Loveless suggests that the reform process needs to be made by restraining state involvement on issues that should be decided at school sites and by breaking up researchers’ and policy makers’ monopoly over new knowledge.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Luft, J.A., and Cox, W.E. (2001). Investing in Our Future: A Survey of Support Offered to Beginning Secondary Science and Mathematics Teachers. Science Educator.10(1), 1-9.

This study reports on survey results of the extent and quality of pre-service and induction programs for beginning secondary science and mathematics teachers in Arizona. The study discusses the findings from several surveys. First is a statewide survey of Arizona school district induction programs. Second is a survey of beginning secondary science and mathematics teachers about their perceptions of their teacher preparation and induction programs. The surveys were conducted in the spring of 1998. The survey response rates varied. While the district survey response rate was a solid 74 percent, the teacher survey response rate was only 47 percent. The quantitative analysis methods appeared appropriate. The authors found that most districts did not have any induction system for new science and mathematics teachers. About 20 percent had formal mentoring programs, the most common form of induction. Of these, 68 percent lasted for only one year. Only 24 percent of beginning teachers in small districts and 59 percent in large districts reported participating in induction programs. Thus there is relatively little assistance given to most beginning mathematics and science teachers. Even in districts with formal mentor programs, one-third of teachers did not receive mentors and only half of those who did receive mentors received same-discipline mentors. In responses to questions about their pre-service experience, 40 percent of beginning science teachers reported they did not major in a science, which is consistent with national data. Further, many teachers reported that their pre-service program did not provide them with an adequate understanding of the national standards, which was just about the lowest rating they gave to any aspect of their pre-service program. These results suggest that pre-service experiences of teachers three years after the introduction of the standards did not inform participants adequately about the standards. Further, while the research indicates that teachers not supported as they begin teaching will resort to more traditional strategies as they encounter the challenges of day-to-day teaching, there are relatively weak supports for teachers in terms of induction and mentoring experiences to help them navigate their first teaching experiences.

Luhm, T., Foley, E., and Corcoran, T. (1998, April). The Accountability System: Defining Responsibility for Student Achievement. Children Achieving: Philadelphia’s Education Reform. Progress Report Series 1996-1997. Philadelphia, PA: Consortium for Policy Research in Education.

Lynch, S. (2001, May). Conclusion: “Science for All” Is Not Equal to “One Size Fits All”: Linguistic and Cultural Diversity and Science Education Reform. Journal of Research in Science Teaching.38(5), 622.

Lynch argues that despite the best intentions to promote equity and to close achievement gaps, the science education reform movement has failed to respond adequately to the diversity of the student population. It has become increasingly obvious that “science for all” does not necessarily mean that “one size fits all”—curriculum, instruction, or assessment. As the reform has created forces for change in schools, so the goal is for linguistically and culturally diverse students to benefit from these pressures, rather than being attended by them. In order for this to happen, the author suggests four necessary factors. The science education research community might consider these needs:

  • Research informed by classroom practice leading to daring but robust theory building that can guide curriculum and instruction for linguistically and culturally diverse learners.

  • Credible, valid research on effective instructional programs in science for culturally and linguistically diverse student populations. These approaches to research must more often go beyond qualitative data. Quantitative research is also needed in order to generalize to larger populations and foster systemic change. Policy makers will not back serious, large-scale reform interventions without such data.

  • A better understanding of the nature of science and its interplay with teaching and learning.

  • A willingness to confront the institutionalized inequities in opportunity to learn, mostly still untouched by the reform and resulting in indifference to students’ lives and futures.

The author also points out that elegant theory and painstaking ethnography sometimes simply reveal the plain-as-mud inequities that exist for culturally and linguistically diverse students in many schools.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Massell, D. (1998, July). State Strategies for Building Local Capacity: Addressing the Needs of Standards-Based Reform. CPRE Policy Briefs: RB-25. Available at: http://www.cpre.org/Publications/rb25.pdf [August 8, 2002].

Massell, D. (1998, October). State Strategies for Building Capacity in Education: Progress and Continuing Challenges. CPRE Research Report: RR-041. Available at: http://www.cpre.org/Publications/rr41.pdf [August 8, 2002].

Massell, D. (2001). The Theory and Practice of Using Data to Build Capacity: State and Local Strategies and Their Effects. In S.H. Fuhrman (Ed.), From the Capitol to the Classroom: Standards-Based Reform in the States, The One Hundredth Yearbook of the National Society for the Study of Education, Part 2, pp. 148-169. Chicago: University of Chicago Press.

Using the same CPRE 1998-99 data as Margaret Goertz did in her accountability system analysis of eight states and 23 school districts, Massell analyzes the data set to better understand state and local efforts to use data to build capacity for reform. Not surprisingly, she found that states play the lead role in generating data (from state testing) and in developing accountability systems and incentives. Massell concurs with Goertz that in the new accountability models, schools are the main units of accountability, and student performance data are the main indicator used for accountability. Schools and districts rely heavily upon both state data and assistance; states are a prime source of professional development for increasing school capacity to use data to meet state performance requirements and for school improvement planning. Districts and schools supplement state data to measure continuous progress toward standards, to gain feedback for instructional improvement, and to evaluate programs. Both states and districts use performance data to identify low-performing schools, and some use these data in teacher and administrator evaluations. Massell notes that school reform efforts appear to have increased the use of data, particularly the use of student performance data. While there is some evidence that at “data-intensive” schools, data are used extensively to align teaching and learning with the assessments and standards and to improve instruction, there is little evidence that such in-depth use of data is widespread. Massell speculates that “intensive” use of data is the result of a combination of local factors: (1) accountability pressures such as consequences that have a direct impact on the organization, and (2) school leaders that value outcomes and performance goals and believe that data can be used effectively to inform decision-making and school improvement. Data use for accountability at the state, district, and school levels remains fragmented: “there is often a disconnect between state and local standards and assessments or across state policies themselves.” While new models of data-based decision making are emerging at the school level, old ways of utilizing data for mere compliance or surface-level alignment of curriculum to assessments are still prevalent. Massell recommends that further professional development is needed to effectively align learning to standards and to connect data to improving classroom instruction at a deeper level. Massell also cautions against quick fixes or simplistic use of data, or expecting data to provide a one-size-fits-all solution; she recommends further study of how data can best be utilized in accountability systems to build capacity and shed light on standards-based reform.

Massell, D., Kirst, M., and Hoppe, M. (1997). Persistence and Change: Standards-Based Reform in Nine States. CPRE Research Report. Philadelphia, PA: Consortium for Policy Research in Education.

The authors investigate the development and progress of standards-based reform in nine states and 25 school districts during 1994-95. The three elements of standards-based systemic reform are: (1) establishing challenging academic standards for what all students should know and be able to do; (2) aligning polices—such as testing, teacher certification, and professional development—and accountability programs to standards; and (3) restructuring the governance system to delegate overtly to schools and districts the responsibility for developing specific instructional approaches. Major findings of the study included:

  • Standards-based, systemic change remained a key feature of all nine states’ education policies and 20 out of 25 districts used standards-based reforms for improving curriculum and instruction.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
  • Difficulties in achieving professional and/or public consensus about the nature and design of standards slowed the pace of reform.

  • Newer practices such as including affective outcomes, constructivist practices, and performance-based assessment were criticized by religious and conservative groups and also by the general public and educators. State and district policy makers have responded by seeking balance between new and older approaches, rather than calling for wholesale return to conventional practices.

  • State standards are intentionally broad for both political and pedagogical reasons, but district administrators and teachers often wanted more guidance and support.

  • More than half of the districts located in states with standards in place reported that the standards initiatives had influenced their own instructional guidance efforts.

  • National-level projects, including national standards documents, influenced local standards.

  • There is a concern about the lack of coherence of messages about good practices that local officials receive from the variety of state and local groups promoting standard-based reform. Policy makers have begun to tie licensure and professional development activities to reform.

Matson, B. (1998). A Case Study of Vermont’s SSI (VISMT), 1992-1997. In P.M. Shields and A.A. Zucker (Eds.), SSI Case Studies, Cohort 2: California, Kentucky, Maine, Michigan, Vermont, and Virginia. Menlo Park, CA: SRI International.

This is a case study of Vermont’s five-year, 1992-97, Statewide Systemic Initiative (SSI), which was funded in part by the National Science Foundation. A critical event towards the end of this period was the adoption of the state’s Framework of Standards and Learning Opportunities. These state standards, greatly influenced by the National Science Education Standards, gave focus and impetus to the statewide initiative. Prior to the Framework of Standards, the SSI had tried multiple strategies to achieve state reform in science and mathematics. At the beginning, the initiative was focused on components including curriculum, assessment and accountability, and professional development that were not well coordinated. In the early years, the work was standards-driven and included other complementary visions of school reform, such as competitive grants for local curriculum projects in science, mathematics, and technology awarded to schools in the first two years. A state science assessment was piloted in 1996 with the intent of administering it in alternate years in grades 6 and 11. The VISMT (Vermont Institute for Science, Mathematics, and Technology) worked with a commercial testing company to modify an available standardized science test to be aligned with the state standards. A standards-based, integrated, hands-on science, mathematics, and technology assessment was developed and piloted in 40 schools in 1996-97. Vermont had not had a state assessment prior to the SSI. No information was given on assessment results nor was descriptive information on the development and activities of the SSI and state department of education provided.

McGinnis, J.R., Shama, G., McDuffie, A., Huntley, M.A., and King, K. (1996, March). Researching the Preparation of Specialized Mathematics and Science Upper Elementary/Middle-Level Teachers: The 2nd Year Report. Source unknown.

This report is divided into two sections. The first section familiarizes readers with the Maryland Collaborative for Teacher Preparation (MCTP), an NSF-funded statewide undergraduate, teacher-development program for mathematics and science upper-elementary/middle school teachers. The second section provides summaries of four longitudinal research studies of knowledge growth in undergraduate mathematics and science teacher education being conducted within the project. Numerical data derived from two Likert-type surveys, and qualitative data derived from ongoing semi-structured interviews with MCTP participants, class observations, participant journals, and MCTP course materials, were collected and documented.

While all four studies focus on teacher development in response to the introduction of the national standards, the fourth study also somewhat addresses teaching practice as the result of that teacher development. This study examines the perceptions of five pre-service teachers and their mathematics professor, as participants in a reform-style classroom. The purpose of the research was to see if the participants perceive the instruction in their class as modeling teaching and learning consistent with the goals set within the reform documents. Ongo

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

ing student and instructor interviews, and classroom observations were conducted. Analysis of the data indicated that the students and instructor had a clear image of what ideal teaching and learning should be, and that the instructor’s practice was consistent with this vision. The research also showed that discussions about pedagogical issues were limited within the content classes.

The research reported here suggests that college students who experience standards-based instruction in their content courses recognize, and may have a better understanding of, reform pedagogy, even if that pedagogy is not made explicit in their classes.

McKeon, D., Dianda, M., and McLaren, A. (2001). Advancing Standards: A National Call for Midcourse Corrections and Next Steps.Washington DC: National Education Association.

In support of standards-based education reforms first introduced over 15 years ago, the National Education Association (NEA) proposes recommendations for interim corrections to current accountability and assessment systems. The authors highlight the “missteps” of implementing standards-based reform, claiming that the reform expectations for education have been raised without the sufficient supports necessary to implement and achieve them. They focus on the inadequacy of the accountability systems that depend on high-stakes testing; advocate the use of multiple measures for promotion, placement, and graduation; suggest the alignment of standards, curriculum, instruction, and assessment be reexamined; and propose a review of equity safeguards, opportunities-to-learn, and the fairness of standards’ impact on all students. The article includes both an NEA “call to action” and recommendations for modifications to be guided by NEA-developed evaluation criteria and an audit tool. Criteria for evaluating and improving standards-based education included in the “Tool for Auditing Standards-based Education” consist of 10 key standards. The audit tool is touted as a guide for discussion, data collection, and analysis for educators, parents, and others to use in evaluating state implementation of standards. This article appears to be a position statement framed as an introduction for the NEA evaluation tool. While references to a variety of Education Week articles and the American Federation of Teachers Making Standards Matter 1999 are provided, there are no direct citations made in the narrative to suggest that the authors’ conclusions and recommendations are research based.

Moore, P. (1994, April). K-12 Science Education: A Teacher’s View. In Scientists, Educators, and National Standards: Action at the Local Level, Sigma Xi Forum Proceedings, Sigma XI, The Scientific Research Society, Research Triangle Park, NC, April 14-15.

Morse, P.M., and the AIBS Review Team. (2001). A Review of Biological Instructional Materials for Secondary Schools.Washington, DC: American Institute of Biological Sciences. Available at: http://www.aibs.org [August 8, 2002].

This report describes the results of a review of instructional materials in biology at the secondary level undertaken by the American Institute of Biological Sciences (AIBS). The purpose of the project was to evaluate instructional materials in biology education to inform school-based adoption decisions. A nine-person team of scientists, teachers, and science educators developed an instrument and procedures based on the National Science Education Standards to evaluate 10 biology programs with publication dates from 1997-2000. The evaluation criteria were based on the life science standards, other content standards (other than physical science and earth/space science), pedagogical standards, and program/system standards and the materials were examined for content accuracy and currency. Six separate reviews were conducted for each program. During the review process, the team met to compare results and to calibrate the rating system.

AIBS grouped the instructional materials into three categories: (1) traditional instructional materials that do not particularly respond to the standards (three programs), (2) innovative instructional materials that are specifically designed to meet all of the National Science Education Standards (three programs), and (3) mixed instructional materials that come from the traditional background, but have responded to some or all of the pedagogy and other standards in presentation (three programs). Results of the study include: (1) there is great variability in how well different programs address standards-based science content, (2) most textbooks simply add more content to address new standards, covering too much content with too little focus, (3) nine out of 10

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

programs adequately represented important topics in biology, but more attention is needed in creating environments that foster learning and in meeting the other content standards and the pedagogy standards, and (4) no programs were considered overall to be exemplary, but nine of the 10 programs ranged between adequate and excellent. The reviewers found that while the life science content was present, accurate, and up-to-date in these programs, there is vast room for improvement in the treatment of other content standards and the use of standards-based pedagogy. The report indicated “most books are just too large, still too encyclopedic, and leave too much responsibility on the teachers to figure out how to use them.”

This study raises the issue of what is required for a program to be considered adequately standards-based. None of the biology programs were considered to be exemplary (i.e., fully aligned with all standards, including pedagogy). All programs but one were considered to adequately address important life science content as designated in the National Science Education Standards. However, there was significant variability in the degree to which the programs met the “less traditional” content standards (inquiry, history and nature of science, science and technology, personal and social dimensions). There also was considerable variability in addressing the teaching standards (approach to learning, learning environment, and instruction). The AIBS study briefly refers to an AAAS study that also evaluated biology textbook programs, which did not find any biology programs to be of high quality, based upon standards. To judge a program as “standards-based,” therefore, significant questions remain: (1) To what extent must a program address all content standards (beyond traditional disciplinary content)? (2) To what extent must instructional materials explicitly espouse and provide concrete support for a particular approach to teaching?

Mullis, I., Martin, M.O., Beaton, A.E., Gonzalez, E.J., Kelly, D.L., and Smith, T.A. (1998, February). Mathematics and Science Achievement in the Final Year of Secondary School: IEA’s Third International Mathematics and Science Study (TIMSS).International Association for the Evaluation of Educational Achievement. Chestnut Hill, MA: Boston College, Center for the Study of Testing, Evaluation, and Educational Policy. Available at: http://timss.bc.edu/isc/isc_publications.html [August 8, 2002].

Muscara, C. (1998, May). A Discussion of Some U.S. Evaluation Efforts for Programs and Resources in Mathematics and Science. In Office of Educational Research and Improvement Working Papers, Vol. 1, Learning from Consumer-Oriented Review Efforts to Guide the Development of a System of Expert Panels to Identify and Share Promising and Exemplary Products and Programs. Tab K. Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement.

This report investigates the issue of the evaluation of science and mathematics programs and instructional resources to determine if they are of high quality and standards-based. This is a summary of processes developed by 12 science and mathematics organizations to review preK-12 mathematics and science products. The organizations surveyed included foundations, nonprofit groups, professional societies, states, regional laboratories, and others. The researcher conducted a thorough study to identify all potential organizations engaged in program and resource evaluation in mathematics and science and then identified the key individual involved in each of those evaluation efforts. The researcher conducted an initial interview and follow-up interviews with the contact person for each organization to determine (1) resource-evaluation strategies and (2) program-evaluation strategies. The findings of this report are useful to those who design and conduct projects to evaluate programs and resources and for others who use the results of such evaluations and must judge the quality and credibility of the evaluation process and procedures used.

The report listed five components common to all program and resource evaluation efforts: (1) a focus or purpose of the evaluations, (2) an identified audience for the evaluation effort, (3) criteria used to evaluate, (4) the process employed during each evaluation, and (5) evaluation results. Several evaluation criteria were common across organizations: quality of program, accuracy/currency of content, pedagogical effectiveness, correlation with state/national standards, attention to equity and lack of bias, multiple content connection, and developmentally appropriate.

The report made several recommendations regarding evaluation of programs and resources:

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
  • Any organization undertaking evaluation work would benefit by carefully defining its focus or purpose to avoid unnecessary work and too broad a scope.

  • Defining the audience for an evaluation effort also helps define the populations from which to draw the evaluation sample.

  • Criteria should be developed by a variety of experts. They should be written in clear language and be described so that users understand each criterion’s meaning and purpose. To be most effective, each criterion should be matched to evidence from the resource or program.

  • Because evaluation efforts are restricted by funding, time, resources, and other considerations, each effort will be different. The more varied the relevant expertise involved, the more complete the evaluation.

  • Evaluation results need to be disseminated to be valuable. Because they are time dependent, a most-recent-evaluation date is critical for the users.

National Center for Education Statistics. (2001). The Nation’s Report Card. Available at: http://nces.ed.gov/nationsreportcard/ [August 22, 2002].

National Center for Improving Science Education. (1989). Science and Technology for the Elementary Years: Frameworks for Curriculum and Instruction.Washington, DC: Author.

This was a study conducted by the National Center for Improving Science Education and the Biological Sciences Curriculum Study, with support from the National Science Foundation, to design a framework for elementary school science. The report discussed the current situation of elementary school science, the distinction between science and technology, the goals and rationale for elementary school science, a framework for curriculum, a framework for instruction, and an overview of the educational environment. The report indicates that “the curriculum should consist of hands-on activities, each of which should relate to the students’ world … rather than skimming a great many concepts, the students will be able to study a few concepts in great depth … students should be able to construct their concepts and skills through a variety of experiences” (p. vi). The report identified nine major concepts for the elementary science program: organization (or orderliness), cause and effect, systems, scale, models, change, structure or function, discontinuous and continuous properties (variations), and diversity.

National Center for Improving Science Education. (1990). Science and Technology Education for the Middle Years: Frameworks for Curriculum and Instruction.Washington, DC: Author.

This was a study conducted by the National Center for Improving Science Education and the Biological Sciences Curriculum Study, with support from the National Science Foundation, to design a framework for middle-school science education. This report discusses the nature of the early adolescent learner, issues related to middle-level education, the status of science education at the middle level, a conception of science and technology for middle-level education, goals for middle school science and technology, student outcomes, an instructional model, the learning environment, and a framework for middle-level science and technology curriculum and instruction. The report recommends that middle-level science and technology programs “include the use of: the middle school concept as the basis for design; a program based on both science and technology; a program for the entire middle-level sequence; an instructional model; a curriculum emphasis for each unit; a variety of activities; an integration of other disciplines; a progression from personal to social, local to global, questions to explorations, and problems to solutions; an articulation with elementary and high school programs; assessment that is consistent with the goals of the curriculum; and assessment that includes evaluation of higher order thinking, attitudes, and problem solving skills” (p. 107).

National Center for Improving Science Education. (1991). The High Stakes of High School Science.Washington, DC: Author.

This was a study conducted by the National Center for Improving Science Education and the Biological Sciences Curriculum Study, with support from the National Science Foundation, to design a framework for high

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

school science. The report included sections on the rethinking of the high school science program, engineering the assessment revolution, the learner and teaching, and promoting change in teachers and schools. The report recommends that all students take science courses during all four years of high school. High school science programs would: meet national expectations for science of high quality; help all students attain the personal empowerment that derives from understanding the natural sciences and their applications; better prepare students to succeed in a workplace that demands greater competence in science and technology; better prepare students to use scientific and technological information when they make personal and social decisions; increase the amount and quality of science instruction for students bound for the workplace; and allow students to keep their options to study science open throughout the high school years.

National Commission on Excellence in Education. (1983). A Nation at Risk: The Imperative for Educational Reform.Washington, DC: U.S. Government Printing Office.

National Council of Teachers of Mathematics. (1989). Curriculum and Evaluation Standards for School Mathematics.Reston, VA: Author.

National Education Goals Panel. (1996a). Building a Nation of Learners. The National Education Goals Report. Washington, DC: U.S. Government Printing Office.

National Education Goals Panel. (1996b). Commonly Asked Questions About Standards and Assessments, Executive Summary. The National Education Goals Report.Washington, DC: U.S. Government Printing Office.

National Education Goals Panel. (1996c). Profile of the 1994-95 State Assessment Systems and Reported Results. National Education Goals Panel 96-05, June. Washington, DC: U.S. Government Printing Office.

National Education Goals Panel. (1998a). Mathematics and Science Achievement State by State, 1998.Goal 3: Student Achievement and Citizenship. Goal 5: Mathematics and Science. Available at: http://www.negp.gov [August 8, 2002].

National Education Goals Panel. (1998b). Promising Practices: Progress Toward the Goals, 1998.Lessons from the States, 1998. Available at: http://www.negp.gov [August 8, 2002].

In 1998, the National Education Goals Panel (NEGP) used data from its annual report to identify states that demonstrated promising practices and progress towards achieving the eight national education goals. Interviews were then conducted with educators and policy experts from each of the identified states to describe the “stories” behind successful practice and to explain the “lessons learned” on the way. The report is organized around the eight national education goals, detailing individual goals and their associated objectives and indicators. Also, for each goal, data on the highest-performing states and most improved states are provided along with profiles and lessons from a few of the top-performing states in each goal category. Goal Number 5, which calls for the United States to be first in the world in mathematics and science by the year 2000, is most relevant to understanding recent progress in science learning in the United States. Focusing on achievement in eighth-grade science, the NAEP and TIMMS data reveal that in comparison to 41 countries, students in 14 U.S. states would be expected to outperform students in 40 of those countries (with the exception of Singapore). The report highlights professional and leadership development programs in Connecticut and Wisconsin that promote science mastery and integration of science standards and instruction with other content areas. The report does not pretend to be the definitive source on the current status of the states in science education; instead, the report cites the work of the American Federation Teachers in its report Making Standards Matter and the Council of Chief State School Officers (CCSSO) 1997 report State Indicators of Science and Mathematics Education. The value of this report lies in the emphasis placed on improving mathematics and science education as one of eight national education goals, and the information it gives on the progress toward reaching that goal.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

National Research Council. (1996). National Science Education Standards.Washington, DC: National Academy Press.

National Research Council. (1999a). Global Perspectives for Local Action: Using TIMSS to Improve U.S. Mathematics and Science Education.Washington, DC: National Academy Press.

National Research Council. (1999b). High Stakes: Testing for Tracking, Promotion, and Graduation.Committee on Appropriate Test Use. J.P. Heubert and R.M. Hauser, editors. Board on Testing and Assessment, Commission on Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.

National Research Council. (1999c). Selecting Instructional Materials.Committee on Developing the Capacity to Select Effective Instructional Materials. M. Singer and J. Tuomi, editors. Center for Science, Mathematics, and Engineering Education, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.

For this study, the National Research Council established a committee to investigate issues related to selecting effective instructional materials. The goal of the Committee “was to produce a tested standards-based instrument that would be helpful to people who select instructional materials for use in the science classroom” (p. 3). The researchers reviewed extant procedures and instruments developed for curriculum review in science developed by several organizations, including the American Association for the Advancement of Science, the National Science Foundation, the National Science Resources Center, the U.S. Department of Education, and the Center for Science, Mathematics, and Engineering Education. The report includes a section describing the project, its rationale, and the review of national efforts to evaluate instructional materials and a section on recommended processes and tools. The appendix includes an instrument for evaluating instructional materials in science.

National Research Council. (2000a). Educating Teachers of Science, Mathematics, and Technology: New Practices for the New Millennium.Committee on Science and Mathematics Teacher Preparation. Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.

This report of the National Research Council’s Committee on Science and Mathematics Teacher Preparation provides a thorough review of the standards movement and the context within which today’s reforms are taking place and calls for fundamental restructuring of teacher preparation and professional development. The report opens with today’s educational context and the evolution of the standards movement over the past decade. Summarizing a variety of research studies that explore aspects of the relationships between teacher learning, teacher practice, and student learning, the report argues that high-quality teaching matters and that teacher quality is related to student achievement in science and mathematics. The report’s authors advocate that teacher education be reconceived as a professional continuum rather than a disjointed sequence starting as pre-service and continuing as in-service. The report concludes with recommendations for a variety of actors, including the government, K-12 community, higher education community, and professional and disciplinary organizations.

National Research Council. (2000b). How People Learn: Brain, Mind, Experience, and School.Committee on Developments in the Science of Learning and Committee on Learning Research and Educational Practice. J.D. Bransford, A.L. Brown, and R.R. Cocking, editors. Commission on Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.

National Research Council. (2001a). Classroom Assessment and the National Science Education Standards: Addendum.Committee on Classroom Assessment and the National Science Education Standards. J.M. Atkin, P. Black, and J. Coffey, editors. Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

National Research Council. (2001b). Knowing What Students Know: The Science and Design of Educational Assessment.Committee on the Foundations of Assessment. J. Pelligrino, N. Chudowsky, and R. Glaser, editors. Board on Testing and Assessment, Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.

This book draws upon the latest research in learning, cognition, and measurement to inform classroom and large-scale assessment practices. The book, prepared by the Committee on the Foundations of Assessment of the National Research Council and funded by the National Science Foundation, identifies three underlying factors to be considered in any assessment. One factor is a model of how students represent knowledge and develop competence in the content area. A second factor is the tasks or situations that allow one to observe students’ performance. The third factor is a method of interpretation that makes it possible to draw inferences from responses produced by the students. The authors claim that most current assessment practices are based on old conceptions of learning and that assessment practices should be based on the most modern and the best models of human cognition and learning available. Research continues to reveal more about how students learn, the variations among individuals, and students’ lack of a uniform progression in learning. Assessments should seek to identify the specific problem-solving strategies students employ, where these strategies are situated on the developmental spectrum, and the appropriateness of these strategies for the particular domain of knowledge and skill being tested. In addition to advances in understanding cognition, advances in measurement and statistical modeling have strong implications for assessment practices. Statistical models exist that reduce the dependency on reporting only a single score and that make it easier to report multiple aspects of proficiency and track students’ progress over time. This book is not only very comprehensive, but also is an excellent resource for anyone involved in assessment. The book recognizes the National Science Education Standards and their emphasis on assessment as being a fundamental part of teaching and learning. Some assessment samples in the book are drawn from science. However, the main thrust of the book is on assessment in general and the need for any assessment—classroom or large-scale—to be developed and interpreted in light of the most recent understandings of how students learn and how measurement advances.

National Research Council. (2002). Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education.Committee on Understanding the Influence of Standards in K-12 Science, Mathematics, and Technology Education. I.R. Weiss, M.S. Knapp, K.S. Hollweg, and G. Burrill, editors. Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.

National Science Foundation. (1994a). Foundation for the Future.Arlington, VA: Author.

This report summarizes information about several NSF-funded programs designed to support the reform of science and mathematics education. NSF reported, “in 1993, approximately 12 percent of the 42 million K-12 students across the country used mathematics and science curricula developed through the Instructional Materials Development program” (p. 6). Programs described in the report include:

  • The Interactive Math Program (IMP) for 9-12

  • Used Numbers program for K-6

  • Algebra I Project

  • A River Runs Through It (secondary science)

  • Calculus Leading the Way

  • Air, Earth, Fire, Water

  • Educating the Technical Work Force for the 21st Century (associate degree program)

  • Promoting Technology Transfer

  • Hampton University Spearheads Increased Production of Doctorates in Science and Education

  • Isolated Colleges Ride the Information Highway

  • Cognitive Guided Instruction: You Take What You Know and Build from There

  • Science Comes to Television: Bill Nye the Science Guy and CRO with Science Kits Too

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
  • 180 Students Demonstrate the Art and Science of Engineering—Some Even Invented Equipment for the Disabled

  • Students in the Global Laboratory Make their School a Safer Place

  • NSF Projects Engage the Public in the Science of Birds and Bogs

  • Physics Is Fun: Toys, and Games for Girls in Missouri

  • Hands-On Science Curriculum Helps Students, Teachers, and Parents “Find Out”

  • No Substitute for Well-Prepared Teachers

  • Twenty-Percent of Full-Time Physics Teachers Learn How to Change the Way They Teach

  • Workshops Work for College and University Faculty

  • Understanding Epileptic Seizures

  • Blind Physicist Develops New Braille Technology for Science and Mathematics

  • U.S. Senators Laud NSF Project Selected as the 1992 Anderson Gold Medalist winner

  • Experimental Program to Stimulate Competitive Research Builds Science and Technology Competitiveness

  • Urban Systemic Initiative: A Revolutionary Transaction

  • Urban Systemic Initiative: Chicago Planning Award

  • Statewide Systemic Initiatives Program Having Major Impact on States

  • New Rural Initiative Completes the Educational Systemic Reform Trilogy

  • Mississippi AMP Program

  • Inventing Systemic Evaluation

National Science Foundation. (1994b). SSI: Statewide Systemic Initiatives in Science, Mathematics & Engineering. 1994-1995. State Profiles. Arlington, VA: Author.

This is the second edition of state profiles of individual state systemic initiatives funded by the National Science Foundation (NSF). The SSI Program encourages improvement in science, mathematics, and engineering education through comprehensive systemic reform in the education systems of the states. At the time of this report (1994), 24 states and Puerto Rico had received five-year awards from NSF. This report provides information on each of the projects funded by the SSI Program, but provides no analysis or summary of the results overall. Each state profile lists contact person information, state background, vision, strategy, accomplishments, and important partners and alliances.

National Science Foundation. (1997). Review of Instructional Materials for Middle School Science.NSF 97-54. Arlington, VA: Author

National Science Foundation. (1999). Program Solicitation and Guidelines: Elementary, Secondary, and Informal Science Education.NSF 99-92. Arlington, VA: Author.

National Science Resources Center. (1997). Science for All Children: A Guide to Improving Elementary Science Education in Your School District.Washington, DC: Author.

This book describes the National Science Resource Center’s (NSRC) strategy for bringing about district-wide elementary science reform consistent with the NSES. The NSRC’s model views elementary science as a cohesive system that includes inquiry-centered science curriculum, professional development, materials support, appropriate assessment, and system and community support. The first part of the book explains the rationale for this model. The second part describes how the model can be implemented. The third part contains eight case studies of districts’ efforts to implement the NSRC model. The eight districts are Montgomery County, Maryland; Spokane, Washington; East Baton Rouge Parish, Louisiana; Cupertino, California; Huntsville, Alabama; Pasadena, California; San Francisco, California; and Green Bay, Wisconsin. The eight case studies include descriptions of the professional development strategies of the districts, which are consistent with the NSES’ approach to teacher training (ongoing, intensive, content-based, inquiry-oriented, providing ready access to materials, in some cases the development of lead, or master, teachers, and the involvement of professional

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

scientists). The case studies are descriptive and are not designed to provide evidence of the impacts of these programs on either the professional development systems of these districts or the professional knowledge and skills of the participating teachers.

National Science Teachers Association. (1992). The Content Core: A Guide for Curriculum Designers.Arlington, VA: Author.

Nelson, G.D. (2001). Counterpoint: Biology Teachers Deserve Better Textbooks. American Biology Teacher. 63(3), 146-147.

Project 2061 produced a review of 10 high school biology textbooks, two of which were developed by BSCS. Project 2061 disagrees with the statement by Rodger Bybee, executive director of BSCS, that because the study finds all the textbooks to be unsatisfactory, the analysis itself is unacceptable. Bybee criticizes the Project 2061 review as limiting textbook adoption choices. Nelson notes, “To the contrary, Project 2061’s evaluation adds information into the system that educators can use to make more sophisticated decisions, based on the specific strengths and weaknesses of the texts. Once a textbook adoption decision is made the Project 2061 data can help define the kinds of supplementary materials and instruction that may be needed to make up for any shortcomings. For example, none of the textbooks adequately accounts for students’ prior knowledge or for their preconceptions or misconceptions, although these are known to be major factors in student learning…. We recommend, for example, that educators use some of the excellent trade books on the market that have been published on science topics to compensate for unsatisfactory textbooks” (p. 146). He also says, “A concern we share with Dr. Bybee is that our reviews will encourage teachers and schools to develop their own biology materials…. We agree that “home-built” curricula would be unlikely to fair well on our analysis” (p. 147).

Nesbit, C.R., Wallace, J.D., Pugalee, D.K., Miller, A., and DiBiase, W.J. (Eds.). (2001). Developing Teacher Leaders: Professional Development in Science and Mathematics.Columbus, OH: ERIC Clearinghouse for Science, Mathematics, and Environmental Education.

Office of Educational Research and Improvement. (1994). Promising Practices in Mathematics & Science Education: A Collection of Promising Educational Programs & Practices from the Laboratory Network Program.Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement.

This is a report of 66 projects selected by the 10 regional education laboratories (funded by the U.S. Department of Education) as being aligned with national curriculum standards, having evidence of effectiveness, and being transferable to other settings. The collection of programs was identified through a thorough search and review process involving educators throughout the nation. The promising programs span elementary, middle, and secondary levels in science, mathematics, technology, or interdisciplinary subjects. Each program description includes a general description and a description of teaching and assessment strategies and of the alignment of the program with the framework developed by the National Center for Improving Science Education (because the NSES were not as yet released).

Ogbu, J.U. (1982). Understanding Cultural Diversity and Learning. Educational Researcher.21(8), 5-14.

Ohio State University Research Foundation. (1994). The Biological and Earth Systems Science Curriculum. Report to the Worthington Board of Education.

This is a report on a project to develop The Biology and Earth Systems Science Curriculum (BESS), a two-year program for ninth- and tenth-grade students. This report describes the history, purpose, goals, implementation plans, evaluation procedures, and plans for improvement. This is a curriculum developed by and for the Worthington School District. The report also provided a summary of the evaluation results including student achievement data, student survey data, and parent survey data. The results indicated that the BESS project was having a positive impact. However, because the project was conducted prior to the release of the NSES, the project did not address alignment of the BESS curriculum with the NSES.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Olson, L. (2001, January). Finding the Right Mix. In Education Week Special Report: Quality Counts 2001: A Better Balance: Standards, Tests, and the Tools to Succeed.Seeking Stability for Standards-Based Education. 20(17), January 11, 2001.

Parker, V., and Gerber, B. (2000, May). Effects of a Science Intervention Program on Middle-Grade Student Achievement and Attitudes. School Science & Mathematics.100(5), 236-42.

Pasley, J.D. (Ed.). (2002). The Role of Instructional Materials in Professional Development: Lessons Learned from the LSC Community.Chapel Hill, NC: Horizon Research.

Pate, E.P., Nichols, S.E., and Tippins, D.J. (2001). Preparing Science Teachers for Diversity Through Service Learning. Science Educator.10(1), 10-18.

In this article, the authors argue that service-learning projects will help prepare prospective science teachers to teach learners of diverse backgrounds because service learning connects meaningful community service experiences with academic learning, personal growth, and civic responsibility. The authors link the goals of service learning to more authentic representation of the nature of science and the self-generation of questions for inquiry that are promoted by the standards. The authors describe the four steps generally found in service learning: preparation, service, reflection, and celebration. They then describe the service-learning projects of two prospective science teachers, quoting their journal entries as evidence of the learning and value of their experiences. The authors argue that prospective teachers can gain understanding of culture as the way groups of people socially negotiate their everyday living circumstances in local settings.

Paulu, N. (1994). Programs for the Improvement of Practice. Improving Math and Science Assessment.Report on the Secretary’s Third Conference on Mathematics and Science Education, June 1994. Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement.

Peak, L. et al. (1996, November). Pursuing Excellence: A Study of U.S. Eighth-Grade Mathematics and Science Teaching, Learning, Curriculum, and Achievement in International Context.Initial Findings from the Third International Mathematics and Science Study (TIMSS). Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement .

Pissalidis, C., Walker, T., DuCette, J., Degnan, J., and Lutkus, A. (1998, April). Observational Methods for Evaluating Changes in Student-Teaching as a Result of a Large Scale Teacher Intervention Program. Paper presented at the Annual Meeting of the American Educational Research Association, San Diego, CA.

This paper describes a collaborative effort by two universities and a school district to develop a new model for science and mathematics K-12 teacher preparation. The paper focuses on the conceptualization of the model. The authors’ description of their framework for pre-service education is in many ways consistent with the elements advocated in the NSES. For example, the authors describe their vision as based on construction rather than transmission of knowledge, cooperative learning, and authentic assessment. However, the authors do not cite any of the national standards bodies as providing a basis for, or having influenced the development of, their model. The authors also envision a multistage evaluation process to gauge the learning of students throughout their pre-service experience; the evaluation process will rely on expert-rated videotapes of classroom instruction, surveys with authentic assessment measures, and cooperating teacher evaluations. The authors state that subsequent papers will describe the actual implementation of the model and its effects on participating novice teachers.

Porter, A. (1993, September). State and District Leadership for Implementation of Project 2061.Project 2061 Policy Blueprint. Washington, DC: American Association for the Advancement of Science.

This paper was prepared as a policy blueprint for AAAS. The paper provides an overview of Project 2061. The paper also describes four models of K-12 science programs developed by six participating school districts.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Porter goes on to project the nature of what a Project 2061 school science program will be if the vision is achieved. Porter identifies challenges for implementation of Project 2061, including: (1) acceptance of the reform objectives of making the content challenging and useful and accessible to all students, (2) understanding the changes needed in instruction, (3) believing that change is possible, and (4) removing obstacles to change that come from the educational hierarchy. The paper provides suggestions for approaches to encouraging implementation of the Project 2061 vision in local school programs.

Porter, A.C. (1998). The Effects of Upgrading Policies on High School Mathematics and Science. In D. Ravitch (Ed.), Brookings Papers on Education Policy: 1998, pp. 123-172. Washington, DC: Brookings Institution.

This study investigated the impact of two initiatives to upgrade high-school science and mathematics: (1) polices that increase the number of credits of mathematics and science required to graduate from high school and (2) transition courses, primarily in mathematics, designed to assist low-achieving students to take and successfully complete college preparatory courses. The data for the study were from the 1989-1990 and 1990-1991 school years. This information is useful in understanding the baseline of reform in science prior to the NSES. Porter concluded “the policy to increase high school mathematics and science credits required for credit proved to be effective. On the one hand, no negative effect was found on the percentages of students graduating from high school. On the other hand, teachers did not water down the curriculum to accommodate the large influx of students, who were, on average, low-achieving students” (p. 162).

Porter, A. and Chester, M. (2001, May). Building a High-Quality Assessment and Accountability Program: The Philadelphia Example. Paper prepared for Brookings Institution Conference on Accountability and Its Consequences to Students, May 15-16.

At the center of this paper is the debate over the role of high-stakes testing in district-level assessment and accountability systems. Porter and Chester have developed a framework for critiquing district assessment and accountability systems based on their work in Philadelphia, Missouri, and Kentucky. The framework is aligned with the AERA, NCME, and APA standards on testing and the AERA position statement on high-stakes testing. The authors’ position is also supported by a literature review of recent publications on high-stakes testing, accountability, and assessment. The assessment and accountability framework has three parts: (1) setting good targets for instruction, (2) creating a program that makes both schools and students accountable, and (3) creating a program that is fair.

The authors suggest that the framework is also useful in understanding the research literature on high-stakes testing. The authors argue that assessment and accountability programs must be accompanied by the appropriate supports to be successful. They provide detailed examples, in a case study format, of the School District of Philadelphia’s recent efforts to improve their district system. The case study highlights the complexities and inconsistencies of phasing in and adjusting new assessment and accountability systems, while at the same time ensuring the systems promote balanced accountability for students and schools, and are both instructionally relevant and fairly implemented. The authors provide compelling examples, evidence to support their framework, lessons learned, and continuing dilemmas faced in the context and realities of a struggling urban district. This paper constitutes a formative evaluation of one district’s evolving assessment and accountability system using a framework for analysis constructed by the authors. The framework is well substantiated by literature, research, policy and practice—the Philadelphia example demonstrates the utility of the framework as a guideline for critiquing the design and implementation of similar systems. While positive signs of improvement are beginning to emerge in Philadelphia, Porter and Chester caution readers about seeking impact evidence from the programs prematurely, suggesting that the system is still evolving; and the assessments and indicators are under continual refinement, making it difficult to research and judge true changes in instructional practice, student persistence, and student achievement. Moreover, given the wide range of reform initiatives simultaneously implemented in the district, it is difficult to attribute improvements to the accountability and assessment programs alone.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Porter, A.C., Kirst, M.W., Osthoff, E., Smithson, J.L., and Schneider, S.A. (1994, September). Reform of High School Math and Science and Opportunity to Learn. CPRE Policy Briefs: RB-13. Available at: http://www.cpre.org/Publications/rb13.pdf [August 8, 2002].

Porter, A.C., and Smithson, J.L. (2001a). Are Content Standards Being Implemented in the Classroom? A Methodology and Some Tentative Answers. In S.H. Fuhrman (Ed.), From the Capitol to the Classroom: Standards-Based Reform in the States, The One Hundredth Yearbook of the National Society for the Study of Education, Part 2, pp. 60-80. Chicago: University of Chicago Press.

This chapter presents a framework for analyzing the impact of standards on the quality of instruction and examines the issues that must be addressed in order to make credible statements about the influence of standards on instruction and ultimately on student achievement.

A number of studies are discussed in the article, including those that have (1) used descriptions of classroom practice, (2) measured alignment between instruction and assessment, and (3) attempted to link instruction to student outcomes. The authors discuss the components of these studies, preliminary findings, and implications for how these studies inform further work in this area. Five studies were cited that have used descriptions of classroom practice. The TIMSS study, which collected a great deal of information on instructional practice, described the U.S. mathematics and science curriculum as “a mile wide and inch deep.” The National Evaluation of the Eisenhower Professional Development Program found that professional development activities with a clear content focus lead to increased emphasis on those topics during instruction. In addition, a number of studies were described that employed the Surveys of Enacted Curriculum, a means for collecting data on teaching practice and content in mathematics and science classes.

A sub-study of those employing the Surveys of Enacted Curriculum was used to illustrate study measures for examining the alignment between instruction and assessment. The assumption of the study was that alignment between the instruction in a state and the state’s test (rather than alignment to tests given in by other states) was an indication of whether standards-based reform is having an effect. While the results indicated that standards-based reform has not yet brought instruction into alignment with the state’s tests, the authors were careful to point out that the results were illustrative only given the study limitations, but provide an indication of utility that such a study would hold.

Porter, A., and Smithson, J. (2001b). Defining, Developing, and Using Curriculum Indicators. CPRE Research Report Series: RR-048. Available at: http://www.cpre.org/Publications/rr48.pdf [August 8, 2002].

The focus of this study was to examine the relationships between what is taught and the standards and assessments that are set to guide instruction. The paper provides a brief summary of the “Reform Up Close” study of 300 high school classrooms in six states. Based on this work, the current study has expanded its conceptual framework to distinguish between (1) the intended curriculum and the assessed curriculum and (2) the enacted curriculum and the learned curriculum. The enacted curriculum is the actual curricular content that students engage in the classroom, while the intended, assessed, and learned curricula are components of the educational delivery system. The intended curriculum is represented in curriculum standards, frameworks, and guidelines. The assessed curriculum is represented by high-stakes tests, in contrast with the intended curriculum (i.e., the difference between what is valued and what is assessed). The learned curriculum represents the knowledge that students acquire, which is insufficiently sampled by current standardized achievement tests. In the instructional surveys, the researchers collected information on modes of presentation, topic coverage, and cognitive demand.

Powers, M.L., and Hartley, N.K. (Eds). (1999). Promoting Excellence in Teacher Preparation: Undergraduate Reforms in Mathematics and Science.Fort Collins, CO: Colorado State University.

This book describes a collaboration among six Colorado universities and community colleges to change their teacher preparation courses in science, mathematics, and technology. The project was funded by the National Science Foundation from 1994 to 1999. Its goals were to develop collaboration between the higher-education institutions, to make the curricula and instruction in teacher-preparation courses more aligned with

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

high-quality mathematics and science instruction (which would become aligned with the mathematics and science standards), and to sensitize faculty to the issues of recruiting and retaining women and ethnic minorities in teaching careers in mathematics and science. The book includes chapters from faculty members in the various institutions about how they restructured their classes with mini-grants and guidance from those leading the collaboration. Relevant chapters include descriptions of changes in instruction for biology, chemistry, geography, and general science for non-majors classes from more traditional didactic delivery to more authentic, group problem-solving and inquiry structures that are consistent with instruction advocated by the national standards. Some of the chapters are descriptive, focusing on changes in the courses and the instructors’ intent behind these changes, but others include survey or interview data that either contrast students’ experiences in these or other, more traditional classes, or describe the influence of these courses on student learning and understanding. A few chapters are focused on issues of diversity. They describe scholarship programs to recruit women and students of color into the pre-service programs, training programs to introduce faculty to multicultural issues, and a project to assist faculty to make changes in the content and pedagogy of their courses to make them more inclusive of all students. Together, the descriptions in the book portray a pattern of changing pre-service experience for many students at these six institutions in Colorado.

Public Agenda. (2000). Leslie Gottlieb and Michael Darden. Survey finds little sign of backlash against academic standards or standardized tests. New York: Author.

Spurred by the belief that a national “parental backlash against academic standards and standardized tests” was growing, Public Agenda conducted a survey of parents of public school students in grades K-12. The national telephone survey consisted of two sample populations: (1) a random sample of parents nationwide (n=803), and (2) an additional “over sampling” of at least 200 parents in each of these urban districts: Boston, Chicago, Cleveland, Los Angeles, and New York City (n=1007). These urban districts were selected because of their emphasis on standards-based reform. The surveys were administered between September 18-26, 2000. The survey questions seek to elicit parental attitudes and beliefs regarding academic achievement, standards, teacher quality, and standardized testing. Some basic demographic information was also solicited. The survey results and findings are presented in a press release. Public Agenda found strong support for continuation of efforts to raise academic standards in public schools and little evidence of a “parental backlash” against standards or standardized testing. A copy of the survey questions shows response rates given for the national sample and by city for each question and answer category. Graphical representations of key findings are presented in tables, pie charts, and bar charts. Caution should be taken when interpreting findings as presented in the press release and graphics; it is not always evident if the results from the two samples are being compared, combined, or are being reported separately. Results from similar studies are incorporated into the press release findings to substantiate findings, but no details are provided on the methodology of these studies.

Quellmalz, E., Hinojosa, T., Hinojosa, L., and Schank, P. (2000). Performance Assessment Links in Science (PALS): An Online Resource Library. Draft Final Project Report. SRI International. Available at: http://pals.sri.com/papers/finalreport [August 8, 2002].

Quellmalz, E., and Kreikemeier, P. (2002, April). Validities of standards-based science inquiry assessments: implementation study. Paper presented at the American Educationalal Research Association Annual Meeting, New Orleans, LA.

Quellmalz, E., Schank, P., Hinojosa, T., and Padilla. C. (1999). Performance Assessment Links in Science (PALS). ERIC Clearinghouse on Assessment and Evaluation Digest Series EDO-TM-99-04. College Park, MD: University of Maryland.

This is a final report to the National Science Foundation that summarizes the activities and products produced by a grant to SRI International to develop Performance Assessment Links in Science (PALS). PALS is an online performance science assessment resource library containing performance assessment tasks for elementary, middle, and secondary levels. Two sets of performance assessments tasks are available. One set is for

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

general used by teachers and professional development organizations. The use of the second set of tasks is restricted to state assessment programs and systemic initiatives programs and is password-protected. The NSES were used to index the assessment activities and to identify topics underrepresented in the resource library. As of September 30, 2000, there were approximately 170 science performance assessment tasks posted on the Web site. The assessment activities were attained from a variety of sources including states, the Council of Chief State School Officers, and projects engaged in assessment development. By request from users, the PALS tasks were indexed by selected state and curriculum frameworks by mapping those onto the NSES. Any framework that is mapped to the NSES can be entered in PALS and then can be used to retrieve activities. The PALS project used a range of methods to gather information about the quality and usability of the resource including usage statistics, interviews, online ratings, surveys, telephone interviews, and curriculum program evaluations. An interview questionnaire was developed by the external evaluator to assure the development process and technical quality of the assessment activities. The resource development was informed by responses to user surveys and appropriate changes were made to the Web site. The organizational structure of this research-based tool was directly influenced by the NSES. The developers identified NSES topics with low numbers of corresponding activities and targeted these topics for acquiring additional activities. The report provides a good description of the PALS at the time it was written but is dated because the resource has continued to be expanded.

Regional Laboratory for Educational Improvement of the Northeast & Islands. (1993). Science and Math Assessment in K-6 Rural and Small Schools. Small Schools Network Information Exchange.Number 14, Spring.

Resnick, L.B. (1993). Standards, Assessment, and Educational Quality. Stanford Law and Policy Review.Winter 1992-93, 53-59.

Rhoton, J., and Bowers, P. (Eds.). (2001). Professional Development Leadership and the Diverse Learner: Issues in Science Education.Arlington, VA: National Science Teachers Association.

This edited book, published by the National Science Teachers Association, brings together a series of chapters that collectively focus on the role of leadership and diversity in efforts to reform science professional development. The book includes six chapters on the role of leadership in implementing standards-based science programs, discussing leadership from a wide variety of perspectives and positions, including both formal and informal leadership, and leaders at all levels, including teachers, supervisors, consultants, coordinators, administrators, higher education faculty, and policy makers. These chapters provide an argument for the importance of leadership, based upon the rich research base that overwhelmingly points to the importance of leadership in the implementation of virtually any initiative. They also present characteristics of effective leadership, again referring to the research base on educational and other organizational leaders. Several of the chapters describe programs or technical assistance models, like the North Carolina Fund for the Improvement and Reform of Schools and Teaching Initiative and the Technical Assistance Academy for Mathematics and Science Services, which are taking approaches to developing and supporting leaders that are consistent with the conceptions of the standards and employ strategies of teaching science leadership consistent with the standards’ methods of providing adult learning opportunities. The second part of the book contains seven chapters that focus more on developing leadership in a multicultural world and a more diverse set of leaders. Similar to the first set of chapters, these largely describe and portray different programs and organizations that are providing professional development to an array of leaders as well as preparing leaders to work with a diverse population of teachers and students. Together, these chapters depict a landscape of leadership programs that reflect the goals of the standards, but provide relatively little evidence that these approaches are influencing the practices of leaders in ways that are consistent with the standards.

Ridgway, J.E., Zawojewski, J.S., Hoover, M.N., and Lambdin, D.V. (2002). Student Attainment in the Connected Mathematics Curriculum. In S.L. Senk and D.R. Thompson (Eds.), Standards-Based School Mathematics Curricula: What Are They? Do They Work?Mahwah, NJ: Lawrence Erlbaum Associates.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

This study reports on the development of the Connected Mathematics curriculum and the effect of this mathematics curriculum on student achievement for grades 6, 7, and 8. Ridgway, Zawojewski, Hoover, and Lambdin found that the Connected Mathematics curriculum was effective in raising the achievement of students on challenging open-response items that emphasize reasoning, communication, connection, and problem-solving as compared with students in curricula less aligned with the NCTM standards. The Connected Mathematics curriculum was developed to promote changes both in the mathematics content taught and in the teaching of that mathematics. It was designed to integrate mathematics content and processes. The curriculum includes interesting problem settings—activities designed to involve groups of students with mathematical concepts and applications as well as discourse and reflective writing about these ideas. The materials call for an instructional model in the classroom that encourages higher-level thinking and problem solving; the emphasis is on making sense of mathematics and its use. Finally, the authors suggest that there was evidence of long-term gains on student achievement afforded by the Connected Mathematics curriculum when the performance over time was studied in a particular school and when the curriculum was the sole curriculum for all of the middle grades. This evidence was supported by a longitudinal study of the state test results. The authors also discuss the importance of the curriculum evaluation, especially when the curriculum broadens and heightens the expectations from what has previously been expected from students. They pose the following questions: (1) What sorts of implementation of the curriculum are responsible for the student achievement findings reported? (2) How will revision of the Connected Mathematics materials subsequent to the large-scale study affect student achievement findings? (3) Are there differential effects of the Connected Mathematics curriculum on different populations? (4) Are the long-term gains observed in this study generalizable to other schools?

Rigney, S. (2002, April). The Bush Accountability and Assessment Agenda: New Opportunities and Challenges. An invited address at the National Council on Measurement in Education Annual Meeting, New Orleans, LA.

Rodriguez, A.J. (1997, January). The Dangerous Discourse of Invisibility: A Critique of the National Research Council’s National Science Education Standards. Journal of Research in Science Teaching.34(1), 19-37.

Rodriguez documents the ways in which children’s learning is influenced by language, culture, identity, and motivation. Rodriguez argues that issues of how ethnic, gender, and SES issues influence science education are largely invisible in the NSES. The author emphasizes that the NSES ought to provide strong arguments and evidence in support of the reasons why “equity” should be a guiding principle in science education reform. The author contends that the invisibility of equity-related discourse dangerously compromises the well-intended goal of the National Research Council by not directly addressing the ethnic, socioeconomic, gender, and theoretical issues that influence the teaching and learning of science in today’s schools. There is the urgent need to conduct more critical and in-depth analysis of the academic performance of various students within various ethnic groups. The author also argues that the NSES should have a more explicit and active role in promoting innovative, multicultural, and student-centered practices. By providing visible theoretical frameworks and arguments in support of learning science for understanding and for teaching science in more inclusive and multicultural ways, the NSES could contribute to encouraging pre- and in-service teachers to take risks and move away from traditional teacher-centered practices.

Roeber, E.D. (1993). Using New Forms of Assessment to Assist in Achieving Student Equity: Experiences of the CCSSO State Collaborative on Assessment and Student Standards. (ED 361 368). Washington, DC: Council of Chief State School Officers.

This paper describes the formative years of the Council of Chief State School Officers’ (CCSSO) effort to form state collaboratives. One collaborative of 14 states strived to develop science education assessment measures for K-12 science. Even though each project included a research and professional development component, this paper does not report any research. At the time this paper was written, the NSES had not been published and were not noted in the paper. The K-12 science education collaborative sought to develop and validate assessment measures along with research and professional development activities. These assessments and scoring rubrics were planned to be related to a consensus map of state outcomes and to be combined with “emerging”

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

(p. 15) national content standards in science. Although not included in the report, in 2002 the group released a CD with over 14,000 pages of science assessments and supporting documents and were in the process of analyzing the alignment of the assessments and the NSES.

Rosebery, A.S., Warren, B., Ballenger, C., and Ogonowski, M. (2002). The Generative Potential of Students’ Everyday Knowledge in Learning Science.Madison, WI: University of Wisconsin, National Center for Improving Student Learning and Achievement in Mathematics and Science.

Rosebery, Warren, Ballenger, and Ogonowski studied the conceptual, linguistic, and imaginative resources that children bring to the study of science, and the ways these can support deep learning and robust achievement among students from diverse backgrounds in three case studies. The classroom research focused on understanding the generative potential of students’ everyday experience and language in science learning and teaching. The students in these classrooms were from heterogeneous backgrounds, which included significant percentages of children from low-income and racial-, ethnic-, and linguistic-minority backgrounds. The authors report that teaching that explicitly addressed these issues and built on the cultural and intellectual resources of disadvantaged children produced substantial benefits for their learning. The students who participated in the design studies answered a mean of 87 percent of test items (234 of 269) correctly. Performance in individual classrooms ranged from 74 percent (14 of 19) to 98 percent (39 of 40) correct. Notably, children in grades 1 through 8 outperformed the international results for eighth grade for TIMSS problems targeting kinematics, gravity, and the mathematics of change; children in third and fourth grade outperformed the international results for third and fourth graders for TIMSS problems targeting plant growth and development. These studies also presented detail about the deep understandings the children developed through their classroom works and activities. With regard to science standards, the accomplishments of these children and teachers exceeded the expectations set forth in national and state frameworks. In each case, children developed robust understanding of significant, rigorous scientific ideas and practices typically taught to older students.

Scannell, M.M., and Metcalf, P.L. (2000). Autonomous Boards and Standards-Based Teacher Development. In K.S. Gallagher and J.D. Bailey (Eds.), The Politics of Education Reform. The National Commission on Teaching and America’s Future. Thousand Oaks, CA: Corwin Press.

Scantlebury, K., Boone, W., Kahle, J.B., and Fraser, B.J. (2001, August). Design, Validation, and Use of an Evaluation Instrument for Monitoring Systemic Reform. Journal of Research in Science Teaching.38(6), 646-662.

Scantlebury, Boone, Kahle, and Fraser administered newly developed questionnaires to 3249 middle school students in 191 classes over a three-year period. The questionnaire development was associated with the Statewide Systemic Initiative (Ohio’s Project Discovery). The instrument measured student attitudes and several environment dimensions (standard-based teaching, home support, and peer support) using a three-step process that incorporated expert opinion, factor analysis, and item response theory. The authors investigated the influence of the class, home, and school environments on the two student outcomes of science achievement and attitudes toward science. An important result is that the classroom environment (standards-based teaching practice) accounted for variance in both achievement and attitude scores over and above that attributable to either the home or peer environment. Therefore, this study supports the advantageous efforts of standards-based teaching. The findings were remarkably consistent across three years (1995, 1996, and 1997). Student achievement (as measured by performance on a test consisting partly of publicly released NAEP items) and student attitudes toward science were positively correlated with the questionnaire’s measure of standards-based teaching practices. The correlations between student achievement and the questionnaire’s measures of home support and peer environment were not significant, although there were positive correlations between the home support and peer environment measures and student attitudes toward science. All three environments accounted for unique variance in student attitudes, but only the environment of the class accounted for unique variance in student achievement. However, the class environment (standard-based teaching practices) was the strongest independent predictor of both achievement and attitude.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Schmidt, W.H. (2001a). Defining Teacher Quality Through Content: Professional Development Implications from TIMSS . In J. Rhoton and P. Bowers (Eds.), Professional Development Planning and Design: Issues in Science Education, pp. 141-164. Arlington, VA: National Science Teachers Association.

This paper, based on the TIMSS study conducted prior to 1996, reviewed the science achievement testing results in the context of the curriculum and instruction provided in 40 countries. The achievement results in science ranged from being tied for second among TIMSS counties at the fourth-grade level, to being just slightly above the international average at the eighth grade, to being at the bottom of the countries at the twelfth grade. When looking at specific topic areas of the science tests, a picture emerges in which on some topics (e.g., organs and tissues), no countries outperformed U.S. students. U.S. students did best in life science and earth science on the grade 4 and grade 8 tests and performed worst in physical science. This pattern is consistent with the emphasis on life science and earth science in the seventh- and eighth-grade curriculum in the United States. The authors conclude that curriculum makes a difference and that the United States does not have a coherent, coordinated view of what we want children to know in science. The U.S. curriculum lacks focus and covers many more topics each year, compared to the rest of the TIMSS countries. This is true of state frameworks that define what children should learn, of textbooks, and of what is actually taught by teachers. Grade 8 textbooks in the United States cover 65 science topics, as compared to around 25 typical of other TIMSS countries. The authors note, “U.S. eighth-grade science textbooks were 700 pages long, hardbound, and resembled encyclopedia volumes. By contrast, many other countries’ textbooks were paperbacks with around 200 pages.” U.S. frameworks and textbooks lack coherence, failing to ideas connected to larger and more coherent wholes. The U.S. curriculum lacked intellectual rigor at the eighth grade and covered many of the same topics that were done in earlier grades.

Schmidt, W. (2001b). Paying the Price of “No Change.”East Lansing, MI: United States National Research Center Third International Mathematics and Science Study (TIMSS).

Schmidt documents that the TIMSS results showed little or no change in the ranking of the United States in either mathematics or science between 1995 and 1999. In science the ranking of the United States actually slipped slightly between 1995 and 1999. Thus there is no evidence that the introduction of standards has helped the United States to gain on other countries with respect to student achievement. The 1995 TIMSS report revealed that the middle school curriculum in both mathematics and science covered elementary topics such as arithmetic, descriptive biology, and earth science to the exclusion of the more advanced topics covered internationally such as algebra, geometry, chemistry and physics. The 1999 report also shows the same patterns. The results indicate that the United States is below the international average in mathematics but not different from it in science. Thus, Schmidt concludes that in many ways we still are where we were in 1995. In addition, the author also reports that for science a large percentage of U.S. students (28 percent) attend classes that mostly emphasize earth science, which is more than twice the international average of the 23 countries participating in both studies. On the other hand, only 5 percent of U.S. students are in classes whose teachers report that physics or chemistry is the most emphasized topic in their eighth-grade science class. The average of the 23 countries is almost five times larger. This implies that internationally one-quarter of the students in a typical country attend a class in which chemistry or physics is the main subject matter for their eighth-grade science class.

Schmidt, W.H., McKnight, C.C., Houang, R.T., Wang, H., Wiley, D.E., Cogan, L.S., and Wolfe, R.G. (2001). Why Schools Matter: A Cross-National Comparison of Curriculum and Learning.San Francisco: Jossey-Bass.

Schmidt, McKnight, Houang, Wang, Wiley, Cogan, and Wolfe try to answer a research question, “How does curriculum affect student learning?” In addition to data on students’ science and mathematics achievement, the Third International Mathematics and Science Study (TIMSS) and the repeat of the TIMSS study (TIMSS-R) data also include extensive information about the teaching practices and professional development of the teachers of the students in the study. This makes it possible to look for associations between teaching practices, curricula, or professional development and student achievement. On the one hand, the authors suggest a conceptual model relating curriculum and achievement. Based on the model, they argue that content standards, textbooks, and

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

teacher content knowledge are closely related to each other. Their original hypothesis was that all of these constructs take the significant effects on student learning in the United States. Based on the result of the 1995 TIMSS, content standards could not be related to the other variables in both mathematics and science. This showed a strong consistency between content standards and textbooks played both directly and indirectly in learning mathematics and science in the eighth grade in the United States. On the other hand, Schmidt et al. found that achievement in specific mathematics topics was related to the amount of instructional time spent on those topics, and that for some topics there was also a positive relationship with teaching practices that could be viewed as moving beyond routine procedures to demanding more complex performances from students, including (1) explaining the reasoning behind an idea; (2) representing and analyzing relationships using tables, graphs, and charts; and (3) working on problems to which there was no immediately obvious method of solution.

Schmidt, W.H., McKnight, C.C., and Raizen, S.A. (1997). A Splintered Vision: An Investigation of U.S. Science and Mathematics Education.Dordrecht, Netherlands: Kluwer Academic.

Schoen, H.L., Fey, J.T., Hirsch, C.R., and Coxford, A.F. (1999, February). Issues and Options in the Math Wars. Phi Delta Kappan.80(6), 444-453.

The article discusses the controversy referred to as “the math wars,” in which current reform efforts in K-12 and undergraduate mathematics are under attack. The article begins by describing the history and the foundation of the recommendations for reform in mathematics education. They describe that the NCTM Standards established a broad national consensus of the needed change. The authors justify the new directions in mathematics content recommended in the NCTM Standards based on changes in available technology (graphing calculators), changes in the ways mathematics are used in the workplace, results of comparisons of the United States with other countries, and recommendations from business and industry. The authors also describe that the reform efforts embrace research-based instructional and assessment practices. The article provides a detailed description of one reform-based curriculum project in mathematics that provides a model for mathematics reform—The Core-Plus Mathematics Project (CPMP). The results of the evaluation of CPMP yielded (1) CPMP students’ average pre-test to post-test growth on mathematical reasoning was nearly twice that of the norm group, and (2) on a mathematics subset of released items from the National Assessment of Educational Progress, the CPMP students’ means were higher on each of the six content and three process subtests than those of a nationally representative sample of students. The evaluation results also show that the CPMP students were more positively disposed toward mathematics and understood and were able to apply many important mathematics ideas significantly better than the traditional students to whom they were compared.

Schukar, R., Johnson, J., and Singleton, L.R. (1996). Service Learning in the Middle School Curriculum: A Resource Book.Boulder, CO: Social Science Education Consortium.

Shepard, L.A. and Bliem, C.L. (1995, November). Parents’ Thinking about Standardized Tests and Performance Assessments. Educational Researcher.24(8), 25-32.

Shields, P.M., Marsh, J.A., and Adelman, N.E. (1998, March). The SSIs’ Impacts on Classroom Practice.Menlo Park, CA: SRI International.

This report examines the impacts of 25 SSIs on classroom practice from 1991-1996. It includes tables with means, standard deviations, and highlighted significant differences. The report also includes an appendix with detailed methodology notes.

The report includes researchers’ analyses of case studies that involved visits to 12 states over a two-, three-, or four-year period. Overall, 10 to 20 person-days were spent at each site each year. Each study involved sampling three local districts that varied in socioeconomic status, urbanicity, and capacity for change. Within each district, up to three schools, representing a range of grade levels, were studied. Typically a sample of three to four teachers in each school was drawn, with teachers varying in their level of participation in their state’s SSI. Trained site visitors interviewed each of these teachers. In addition to these case studies, the researchers

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

analyzed SSI reports submitted to NSF, as well as teacher survey data collected by individual SSI internal evaluation teams.

The researchers found that there was general agreement among the SSIs on the problems in mathematics and science instruction, as well as the reforms in curriculum content and instructional strategies necessary for improvement. Researchers found that about 10 percent of the teachers participated directly and intensively in the SSI, but that contextual factors influenced the ability of the SSIs to impact classroom practice. Data showed that SSIs had some success in changing teachers’ attitudes, beliefs, and intentions, but that classroom impacts across and within SSIs were uneven. In the cases where classroom impact was demonstrable, it appeared to have less to do with adopting specific strategies and more to do with the quality of the design and implementation of those strategies.

Shields, P.M., Marsh, J.A., Marder, C., and Wilson, C.L. (1998). A Case Study of California’s SSI (CAMS), 1992-1997. In P.M. Shields and A.A. Zucker (Eds.), SSI Case Studies, Cohort 2: California, Kentucky, Maine, Michigan, Vermont, and Virginia. Menlo Park, CA: SRI International.

The case study describes the work and impacts of the California Statewide Systemic Initiative, which was largely focused on two teacher networks, Mathematics Renaissance (MR—middle school mathematics) and California Science Implementation Network (CSIN–K-5 science). The two networks provided and supported professional development to support implementation of the California mathematics and science curriculum frameworks. The California frameworks were strongly related to the first NCTM Standards and the Project 2061 Benchmarks. CSIN reached approximately 25 percent of the state’s K-5 teachers; MR reached about 50 percent of the state’s middle-grades mathematics teachers.

The professional development that the networks have provided is a partnership of universities and school districts. The networks are committed to long-term (several years per teacher), sustained, and intensive professional development. The report presents a vignette of classroom practice from one CSIN and one MR teacher. Themes from these two vignettes (and presumably others) are highlighted, with the conclusion being that some changes toward standards-aligned practice are evident, but room for growth remains. These vignettes are contrasted with a quotation from a teacher who is less apt to change practice due to lack of content background.

Science assessments, developed in a companion project to CSIN, were administered to 25,000 students in the state in grades 5 and 8 in 1996. Students in schools involved in CSIN for three or more years scored better than students in schools involved for two or fewer years on all three scales, but the comparability of these schools and students was not detailed. Also in 1996, MR used the New Standards Reference Examination to test mathematics achievement. In this case, scores of 3,250 students from a sample of MR classrooms were compared with scores of students in other states. On the exam’s three scales, a slightly larger percentage of MR students than comparison students scored “met the standard” or above on conceptual learning and problem-solving, and the advantage of MR students was fairly substantial on skill learning.

Shymansky, J.A., Yore, L.D., Dunkhase, J.A., and Hand, B.M. (1998, April). Do Students Really Notice? A Study of the Impact of a Local Systemic Reform. Paper presented at the Annual Meeting of the National Association for Research in Science Teaching, San Diego, CA.

The Science: Parents, Activities and Literature (i.e., PALs) project aimed to increase teachers’ content and pedagogical knowledge in order to move them towards an interactive-constuctivist model of teaching and learning in line with the National Science Education Standards. A professional development program was designed that provided teachers an experience with the interactive-constructivist approach as well as problem-centered inquiry. By the end of the four years, 70 percent of the elementary teachers in the district had participated in the PALs program.

To evaluate the success of the PALs program, comparison groups were formed of participant and nonparticipant teachers. The students of these teachers were given surveys that reflected constructivist learning environments and elements of the PALs program to assess (1) their perceptions of science teaching and (2) their attitudes toward science learning. The research questions focused on the influence of teachers’ years of experi

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

ence in PALs, students’ grade level, students’ gender, and any interaction effects of these three on students’ perceptions and attitudes.

Results suggest that teachers may require more than two years of experience implementing a standards-based reform before increases in student results (their perceptions of and attitudes towards science instruction) are evident. A competing hypothesis (not noted by the authors) is that those with more than two years of experience in PALs were early recruits and were teaching in ways consistent with the standards prior to their involvement. The results also suggest that students who have experienced more traditional science instruction in earlier grades may not respond positively to standards-based instruction in upper elementary grades.

Shymansky, J.A., Yore, L.D., Henriques, L., Dunkhase, J.A., and Bancroft, J. (1998, April). Students’ Perceptions and Supervisors’ Rating as Assessments of Interactive-Constructivist Science Teaching in Elementary School. Paper presented at the Annual Meeting of the National Association for Research in Science Teaching, San Diego, CA.

This is a verification study testing the use of student perceptions and attitudes along with supervisors’ expert ratings to measure teachers’ implementation of constructivist classroom strategies. This study took place within the context of a four-year local reform effort entitled Science: Parents, Activities and Literature (Science PALs), a collaborative endeavor undertaken by the University of Iowa and the Iowa City Community School District. The goal of the PALs project is to enable teachers to move toward an interactive-constructivist approach to teaching and learning that is in line with the National Science Education Standards and other reform documents of recent years.

This publication contains extensive discussion of constructivist practice and a detailed accounting of instrument development and verification. A pilot study demonstrated the usefulness of expert ratings combined with students’ perceptions and attitudes as a way of documenting science instruction. The final expert rating instrument was developed through use of literature along with internal and external consultation. This checklist, used by the science supervisor during classroom observation, consisted of eight dimensions reflecting features of constructivist approaches, interactive-constructivist strategies, and the PALs model. This instrument was intended to rate the use of interactive-constructivist approaches by teachers. The student perception and attitude items were developed in a similar manner. The student instrument was intended to assess the impact of teachers’ approaches on their students. It was determined that the two instruments had acceptable validities for exploratory research of the manner undertaken here, although the authors do not substantiate these claims.

The sample used to verify these instruments was a convenience sample of 52 elementary science teachers identified by the science supervisor. This sample represented all 16 elementary schools in the district, with fairly even distribution among grades 1 to 6. The teachers were either third-year participants in the PALs project or non-participants, but the number in each of these two subgroups was not specified. A total of 1,315 students completed the student survey. Data analyses yielded descriptive data, ANOVAs, and t-tests. The results of this verification study indicated that student perceptions and attitudes along with expert ratings of constructivist science teaching have only marginal validity.

Simon, E., Foley, E., and Passantino, C. (1998). Making Sense of Standards: Implementation Issues and the Impact on Teaching Practice.CPRE Research Reports. Available at: http://www.cpre.org/Publications/careport03.pdf [August 8, 2002].

This publication reports the results of a formative evaluation of a standards-based, district-wide school reform project in Philadelphia entitled Children Achieving. The basic tenets of this reform were standards, a system for accountability, decentralization of decision-making, and support for teachers and students. In this effort, standards were viewed as both a system of accountability and an approach to instruction. This report focuses primarily on implementation of the latter.

Reported findings were based on surveys, observations, and interviews and dealt with the influences on implementation of the reform at the district, school, and classroom level. At the district level, there were competing visions regarding the amount and kinds of guidance the district should provide about the curricula. This

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

confusion led to slower implementation efforts with teachers seeing little alignment and/or support from the central office.

At the school level, when leadership understood and supported standards-based instruction by focusing curriculum revisions on standards and by providing time and assistance to teachers for curriculum development, then teacher understanding and classroom practice were favorably influenced. Conversely, when school leadership focused on the accountability system involved in Children Achieving (i.e., the Stanford-9 Achievement Test), then teachers largely equated this test with the standards. Also, when existing school-based programs were standards-based, these contributed to shaping teachers’ practice to fit the standards; in schools with unfocused or competing programs, the standards became merely one program among many.

At the classroom level, teachers were generally aware of the standards, believed they understood their purpose, and supported their potential benefits for students. Nonetheless, most teachers believed that their current practice was effective and that they did not need to change their practice to meet the standards. Lastly, findings from classroom observations revealed that many classrooms were in transition. In general, a constructivist, standards-based approach was more prevalent in the lower grades. Even so, when the structure of innovative practice was in place, there was often a lack of deep student engagement with the content.

Singer, J., Marx, R.W., Krajcik, J., and Chambers, J.C. (2000, April). Designing Curriculum to Meet National Standards.Arlington, VA: National Science Foundation.

This is an evaluation report of a project to develop curriculum materials that serve diverse populations in an urban setting (Detroit Public Schools), which promote inquiry, connect with research on how people learn, and make extensive usage of learning technologies. The project evaluation of student learning using a pre-post test of content and processes yielded significant positive effect sizes for four different curriculum units (which were in development). The authors noted that the evaluation was not a controlled experiment and that there were large differences in effects among teachers for each unit. The authors propose several variables that might affect the results: the teacher, instruction, social-economic context, instructional resources, and administrative support. In addition, the authors found that it takes several iterations of curriculum revision to produce effective materials. Areas needing additional research and development include: supports to promote discourse among students, supports to help students learn from inquiries, and the role that instructional materials play in teacher learning.

The curriculum units were developed by a collaborative team—teachers, school and district administrators, university scientists, educational researchers, and curriculum specialists. Their curriculum approach is based on four elements of social constructivism: active construction of knowledge, situated cognition, community of learners, and discourse. The project uses the following curriculum design principles: contextualized learning; standards-based content; extended inquiry; collaboration among students, teachers, and scientists; usage of learning technology; artifacts as learning products; and scaffolds for teaching and learning. The authors also describe a project on “What Affects the Quality of Air in My Community” as an example of their curriculum development efforts. The goal of the unit is to help students learn core science content and to develop inquiry abilities. The authors employ multiple instructional strategies to engage students in learning. Learning technology for this unit provides a database of air pollution and an opportunity for the students to investigate changes in air pollution levels at different locations over time. Students are asked to identify variables, make comparisons, explore hypotheses, and form conclusions. They also use “Model Builder” to make qualitative models of cause-and-effect relationships for air pollution and “e-chem,” a visualization tool to construct and rotate three-dimensional representations of molecules.

This paper provides a good model for designing standards-based curriculum materials. It begins with identifying key principles of the Standards (goals, learning, teaching, assessment), collaboratively designs instructional materials, pilots the materials with multiple teachers, undertakes one or more cycles of revision and testing, and evaluates the effectiveness of the materials by examining student learning of science content and science inquiry.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Smith, M.S. (1994, April). The National Education Reform Movement. In Scientists, Educators, and National Standards: Action at the Local Level, Sigma Xi Forum Proceedings, Sigma XI, The Scientific Research Society, Research Triangle Park, NC, April 14-15.

Smith, P.S., Banilower, E.R., McMahon, K.C., and Weiss, I.R. (2002). The National Survey of Science and Mathematics Education: Trends from 1977 to 2000.Chapel Hill, NC: Horizon Research.

Smithson, J.L., Porter, A.C., and Blank, R.K. (1995, March). Describing the Enacted Curriculum: Development and Dissemination of Opportunity to Learn. Indicators in Science Education.Washington, DC: Council of Chief State School Officers.

Solano-Flores, G., and Nelson-Barber, S. (2001). On the Cultural Validity of Science Assessments. Journal of Research in Science Teaching.38(5), 553-573.

This article makes the case that cultural validity should be considered as a form of test validity in science assessment. Cultural validity is the effectiveness with which science assessment addresses the sociocultural influences that shape students’ thinking and the ways in which students make sense of science items and respond to them. The authors draw upon a large body of literature to make their argument that sociocultural influences will affect students’ views of science as well as how students respond to assessment activities— through student epistemology, student language proficiency, cultural world views, cultural communication and socialization styles, and student life context and values. Specific examples of student responses to science assessment activities are presented in the article to illustrate how students’ cultural and world views affect student responses to assessment items that do not accurately reflect their scientific understanding. Examples also are given to illustrate the exclusion from standards documents, including the NSES, of topics such as body-based measurement skills that are very relevant to many indigenous cultures. The article is a very thoughtful presentation of the issues related to sociocultural influences on students’ thinking. The authors stress the importance of taking these issues into consideration in assessment development, a process that generally does not give a great deal of attention to student diversity. Student diversity is more often considered in the weeding of assessment activities. For cultural validity to be fully incorporated into assessment, the measurement of cultural minority students needs to focus on understanding student thinking and the sociocultural influences that shape this thinking.

Spillane, J. (2000, February). District Leaders’ Perceptions of Teacher Learning. CPRE Research Report Series: OP-05. Available at: http://www.cpre.org/Publications/op-05.pdf [August 8, 2002].

This paper reports on part of a five-year study that examined relations between state and local government policy making and mathematics and science instruction. This particular paper focuses on the perceptions of 40 district policy makers in nine Michigan school districts about teacher learning and the learning opportunities that were provided for teachers in these districts. The paper includes a careful description of both the way that districts were selected for participation, the methods of data collection, and the analytical techniques. The qualitative methods employed by the author appear appropriate. The author uses a theoretical framework of three distinct approaches about learning to situate the beliefs of district policy makers. Based on interview responses, the author places policy makers in either a behaviorist perspective, a situated perspective, or a cognitive perspective. The behaviorist perspective, held by the overwhelming majority (85 percent) of the district leaders, maintained the traditional perspective that knowledge was transmitted by teachers and received, not interpreted, by students. The situated perspective, held by 13 percent of the district leaders, viewed learning as the development of practices and abilities valued in specific communities and situations. The cognitive perspective, held by only one leader in a suburban district, viewed learning as the active reconstruction of existing knowledge. The author traces how these views translated into the learning opportunities and curriculum of professional development (i.e., content, delivery method, materials) that were provided to teachers in the districts, and how this shaded district leaders’ perspectives on providing motivation for teachers to pursue learning opportunities. The author concludes the study by hypothesizing about the structural influences of their

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

work and external pressures that contribute to district leaders’ perceptions about teaching and learning and consequently about the types of learning opportunities that they provide for teachers in their districts.

Spillane, J.P. (2001). Challenging Instruction for “All Students”: Policy, Practitioners, and Practice. In S.H. Fuhrman (Ed.), From the Capitol to the Classroom: Standards-Based Reform in the States, One Hundredth Yearbook of the National Society for the Study of Education (Chapter 11, pp. 217-241). Chicago: University of Chicago Press.

Spillane, J.P., and Zeuli, J.S. (1999). Reform and Teaching: Exploring Patters of Practice in the Context of National and State Mathematics Reforms. Educational Evaluation and Policy Analysis.21(1), 1-27.

This article investigated 25 classroom teachers’ patterns of mathematics instructional practice in the context of national, state, and local efforts to reform mathematics education. The goal of the study was to look carefully within practice to understand progress of reform, identifying efforts that are in the direction of reform and those that remained unchanged. Both quantitative and qualitative methods were used to collect the data. The TIMSS questionnaire, with a set of items related to the reforms identified, was administered to 640 third-, fourth-, seventh- and eighth-grade teachers from nine Michigan school districts in mid-size city, suburban, and rural areas; 283 teachers responded (44 percent). A subsample of 25 teachers (18 third/fourth-grade and 7 seventh/ eighth-grade mathematics teachers) who reported practice that was fairly well aligned with the reform vision were interviewed and observed.

The analysis focused on the intersection of classroom tasks and discourse patterns with principled and procedural mathematics knowledge; three distinctively different patterns of instruction were identified, with some dimensions of practice found to be more responsive to reform than others. Pattern one, found in four of the 25 classrooms, was the closest to reform practices. It involved principled knowledge tasks and principled knowledge discourse. Pattern two, observed in 10 classrooms, was not as closely aligned with reform. While it highlighted principled knowledge tasks, the discourse focused more on procedural knowledge. Pattern three, evident in 11 classrooms, included aspects of reform such as group work and use of manipulatives; however instruction was primarily grounded in procedural knowledge tasks and discourse. This study highlights the need for caution in interpreting self-report data on standards-based practice; the authors noted that even when teachers report teaching in ways consistent with mathematics reforms, they create diverse responses to the reforms because of their beliefs, knowledge, and experiences.

Spiri, M.H. (2001). Children Achieving: School Leadership and Reform: Case Studies of Philadelphia Principals. The Evaluation of the Annenberg Challenge in Philadelphia. Philadelphia, PA: Consortium for Policy Research in Education.

SRI International (1998). “Appendix” Evaluations of Student Outcomes in Seven SSIs. In K.G. LaGuarda, Assessing the SSI’s Impact on Student Achievement: An Imperfect Science. Menlo Park, CA: Author.

Stecher, B.M., Barron, S., Kaganoff, T., and Goodwin, J. (1998). The Effects of Standards-Based Assessment on Classroom Practices: Results of the 1996-97 RAND Survey of Kentucky Teachers of Mathematics and Writing.CSE Technical Report 482. Los Angeles: CRESST.

This is the first report of a multiyear research project in Kentucky investigating the consequences of standards-based assessment reform at school and classroom levels. The influence of the Kentucky standards-based reform, driven by the Kentucky Education Reform Act (KERA), on teachers’ classroom practices in mathematics and writing was studied. A random sample of about 400 teachers from Kentucky responded to a written questionnaire on their classroom practices. Researchers selected a stratified random sample of 280 schools, grouped by gain in mathematics or writing biennial scores (1992-1994 vs. 1994-1996) (low, medium, and high) and by size (small and large). Four samples of 70 schools were selected, one each for grade 4 writing, grade 5 mathematics, grade 7 writing, and grade 8 mathematics. Seventy percent of the teachers sampled responded to the written survey. A closed-form question was used for most questions. Teachers were asked

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

about current practices and change in practices over the past three years. Statistical differences between responses for teachers in low- and high-gain schools were computed using chi-square and t-tests. Over one-third of the elementary teachers reported increasing the amount of time spent on science to four hours a week. Over half of the elementary teachers said they increased the frequency of the times when they had integrated mathematics with science. These are the only two findings related to science. Most teachers of mathematics felt that the changes in the school mathematics program did not have a large impact on state assessment scores; rather, improved performance was more related to greater familiarity with the test format. However, a greater number of teachers from schools with high gains than from those with low gains attributed higher student scores to improved practices associated with the state reform. Two-thirds of the grade 8 mathematics teachers from the high-gain schools reported that the NCTM Standards had a great deal of influence over content and teaching strategies compared to 37 percent of grade 8 mathematics teachers from low-gain schools. Teachers reported that the state assessments and the curriculum materials provided by the state had a strong influence on mathematics instruction. This is a comprehensive study based on teacher self-report information. Findings contrasting high- and low-gain schools are subjects for review and can be biased due to selection on the dependent variable.

Stefanich, G.P., and Egelston-Dodd, J. (Eds.). (1994). A Futures Agenda: Proceedings of a Working Conference on Science for Persons with Disabilities.Missoula, MT: Montana University Affiliated Rural Institute.

St. John, M., Carroll, B., Century, J., Eggers-Pierola, C., Hirabayashi, J., Houghton, N., Jennings, S., Tibbitts, F., and Von Blum, R. (1999, April). The Quality of the Teaching of Mathematics, Science and Technology in K-12 Classrooms in New York State. A Summary of Findings.Inverness, CA: Inverness Research Associates. Available at: http://www.inverness-research.org [September 3, 2002].

This report summarizes the findings of The New York State Landscape Study, a component of the New York Statewide Initiative (NYSSI) funded by the National Science Foundation (NSF) and evaluated by Inverness Research Associates. The purpose of the study was to determine the current status and quality of mathematics, science, and technology instruction in K-12 classrooms. The evaluation sample included seven randomly selected districts of varying types; a total of 156 K-12 classroom observations of mathematics, science, and technology (MST) lessons were conducted using an observation protocol developed by Horizon Research, Inc. In addition to summarizing the quality of MST teaching, this report provides data summaries that describe differences between MST lessons, and differences in quality between grade levels and different district types.

The findings from the classroom observation data indicate that only a small fraction of MST lessons reflected the vision for classrooms as stated in the national standards documents. The underlying culture of the classrooms interfered with student learning, and the lessons were not likely to enhance student ability and interest in the discipline. In comparing subject-specific lessons, the researchers found that technology lessons were rated favorably overall, with only minor differences between mathematics and science lessons. The variation in quality of lessons was found to be greater within each district than across districts, however significant differences were seen between urban and non-urban districts.

Concluding comments indicate that MST instruction in New York K-12 classrooms is merely in the beginning stages of effective implementation. The authors argue for ongoing examinations of the quality of teaching in real classrooms, in hopes that they can provide incentives and guidance for improvements in instruction.

Stepanek, J. (1997, June). School Improvement Program, Science and Mathematics Standards in the Classroom: It’s Just Good Teaching.Portland, OR: Northwest Regional Educational Lab.

Stevens, F.I. (1996). Opportunity to Learn Science: Connecting Research Knowledge to Classroom Practice. Mid-Atlantic Laboratory for Student Success.Philadelphia, PA: National Research Center on Education in the Inner Cities.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Stevenson, H.W. (1998, March). A Study of Three Cultures: Germany, Japan and the United States—An Overview of the TIMSS Case Study Project. Phi Delta Kappan.79(7), 524-29.

This article summarizes the results of the three case studies of mathematics and science teaching in the United States, Germany, and Japan. The studies used a quasi-ethnographic methodology that involved observations and interviews with families and teachers and information obtained from school authorities and government policy experts. The study focused on: national standards, teacher training and teachers’ working conditions, attitudes toward dealing with differences in ability, and the place of school in adolescents’ lives. Careful attention was given to the selection of research sites, hiring of researchers, and devising research procedures. Major findings included the following. The amount of national control of the science curriculum varies among the three nations. In the United States, there is no mechanism at the federal level for controlling the curriculum. Even though state and voluntary national standards do influence school curricula, there is a strong drive for local decision making in what is taught. In the United States, textbooks are the de facto curriculum, with publishers producing books that maximize sales. In Germany, the Conference of Ministers of Education, with representatives from each state, oversees the educational polices and coordinates the structure, institutions, and graduation requirements. This national-level effort forms a basis for a degree of comparability across the states. In Germany, the textbooks must conform to state guidelines and be approved by a state committee. Textbooks establish the content and organization of the courses, but the German teacher is able to develop his or her own course material. In Japan, the Ministry of Education develops national curricular guidelines and standards, but flexibility is given to schools to decide exactly what is to be taught at each grade level. The Ministry of Education approves the textbooks to ensure their adherence to the curriculum guidelines and quality of presentation.

Supovitz, J.A. (2001). Translating Teaching Practice into Improved Student Achievement. In S.H. Fuhrman (Ed.), From the Capitol to the Classroom: Standards-Based Reform in the States, The One Hundredth Yearbook of the National Society for the Study of Education , pp. 81-98. Chicago: University of Chicago Press.

Supovitz, J.A., Mayer, D.P., and Kahle, J.B. (2000). Promoting Inquiry-Based Instructional Practice: The Longitudinal Impact of Professional Development in the Context of Systemic Reform. Educational Policy.14(3), 331–356.

Supovitz, J.A., and Turner, H.M. (2000). The effects of professional development on science teaching practices and classroom culture. Journal of Research in Science Teaching.37(9), 963-80.

This study reports a strong and significant relationship between professional development and a teacher’s practice and classroom cultures. Both teaching practices and classroom cultures were affected most deeply after intensive and sustained staff development activities. Supovitz and Turner found that teachers’ self-reports of inquiry teaching practices and investigative classroom cultures depended on the quantity of professional development in Local Systemic Change projects. It was only teachers with more than two weeks of professional development who reported teaching practices and classroom cultures above average. It appears that it was somewhat more difficult to change classroom culture than teaching practices. The positive results came for teachers who had spent 80 hours in focused professional development. The best change in investigative culture came only after 160 hours of in-service education. Supovitz and Turner argue that standards-based classroom practices require substantial investments in standards-based curricula or professional development. All the LSC projects have a heavy standards emphasis and are required to use NSF-approved curriculum materials in support of their initiatives. Teachers in this study were provided with curriculum materials of grade-level appropriate and content-rich activities linked to larger science concepts as well as sequenced to meet national standards. The authors also argue that the most powerful predictors of reform teaching are (1) content preparation as an individual teacher factor and (2) school factors such as differences in class size, discipline, and time allocations.

Thiessen, D. (2000). Developing Knowledge for Preparing Teachers: Redefining the Role of Schools of Education. In K.S. Gallagher and J.D. Bailey (Eds.), The Politics of Education Reform, pp. 129-144. The National Commission on Teaching and America’s Future. Thousand Oaks, CA: Corwin Press.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Thompson, B. (2002). What Future Quantitative Social Science Research Could Look Like: Confidence Intervals for Effect Sizes. Educational Researcher.31(3), 25–32.

Thompson, D.L., Spillane, J., and Cohen D.K. (1994). The State Policy System Affecting Science and Mathematics Education in Michigan.East Lansing, MI: MSSI Policy and Program Review Component, Michigan Partnership for a New Education.

Thorson, A. (Ed.). (2000). Assessment That Informs Practice. Eisenhower National Clearinghouse for Mathematics and Science Education. Enc Focus.7 (2). Available at: http://enc.org/focus/assessment [August 8, 2002].

Tuomi, J. (1994, April). Teachers: The Vision Supported. In Scientists, Educators, and National Standards: Action at the Local Level, Sigma Xi Forum Proceedings. Sigma XI, The Scientific Research Society, Research Triangle Park, NC, April 14-15.

Underhill, R.G., Abdi, S.W., and Peters, P.F. (1994, January). The Virginia State Systemic Initiative: A Brief Overview of the Lead Teacher Component and a Description of the Evolving Mathematics and Science Integration Outcomes. School Science & Mathematics.94 (1), 26-29.

This article describes the Lead Teacher Component of an NSF-funded State Systemic Initiative, called Virginia’s Quality Education in Science and Technology (V-QUEST). Noting that both AAAS’ Project 2061: Science for All Americans and NCTM’s Curriculum and Evaluation Standards for School Mathematics urge schools to prepare mathematically and scientifically literate students, the authors argue that the traditional practice of teaching mathematics and science separately hinders students’ ability to develop into citizens who are literate in mathematics and science. After briefly describing the beliefs of the project’s planning team, the article explains how the lead teacher component of V-QUEST includes classroom activities that are designed to help teachers integrate the two subjects. The article goes on to share more details about the V-QUEST project as a whole, including its guiding principles, objectives, and strategies.

The article also shares some insights gained from the project’s pilot year and first summer institutes efforts; for example, they found that “our approach of focusing on conceptions and projects has been beneficial but inadequate.” It does not describe the evidence upon which these statements are based. While many of the project beliefs are consistent with national standards, integration of mathematics and science is the centerpiece of this reform initiative, but not central to the national standards documents.

Valverde, G.A., and Schmidt, W.H. (1997). Refocusing U.S. Math and Science Education. Issues in Science and Technology Online.Winter 1997. Available at: http://ustimss.msu.edu [August 8, 2002].

This is a report summarizing results from the Third International Mathematics and Science Study (TIMSS) that pertain to the status of the science curriculum in the United States. The achievement results in science ranged from being tied for second among TIMSS countries at the fourth-grade level, to being just slightly above the international average at the eighth grade, to being at the bottom of the countries at the twelfth grade. When looking at specific topic areas of the science tests, a picture emerges where on some topics (e.g., organs and tissues), no countries outperformed U.S. students. U.S. students did best in life science and earth science on the grade 4 and grade 8 tests and they performed worst in physical science. This pattern is consistent with the emphasis on life science and earth science in the seventh- and eighth-grade curriculum in the United States.

The authors concluded that curriculum makes a difference, and that the United States does not have a coherent, coordinated view of what children are to know in science. The U.S. curriculum lacks focus and covers many more topics each year, compared to the rest of the TIMSS countries. This is true of state frameworks that define what children should learn, of textbooks, and of what is actually taught by teachers. Grade 8 textbooks in the United States cover 65 science topics as compared to around 25 typical of other TIMSS countries. The authors note that “U.S. eighth-grade science textbooks were 700 or more pages long, hardbound, and resembled encyclopedia volumes. By contrast, many other countries’ textbooks were paperbacks with less than 200 pages”

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

(p. 3). U.S. frameworks and textbooks lack coherence, failing to connect ideas to larger and more coherent wholes. The U.S. curriculum lacked intellectual rigor at the eighth grade and covered many of the same topics that were done in earlier grades.

Van Zee, E.H., Iwasyk, M., Kurose, A., Simpson, D., and Wild, J. (2001). Student and Teacher Questioning During Conversations About Science. Journal of Research in Science Teaching.38(2), 159-190.

Vermont State Department of Education. (1996). Vermont’s Framework of Standards and Learning Opportunities. Montpelier, VT: Author.

This report describes Vermont’s framework of standards and learning opportunities. The document is to be used to provide structure for the development, organization, implementation, and assessment of curricula; to provide the basis for the development of a state, local, and classroom comprehensive assessment system; and to specify what may be included in statewide assessments of student learning. The framework has four main parts: vital results standards, field of knowledge standards, learning opportunities, and appendices that describe how the framework was developed and is to be used. Vital Results Standards include communication, reasoning and problem-solving, personal development, and civic/social responsibility. Fields of Knowledge Standards are provided in the following areas: (1) arts/language and literature, (2) history and social sciences, and (3) science, mathematics, and technology. Learning opportunities refer to issues of access, instruction, assessment and reporting, connections among subjects, and best practices in the fields of knowledge. The development of the framework began in 1993 and was completed in 1996, concurrent with the development of the NSES. Teachers, school administrators, school board members, parents and community members, health and human services staff, business and higher education representatives, consultants, staff of the Vermont Institute for Science, Mathematics, and Technology, and school improvement teams at the Vermont Department of Education were involved in the development of the framework. An effort also was made to reflect the work of the New Standards project in the Vermont Standards.

Von Driel, J.H., Beijaard, D., and Verloop, N. (2001). Professional Development and Reform in Science Education: The Role of Teachers’ Practical Knowledge. Journal of Research in Science Teaching.38(2), 137-158.

In this article, professional development focused on developing teachers’ practical knowledge is discussed in light of the current education reforms in science, including the NSES in the United States and reform documents in other western countries. Teachers’ practical knowledge is defined as the combination of experiential knowledge, formal knowledge, and personal beliefs held in the context of the teachers’ work. On the basis of a literature review, the authors argue that many reform efforts have been unsuccessful because teachers’ practical knowledge was rarely taken into account. The authors provide only skeletal detail about the studies they used.

Based on their review, the authors suggest that future studies with multi-method designs are needed to understand this complex type of knowledge. It is recommended that reform efforts take into account teachers’ practical knowledge from the start, and that changes in this knowledge be monitored throughout reform projects. The authors also conclude that long-term professional development programs are the best option for lasting change in teaching practices, with the following strategies showing the most potential: (1) learning in networks, (2) peer coaching, (3) collaborative action research, and (4) the use of cases.

Von Secker, C.E., and Lissitz, R.W. (1999). Estimating the Impact of Instructional Practices on Student Achievement in Science. Journal of Research in Science Teaching.36(10), 1110-1126.

Von Secker and Lissitz report on analyses of data on science achievement from the 1990 High School Effectiveness Study. They found that traditional teacher-centered instruction was related to lower average science achievement. There was a positive correlation between tenth-grade science achievement, as measured by science tests constructed by the Educational Testing Service, and laboratory-centered instruction. There is a positive relationship with individual environment and differences such as SES, gender, and minority. This study uses a hierarchical linear model (HLM) to estimate direct and indirect effects of instructional practices recommended by the NSES on individual achievement. It applied unconditional HLM and unconditional Within-School

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

HLM, as well as conditional Between-School HLM. These results suggest that the NSES are more likely to promote equity if they are supported by national, state, and local efforts to provide equal opportunities for access to laboratory facilities, equipment, and supplies. De-emphasizing traditional teacher-centered instruction is expected to increase average science achievement and minimize gaps in achievement between individuals of different socioeconomic status. However, from the HLM results, teacher-centered instruction does not cause inequity in achievement associated with SES, and multiple explanations for this association are reasonable. The findings suggest that instruction matters. School excellence and equity can be positively or negatively affected by the way science is taught.

Ware, M., Richardson, L., and Kim, J.J. (2000, March). What Matters in Urban School Reform. How Reform Works: An Evaluative Study of National Science Foundation’s Urban Systemic Initiatives. Study Monograph No. 1. Available at: http://www.systemic.com/publication.cfm#usi [August 8, 2002].

Warren, B., Ballenger, C., Ogonowski, M., Rosebery, A.S., and Hudicourt-Barnes, J. (2001, May). Rethinking Diversity in Learning Science: The Logic of Everyday Sense-Making. Journal of Research in Science Teaching. 38(5), 529-552.

Warren, Ballenger, Ogonowski, Rosebery, and Hudicourt-Barnes argue that it is crucial to understand children’s diverse sense-making practices as intellectual resources in science learning and teaching. The authors discuss how the relationship between everyday and scientific knowledge and ways of knowing has been conceptualized in the field of science education research. It is important to take seriously the ideas and ways of talking and knowing that children from diverse communities bring to science. Science learning is not simply the accumulation of different ways with words and ways of seeing. Rather, it is from different perspectives as a creative critical process, in which diverse ways with words and ways of seeing are probed, challenged, and perhaps even transformed to the benefit of all students. The authors suggest that the diverse ideas and ways of talking and knowing of all children be brought into contact with each other as well as with recognized canonical views and modes of organizing explanations and arguments. Too little attention has been paid by researchers and teachers alike to the potentially profound continuities between everyday and scientific ways of knowing and talking, and thus to the pedagogical possibilities that may be derived from such an analysis, especially for typically marginalized children. It is necessary to have a framework for understanding the everyday sense-making practices of students from diverse communities as an intellectual resource in science learning and teaching. Two case studies illustrate this point of view. Through analysis of Haitian American and Latino students’ talk and activity, the authors show how the students work to understand metamorphosis and experimentation with diverse sense-making practice.

Watson, S., Foley, E., Tighe, E., and Wang, A. (2001). Children Achieving: Recruiting and Retaining Teachers: Keys to Improving the Philadelphia Public Schools.Philadelphia, PA: Consortium for Policy Research in Education.

Webb, N.L. (1992). Assessment of Students’ Knowledge of Mathematics: Steps Toward a Theory. In D.A. Grouws (Ed.), Handbook of Research on Mathematics Teaching and Learning, pp. 334-368. New York: Macmillan.

Webb, N.L. (1997, April). Criteria for Alignment of Expectations and Assessments in Mathematics and Science Education. Research Monograph No. 8. Madison, WI, and Washington, DC: National Insitute for Science Education and Council of Chief State School Officers.

This monograph presents a conceptual framework for thinking about and analyzing the alignment among expectations and assessments. Alignment is defined as “the degree to which expectations and assessments are in agreement and serve in conjunction with one another to guide the system toward students learning what they are expected to know and do” (p. 3). Alignment is distinguished from validity because it is an attribute of the relationship between expectations and assessments rather than an attribute of an assessment only. Twelve criteria for judging alignment grouped into five general categories are specified: content focus, articulation across grades and ages, equity and fairness, pedagogical implications, and system applicability. Most commonly,

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

alignment has been thought of only as content focus, with the other categories being ignored. Explanations and illustrative examples of the 12 different criteria are drawn from research and literature in science and mathematics education. A content analysis of the NSES and the Benchmarks for Science Literacy is used to illustrate an expert review approach to studying alignment—in this case, alignment between two documents. The conceptual framework draws upon research and was developed with the input of an expert panel formed as a cooperative effort between the Council of Chief State School Officers (CCSSO) and the National Institute for Science Education (NISE) funded by the National Science Foundation.

Webb, N.L. (1999, August). Alignment of Science and Mathematics Standards and Assessments in Four States. Research Monograph No. 18. Madison, WI, and Washington, DC: National Institute for Science Education and Council of Chief State School Officers.

Reviewers analyzed the alignment of assessments and standards in mathematics and science from four states at a four-day institute. Six reviewers compared the match between assessment items and state standards in mathematics, and seven compared the match in science. Data from these analyses were processed and used to judge the degree of alignment on the basis of four criteria: categorical concurrence, depth-of-knowledge consistency, range-of-knowledge correspondence, and balance of representation. In science, seven analyses were performed—at two grade levels for two states and three grade levels for one state. The three states varied in the proportion of the standards found to be aligned with the assessments, but within each state there were only small differences among the grade levels. In general, the science standards and assessments were found to be aligned on three of the four criteria—categorical concurrence (number of items per standard), range-of-knowledge correspondence (proportion of objectives of standard assessed), and balance of representation (emphasis given to specific objectives on the assessment). The standards and assessment were less aligned on the depth-of-knowledge consistency criterion. A major goal of the study was to develop a valid and reliable process for analyzing the alignment among standards and assessments. The process did produce credible results that distinguished among the different attributes of alignment and detected specific ways in which alignment could be improved. The states that participated volunteered to be a part of the study and wanted the information in order to achieve better alignment of their assessments and standards. The study employed content analysis to derive the results and the researcher acknowledged that full alignment is determined by the degree to which standards and assessments work together to improve student learning.

Weiss, I.R. (1994) A Profile of Science and Mathematics Education in the United States: 1993.Chapel Hill, NC: Horizon Research.

This report presents results of the 1993 National Survey of Science and Mathematics Teaching conducted by Horizon Research, Inc. Six thousand teachers in grades 1 through 12 at 1,250 schools completed the survey after a process of sampling was used to select teachers who would accurately estimate the national population. An 88 percent response rate was obtained for school program representatives and 84 percent for science and mathematics teachers. Teachers gave information about their teaching practices, beliefs, and background. School representatives answered questions about the types of courses offered, money spent for different types of educational materials, and problems/obstacles that faced the school. The findings of this study include the movement of science and mathematics education toward current reform ideas. Specifically, hands-on activities have increased, especially in elementary mathematics. However, the goal of quality education for “all students” is still not in sight as inadequate facilities, equipment, and the lack of money to purchase consumable supplies are still formidable barriers. Lack of content preparedness is another obstacle for elementary teachers, although most high school teachers have more extensive backgrounds than their counterparts at lower grades. There is evidence that more teachers are participating in science and mathematics in-service activities, but the small amount of time spent on these activities apparently did not address teachers’ expressed needs for content preparedness and preparedness to teach a diverse student population (e.g., students of different ethnic groups, English Language Learners, and learning disabled).

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Weiss, I.R. (1997, June). The Status of Science and Mathematics Teaching in the United States: Comparing Teacher Views and Classroom Practice to National Standards. NISE Brief.1(3).

The brief addresses teacher attitudes about and classroom implementation of the NCTM Standards and the NSES, using data from the 1993 National Survey of Science and Mathematics Education conducted by Horizon Research, Inc. The 1993 National Survey involved a probability sample of 1,250 schools and approximately 6,000 teachers in grades 1-12 throughout the United States. Teachers were asked to provide information about their qualifications and preparedness, participation in professional activities, and beliefs about math and science instruction. Department heads or teacher-leaders were also asked to report about their school’s science and mathematics programs. The author focuses on the findings that although teachers typically report instructional objectives in line with the vision of the standards, classroom activities are often not well aligned with the recommendations of NCTM and NRC standards, and students do not have equal access to quality education as envisioned by the reform agenda. Support for these findings include the high proportion of classroom time spent learning basic facts and terminology and preparing for standardized tests, and evidence that classes with high percentages of minority students do not have access to the same resources as other classes. Based on the survey data, the author concludes that many teachers do not feel well prepared to teach various content areas or to use the recommended instructional strategies, nor do they feel they get the support they need to implement the recommendations. While many teachers reported support for pedagogical reform, the instructional strategies they reported using leave classroom practice far behind the vision described in the NSES, and the goal of “quality education for all” has not been reached. Implications of these findings and recommendations of the research for the education system include improving teacher preparation such that teachers are grounded in the content they are expected to teach; provided with models of effective standards-based instruction; and given the materials, facilities, and support they need to implement such instruction.

Weiss, I.R., Banilower, E.R., McMahon, K.C., and Smith, P.S. (2001). Report of the 2000 National Survey of Science and Mathematics Education.Chapel Hill, NC: Horizon Research.

This report summarizes data collected as part of two national surveys—one in 1993, another in 2000—of science and mathematics teachers in grades K-12 public and private schools. Both studies involved national probability samples. The 1993 study sampled 6,000 teachers, and the 2000 study sampled 9,000. Both samples allowed calculations of national estimates. In addition to the questionnaires completed by teachers, science and mathematics program representatives at each study school (approximately 1,000 in each study) completed a questionnaire.

Weiss, I. R., Banilower, E. R., Overstreet, C. M., and Soar, E. H. (2002). Local Systemic Change Through Teacher Enhancement: Year Seven Cross-Site Report.Chapel Hill, NC: Horizon Research.

Weiss, I.R., Matti, M.C., and Smith, P.S. (1994). Report of the 1993 National Survey of Science and Mathematics Education.Chapel Hill, NC: Horizon Research.

Weiss, I.R., and Raphael, J.B. (1996). Characteristics of Presidential Awardees: How Do They Compare with Science and Mathematics Teachers Nationally?Chapel Hill, NC: Horizon Research.

Wiggins, G. (1989, May). A True Test: Toward More Authentic and Equitable Assessment. Phi Delta Kappan.70 (9) , 703-713.

Wilcox, J., Hoover, J., and Burthwick, P. (1999, March). Disability Research Encompassing Native Americans in Math and Science: A Demonstration Inclusion Project. In Rural Special Education for the New Millennium, Conference Proceeding of the American Council on Rural Special Education (ACRES), pp. 185-190. Albuquerque, NM: ACRES.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

Wilcoxson, C. (1997, October). Achieving the Vision of the National Standards in Nebraska: A Framework as a First Step to Classroom Implementation. School Science & Mathematics.97(6), 311-315.

Wilson, S.M., and Floden, R.E. (2001). Hedging Bets: Standards-Based Reform in Classroom. In S.H. Fuhrman (Ed.), From the Capitol to the Classroom: Standards-Based Reform in the States, The One Hundredth Yearbook of the National Society for the Study of Education, Part 2, pp. 193-216. Chicago: University of Chicago Press.

This paper provides a preliminary analysis of a three-year study conducted by the Consortium for Policy Research in Education (CPRE), in which researchers tracked curriculum and assessment reforms in 23 school districts in eight states. Interviews were conducted with teachers, principals, and district staff from these 23 school districts “as they responded to local, state, and national pressures to reform teaching and learning.” In addition, four states were chosen for more intensive interviewing and observations, and all teachers were surveyed in the study’s third year.

The goal of the study was to determine the impact of standards-based reform by looking at two questions: (1) What varieties of standards-based reform do teachers encounter in schools? and (2) What is the impact of those reforms? In addressing these questions, the paper first describes the experiences of four schools that are representative of the view of standards-based reform. Then it examines three critical issues—teaching and learning, accountability, and communication—concerning standards-based reform and its impact. The analysis reveals two findings. First, the concept of standards-based reform is interpreted in a wide variety of ways, with perceptions differing even within schools. For some educators, it is hardly noticeable among the other reforms, but for others it has provided a clarity and language for thinking about instruction. Second, teacher interviews, classroom observation, and teacher survey data indicate that classroom practice reflects a balance between traditional and standards-based practices. Instruction still looks traditional, with a mix of reform-oriented practices.

Based on these findings, the authors highlight the hopes and concerns for standards-based reform, suggesting that while the rhetoric would make people believe it has the potential for transforming teaching and learning, the evidence is showing otherwise. Elements of reform may be evident, but traditional teaching is prevalent.

Wolf, R.M. (1998, May). National Standards: Do We Need Them?Educational Researcher.27(4), 22-25.

Wright, J.C., and Wright, C.S. (1998). A Commentary on the Profound Changes Envisioned by the National Science Education Standards. Teachers College Record.100(1), 122-143.

In this conceptual paper, the authors, from the perspective of a university faculty member who teaches physical sciences, voice their opinions about the nature of science literacy and how to attain it. The authors point out the difficult challenge of educating our students to achieve science literacy while simultaneously developing the capacity of science teachers to change the nature of the teaching and learning experience. They stress that the standards fail to define the problem they are trying to solve and do not define scientific literacy with sufficient precision required to guide classroom practice. They call for more specific, detailed descriptions of goals of science literacy and of the nature of teaching and learning than are found in the NSES.

The authors explain that while the NSES are a brilliant definition of what success is, they do too little to address the issue of implementation of the change required to achieve that vision. The authors believe that science faculty will see different messages about the goals and attitudes underlying the NSES based on their own perceptions of science literacy. The authors call for small-scale, authentic, inquiry-based projects to investigate strategies for implementing reform as a better approach than large-scale systemic reform efforts. They find that teachers and administrators need data, teaching toolkits, menus of approaches, good assessment tools, and clear examples of how changes are implemented and how they work before they will be prepared to tackle wholesale reform. The authors propose that active learning is the lever for moving along reform and that reform should shift from a focus on issues of control to the new paradigm of ownership.

The paper questions that the potential impact of standards on science curricula will be constrained unless: (1) science literacy is clearly defined and understood by all stakeholders, (2) reformed curricula develop higher-level conceptual understanding and problem-solving skills, (3) the student is given ownership and responsibility

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

for learning, (4) stakeholders change their attitude and understanding as to the nature of science literacy and how to achieve it, and (5) approaches to teaching, learning, and assessment must change.

Yager, R.E., Lutz, M.V., and Craven III, J.A. (1996, June). Do National Standards Indicate the Need for Reform in Science Teacher Education?Journal of Science Teacher Education.7(2), 85-94.

Yin, R.K., Noboa-Rios, A., Davis, D., Castillo, I., and MacTurk, R. (2001). Update and Ongoing Work: Cross-Site Evaluation of the Urban Systemic Program.Bethesda, MD: Cosmos.

This report describes the cross-site evaluation of the National Science Foundation’s Urban Systemic Program (USP). The USP is currently in 18 sites in two cohorts. The report describes both the formative and the summative components of the cross-site evaluation, including the research design, logic model, and research questions. The report describes a logic model that would explain different stages of systemic reform and proposes an evaluation design that would capture the “systemicness” of each site and the program as a whole. After discussing various traditional evaluation designs, the authors propose a replication design in which each site is considered to be a naturally occurring experiment and cross-site patterns are seen as evidence of replication. The evaluation design focuses on the components in each site that make them systemic. Proposed data collection includes interviews with key officials, document analysis, and direct field observations. The authors also report on their first year of field work with the five first cohort sites. They report early signs of “systemicness” around strategic vision, assessment, professional development, parent and community roles, pre-service education, resource convergence, and partnering. They also discuss the threat of external events to continued progress.

Yinger, R.J., and Hendricks-Lee, M.S. (2000). The Language of Standards and Teacher Education Reform. In K.S. Gallagher and J.D. Bailey (Eds.), The Politics of Education Reform, pp. 94-106. The National Commission on Teaching and America’s Future . Thousand Oaks, CA: Corwin Press.

Yoon, B., and Young, M.J. (2000, October). Validating Standards-Referenced Science Assessments.CSE Technical Report No. 529. Los Angeles: California University, Center for the Study of Evaluation. Center for Research on Evaluation, Standards, and Student Testing.

Zucker, A.A., Shields, P.M., Adelman, N.E., Corcoran, T.B., and Goertz, M.E. (1998, June). A Report on the Evaluation of the National Science Foundation’s Statewide Systemic Initiative (SSI) Program.Menlo Park, CA: SRI International.

This report is intended primarily for individuals with an interest in federal education policy. The final report in a series of more than 15 reports, this report summarizes and synthesizes findings from all other reports on a national evaluation of NSF’s Statewide Systemic Initiative (SSI). Through SSI, the National Science Foundation provided funding for five years to selected states undertaking ambitious system-wide reforms in science, mathematics, and technology education. Each state adopted different reform strategies for improving instruction in mathematics and science for all students. The appendices in this report summarize the implementation strategies and impact of the SSI for each state. The authors developed a conceptual model of systemic reform, both to incorporate all the elements that would play a role in achieving SSI’s objectives and to frame their evaluative process. To complete their final assessment, the authors pooled data from a variety of sources: quantitative data gathered annually from the principal investigators in each SSI, repeated site visits in every SSI and subsequent phone interviews, and secondary data analysis of data sets gathered by many SSIs to evaluate their own efforts. The analytical methodologies were not reported.

The authors examined the accomplishments and lessons learned by the SSI program and their application to standards-based reform efforts. The following accomplishments were observed: increases in inquiry-based instruction, development and use of high-quality instructional materials, improved professional development, standards-based state curriculum policies, assessments aligned with curriculum, improved student achievement, additional funding sources and mobilized stakeholders, and more highly developed leadership pools. The

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×

authors point out that these accomplishments only affected a small fraction of teachers and students within the states and more time is required to see reform efforts reach a larger population. The lessons learned from the SSI program and described in detail in this report will aid reform efforts in the future. As confirmed by the authors, the SSI program created a partnership between federal and state agencies and helped jump-start the movement toward standards-based reform in mathematics and science education.

Zucker, A.A., Shields, P.M., Adelman, N.E., and Humphrey, D. (1997). Reflections on State Efforts to Improve Mathematics and Science Education in Light of Findings from TIMSS.Menlo Park, CA: SRI International.

The purpose of this study was to investigate how states are implementing their standards. The data for this study came from data sets collected for prior investigations of State Systemic Initiatives and evaluations of the Dwight D. Eisenhower Mathematics and Science Education Curriculum Framework Projects. This report by SRI International summarizes the general findings from TIMSS and found similarities with SRI studies: The science curriculum tries to cover a great many topics but sacrifices intensity of coverage, and deeper understanding, by doing so. SRI studies have found that instructional materials are the weak link, especially in high school science.

Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 127
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 128
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 129
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 130
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 131
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 132
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 133
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 134
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 135
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 136
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 137
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 138
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 139
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 140
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 141
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 142
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 143
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 144
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 145
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 146
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 147
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 148
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 149
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 150
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 151
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 152
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 153
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 154
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 155
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 156
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 157
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 158
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 159
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 160
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 161
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 162
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 163
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 164
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 165
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 166
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 167
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 168
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 169
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 170
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 171
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 172
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 173
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 174
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 175
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 176
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 177
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 178
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 179
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 180
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 181
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 182
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 183
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 184
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 185
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 186
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 187
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 188
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 189
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 190
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 191
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 192
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 193
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 194
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 195
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 196
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 197
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 198
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 199
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 200
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 201
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 202
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 203
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 204
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 205
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 206
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 207
Suggested Citation:"8. Annotated Bibliography." National Research Council. 2003. What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10618.
×
Page 208
What Is the Influence of the National Science Education Standards?: Reviewing the Evidence, A Workshop Summary Get This Book
×
Buy Paperback | $47.00 Buy Ebook | $37.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

In 2001, with support from National Science Foundation, the National Research Council began a review of the evidence concerning whether or not the National Science Education Standards have had an impact on the science education enterprise to date, and if so, what that impact has been. This publication represents the second phase of a three-phase effort by the National Research Council to answer that broad and very important question.

Phase I began in 1999 and was completed in 2001, with publication of Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education (National Research Council, 2002). That report provided organizing principles for the design, conduct, and interpretation of research regarding the influence of national standards. The Framework developed in Phase I was used to structure the current review of research that is reported here.

Phase II began in mid-2001, involved a thorough search and review of the research literature on the influence of the NSES, and concludes with this publication, which summarizes the proceedings of a workshop conducted on May 10, 2002, in Washington, DC.

Phase III will provide input, collected in 2002, from science educators, administrators at all levels, and other practitioners and policy makers regarding their views of the NSES, the ways and extent to which the NSES are influencing their work and the systems that support science education, and what next steps are needed.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!