3
Evidence of the Influence of the National Science Education Standards on the Professional Development System

Jonathan A. Supovitz

University of Pennsylvania

Consortium for Policy Research in Education

The National Science Education Standards, first introduced in 1996, call for teachers to focus on the “big ideas” in science, use inquiry-based strategies, employ an array of pedagogical approaches ranging from didactic teaching to extended explorations, guide and facilitate the learning of diverse student populations, teach for understanding, and focus on students’ application of knowledge. The implications of this vision of standards-based instruction on the preparation of teachers are enormous. Training teachers to meet the challenges implicit in this vision of standards-based instruction indicates that teacher-preparation policies and programs need to improve the content knowledge and pedagogical strategies of teachers; improve their understanding of the diverse ways that students learn and understand; and enhance their abilities to frame questions, choose activities, and assess student learning appropriately.

For this paper, I have been asked to examine the extent to which the National Science Education Standards (NSES) have influenced the system of professional development. I investigate the evidence that the NSES have influenced various components of the professional development system that shape, construct, and deliver professional development at the national, state, and local levels. I also attempt to characterize the differing quality of evidence that contributes toward any conclusion of the influence of the NSES on the system of professional development. Rather than examining the influence of the NSES on particular professional development programs or on the practices of individual teachers, in this paper I take a macro perspective for examining the influence of the NSES on the various aspects of the system of professional development. For this analysis, I have examined and report primarily on the influence of the National Science Education Standards. When discussing the effects of standards at the state level, I also refer to a study by Cohen and Hill (2000) of the mathematics standards published by the National Council of Teachers of Mathematics, which were published more than five years before the National Science Education Standards.

Overall, I found that the influence of the NSES on the system of professional development appears uneven. On the one hand, there seems to be substantial evidence that they have influenced a broad swath of in-service professional development programs. Most of the evidence points toward the influence of the National Science Foundation and Title II of the old Elementary and Secondary Education Act, the Eisenhower program. On the other hand, there is less evidence that the NSES have successfully influenced the state and district policy structures that leverage more fundamental changes in such areas as professional development standards, teacher licensing, or re-certification requirements. Additionally, the evidence is thin that institutions of higher



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 64
3 Evidence of the Influence of the National Science Education Standards on the Professional Development System Jonathan A. Supovitz University of Pennsylvania Consortium for Policy Research in Education The National Science Education Standards, first introduced in 1996, call for teachers to focus on the “big ideas” in science, use inquiry-based strategies, employ an array of pedagogical approaches ranging from didactic teaching to extended explorations, guide and facilitate the learning of diverse student populations, teach for understanding, and focus on students’ application of knowledge. The implications of this vision of standards-based instruction on the preparation of teachers are enormous. Training teachers to meet the challenges implicit in this vision of standards-based instruction indicates that teacher-preparation policies and programs need to improve the content knowledge and pedagogical strategies of teachers; improve their understanding of the diverse ways that students learn and understand; and enhance their abilities to frame questions, choose activities, and assess student learning appropriately. For this paper, I have been asked to examine the extent to which the National Science Education Standards (NSES) have influenced the system of professional development. I investigate the evidence that the NSES have influenced various components of the professional development system that shape, construct, and deliver professional development at the national, state, and local levels. I also attempt to characterize the differing quality of evidence that contributes toward any conclusion of the influence of the NSES on the system of professional development. Rather than examining the influence of the NSES on particular professional development programs or on the practices of individual teachers, in this paper I take a macro perspective for examining the influence of the NSES on the various aspects of the system of professional development. For this analysis, I have examined and report primarily on the influence of the National Science Education Standards. When discussing the effects of standards at the state level, I also refer to a study by Cohen and Hill (2000) of the mathematics standards published by the National Council of Teachers of Mathematics, which were published more than five years before the National Science Education Standards. Overall, I found that the influence of the NSES on the system of professional development appears uneven. On the one hand, there seems to be substantial evidence that they have influenced a broad swath of in-service professional development programs. Most of the evidence points toward the influence of the National Science Foundation and Title II of the old Elementary and Secondary Education Act, the Eisenhower program. On the other hand, there is less evidence that the NSES have successfully influenced the state and district policy structures that leverage more fundamental changes in such areas as professional development standards, teacher licensing, or re-certification requirements. Additionally, the evidence is thin that institutions of higher

OCR for page 64
education, where pre-service professional development largely resides, have substantially changed their practices and programs since the introduction of the NSES. In the rest of this paper, I discuss how I arrived at these conclusions. After this introduction, I briefly describe the body of evidence that I examined, how it was compiled, and the framework I developed to conduct these analyses. I have organized the findings of this paper into three major sections, modeled after the National Research Council’s (NRC) framework for research in mathematics, science, and technology education (NRC, 2002). First, I examine the evidence of the influence of the NSES on policies and policy systems related to professional development. Second, I investigate the evidence of the influence of the NSES on the pre-service delivery system. Third, I explore the evidence of the influence of the NSES on the in-service professional development delivery system. The paper concludes with a discussion of how research can better investigate the relationship between the NSES and different components of the professional development system. CONCEPTUAL FRAMEWORK Thinking about how to weigh the evidence that could substantiate a case that the NSES have influenced the components of the professional development system, I refined a framework that was developed at the Consortium for Policy Research in Education (T.B. Corcoran, personal communication, 2001). It is displayed in Figure 3-1. In the figure, our confidence in any research-based knowledge is predicated by two factors. The first factor is the quality of the research that has been conducted to address a particular hypothesis. The second factor is the replicability of these findings. Thus, if one case study reaches a certain conclusion, we have little confidence in the generalizability of these results. However, if the results are confirmed repeatedly in studies that employ multiple research strategies, we can have increasing confidence that their findings are generalizeable. Thus, as the conceptual framework in Figure 3-1 implies, in order to consider how the NSES have influenced the various components of the professional development system, it is important to identify both the quality of the evidence and the extent to which studies reinforce each other (replicability) in order to assess the strength of the evidence of the influence of the NSES on professional development. FIGURE 3-1 Framework for building a body of research evidence.

OCR for page 64
THE BODY OF POTENTIAL EVIDENCE The body of potential evidence that I considered for this review was the set of papers provided to me by the NRC. These papers were culled from the literature base and are considered to be the primary evidence available about the effects of the NSES on professional development from the field since they were released in 1995. A broad range of articles, papers, reports, and books were reviewed for this paper. Collectively, they represent a wide array of documents, ranging from peer-reviewed journal articles, to small- and large-scale evaluation reports on a variety of local and national professional development projects, to policy briefs put out by various organizations, to edited books, to policy reports. Because of the fragmentation of American educational research—where work is being done in universities, by various organizations, as well as by private consultants and nonprofit evaluation companies—pulling together a comprehensive set of the literature is a monumental task. There are likely to be important pieces of work that were not considered in this review. However, since this is a macro perspective of the landscape, I believe this is a fair representation of the state of knowledge in the field. Adding further pieces of high-quality work would certainly influence the details of the story I am about to tell, but would be unlikely to change the pattern that emerges as one looks across the literature. A more difficult task was deciding where to draw the boundaries within the literature that was collected. This challenge was made easier by the system developed by NRC for this review. While many papers may touch upon influences of the professional development system, only those papers that were considered by the NRC to have professional development as their primary focus were considered. Thus, if a paper was primarily about a new curriculum or assessment system and described the professional development that surrounded that effort, it was not considered here. As another example, papers that analyzed the influence of professional development on teachers’ practices and describe the professional development experience that produced the instructional practices as the context for analyzing influences on practice were considered outside of the purview of this analysis. Of course, this distinction is a little bit messy because many papers and reports have multiple purposes and therefore some evidence may have been overlooked. Attribution is another particularly difficult issue. For example, in some cases authors would describe a professional development program that contained elements that seemed aligned with the NSES, but the authors did not mention the NSES as an influence. In these cases, I adopted a broadminded perspective, considering all that appeared to be consistent with the NSES to be so. ANALYSIS FRAME In developing a framework for investigating the influence of teacher development as a channel of influence on the NSES, the NRC (2002) considered three areas of focus: initial preparation of teachers, certification and licensure, and ongoing professional development. I have used these three categories as a basis for thinking about how to organize the literature reviews that I conducted. In the first category were those papers that discussed influence of the NSES on the policy domain more generally, although I look specifically at issues of certification and licensure. In the second category were those papers that focused on pre-service, or the systems that provided training to potential teachers, usually through their college or university experiences. The third topical category I created included papers on the in-service professional development system. More critical to the conclusions of this analysis are what I considered to be compelling evidence of the influence of the NSES on the different foci of professional development. To address this, I decomposed the papers into four different classes that present evidence of the influence of the NSES on professional development. The first class of paper presented some manner of empirical evidence about the influence of the NSES on some aspect of the professional development system. Within this class, different authors used a variety of qualitative or quantitative research methods to demonstrate some relationship between a program or intervention and its influence. Within this class, papers employed a range of research methods and strategies that could be considered of varying levels of rigor and thus persuasiveness. The second set of papers consisted of summa

OCR for page 64
ries of research done by others. Rarely did these papers include criteria for the evidence they considered, so it is difficult to disentangle the beliefs and assumptions of the authors and the evidence they marshal to support their claims. In the third category of papers, authors described some process or experience that they were involved in, but these papers were not intended as evidence of the impact of these experiences. The final class of papers was those where the authors made claims or statements that were not substantiated by any form of evidence. I considered the papers that presented empirical evidence and research summaries to be more convincing than descriptions or unsubstantiated claims. Results Based upon these two categorizations—focus and class—I constructed a matrix to examine how the papers are distributed. As can be seen in Table 3-1, the distribution reveals many interesting things about the evidence base underlying various dimensions of the influence of the NSES on the professional development system. The organization of the articles by topical category and quality of the evidence reveals many interesting patterns. First, it becomes obvious that the strongest body of evidence, both in terms of the sheer number of articles and those that are empirical, is around the influence of the NSES on the in-service professional development system. Conversely, evidence of the influence of the NSES in the policy realm and on pre-service is relatively sparse. Third, while most of the articles contained some form of evidence, or summarized research evidence, there were a substantial number that were either descriptive or contained largely unsubstantiated claims. Finally, even within the set of papers that presented empirical evidence of the influence of the NSES on professional development, there was wide variation in quality. One indicator of this was that, of the 35 papers I examined, only six were peer-reviewed, which is the traditional “stamp of quality” in the research field. More directly, many of the empirical studies I examined had flaws that reduced my confidence in their findings. For example, some were hampered by small sample sizes. Other studies had poor survey response rates that brought into question any findings as a result. Others did not describe their methodologies, making it difficult for me to determine the validity of the results. In other cases, authors over-reached their data, attempting to draw conclusions that were simply not supported by the evidence at hand. In the sections that follow, I describe and summarize the evidence of the impact of the NSES on each of the three categories of professional development—policy, pre-service, and in-service. To take into account the differing quality of the empirical evidence, I include a discussion of the quality of the empirical evidence base at the end of each section. TABLE 3-1 Matrix of the Quality of the Evidence and Different Components of the Professional Development System   Quality of Evidence   Papers That Present Empirical Evidence Papers That Primarily Summarize Research of Others Papers That Describe a Process or Papers That Make Unsubstantiated Experience Claims TOPICAL CATEGORIES Policy Influence of Professional Development 4 2 1   Pre-Service Professional Development System 4 1 2 1 In-Service Professional Development System 8 3 5 2

OCR for page 64
EVIDENCE OF THE INFLUENCE OF THE NSES ON PROFESSIONAL DEVELOPMENT POLICIES The evidence of the influence of the NSES on state and local professional development policies is thin. Much of the evidence that does exist comes from evaluations of the various Statewide Systemic Initiatives (SSIs) that were funded in the 1990s by the National Science Foundation (NSF). Corcoran, Shields, and Zucker (1998) conducted a cross-SSI analysis of the impact of the SSIs on various aspects of professional development. Using longitudinal case studies of 12 of the SSIs, site visits to the other SSIs, internal SSI documentation and evaluation reports, and monitoring reports from an external monitor, they compiled several findings relevant to policy. One strategy they reported that SSIs conducted was to change their state’s professional development system by “revising state policies for new teachers and recertification and building state delivery systems to provide professional development” (p. vi). They found that in almost all cases the SSIs’ professional development structures were set up outside of the states’ existing professional development infrastructures and consequently had less influence on the infrastructures that provided most of the learning opportunities for teachers. They concluded that the SSIs did not have the leverage or resources to have a widespread influence on the professional development system and, consequently, the system is still in need of restructuring, which reduces the ability to have broad influence. In a summary of the findings from across the SSIs, Blank (2000) reiterated the findings of Corcoran, Shields, and Zucker. Blank found that few states had directly linked the NSES for student learning in any subject to state policies regarding recertification, state and local funding for continuing education, or professional development of teachers. There were, however, some exceptions. Goertz and Carver (1998) described the Michigan Statewide Systemic Initiative’s (MSSI) strategy of working with policy makers to incorporate the principles of high-quality professional development into state policy. They pointed out that the MSSI focused less on providing direct service to teachers than on communicating a standards-aligned paradigm of professional development to those who provided it and supplying professional development to the main providers in the state. They also described how the co-directors of the MSSI’s professional development component played a leadership role in the development of the state’s new professional development standards. Only two papers directly focused on the crucial state policy of teacher licensing. One by Andersen (2000) is a description of Indiana’s certification program, which was in the process of changing from a system based upon completed coursework to one in which teachers would have to provide evidence of competence based on standards developed by the Interstate New Teacher Assessment and Support Consortium (INTASC). INTASC’s standards, the author explains, are based upon the standards of professional organizations, including the NSES. Both Indiana’s system and the INTASC standards appear to be promising reforms, although it appears premature to see evidence of their influence. The Education Trust (1999) presented the results of a national panel’s review of the content of teacher licensing exams in English, mathematics, and science in contrast to the expectations of state and national standards. They argue that if licensing exams are consistent with standards, they should test teacher preparation to teach the standards. The study focused on the two major examinations used in most states, the Praxis series by the Educational Testing Service and state-specific exams designed by National Evaluation Systems. The results of the review were not encouraging. The majority of the tests, the authors reported, were multiple-choice assessments dominated by high school-level material. In a few cases there were essay examinations that required candidates to demonstrate their depth of knowledge. But the essays were used by far fewer states than the lower-level, multiple-choice tests. Further, the reviewers found, knowledge for teaching was a gaping hole in the licensing exams. Despite the fact that the tests were mostly low level, the data on passing rates are fairly low, with between 10 and 40 percent of takers failing the tests. The authors conclude their paper by arguing that the licensing exams are not intended to set high expectations, but rather to establish a floor. The reason for this is due to the potential for litigation. Spillane (2000) offers a thoughtful view of district policy makers’ perspectives on teacher professional learning opportunities. Using interviews with district administrators, he developed a theoretical framework of three distinct approaches about learning to situate the beliefs of district policy makers. The behaviorist perspective, held by the overwhelming majority (85 percent) of the district leaders, maintained the traditional perspective that knowledge was transmitted by teachers and received, not interpreted, by students. The situated per-

OCR for page 64
spective, held by 13 percent of the district leaders, viewed learning as the development of practices and abilities valued in specific communities and situations. The cognitive perspective, held by only one leader in a suburban district, viewed learning as the active reconstruction of existing knowledge. Spillane traces how these views translated into the learning opportunities and curriculum of professional development (i.e., content, delivery method, materials) that were provided to teachers in the districts, and how this shaded district leaders’ perspectives on providing motivation for teachers to pursue learning opportunities. He concluded that the behaviorist perspective is in many ways inconsistent with the beliefs of effective teacher learning that are represented in the standards. The Quality of the Evidence In sum, although the number of studies that examined the influence of the NSES on professional development policies was quite small, the quality of these pieces was generally high. The SSI studies, the Education Trust report, and the Spillane piece were all examples of solid educational research. Together, they suggest that the NSES have had only a weak and variable influence on the policy structures that play a crucial role in providing guidance to a variety of implementing agencies. EVIDENCE OF THE INFLUENCE OF THE NSES ON PRE-SERVICE DELIVERY SYSTEMS In the articles that I reviewed, seven focused primarily on the system of preparing teachers for entering the teaching profession. Four of these contained empirical data, while three were descriptive or made arguments without data to back them up. Overall, these studies left the impression that the NSES had not made substantial inroads into changing the practices in the institutions of higher education that are the primary deliverers of pre-service professional development to teachers. Several studies reinforce the notion that the colleges and universities that prepare teachers have not incorporated the NSES into their teacher preparation programs. Luft and Cox (2001) conducted a survey of first-year teachers in Arizona, which included questions about their pre-service experience. The results of the survey must be interpreted with caution, since only 47 percent of the teachers who were sampled responded. Many teachers reported that their pre-service program did not provide them with an adequate understanding of the national standards, which they rated amongst the lowest aspects of their pre-service program. The NRC (2000a) reviewed research on the state of pre-service professional development and reported, “the preparation of beginning teachers by many colleges and universities does not meet the needs of the modern classroom” (p. 31). Together, these studies suggest that pre-service experiences of teachers, five years after the introduction of the NSES, did not inform participants adequately about the NSES. In the evaluation reports of the SSIs, few reported that they seriously tackled the difficult challenge of influencing the higher education system that overwhelmingly provides pre-service experiences to teachers. Since many of the SSIs were housed in institutions of higher education and most provided training to teachers, there clearly must have been some influence on higher education faculty members. However, what I was looking for, but did not find, was broad evidence that the SSIs had systematically tackled the pre-service systems in their states, and to what effect. There were, however, a few cases where pre-service was a focus of the work of an SSI. For example, Goertz, Massell, and Corcoran (1998), in their case study of Connecticut’s Statewide Systemic Initiative reported that, although the SSI lacked leverage with higher education institutions, they instigated conversations about the preparation of teachers and the pre-service structures in the state, and several institutions altered courses and institutionalized co-teaching. There also were a few papers and books that described plans and efforts by universities to redesign their teacher preparation programs to align them with the conceptions of teaching and learning underlying the standards movement. However, the evidence of the effects of these efforts was mostly lacking. Pissalidis, Walker, DuCette, Degnan, and Lutkus (1998) described a framework that they planned to use in Philadelphia, Pennsylvania, for pre-service education, which is in many ways consistent with the elements advocated in the NSES, based

OCR for page 64
on construction rather than transmission of knowledge, cooperative learning, and authentic assessment. Powers and Hartley (1999) edited a book that described the collaboration between six Colorado universities and community colleges, funded by the National Science Foundation, to change their teacher preparation programs in science, mathematics, and technology. The book includes chapters from faculty members in the various institutions about how they restructured their classes with mini-grants and guidance from those leading the collaboration. Relevant chapters include descriptions of changes in instruction for biology, chemistry, geography, and general science for nonmajors classes from more traditional didactic delivery to more authentic, group problem-solving and inquiry structures that are consistent with instruction advocated by the NSES. Some of the chapters are descriptive, focusing on changes in the courses and the instructors’ intent behind these changes, but others include survey or interview data that either contrast students’ experiences in these or more traditional classes, or describe the influence of these courses on student learning and understanding. Finally, a few intriguing studies shed some light on the implications of aligning the NSES and pre-service experiences for teachers. Hammrich (1997) described her attempts to engage students in her teacher preparation classes in activities that gave them practice in applying the NSES to their classroom lessons. Using qualitative methods and a quasi-experimental design to detect influence, she found that teacher-candidates’ conceptions of effective science instruction were directly influenced by their conception of science, that they had differing views on the teachers’ role in students’ construction of knowledge, and that the principles reflected in the national reform initiatives were viewed as beneficial, but time-consuming, and may not be worth the time investment. She concludes that pre-service experiences of teachers must be dramatically changed in order for teachers to apply the principles of the NSES in the classroom. Pate, Nichols, and Tippins (2001) argue that service learning is a way to develop a more authentic representation of the nature of science and the self-generation of questions for inquiry that are promoted by the NSES. Using artifacts generated by a small number of pre-service teachers, they contend that prospective teachers can gain understanding of culture as the way groups of people socially negotiate their everyday living circumstances in local settings. The Quality of the Evidence Overall, the evidence base of the influence of the NSES on pre-service professional development is extremely thin. There were no empirical studies that examined changes in pre-service professional development systems that could in any way be attributed to the introduction of the NSES. The two studies that did describe attempts to change pre-service institutions were descriptive, not analytical, in nature. The remaining studies that examined influences in pre-service were small-scale studies of the implications of the NSES on different types of pre-service experiences (service learning and the implications of applying the principles underneath the NSES to the classroom). Thus, beyond pockets of clear influence, we are left to wonder the extent to which the NSES have changed the way that the pre-service industry prepares tomorrow’s teachers. INFLUENCE OF THE NSES ON IN-SERVICE PROFESSIONAL DEVELOPMENT PROGRAMS The largest body of evidence related to the impact of the NSES on teachers’ professional learning opportunities resides in the area of in-service professional development. There was a fairly broad set of research evidence that indicates that the NSES have had an influence on the professional learning experiences that many current teachers receive. Several major research studies conducted at the national, state, and local levels collectively provide a substantial base of evidence that the NSES have influenced the learning opportunities of a substantial number of teachers, mostly through federally funded programs. Thus, as we saw in the earlier chapter on curriculum, federal funding appears to have deepened the implementation of the NSES. By contrast, the evidence suggests that the NSES have been less successfully incorporated into the existing state and district in-service delivery systems. Although there are many ways the studies that informed this conclusion could be organized and presented, the level of influence—national, state, or local—seemed to be an appropriate way to sort them, so I have used this as an organizing heuristic.

OCR for page 64
The Influence of the NSES on In-Service Professional Development Nationally Two national evaluations of major federal initiatives, the National Science Foundation’s Statewide Systemic Initiative (SSI) and the Eisenhower mathematics and science professional development program, suggest that their focus and emphasis and their reach in terms of the proportion of teachers served were consonant with the vision of the NSES even before these documents had been widely disseminated. Corcoran, Shields, and Zucker (1998) conducted an evaluation of a variety of dimensions of the 25 SSI professional development programs. Using longitudinal case studies of 12 of the SSIs, site visits to the other SSIs, internal SSI documentation and evaluation reports, and monitoring reports from an external monitor, they compiled several findings. First, they concluded that the SSIs invested heavily in professional development. They also found that the learning opportunities provided by the SSIs met contemporary standards of quality, for which they included many components consistent with the NSES, including subject-matter focus; research-based, coherent, and sustained experiences; active learning; and teacher involvement in design, emphasizing teacher subject-matter knowledge. They also found that the reach of SSI professional development, although they served “tens of thousands” of teachers, “in most states only touched a small proportion of the teaching population because the SSI professional development, for the most part, was not integrated into the states’ professional development infrastructure” (p. v). There were two evaluation reports of the federal government’s Eisenhower mathematics and science professional development program that provided evidence of its national scope and influence. The Eisenhower program is Title II of the Elementary and Secondary Education Act (ESEA), which is the federal government’s largest investment in teacher professional development. The first report, by Birman, Reeve, and Sattler (1998), described six exploratory district case studies conducted in the spring of 1997. The authors viewed these case studies primarily as a way to familiarize themselves with some of the sites and to identify themes for more in-depth exploration. The findings of the report are organized around 10 emerging themes. The themes, or findings, are quite broad. For example, the authors report that the program supported a wide variety of activities, that most efforts went toward mathematics and science professional development, that most of the professional development that the funding supported was consistent with standards for high-quality professional development, and that the reliability of the Eisenhower funding allowed districts to engage in long-term planning and to leverage other funds. Overall, the authors conclude that the Eisenhower-funded activities emphasized several elements of high-quality professional development, including sustained and intensive professional development, the use of teachers as leaders, and promotion of alignment with high standards. They found that the Eisenhower coordinators were able to identify some components of high-quality professional development. The second report, a follow-up of the first by Garet, Birman, Porter, Desimone, Herman, and Yoon (1999), synthesized the lessons from the Eisenhower mathematics and science professional development program. The second-year evaluation was based upon a sophisticated sample and analysis of the survey results of a nationally representative probability sample of teachers in districts, 10 in-depth case studies in five states, and an ongoing longitudinal study of teacher change. The Eisenhower program is large; its 1999 appropriation was $335 million, providing funds through state education agencies to school districts, institutions of higher education, and nonprofit organizations. Beyond this, the report does not estimate the reach of the Eisenhower program. The results on the effectiveness of the Eisenhower program were mixed. On the survey, about 70 percent of teachers who participated in the programs reported effects on their knowledge of mathematics and science, but only roughly half of the teachers in the sampled districts reported influence. The findings relative to the quality of Eisenhower-assisted activities suggest that most were traditional workshops rather than alternative forms of learning opportunities such as study groups, networks, or mentorships. The authors also found that relatively few of the activities emphasized collective participation of teachers in schools or districts, but mostly focused on individual teachers. Finally, content emphasis, active learning, and coherence were evident in about 60 percent of activities observed. The report also discusses district and higher-education-institution management of Eisenhower-assisted activities and finds that co-funding, alignment, continuous improvement, and teacher involvement in planning lead to higher-quality professional development.

OCR for page 64
The Influence of the NSES on In-Service Professional Development at the State Level The major source of evidence surrounding the influence of the NSES on in-service professional development at the state level was the individual evaluation reports of the SSIs. These evaluations show generally wide reach of the SSIs in states, but mixed influence on the structural elements of the states’ systems. For example, Corcoran and Matson (1998) conducted a case study of Kentucky’s SSI, called the Partnership for Reform Initiatives in Science and Mathematics, or PRISM. Drawing on extensive visits to the state and interviews, the case study describes the main strategy employed by PRISM as developing regional cadres of specialists in mathematics, science, and technology who would model and spread the new approaches to teaching and learning aligned with the NSES. Although PRISM reached nearly 2,500 teachers with its various initiatives, the authors find that the designers of the SSI made flawed assumptions that impeded the implementation of their strategy. They assumed that the specialists would be willing and able to provide professional development to their peers. They also assumed that local administrators would value the specialists and provide opportunities for them to work with their peers and play leadership roles in their schools. The fact that PRISM essentially set up a professional development system outside of existing professional development providers in the state raises questions about how deeply the NSES influenced the existing professional development apparatus in the state. Goertz, Massell, and Corcoran (1998) conducted an evaluation of Connecticut’s SSI, called CONNSTRUCT. The authors report that two of the SSI’s major strategies were to develop an independent academy to serve as a catalyst, advocate, and broker for reform and to focus assistance on 19 urban and rural disadvantaged districts. Overall, the authors concluded that the results of these in-service strategies were variable, due to the weak position of the SSI outside of the state’s system and its dependence on the willingness and capacity of districts and schools to identify their need, tap the resource networks, and use resources to institute curricular and instructional changes. Luft and Cox (2001) reported the results of a survey of district administrators in Arizona. The district survey was focused on the extent to which districts had induction systems to support science and mathematics teachers in their early years of teaching. Luft and Cox argued that teachers who are not supported as they begin teaching will resort to more traditional strategies as they encounter the challenges of day-to-day difficulties of teaching. Through the survey, which had a response rate of 74 percent, the authors found that most districts did not have any induction system for new science and mathematics teachers. About 20 percent had formal mentoring programs, the most common form of induction. Of these, 68 percent lasted for only one year. Only 24 percent of beginning teachers in small districts and 59 percent in large districts reported participating in induction programs. Thus, there is relatively little assistance given to most beginning mathematics and science teachers. Even in districts with formal mentor programs, one-third of teachers did not receive mentors and only one-half of those who did receive mentors received same-discipline mentors. Interestingly, studies of the influence of the National Council of Teachers of Mathematics (NCTM) Standards have produced similarly weak influence at the state level. Cohen and Hill (2000) examined the alignment between the learning opportunities that teachers in California had experienced after the introduction of the state frameworks, which were heavily influenced (and thus presumably aligned) with the NCTM Mathematics Standards. The study suggests two important things about the relationship between standards and professional development. First, about half the teachers in the study reported attending some professional development consistent with the frameworks that suggested that the Mathematics Standards had just started to create expanded opportunities to receive reform-oriented professional development. However, while the content of professional development opportunities was appropriate, teachers were not given the depth of opportunities necessary for widespread changes in practice, as most teachers were still attending short workshops. Second, the authors did demonstrate a relationship between curriculum-specific professional development and changes in practice, while generic workshops (e.g., cooperative learning, Family Math) did not have an influence on practice. This provides evidence of the importance of focusing on increasing the content knowledge of teachers and providing ongoing and sustained experiences that are advocated in the NSES.

OCR for page 64
The Influence of the NSES on In-Service Professional Development Locally Several studies speak to the quality, reach, and influence of professional development at the local level. A book published by the National Science Resources Center (NSRC, 1997) described the organization’s strategy for bringing about district-wide elementary science reform consistent with the NSES. The NSRC’s model views elementary science as a cohesive system that includes inquiry-centered science curriculum, professional development, materials support, appropriate assessment, and system and community support. The book also contains eight case studies of districts’ efforts to implement the NSRC model, written by the leaders of the district reform efforts. The eight districts are Montgomery County, Maryland; Spokane, Washington; East Baton Rouge Parish, Louisiana; Cupertino, California; Huntsville, Alabama; Pasadena, California; San Francisco, California; and Green Bay, Wisconsin. The eight case studies include descriptions of the professional development strategies of the districts, which are consistent with the NSES approach to teacher training (ongoing, intensive, content-based, inquiry-oriented, providing ready access to materials, in some cases developing lead or master teachers and involving professional scientists). The case studies are descriptive and are not designed to provide evidence of the influence of these programs on either the professional development systems of these districts or the professional knowledge and skills of the participating teachers. Huinker, Pearson, Posnanski, Coan, and Porter (1998) reported as part of the formative evaluation of the first year of the National Science Foundation-sponsored Milwaukee Urban Systemic Initiative (MUSI). The main strategy of the MUSI was to develop a cadre of mathematics/science resource teachers that each served two schools in order to build capacity for change at the classroom, school, and district levels. The report does not describe other aspects of the MUSI structure. The researchers took the resource teacher reports and organized the data into themes, which included how the resource teachers assessed the needs of their schools, developed strategies to meet the needs of their schools, provided professional development in their sites, contributed to a district community of learners, and worked with principals. The authors conclude that, through teachers’ self-reports, the resource teachers demonstrate that they have been actively involved in improving mathematics and science teaching and learning in a variety of communities, including the classroom, school, and district. The variety of professional development activities offered by the resource teachers reflected many aspects of the NSES, including offering formal staff in-service, mentoring at grade level, facilitating the development of school action plans, assisting teachers to prepare students for high-stakes testing, participating with teachers in other professional development activities and then helping them reflect and discuss implications for instructional practice, and arranging teachers to visit and observe each other’s practice. Kim, Crasco, Blank, and Smithson (2001) conducted an analysis of surveys completed by elementary and middle school teachers in eight Urban Systemic Initiative (USI) sites in 1999 and 2000. The survey instrument used, called the Survey of Enacted Curriculum, is a sophisticated self-report survey instrument developed at the University of Wisconsin-Madison by Andrew Porter and John Smithson. The survey asked teachers about their curriculum coverage, classroom practices, and professional development experiences. The response rate reported in 1999 was 61 percent. The authors do not report the response rate for 2000, although they do say it was better than in 1999. Relevant to this chapter are the authors’ findings that 80 to 90 percent of the USI teachers were actively involved in professional development, which they reported was focused on content standards, in-depth study of content, curriculum implementation, multiple strategies for assessment, and new methods of teaching. Teachers also reported that the professional development they received was being used and applied in the classroom and that state and district standards and frameworks influenced their curriculum. Adams and Krockover (1999) sought to relate a single science teacher’s use of the Secondary Science Teaching Analysis Matrix (STAM), which is consistent with the style of teaching advocated by the NSES, with his development over time from a didactic to a more constructivist teacher. Citing others, the authors argued that, despite pre-service experiences, beginning teachers often adopt “survival strategies” rather than those advocated by the NSES. Using a mechanism like STAM, they argue, teachers can conduct self-assessment and have a heuristic to guide them toward more student-centered styles of teaching. The authors analyzed their data with several qualitative analytical techniques, including analytic induction, extensive use of memos, and synthesis of the various data sources. The authors inferred that, since both the subject of the case (named Bill) and their own data pointed to the influence of the STAM as a roadmap for Bill’s progression from a didactic to a

OCR for page 64
constructivist teacher, the use of such an instrument can help novice teachers reflect on and change their teaching practice. The Quality of the Evidence Overall, both the quantity and the quality of the evidence on in-service professional development increase our confidence that the NSES have influenced the way that science professional development is provided to a large number of current teachers. Although it is hard to get a handle on the proportion of teachers that have received standards-based science professional development, the large scope of both the Eisenhower and the NSF programs suggest that this influence has been extensive, although still only accounting for a small proportion of the national population of teachers of science. Our confidence in the influence of these in-service programs is further enhanced by the quality of the research. Both the Eisenhower and the SSI evaluations are high-quality, mixed-method studies that report broad national influence. The various studies that reported survey results appeared to have reasonable designs, response rates, and analytical techniques. By contrast, the studies of local impact were descriptive or in their early stages, leaving uncertain the influence of the NSES on district professional development infrastructures. WHAT COMES NEXT IN THE RESEARCH The body of literature I reviewed came from a diversity of sources and had a multiplicity of purposes. Few of the authors explicitly set out to establish a relationship between the NSES and any aspect of the professional development system in the United States. Some were intended to be empirical works, while others were designed to lay forth arguments about the importance of reforms advocated by their authors. As this review shows, if we strip away many of the latter pieces and just consider empirical evidence that establishes a reasonable link between the NSES and the professional development system, then the evidence of the influence of the NSES on the system of professional development is variable. Although the NSES have unquestionably influenced in-service professional development for large numbers of teachers, the evidence is unconvincing that there have been structural changes in either the policy system, the institutions of higher education that largely provide training to prospective teachers as they prepare to enter the profession, or the existing structures that provide large amounts of in-service training to teachers. Even this finding may be overstated because of the fact that much of the examined research focused on those places where reform is going on, thus increasing the likelihood of finding effects that are unrepresentative of the nation as a whole. However, if we adopt a broader view and consider all of the products, regardless of their purpose, as evidence that the NSES are influencing the discourse around how to construct a professional development system in support of the NSES, then we might reach a different conclusion. For taken together, after reading all of the papers, briefs, reports, and journal articles, one cannot help but to have the impression that the NSES have focused the conversation and contributed to a freshly critical evaluation of the systems and policies that prepare and support teachers to deliver the kinds of instruction advocated by the NSES. What is lacking is empirical evidence that the NSES have had a deep influence on the structures and systems that shape professional development in this country. There may be two reasons for this lack of evidence. First, it may be premature, just six years after the release of the NSES, to expect that the leaders of systems as slow changing as policy structures and pre-service institutions will have made structural reforms. Second, there seem to be few research studies that conduct the kinds of policy and organizational research that would provide evidence of these changes should they exist. If the first of these two reasons is predominantly true—that deep-rooted changes have not yet occurred, particularly in the policy and pre-service areas—then conducting better research will only further substantiate these preliminary conclusions. However, if changes are beginning to occur, then we clearly need more targeted and better quality research to explore how the landscape is changing and how the NSES have influenced that process.

OCR for page 64
In the research I reviewed, a few studies stood out as the kinds of research that are needed. Yin, Noboa-Rios, Davis, Castillo, and MacTurk (2001) described a logic model developed as part of a plan to conduct a cross-site evaluation of NSF’s Urban Systemic Initiative that would explain different stages of systemic reform. The evaluation design is intended to capture the “systemicness” of each site and the program as a whole using a replication design in which each site is considered to be a naturally occurring experiment, and cross-site patterns are seen as evidence of replication. Although it was too early in the work for Yin et al. to report results, the model is a promising approach to capturing some of the policy and structural influences of the NSES on the systems that undergird the delivery of professional development. Likewise, the Eisenhower evaluations and SRI’s cross-site SSI evaluation were exemplars of high-quality, thoughtful studies that provided substantial evidence of where and why the NSES have and have not influenced the different aspects of the professional development system. Additionally, studies like Spillane’s investigation of how policy makers’ beliefs about learning influence their policy strategies provide fresh insight into the often superficial levels of understanding of those leaders charged with enacting the NSES and the profound influence of local culture and context on the implementation process. There are also several important areas where research is largely silent. There are several professional organizations that have traditionally provided guidance to professional developers, but we know little about the influence of the NSES on the way these organizations provide leadership for their members. For example, there are several organizations that accredit universities to provide pre-service education, such as the National Council for Accreditation of Teacher Education and the Teacher Education Accreditation Council, as well as the new Interstate New Teacher Assessment and Support Consortium. It would be worthwhile to specifically study whether and how these organizations have changed their systems since the advent of the NSES. Additionally, there are also professional organizations (e.g., Association for Supervision and Curriculum Development, National Staff Development Council) that provide guidance to a large number of in-service professional developers. How have these organizations been influenced by the NSES? At the beginning of this paper, I presented a framework for developing robust research-based evidence. Within this framework, the goal for researchers and the sponsors of research is to develop a more coordinated body of evidence in order to systematically build a strong case in support of a particular hypothesis (in this case, the influence of the NSES on policies, pre-service professional development, or in-service professional development). Building a strong evidence base requires multiple examples of quality research employing appropriate methods that together provide confirmatory findings. The evidence examined in this study suggests that the current research base is of variable quality and provides too few reinforcing results. While there are an incredible number of talented researchers across the nation, our efforts are largely unfocused and idiosyncratic. The current educational research system lacks commonly accepted standards of quality research (regardless of methodology), poor coordination, and too few incentives that would allow us to build a systematic evidence base around important questions like the influence of the NSES on the system of professional development.