Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 210
Assessing Accomplished Teaching: Advanced-Level Certification Programs 10 The Effects of Certification on the Education System In A Nation Prepared, the Carnegie Task Force on Teaching as Profession documented their conviction that the overall quality of the nation’s teaching force needed to be improved, and they endorsed the creation of the National Board for Professional Teaching Standards (NBPTS) as one mechanism for accomplishing these changes. Task force members hoped that the influence of the NBPTS would reach well beyond any impact that individual board-certified teachers might have on their students. They hoped that the board’s standards for accomplished teaching would be widely influential and that the demand for board-certified teachers would lead to improvements in working conditions for all teachers. The founders envisioned (Carnegie Task Force on Teaching as a Profession, 1986; National Board for Professional Teaching Standards, 1991): A growing cadre of board-certified teachers would serve as leaders in their schools and districts, working to improve instruction and sharing their expertise with other teachers through informal collegial relationships, formal mentoring activities, and participation in professional development programs. Schools, districts, and states would value board-certified teachers. They would use the standards defined by the board as a guide in hiring and making teaching assignments and would work to provide teaching environments conducive to the national board approach. Teacher preparation programs would focus on the standards articulated by the national board and be influenced by its portfolio
OCR for page 211
Assessing Accomplished Teaching: Advanced-Level Certification Programs based assessment format, so that entry-level teachers would learn the foundational skills that lead to accomplished practice. Professional development and inservice programs for teachers would focus on national board standards and practices. Eventually all teachers—not just those who became board certified—would learn the skills and practices endorsed by the board. In the context of program evaluation, these sorts of far-reaching impacts are referred to as “spillover effects,” although that term is not intended to imply that they are extraneous or unimportant. They are also sometimes referred to as systemic or secondary effects or externalities (Rossi, Lipsey, and Freeman, 2004). In this chapter, we evaluate the extent to which spillover effects are evident. These kinds of effects were important elements in the overall goal the task force hoped to achieve with its multipronged reform approach. It is important to note, however, that the task force cautioned that none of its proposed strategies (of which the assessment-based certification program was one) “will succeed unless all are implemented” (Carnegie Task Force on Teaching as a Profession, 1986, p. 57). This chapter addresses our seventh question: Question 7: Beyond its effects on candidates, to what extent and in what ways does the certification program have an impact on the field of teaching, the education system, or both? Figure 2-1 shows where this question fits within our evaluation framework. We identified several kinds of influence an advanced-level certification program for teachers might have and framed specific questions about them: What are the effects of having one or more board-certified teachers in a school or district? Has the board-certification program had any effects on: the course content, methods of preparation, and assessments used in teacher education programs or the content of and strategies used in inservice training and professional development for practicing teachers? Has the board-certification program had any effects on the applicant pool for teacher education programs? Since the board came into existence, have there been changes in the numbers of individuals entering teacher education programs or the characteristics of the applicants? Has the existence of board certification had an impact on the allocation of teachers across districts and schools? Has the program
OCR for page 212
Assessing Accomplished Teaching: Advanced-Level Certification Programs been a useful tool for increasing the numbers of accomplished teachers in high-needs schools? We begin with a discussion of the challenges associated with this aspect of our evaluation. We then discuss what can be learned from the existing studies, which primarily relate to Subquestion a, and we close the chapter with our conclusions. THE CHALLENGES OF EVALUATING SPILLOVER EFFECTS Evaluating systemic change is difficult in any context. Systems are complex, and the many factors that may affect outcomes interact in complicated ways. Detecting and isolating effects is correspondingly complex because researchers cannot manipulate conditions or use experimental controls, as is done in other kinds of research. Without these tools, it is usually not possible to isolate a single factor as the cause of any observed change. Thus, for example, if we were to observe a change in the content of teacher education programs, it would be nearly impossible to attribute it directly to the NBPTS or any action it has taken. Researchers may see signs that particular changes have occurred but lack suitable indicators with which to measure them. For example, suppose that board-certified teachers were indeed influencing their colleagues in a positive way or that increasing numbers of teacher education programs were relying on NBPTS standards in developing their curricula. How would one measure such change? The changes may be so gradual that they are difficult to detect, or there may be very few reliable criteria to use in calibrating the “before” and the “after” effects. Systemic change also happens slowly, whether the desired change is the reduction of a behavior linked to public health problems or a shift in the culture of a large organization. It takes time for each element of a system to respond to an intervention and for the relationships among different elements to adapt, and it takes time to change behavior. Education is no exception, and those who study education reform have written about the challenge of engaging each of the necessary partners (teachers, administrators, state and local political leaders, etc.) in enacting changes, and also about the inertia that reformers often face (see, e.g., Datnow and Stringfield, 2000; Fullan, 2007; Goertz, Floden, and O’Day, 1995). Schools and teachers have their own cultures, traditions, and habits. Educators must operate within a complex network of rules, regulations, and policies imposed by the district, the state, and the federal government, all of which may interfere further with efforts to introduce change. Many observers of education reform have pointed to the difficulty of bridging the critical gap between presenting prescriptions for improvement and affecting teachers’ day-to-day practice (e.g.,
OCR for page 213
Assessing Accomplished Teaching: Advanced-Level Certification Programs Stevenson and Stigler, 1992; Tyack and Cuban, 1996). The title Tinkering Toward Utopia, which describes generations of reforms that have passed lightly over the surface of public education without fundamentally changing it, is an apt phrase to describe the imperviousness of educational institutions to change (Tyack and Cuban, 1996). For the Carnegie task force’s vision to be realized, fairly dramatic changes would be needed in how teachers do their work and in how they think about their roles. Yet teachers as a group have tended to be reluctant to stand out or to seem to claim that they are superior in some way to their colleagues. Lortie (2002), among others, has described teachers as viewing their field in an egalitarian way and as resisting professional status distinctions. As we discussed in Chapter 3, teachers in the United States also tend to have an individualistic orientation, in contrast to those in other countries. They tend to work in isolation and determine for themselves what is best for their students. Although schools create opportunities for certain kinds of collaboration, such as meetings for those teaching a particular grade level or subject matter, U.S. teachers rarely observe one another in the classroom or critique one another’s practice (Little, 1990; Lord, 1994; McLaughlin and Talbert, 2001). Yet the national board prizes collaboration and reflection and also identifies and rewards exemplary teachers. Their goal is to create an occupational status distinction that places teachers who earn the credential above other teachers. Thus the board faced an uphill climb as it set out to alter fundamental aspects of the way U.S. teachers approached their practice. Despite the challenges of investigating spillover effects, we have included these questions in our evaluation for several reasons. First, the board clearly viewed stimulating systemic change in the field of teaching as a critical goal. Second, we think that spillover effects are important, even if they are difficult to pinpoint. Including them in our evaluation allowed us to step away from detailed technical questions to consider broader questions about the program’s impact and significance that we think are an important aspect of this evaluation. Finally, although little research is available on spillover effects associated with the NBPTS, we think it is possible to conduct this kind of research. These kinds of studies take time. Researchers must plan in advance to take advantage of opportunities to collect data, and they must wait for longitudinal data to accumulate. We think that evidence of spillover effects could be collected, if studies based on thoughtful hypotheses and well-planned data collection were undertaken. AVAILABLE EVIDENCE Among the NBPTS-related studies, we identified three that provided information relevant to spillover effects, although they focus only on the
OCR for page 214
Assessing Accomplished Teaching: Advanced-Level Certification Programs issue addressed by our Subquestion a: Koppich et al. (2006), Yankelovich Partners (2001), and Sykes et al. (2006). Key characteristics of these studies are highlighted in Table 10-1, and more complete descriptions appear in Appendix A. Two of these studies are very comprehensive and make use of multiple methods for obtaining and analyzing data. Koppich, Humphrey, and Hough (2006) studied the impact of board certification in six states, focusing on what board-certified teachers do after becoming certified and what it takes for them to make a difference in a school. They collected data using a mail survey of board-certified teachers, focus groups, interviews, and site visits to 18 selected case study schools. Sykes and colleagues (2006) report findings from three interrelated studies of board-certified teachers and their influence in their school systems. The studies include a school-level survey, a state-level survey, and a four-school field study of teachers in South Carolina and Ohio. The third study offers less robust evidence. An NBPTS survey (Yankelovich Partners, 2001) queried teachers about the types of activities in which they participated after earning the credential. We were hesitant to draw any firm conclusions from this survey because the report consists only of a tally of the survey responses and provides minimal discussion of the methodology and findings. No research has been done on the impact of the NBPTS standards on teacher preparation or teacher professional development. A full-scale study of this issue would have been beyond the scope of our evaluation and would have required far more time and resources than were available. However, as described in Chapter 2, we held a panel discussion at our third meeting at which we heard testimony from three teacher educators with regard to this issue. While their commentary provides only anecdotal accounts of the kinds of influences that NBPTS standards might have, we think that they provide a basis for conceptualizing additional research in this area. EFFECTS OF HAVING BOARD-CERTIFIED TEACHERS IN SCHOOLS Findings from surveys conducted by Sykes et al. (2006) and Yankelovich Partners (2001) provide some evidence that board-certified teachers participate in mentoring and leadership activities within their school system. In both studies, the majority of survey respondents indicated that they are involved in such activities as mentoring other teachers, serving as team leaders in their schools, developing curriculum materials for the school system, providing professional development activities, and supporting other national board candidates as they undergo the certification process. Sykes et al. also report that teachers’ participation in such activities seems to increase over
OCR for page 215
Assessing Accomplished Teaching: Advanced-Level Certification Programs time once teachers obtain board certification, although they note that the more experienced teachers in a school system are typically the ones given leadership roles in any case. One point that cannot be discerned from these studies is the extent to which earning the credential caused teachers to participate in such activities. There is no evidence in Sykes et al. as to how active teachers were in leadership activities prior to becoming board certified, but the majority of respondents to the surveys by Yankelovich Partners (2001) and Koppich et al. (2006) said that they had participated in these kinds of activities prior to earning the credential. Thus, it may be that teachers who decide to pursue board certification are those who are already leaders in the school system, and that the credential simply signals their leadership skills. The case studies conducted by Koppich et al. and Sykes et al. provide additional insights about board-certified teachers’ experiences in their school systems. Taken together, the results from these two studies indicate that, in the regions studied, board-certified teachers are not having the desired effects in their school systems. Koppich and her colleagues asked board-certified teachers about the support they received from their administrators and the ways in which their skills are used. Overall, they found little evidence of schools relying on board-certified teachers to serve as mentors or in leadership positions. They found that many of the board-certified teachers were teaching in situations that were not supportive of efforts to take on a leadership role or to move beyond the conventional obligations of classroom teaching. The authors report that more than 90 percent of the teachers surveyed said they were no more influential than other teachers on such matters as selecting curriculum and materials, advising on professional development programs, teacher hiring and evaluation, advising on budget, and determining the focus of school reform efforts. Interviews with board-certified teachers and their colleagues led the authors to conclude that there is a culture of “individualism and egalitarianism that remains alive in the profession.” Board-certified teachers reported that they are often given the cold shoulder by nonboard-certified teachers, and nearly 43 percent agreed that “my school culture is not welcoming of teachers stepping into leadership positions.” The authors found that board-certified teachers actually go to considerable lengths to downplay any distinctions between themselves and their nonboard-certified colleagues, sometimes even concealing the fact that they have earned the credential. In one school, the authors found that board-certified teachers actually declined requests to participate in leadership activities, despite encouragement from the principal. At this school, there was a history of negative attitudes toward board-certified teachers. The board-certified teachers said they were
OCR for page 216
Assessing Accomplished Teaching: Advanced-Level Certification Programs TABLE 10-1 Studies Examining Effects on the Teaching Profession and the Education System Study Population Studied State(s) Sampling Method Koppich, Humphrey, and Hough (2006) NBCTs, their colleagues, their administrators CA, FL, MS, NC, OH, SC Stratified random sample Sykes et al. (2006) NBCTs OH, SC Yankelovich Partners (2001) NBCTs who earned certification in 1999 Nationwide Sent survey to all; sample consisted of those who responded by a specific date. willing to lead professional development activities elsewhere but not at their own school with the colleagues with whom they worked each day. Sykes and his colleagues reported similar findings from their case studies. The teachers they interviewed reported that they did not interact with each other about their instructional practices, and board certification was not emphasized. The teachers who had obtained board certification were
OCR for page 217
Assessing Accomplished Teaching: Advanced-Level Certification Programs Sample Size, Response Rate Methods Findings Issues Affecting Validity of the Findings 1,136; responses from 654 (75%) Mail survey, interviews, focus groups, 18 case studies Little evidence that schools rely on NBCTs as leaders or mentors. More than 90% of NBCTs report they are no more influential than other teachers. No concerns. 1,500; responses from 1,153 (77%); 566 from SC and 587 from OH Mail survey, focus groups, interviews, case studies Majority of NBCTs participate in leadership activities. NBCTs and non-NBCTs are reluctant to say that certification signals special competence. Principals are reluctant to favor NBCTs. No concerns. 4,800; responses from 2,100 (45%) Mail survey Majority participate in mentoring and leadership activities. Most had done so prior to becoming board certified. Report is a tally of survey responses; no details are provided about methodology or findings. Sampling methods were questionable. generally positive about the experience, although they were reluctant to state that board certification signaled a level of competence that set them apart from their colleagues. The nonboard-certified teachers tended to think there was no difference between those who were board certified and themselves, sometimes citing stories of well-qualified teachers who tried and did not pass or less qualified teachers who passed. Principals also noted that
OCR for page 218
Assessing Accomplished Teaching: Advanced-Level Certification Programs they were careful about how they made assignments, not wanting to seem to favor the board-certified teachers or to engender resentment from those who were not board certified. Among the 18 case studies conducted by Koppich and colleagues, one stood out as an example of what the founders of the national board likely had in mind. At this elementary school in North Carolina, both the principal and assistant principal were board certified. They enacted a number of changes designed to promote the goals of the certification process. This began with encouragement at the district level for all teachers to pursue certification—the district sponsored weekly and monthly meetings and training sessions and provided other supports. The administrative staff at the school took the lead in changing the school’s teaching culture to one of shared learning and growth, altering the school schedule to allow time for collaborative work and redefining teaching as a public activity in which observation and constructive critique were the norm. However, although this elementary school achieved considerable success with its program, including significant gains in student achievement, it was a “rare bird,” according to the authors. Koppich et al. also cited the ways that Cincinnati public schools made use of national board certification as part of their efforts to create a career ladder for teachers. Their description (p. 15) of this process, paraphrased below, provides another portrait of the factors that seem to be necessary to make board certification a significant benefit in a school or system. In Cincinnati, the creation of the lead teacher position has opened up new roles and opportunities for board-certified teachers. When A Nation Prepared was first released, the school system used the report to guide their attempts to professionalize teaching. Administrators developed a teacher career ladder that included a lead teacher position, with the goal of creating professional leadership roles for teachers that would allow them to remain in the classroom. The school system defined roles for the lead teachers, including such responsibilities as department head, team leader, curriculum specialist, staff development specialist, and peer evaluator. Lead teachers served on committees that made decisions about instruction and resource allocation and on intervention teams for low-performing schools. Initially, the school system used its own assessment procedures but adopted board certification when it became available as a means for earning lead teacher status. Earning board certification is not a requirement for becoming a lead teacher, but having the credential increases a teacher’s chances of receiving this designation. EFFECTS ON TEACHER PREPARATION The Carnegie task force intended that the national board-certification program and its standards for accomplished teachers would have a signifi-
OCR for page 219
Assessing Accomplished Teaching: Advanced-Level Certification Programs cant influence on the preparation of new teachers (Carnegie Task Force on Teaching as a Profession, 1986). They hoped that teacher preparation programs would coordinate their standards with those of the national board and be influenced by the portfolio-based approach, and that entry-level teachers would learn the foundational skills that lead to accomplished practice. Thus, the committee sought information about whether the national board certification program has had any effects on the content, methods of preparation, and assessment of candidates in initial teacher education programs or in advanced programs for teachers, such as master’s degree programs. Two organizations that influence the content of teacher preparation programs have worked to align their standards with those of the national board. The Interstate New Teacher Assessment and Support Consortium (INTASC) was established in 1987 to promote collaboration among states seeking to reform teacher preparation and teacher licensing with the aim of improving the quality of the teaching force. The organization is a consortium of state education agencies (34 states are current members) and professional educational organizations (including the national board, National Council for Accreditation of Teacher Education [NCATE]), and the two largest teachers unions). It defines curriculum standards for programs that prepare beginning teachers, which are aligned with those of the national board (http://www.ccsso.org/projects/Interstate_New_Teacher_Assessment_and_Support_Consortium/). The NCATE, an alliance of 33 professional groups (including the NBPTS as well as teacher educator organizations, teachers unions, and other organizations), promotes high-quality teaching through the accreditation of schools, colleges, and departments of education. NCATE’s standards for teacher education programs are also aligned with those of the national board. Because of the intentional alignment of the standards of these three groups, some have called INTASC, NCATE, and the NBPTS the “three-legged stool” of teacher quality (Bradley, 1997). These efforts lay the groundwork for the NBPTS standards to impact teacher preparation, but there is no research to document the extent of the board’s influence on the content of teacher preparation programs or the standards of individual programs. We explored this issue with the teacher educators at our June 2006 meeting and heard anecdotal accounts of the ways in which the NBPTS standards have been used to make changes at two institutions. Mary Futrell, dean the school of education at George Washington University, served on the original board of directors for the NBPTS and was a strong supporter of the program. She persuaded her faculty to look at the three sets of standards (NBPTS, NCATE, and INTASC) in relation to their curricula. They considered the ways in which their program incorporated the NBPTS standards and what would be needed to bring their program in line with them.
OCR for page 220
Assessing Accomplished Teaching: Advanced-Level Certification Programs Although she encountered considerable resistance, Futrell said, she was eventually able to persuade the faculty to revise the curriculum to bring it in line with the standards, and the five NBPTS propositions (see Chapter 4) are now explicitly incorporated into the curriculum. In addition, teachers enrolled in the university’s graduate special education program are required to assemble a portfolio that is graded on the basis of the NBPTS standards. Their faculty also now work with teachers at a low-performing school in Northern Virginia to help them improve their practices and to encourage them to become board certified. Carol Matern, on the faculty of the teacher preparation program at Indiana University–Purdue University, Indianapolis, became board certified several years ago. She found the certification process to be an exceptional professional development experience. Since earning board certification, she has worked to incorporate NBPTS standards into the courses she teaches. She relies on the standards when preparing her course syllabi, and she explicitly includes reflective writing, analysis of videotaped lessons, and collaborative discussions in her graduate courses for teachers. Although these accounts are clearly anecdotal and describe very localized changes, they indicate the kinds of influences the NBPTS standards can have. They also suggest that both commitment to the NBPTS approach by program administrators and institutional leadership are needed for the board standards to have a noticeable influence on teacher education programs. We caution, however, that these anecdotal accounts did not provide an indication of how representative these changes might be of education programs at other universities, and no data have been collected to indicate the extent to which the more than 1,200 teacher education programs in the United States have been influenced by the NBPTS standards. CONCLUSIONS AND RECOMMENDATIONS At present, little research has been conducted on the extent to which the national board is having spillover effects. At the same time, there is evidence from two investigations that board-certified teachers are having very limited impact in their school systems. The teachers studied by Koppich and by Sykes were reluctant to accept status distinctions in their field. Perhaps out of reluctance to violate an egalitarian tradition in the field, those who earn the credential tend to keep quiet about it, and those who are not board certified minimize its value. These studies also provide glimpses of the circumstances that are necessary for board-certified teachers to have a marked impact, such as engaged administrators at the school and district level who provide leadership opportunities and a shared commitment to changing the teaching culture in a school. Another factor limiting the systemic impact of the NBPTS is the low
OCR for page 221
Assessing Accomplished Teaching: Advanced-Level Certification Programs number of board-certified teachers. Even if all of the 63,800 teachers who have earned board certification were still teaching, this translates to only an average of three board-certified teachers for every five schools and about 2 percent of all the 3.7 million members of the current teaching force. Except in a few districts, the numbers of board-certified teachers are likely to be too small for them to have an impact on their school systems. With regard to the other areas in which spillover effects may have occurred, there simply is no research to draw from. For example, there have been no systematic attempts to evaluate the content of teacher preparation programs to see if changes related to the national board have occurred over the past decade. Such research is difficult to carry out, but it is not impossible. Studies could evaluate the content of course syllabi or curriculum standards and note changes that occur over time. Surveys of administrators of teacher preparation programs could also shed light on these issues. These kinds of studies may not use the stringent kinds of methodologies that would allow one to attribute any detected changes directly to the NBPTS, but they would provide the beginnings of a research base on these questions. We think that such studies lie within the purview of the NBPTS. The board established these goals from the outset and should implement the kinds of research that would make possible evaluation of the extent to which these goals have been realized. Late in our evaluation process, the board embarked on this kind of study, and we encourage the completion of this investigation. There is also no way for us to evaluate the impacts of the national board on professional development programs for teachers because no research has examined the extent to which the NBPTS standards have influenced inservice programs. This type of research could also be conducted. Researchers who have studied the effects of professional development on instruction have found ways to characterize different kinds of professional development, to identify theoretical approaches, and to examine the effects of different approaches on teachers’ classroom practice (Desimone et al., 2002; Garet et al., 2001; Guskey, 2003; Hawley and Valli, 1999; Porter et al., 2000; Wilson and Berne, 1999). This body of work provides research models that could be useful in efforts to trace the influence of the NBPTS approach on professional development programs. This literature also indicates that a consensus seems to be emerging about the key features that make professional development effective, such as opportunities for teachers to work as a group and to develop their learning over an extended period of time; opportunities for active learning; a focus on content; and links among the professional development activities, the curricula with which the teachers are working, and the standards they are using. These newer findings regarding professional development seem to reinforce many of the elements recommended by the national board.
OCR for page 222
Assessing Accomplished Teaching: Advanced-Level Certification Programs Our evaluation framework also includes questions about the impact of the program on the characteristics of teachers who enter the profession. At present there are no data that could be used to address this question. No data have been collected to ascertain whether applications to schools of education have increased since board certification became available or whether the characteristics of applicants have changed over time. The final question in our framework dealt with the allocation of teachers across schools. We were interested in the extent to which principals and administrators use board certification status to assign teachers to high-needs schools or classrooms with the most challenging students. We were not able to answer this question either. In Chapters 6 and 9 we described the problems with the existing data systems (i.e., that there are no ways to determine where teachers currently work or to track their placements on a national level), and we refer the reader to those sections of the report for details about the problems with data collection in these areas. Investigating spillover effects caused us to consider what might look different if states, districts, and schools around the country had actively embraced the board certification program from the start. As Chapter 6 discusses, this has not happened, except in a few places. There is a stark contrast between the particularly ambitious goals of the Carnegie task force and the very modest spread of the national board certification program. We could find no studies or evidence to answer questions about why the national board has not become more deeply ingrained in the U.S. education system, but it is clear that systemic effects go hand in hand with the volume of certified teachers. Having reviewed the evidence on all of these questions, we think that board-certified teachers are unlikely to have a significant impact without broader endorsements by states, districts, and schools of the NBPTS goals for improving professional development, setting high standards for teachers, and actively using the board-certified teachers in leadership roles. Furthermore, we think that the NBPTS program is unlikely to have broad systemic effects on the field of teaching unless greater numbers of teachers become board certified and the Carnegie task force’s other recommendations—for creating a more effective environment for teaching and learning in schools, increasing the supply of high-quality entrants into the profession, and improving career opportunities for teachers—are implemented. Our review of the evidence led us to draw the following conclusion: Conclusion 10-1: There is not yet sufficient research to evaluate the extent to which the NBPTS is having systemic impacts on the teaching field and the education system.