been a useful tool for increasing the numbers of accomplished teachers in high-needs schools?

We begin with a discussion of the challenges associated with this aspect of our evaluation. We then discuss what can be learned from the existing studies, which primarily relate to Subquestion a, and we close the chapter with our conclusions.

THE CHALLENGES OF EVALUATING SPILLOVER EFFECTS

Evaluating systemic change is difficult in any context. Systems are complex, and the many factors that may affect outcomes interact in complicated ways. Detecting and isolating effects is correspondingly complex because researchers cannot manipulate conditions or use experimental controls, as is done in other kinds of research. Without these tools, it is usually not possible to isolate a single factor as the cause of any observed change. Thus, for example, if we were to observe a change in the content of teacher education programs, it would be nearly impossible to attribute it directly to the NBPTS or any action it has taken.

Researchers may see signs that particular changes have occurred but lack suitable indicators with which to measure them. For example, suppose that board-certified teachers were indeed influencing their colleagues in a positive way or that increasing numbers of teacher education programs were relying on NBPTS standards in developing their curricula. How would one measure such change? The changes may be so gradual that they are difficult to detect, or there may be very few reliable criteria to use in calibrating the “before” and the “after” effects.

Systemic change also happens slowly, whether the desired change is the reduction of a behavior linked to public health problems or a shift in the culture of a large organization. It takes time for each element of a system to respond to an intervention and for the relationships among different elements to adapt, and it takes time to change behavior. Education is no exception, and those who study education reform have written about the challenge of engaging each of the necessary partners (teachers, administrators, state and local political leaders, etc.) in enacting changes, and also about the inertia that reformers often face (see, e.g., Datnow and Stringfield, 2000; Fullan, 2007; Goertz, Floden, and O’Day, 1995). Schools and teachers have their own cultures, traditions, and habits. Educators must operate within a complex network of rules, regulations, and policies imposed by the district, the state, and the federal government, all of which may interfere further with efforts to introduce change. Many observers of education reform have pointed to the difficulty of bridging the critical gap between presenting prescriptions for improvement and affecting teachers’ day-to-day practice (e.g.,



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement