5
Generating Continuous Improvement Cycles Through Research and Development
In her presentation, Suzanne Donovan, executive director of the Strategic Education Research Partnership (SERP), said she received her Ph.D. in the economics of public policy, so she was trained to think about education from an economics perspective. A major emphasis within economics is the idea of incentives, she noted, which in the field of education includes such initiatives as merit pay and competitions among schools. This perspective tends to treat what happens inside the classroom as a black box, Donovan said. “It’s not a bad thing to put all that inside a black box,” she said. “It makes life a lot easier. It’s the great discovery of the invisible hand.”
Asking convocation participants to use their cell phones to take an informal survey, Donovan asked them to identify whether they have ever been a K-12 teacher and then to answer the question: “To achieve a high-functioning K-12 education system, what portion of the challenge can be addressed through policies that address incentives and accountability?” Most indicated electronically that they had been teachers, and the large majority of these respondents thought that less than one-quarter of the problem could be solved through incentives and accountability, with a minority choosing between one-quarter and one-half.
Though this sample was small, Donovan acknowledged, the result clearly suggested that treating the classroom as a black box is insufficient. “If we really want to understand how to improve education—and most of you believe this as well—we need to open the black box and see what’s inside,” she said. This growing realization eventually led
Donovan to switch her interests from economics to the learning sciences, she explained. She worked at the National Research Council (NRC) on the publications, How People Learn: Brain, Mind, Experience, and School (National Research Council, 2000) and How People Learn: Bridging Research and Practice (National Research Council, 1999), noting this experience made her and her colleagues realize that to affect policy, research results need to be translated in ways that make sense for policy makers. Similarly, she said, teachers need to know what research results “look like” in the classroom to make use of that knowledge. She explained this conclusion led to a third publication, How Students Learn: History, Mathematics, and Science in the Classroom (National Research Council, 2005), which incorporated topics from history, mathematics, and science into a representative elementary school-level, middle school-level, and high school-level classroom.
Each chapter required a remarkable amount of work to ensure that the underlying principles were well articulated, said Donovan. But the researchers who helped prepare this publication had a harder time doing so than teachers. “When you get a teacher who is able to think deeply about curriculum, there is a richness that is not there with somebody who hasn’t spent years in a classroom,” she observed. This kind of translational research, she said, “treats teachers as targets for learning instead of as targets for changing behaviors. Teachers become a more active part of solving the problem of improving instruction.”
Yet this translational research also has its limits, she noted. Teachers are trying to balance so many things in the classroom that they can be overwhelmed by the task of incorporating translational research into their teaching. They must simultaneously manage their classrooms, implement engaging activities for their students, and engage in diagnostic teaching. Beginning teachers typically spend much of their time on the first and second of these activities. “For these teachers, translational research isn’t going to help,” said Donovan. More accomplished teachers can spend more of their time on implementing engaging activities than on classroom management, but even they tend to spend little time engaged in diagnostic teaching. Only with expert teachers, who represent a small fraction of all teachers, is a substantial fraction of their time devoted to diagnostic teaching.
As many others have pointed out, she said, schools are complex systems, and teacher expertise is only one factor in a system where the parts interact with each other in unpredictable ways. In such systems, said Donovan, “it is very difficult to get education change—and impossible to get it from the outside.” A variety of obstacles can stand in the way of moving toward best practices, which requires that many people be involved in removing those obstacles.
This gradual evolution of thinking about teaching at the National Academy of Sciences led to an effort, staffed by Donovan, to do work on teaching practices in a practice setting, not in research universities or the National Academies. The report Strategic Education Research Partnership (National Research Council, 2003) argued that a new kind of organization was needed to move the research enterprise into school settings, with problems taken not from theory but from practice. The result was the creation of the independent nonprofit Strategic Education Research Partnership (SERP), which Donovan now heads. Its theory of action “puts practitioners, both teachers and administrators, into a much more central role,” she said, explaining it brings together education professionals, researchers, and designers in principled collaborations to work on tools, programs, and practices that can result in achievement gains and knowledge accumulation at scale.
Donovan emphasized the role of designers. They understand problems from a different perspective, she said. They always begin with the user and how a tool, program, or practice is going to influence the user’s behavior. “What’s in it for them, what’s going to make them change their behavior? If the answer is nothing, don’t expect that what you hope they’re going to do is in fact what they’re going to do,” she commented.
The process at SERP involves proposing and designing solutions, testing them in practice settings, finding whether they work or not, and making iterative revisions until something does work. Teachers pilot what works, and according to Donovan, “the extent to which the final product works in practice is largely an artifact of how engaged teachers have been in shaping the final product.”
Donovan demonstrated several examples of products devised through this process. One of the most popular has been a collection of five-inch by eight-inch cards that list the principles that a lesson is designed to achieve and the vital actions expected of students. The idea was to avoid giving teachers huge binders filled with aspirations. Rather, the cards are designed to give teachers something that they can use to figure out why they need to change. They do not have to read large amounts of material to change their teaching. What they need to know in a particular circumstance is on the card. This approach “treats teachers differently,” said Donovan. “It treats them as an essential member of a team and as clients for whom the design team needs to design.”
Classrooms taught using this approach can look more chaotic than normal classrooms, but they also can increase student learning, Donovan said. One challenge is to get principals to “change their taste in instruction” so that they are not put off when they walk into a classroom using these materials. The five-by-eight cards have been critical in this respect,
because they let principals know why a classroom might look different than they expect.
A deeper problem is the need to shift policy makers away from thinking solely in terms of incentives and accountability as the primary drivers of change, said Donovan. “It’s not that [incentives] are irrelevant; it’s just that they’re not going to solve very much of the problem,” she concluded. “The basic problem is building capacity and solving the problems of a complex system. We need to shift to an investment in a very different kind of policy.”
In the discussion during Donovan’s presentation, a workshop participant pointed out that having five-by-eight cards not only for teachers and principals, but also for parents would help parents understand new approaches to teaching. Donavan responded that SERP has not worked on that problem yet, but it is an important problem that probably will receive attention in the future.
Rebecca Sansom, a 2013–2014 Einstein fellow, raised the point that whenever teachers try something new, they are not going to do that new thing as well as they did an older thing, even if the newer approach will ultimately be more effective than the older approach. This is an issue of teacher leadership, she said, “because there is something about teacher leaders that makes them more willing to accept that risk and that vulnerability.”
Donovan pointed out that the key is to clarify the purpose of change. Teachers need to understand the reason for a change and then be supported to make the change and incorporate it into their practice, she said. One useful option, she said, might be to create schools within a district that are focused on innovation, and then put expert master teachers into those schools. New and more experienced teachers then could work with or in these schools to incorporate new materials into their practices. Such an approach could also help bring about an essential change in teaching, said Donovan, toward a profession that is widely viewed as highly skilled.
