allowed CALDER research teams to demonstrate the effect of the teacher on student learning and show how widely individual teachers vary in stheir effectiveness. This feature has also enabled research on the factors that may account for the variation in teacher effectiveness, such as credentials, training, classroom behavior, and experience. Hannaway argued that these studies are important because they help to clarify which factors do indeed promote student achievement, which in turn has implications for cost efficiency. For example, many school districts provide higher salaries to teachers with master’s degrees, but recent CALDER studies indicate that, among elementary school teachers, the presence or absence of a master’s degree does not affect student learning gains.
Third, the databases consist of census data, including all students and teachers in the state public education system. This feature of the databases allows an investigator to conduct multiple comparisons. For example, in her recent study of Teach for America teachers in North Carolina, Hannaway was able to compare Teach for America teachers with other new teachers, with all teachers, and with fully licensed and credentialed teachers. She described the potential of the databases for multiple comparisons as “very important” for policy purposes and as a valuable complement to random assignment studies.
Fourth, the databases incorporate historical records, a feature that is critical to understanding the effects of a change in education policy. For example, when studying Florida’s A Plus accountability policy, Hannaway expected that the policy would have its largest effects on the lowest performing schools. Contrary to expectations, she initially found that the low-performing schools were less likely to change their behavior than the high-performing schools—suggesting that the accountability policy was not working as intended. However, after analyzing additional data from an earlier period, when another policy targeted many of the same low-performing schools, they concluded that the previous policy had already generated behavioral change in the low-performing schools. Without the historical data, Hannaway said, “you could come to a very faulty inference … in policy research.”
In response to a question, Hannaway said that the quality of district and state data varies. For example, one North Carolina school district employing a large number of Teach for America teachers provided her research team with data, after a long delay. When Hannaway compared these data with a separate list provided by Teach for America, she found an overlap of only 25 percent. In contrast, some states, including North Carolina and New York, have invested in their data systems, are working with multiple researchers, and have accurate, reliable data, she said.