One of the most prominent issues in education policy today, accountability is a key element in the success of education improvement systems. Literally the process by which students, teachers, and administrators give an account of their progress, accountability is a means by which policy makers at the state and district levels—and parents and taxpayers—monitor the performance of students and schools. Accountability systems include a range of mechanisms, from simply requiring schools and districts to report on progress to policy makers and the public, to placing consequences—rewards for high performance and sanctions for poor performance—on the results of performance measures.
The 1994 Title I statute includes a number of provisions regarding the establishment of accountability structures. The law requires states to develop measures to determine whether schools are making adequate yearly progress toward the standards, based on the state assessments. It also states that local education agencies shall designate as “distinguished” schools that exceed the state's definition of adequate yearly progress for three consecutive years, and that such schools can be rewarded with Title I funds. At the same time, the law states, local education agencies shall identify schools that fail to make adequate progress and target them for assistance; after three years, such schools are subject to “corrective action,” including the loss of funds, reconstitution of the staff, and the transfer of students.
By focusing on student performance as the measure by which schools and districts will be accountable to states, the 1994 statute reflects a substantial shift in thinking about accountability that has taken place over the past decade (Elmore et al., 1996). In the past, states held schools accountable for following rules set out by legislatures and boards of education, and for spending funds according to those rules. To that end, accountability mechanisms focused on inputs—the number of books in the library, the ratio of certified staff to stu-
dents. These efforts were designed to ensure that schools carried out the mandates issued by state officials.
The new approaches shift the focus to outcomes—the results of all the inputs—and specifically to student achievement outcomes. These new systems reflect what the National Governors' Association (1986) referred to as a “horse trade”: flexibility in exchange for accountability. Believing that those closest to students—schools and districts—knew best how to meet the needs of their student populations, the governors agreed to the idea of relaxing rules and giving schools maximum flexibility to design appropriate instructional programs. However, they said they would do so only so long as the schools produced results, and the states agreed that they would monitor the results, reward improvement, and impose sanctions for failure.
These new accountability schemes were designed to change the incentive structure for teachers and administrators. By placing consequences on the results, accountability was aimed at encouraging teachers and administrators to innovate and to design effective curricular and instructional programs that will improve student performance.
In addition, the accountability mechanisms were aimed at improving the efficiency and effectiveness of state agencies. By determining which schools are succeeding in their basic mission and which schools are failing, states could direct resources and assistance to the schools and districts that need them the most—the ones in which performance measures indicate problems. Otherwise, resources could be wasted, and needs could remain unmet. Students and taxpayers would both benefit under the new systems.
But designing a means of accountability poses a number of challenges. Chief among them is how schools respond to the accountability pressures. The external accountability structures can set ground rules and design incentives, but these processes will have the desired effect only if the internal accountability matches that from the outside. That is, teachers and administrators must hold themselves accountable for the performance of themselves and their students. If there is a mismatch between the internal and external accountability, when accountability knocks, no one may be home.
Who Is Accountable? One of the key design issues in accountability is determining who is to be held accountable. The Title I statute clearly intends for states to hold institutions—schools and school districts—accountable for student performance. Although the law requires states to collect and report data
on individual students, its requirements for identifying schools and districts that have students exceeding (or failing to reach) standards place the locus of accountability on institutions, not individual students or teachers.
In doing so, the law follows the lead of early reforming states, such as Kentucky and Mississippi, that designed mechanisms explicitly for school and district accountability. The argument in those states was that school faculty and staffs, collectively, are responsible for student performance. Although a 4th grade teacher may determine to a large extent what a 4th grade student learns, and how that student performs on a 4th grade test, the student's performance in fact reflects the cumulative knowledge and skills she has learned to that point. Thus all teachers contribute to the students' achievement.
Moreover, school-level accountability was designed to encourage teachers to work together to improve instruction, in contrast to programs such as merit pay, which were seen as fostering competition among school staffs (Clotfelter and Ladd, 1996).
However, placing accountability at the school level may mask some important information. As Willms (1998) found, the variation in student performance within schools was greater than the variations among schools; therefore, reports that made judgments about school performance based only on overall results, without taking into account the variations within schools, could be misleading, since some teachers perform well and some perform poorly.
In addition, placing accountability at the school and district levels leaves out a key piece of the student performance puzzle—the students themselves. Some critics argue that such schemes set up a conflict of interest between students and teachers; teachers have a strong incentive to raise performance, but students, with nothing riding on the results, have little incentive to do their best on the tests, particularly at the high school level. This situation, moreover, reinforces the low levels of motivation high school students have to work hard in school, and masks the consequences for inadequate performance students will face when they get out of school and find themselves unable to find a high-paying job (Bishop, 1994).
In an effort to increase student motivation for schoolwork and hold students accountable for their own learning, a number of policy makers, including President Clinton, have proposed some form of student accountability, such as making promotion from grade to grade or graduation from high school contingent on demonstrating a certain level of performance, usually by passing a test. President Clinton and others have posed the issue as one of ending “social promotion,” or the practice of moving students up the grades to remain with their peers, regardless of their academic performance. As the president stated in his 1998 State of the Union Address: “when we promote a child from grade to grade who hasn't mastered the work, we don't do the child any favors.”
But as a number of studies have shown, schools do children no favors when they retain them in grade and continue to provide them with inadequate
instruction. Students who are retained tend to have lower academic achievement than those who are promoted, and drop out of school at higher rates (National Research Council, 1999a).
Placing high-stakes accountability on students also poses special problems. For one thing, tests that are used to make decisions about schools may be illsuited for decisions about individual students. In addition, states and districts face substantial legal hurdles in using tests to apply consequences to students. Specifically, they need to demonstrate that the tests neither discriminate against any group of students nor deny any student due process. To demonstrate the latter, states need to prove that students have received adequate notice of high-stakes testing requirements and that they have been taught the knowledge and skills the test measures (Debra P. v. Turlington, 1981).
For these and other reasons, the Committee on Appropriate Test Use of the National Research Council (National Research Council, 1999a:279) recommended that “high-stakes decisions [about individual students] such as tracking, promotion, and graduation should not automatically be made on the basis of a single test but should be buttressed by other relevant information about the student's knowledge and skills, such as grades, teacher recommendations, and extenuating circumstances.”
Accountability for What? Determining what students or schools should be held accountable for is no less challenging than determining whom to hold accountable. The Title I statute and the new accountability ideas it reflects hold that the answer is “student performance.” But in practice, this answer leads to a number of interpretations, and the way schools respond to those interpretations affects whether accountability realizes its goals of increasing learning for all students.
As noted above, one of the major purposes of accountability based on performance is to encourage schools to focus their efforts on improving performance above all else. Everyone held accountable has an incentive to ensure that performance increases—or at least to stave off declines.
In the past, though, efforts to raise stakes on tests have not always had the desired effect. In some cases, schools employed inappropriate practices to raise test scores, such as focusing instruction on the format or general content of tests, rather than the concepts and skills the tests were expected to measure. These practices may have boosted scores, at least temporarily, but they did not in fact raise achievement (Koretz et al., 1991). Occasionally, schools resorted to practices that were unethical or illegal, including cheating.
The phenomenon of raising test scores without raising achievement occurs only under certain circumstances, although these circumstances happen to be relatively common. The first is when schools use tests that are not particularly sensitive to instruction. Tests that measure general knowledge and skills, rather
than the knowledge and skills schools are focusing on, do not respond immediately to instructional changes, no matter how effective. So in order to raise scores quickly, schools employ test-based strategies, and achievement does not increase. If, however, schools used instructionally sensitive instruction, they could raise scores and achievement by improving instruction.
Another factor that contributes to inappropriate test preparation strategies is the use of a single test as the basis for rewards and sanctions. Although the Title I statute calls for the use of multiple measures of student achievement, states and districts at this point continue to use one test in designing accountability. Schools get the message that they have to raise scores on that test in order to earn rewards or avoid sanctions. Using multiple measures could encourage schools to focus less on a single measure and more on improving achievement generally.
In an effort to broaden the measure of achievement, some states include additional factors for accountability. Texas, for example, includes graduation rates and attendance rates, along with test scores, in determining ratings for schools. But few schools have earned low ratings because of these factors; as a result, schools continue to focus their attention on the tests (Gordon and Reese, 1997).
The way that states calculate performance also affects schools' responses to accountability. In some states, schools or districts must reach a threshold level of performance in order to earn rewards; that is, a certain percentage of students must attain a passing score or reach a particular level of proficiency. In these states, some schools reason that the most efficient way of meeting those targets is to focus on students who are just below the bar, and provide them with intensive test preparation.
As Willms (1998) found, this strategy may be shortsighted. Examining data from British Columbia, he found that schools that improved performance overall did so by raising the performance of low-performing students. This occurred, he notes, because high-performing students tend to do well in any circumstances; raising the floor also raises overall performance.
Other states have tried to encourage schools to focus their efforts on low-performing students by placing an emphasis on improving the distribution of performance, and reducing the number of low performers as well as increasing the average. In these states, test preparation for a few students will not work; improving instruction across the board will earn them rewards.
Test preparation alone will also be effective only if the objective is to reach a certain level of performance, rather than to improve performance continually. States such as Kentucky, where schools must reach new performance goals every two years, have found that they can raise performance in the early years by focusing on the test; sustaining gains requires instructional improvement—which in turn requires support for professional development.
Rewards and Sanctions. Taking a leaf from the business and organizational literature, designers of accountability mechanisms have sought to create tangible rewards for high performance and significant penalties for poor performance. The goal was to create real incentives for change, rather than relying simply on the good will and best efforts of teachers and administrators.
The practices have varied widely. Nearly all states, and many districts, have simply published performance results, or provided them to newspapers, which published them. By making the information public, officials reasoned, schools would have an incentive to improve to look their best in the media and have an answer for parents and public officials who questioned their performance. These public reporting systems certainly attracted the public's attention; whether they produced any real change is not clear (Elmore et al., 1996).
In other cases, states and districts have tried more stringent methods to spur change. Some 14 states offer rewards to high-performing schools (Education Week, 1998). These include ceremonial honors, such as blue ribbons, as well as cash awards. South Carolina and Mississippi relaxed certain state regulations for schools that performed above a designated level.
The power of these rewards as motivations for change is unclear. In Kentucky, where the state provided cash bonuses to schools amounting to between $1,300 and $2,600 per certified staff member, the bonuses did not appear to have much effect (Elmore et al., 1996). Some teachers doubted whether the bonuses would in fact materialize, citing a previously announced bonus plan that died aborning. Whatever the reason, teachers did not appear to pay much attention to the prospect of cash awards.
More significant in Kentucky, and elsewhere, was the threat of sanctions (Kelley et al., 1998). Some 31 states provide some sort of penalty for failing schools (Education Week, 1998), ranging from requiring a state-approved improvement plan, to reconstitution (replacing the entire faculty and staff), to state takeover. Few states have actually imposed the most dramatic sanctions; the threats themselves appear to have spurred schools into action. The threats have even attracted the notice of schools that are not at risk of intervention (Firestone et al., 1998).
Many states and districts that have not imposed sanctions have offered assistance to troubled schools. Assistance can take the form of technical help in writing school improvement plans, as in Mississippi, or a state-appointed monitoring team that oversees the implementation of a reform plan, as in New York State. These assistance efforts have helped to turn troubled schools around; however, it is not clear whether states or districts have sufficient capacity to assist all schools that need help. A survey by the U.S. Department of Education found that only 9 states report that they can provide support to at least half the schools in need of improvement; 12 states report that they serve less than half of schools in need of improvement; and 24 states say they have more schools in need of improvement than they can serve (U.S. Department of Education,
1999). Even if states that can provide assistance to the lowest-performing schools, few serve schools in the middle of the performance distribution, which tend to receive less attention from the state accountability efforts (Massell, 1998).
The ability of accountability mechanisms to produce desired effects depends on the level of internal accountability within schools (Abelmann and Elmore, 1999). That is, teachers' own judgements about their ability to affect the learning of their students governs the teachers' willingness to take responsibility for improving student learning and to change their practice to make such improvements come about. Misalignment between internal and external accountability may make it less likely that external systems, no matter how strong, will have much effect.
Internal accountability includes the norms by which teachers operate, the expectations they hold about student learning and their role in improving it, and the processes they use to carry out their work. In schools with weak internal accountability, the norms emphasize the individual responsibility of each teacher over student learning, rather than the collective responsibility of the entire school. In those cases, teachers' judgments about whether and how much they could improve learning depend on their understanding of the students' background and lack a perspective of what students could do under different circumstances.
Similarly, the expectations for student learning in such schools are relatively low, since teachers believe that the conditions the students brought to school, rather than their own efforts, exert the greatest influence over their academic performance. Teachers in schools with low internal accountability tend to place a greater emphasis on order, an expectation each teacher shares.
Schools with weak internal accountability therefore tend to respond to external pressures for change by summoning their own individual beliefs, rather than by consulting with colleagues and attempting to work collectively for improvement.
- Accountability should follow responsibility: teachers and administrators—individually and collectively—should be held accountable for their part in improving student performance. Teachers and administrators should be accountable for the progress of their students. Districts and states should be accountable for the professional development and support they provide teachers and schools to enable students to reach high standards.
- Accountability decisions should be based on multiple indicators.
- Accountability mechanisms should be based on a range of measures, including indicators of instructional quality, as well as student outcomes.
- Accountability results should be reported so that the improvements needed are clear to students, teachers, and parents.
- Accountability mechanisms should encourage schools to improve all students' performance.
- Assistance should be aimed at strengthening schools' capacity for educating all students to high standards and to building the internal accountability within schools.
Questions to Ask
- Are teachers and administrators held accountable for the progress of their students, and are districts and the state held accountable for the professional development and support they provide teachers and schools to enable students to reach high standards?
- Are multiple indicators used in determining accountability, including indicators of instructional quality, as well as student outcomes?
- Are accountability results reported so that the improvements needed are clear to students, teachers, and parents?
- Does the accountability mechanism encourage schools to improve all students' performance?
- Does your accountability system provide assistance to build capacity and internal accountability in schools?
Link to Instructional Improvement. The accountability system should be tied directly to the instructional improvement system, so that all schools can learn from the example of the successful schools and poor-performing schools can receive the support they need to improve.
Assistance Before Sanctions. Penalties such as reconstitution and takeovers are not solutions; they are means to implement solutions. State and district efforts should emphasize assistance first, and sanctions only after a period of continual decline in performance and evidence that additional help would be fruitless.
Assistance to All, with Priority for Lowest-Performing Schools.
The poorest-performing schools would benefit most from assistance and should receive it first, but all schools need some form of help in developing teacher capacity and internal accountability.
The following examples come from two states that have shown the largest gains in performance on the National Assessment of Educational Progress—North Carolina and Texas. In each state, accountability has created incentives for improvement. North Carolina's system links accountability directly to improvement for low-performing schools; Texas's system encourages schools to direct their efforts at improving education for all students.
Under North Carolina's accountability system, known as the ABCs of Public Education, the state measures student performance on the state assessment and creates an “expected growth” composite for each school based on statewide average growth and the previous performance of students in the school. The state then adjusts the results statistically to compare student performance from one year to the next.
Schools are designated as “low-performing” if less than 50 percent of their students achieve standards, which is defined as at or above grade level in reading, mathematics, and writing. Low-performing schools are assigned assistance teams of educators who work with school staffs to align the curriculum to state standards.
The state also recognizes schools that have large percentages of high-performing students, or that demonstrate large gains in performance. Schools that meet their expected growth standard and have at least 90 percent of students performing at or above grade level (in K-8) or at above Achievement Level III (in high school) are designated as Schools of Excellence and are recognized at a state luncheon and receive cash awards. Schools with 80 percent of students at or above grade level of Achievement Level III are designated Schools of Distinction. Schools that show exemplary gains—10 percent or more above the statewide average—receive cash awards; the 25 schools that gained the greatest amount are honored at a statewide luncheon. Schools must test at least 95 percent of the student body (98 percent in K-8 schools) in order to be eligible for recognition.
In Texas, students and teachers know that the TAAS (the Texas Assessment of Academic Skills) matters. The tests play a central role in the state accountability system, for students and institutions. Under state law, students must pass each section of the exit-level exam in order to graduate from high school. Students may retake any part of the test they do not pass; students now can take the test up to eight times. When the full battery of end-of-course exams is implemented, students may be able to graduate by passing these tests, rather than the TAAS.
The tests matter to schools and districts, too, because they are judged in large part on their ability to enable students to pass the test. The state has developed an elaborate accountability rating
system that classifies schools and districts by using their TAAS passing rates, dropout rates, and attendance rates.
Under the system, the state rates districts each year in four categories: exemplary, recognized, academically acceptable, and academically unacceptable. The state also rates schools in the following categories: exemplary, recognized, acceptable, and low-performing. The standards for each designation have risen over the past few years, as performance has improved.
Significantly, schools must demonstrate that students in each group—black, Hispanic, white, and economically disadvantaged—as well as students overall, have met the required passing rates in order to earn a status of acceptable or above. In this way, schools cannot attain high ratings if only a small group of their students perform well.
Schools rated as acceptable or low-performing or districts rated academically acceptable or academically unacceptable must show required improvement. To meet that standard, a school or district with a TAAS passing rate below 40 percent in any area must show that, over two years, its rate of change exceeded the rate required to reach a 50 percent passing level within five years. A school or district with a dropout rate above 6 percent must show a two-year rate of change that would meet or exceed the rate needed to reach a 6 percent rate within five years.
The accreditation ratings are used to determine rewards and sanctions. High-performing schools, those designated exemplary or recognized, and those designated acceptable that have demonstrated significant gains in student performance, are eligible to share monetary awards. In 1997–1998, the legislature appropriated a total of $5 million over two years for such financial awards; schools can receive between $500 and $5,000. These financial awards are not considered a significant motivation to improve performance.
The sanctions are considered more important. For districts that are academically unacceptable, the state commissioner may order the district to publish the ratings to all property owners and parents; require the district to develop an improvement plan; appoint a master to oversee the operations of the district or a management team to direct operations in low-performing areas. If districts are rated academically unacceptable for a year or more, the state may replace the school board; if a district is rated academically unacceptable for two years or more, the state may annex the district to a neighboring district.
For schools that are designated low-performing, the state may notify the district of its status; require the school to develop an improvement plan; or appoint a special intervention team to conduct an on-site evaluation and recommend appropriate changes in budget, personnel, or school policies. If a school is designated low-performing for a year or more, the state may appoint a board of managers to assume the school board's authority over the school. A
school designated low-performing for two years or more may be shut down.
Perhaps the most important spur to improvement is the simple publication of results. Not only are accountability ratings made available to the media, but the results are publicly available on the state's Internet home page. In addition, all schools are required to send school report cards to parents. These report cards must include state-generated data related to the accountability system. The public awareness of the results and the accountability ratings that these reports generate play a significant role for school leaders.