Title I of the Elementary and Secondary Education Act is the largest federal effort in precollegiate education. Created in 1965, when the federal government for the first time agreed to provide aid to elementary and secondary schools, the program was designed to level the playing field for disadvantaged students by providing financial assistance to schools to compensate for the advantages enjoyed at schools with students from more affluent families. Now with an annual budget of approximately $8 billion—a fourth of the U.S. Department of Education's total annual budget—the program reaches more than 11 million students in two-thirds of all elementary schools and more than a fourth of all secondary schools.
Although Title I is large by federal standards, the program in fact represents a tiny fraction of the nearly $300 billion spent each year on precollegiate education. Nevertheless, Title I (the program was called Chapter 1 between 1981 and 1994) has exerted a powerful influence on schools and school districts. This is particularly true in the area of testing. From its inception, Title I required the use of “appropriate objective measures of educational achievement” in order to ensure that the program was achieving its goal of reducing the achievement gap between low-income and higher-income students. In carrying out this requirement, states and school districts, for the most part, used standardized norm-referenced tests to measure the achievement of eligible students—both to determine eligibility and to measure gains. As a result, Title I increased dramatically the number of tests that states and districts administered; one district administrator estimated that the Title I requirements doubled the amount of testing in the district (Office of Technology Assessment, 1992). Increasingly, the tests that districts used to report to Title I officials became the basis of the district's testing program. In that way, the relatively modest federal
investment proved to be a lever that moved practice in nearly every school in the country.
As a number of reports and studies have concluded, however, this influence was not altogether beneficial. For one thing, despite the dollars spent and the testing requirements imposed, the achievement gap between disadvantaged and more advantaged students persists. In fact, the most extensive study of the program found that Title I failed even to narrow the achievement gap. In the final report of that study, known as Prospects, Puma et al., (1997) found that “where students started out relative to their classmates is where they ended up in later grades.” The researchers caution, however, that this finding does not indicate that Title I was a failure, particularly since funds did reach their intended beneficiaries. It may be, they point out, that the gap would have widened further if not for Title I assistance.
As a number of commentators have suggested, the testing requirements may have contributed to the failure to produce achievement gains for low-income students. According to the Advisory Commission on Testing in Chapter 1 (1993:13), “There is evidence that Chapter 1 testing procedures may indeed be promoting undesirable instructional practices, limiting the kinds of learning experiences to which students are exposed, or reinforcing outmoded ways of teaching disadvantaged students.” In large part, the questionable testing practices came about in response to federal requirements. In particular, the federal government required schools to test Title I students using nationally normed tests, which compare students' performance to that of a nationally representative norming group, in order to permit comparisons across states and districts. But while these tests may provide information that is useful for program monitoring, they are less useful for providing information about students' knowledge and skills that would help guide instruction. And, because Title I was intended as a compensatory education program, the tests usually measured basic skills only, to provide information on how students participating in the program fared on such tasks. The Advisory Commission found, however, that the reliance on norm-referenced tests of basic skills to produce national data on student achievement encouraged schools and teachers to narrow the curriculum to the material tested and to “spend undue time teaching test-taking skills or low-level basic skills, rather than challenging content” (p. 13).
In response to such concerns, the Congress revamped the Title I law substantially in 1994: perhaps the most far-reaching changes were in the assessment arena. Specifically, the law required states to develop challenging standards for student performance and assessments that measure student performance against the standards. Significantly, the law states that the standards and assessments are expected to be the same for all students, regardless of whether they are eligible for Title I. Thus for the first time, the 1994 statute enshrines into law the principle that Title I students are to be held to the same standards as all other students.
These changes did not come about in a vacuum. To be sure, they represented a response to the well-documented shortcomings of the Title I program as it existed for its first 30 years. But the new law also fit squarely within the reform context of the early 1990s. Specifically, the law's focus on standards for student performance, and its premise that all students are expected to meet challenging standards, conformed to the emphasis in the reform movement on standards as the fulcrum of redesigned schools and school systems. Many of the most prominent reform efforts of the era, notably the Kentucky Education Reform Act, the most sweeping statewide reform statute in history, share this focus on standards and are considered examples of “standards-based reform.” This general category refers to the idea of creating high standards for all students, measuring student performance against such standards, giving schools flexibility in how they design curriculum and instruction to enable students to meet the standards, and holding schools strictly accountable for attaining the standards. By requiring states to develop standards for student performance—the same challenging standards for all students—and to develop assessments linked to the standards, the Title I law in effect required states to adopt standards-based reform.
Moreover, the 1994 Title I statute also reflected the ferment in testing and assessment that has churned up the field since the mid-1980s. At that time, as state testing mandates increased and testing became more prevalent and more prominent in schools, critics became more vocal. Like the critics who focused specifically on Title I testing, including the Advisory Commission cited above, the testing critics charged that the growing use of testing with high stakes attached narrowed the curriculum and encouraged schools to emphasize low-level skills and knowledge at the expense of more challenging abilities. In place of such tests, reformers argued for so-called performance-based assessments, which ask students to demonstrate their knowledge and skill by performing a task, such as writing an essay, completing a science experiment, or explaining their solution to a mathematics problem. The reformers also argued for reporting assessment results based on how well a student performed against expectations for achievement, rather than a comparison with other students' performance.
The Title I law fit into this assessment reform movement by requiring tests that measure performance against standards, rather than those that compare student performance with that of other students. In addition, the law explicitly mandates that states use multiple, up-to-date measures of student performance, thus enshrining in law the demand for reform in assessment. In addition, the law also requires states to:
- Use assessments for purposes for which they are valid and reliable and ensure that such assessments are consistent with relevant, nationally recognized professional and technical standards;
- Administer assessments at least once between grades 3 and 5, and again between grades 6 and 9 and grades 10 and 12;
- Provide disaggregated achievement data that indicate the performance of students by gender, race, income, and other categories;
- Establish at least two levels of achievement, proficient and advanced, and indicate the proportion of students who attain each level, as well as a third level of performance, partially proficient, to provide information about the progress of lower-performing children toward reaching to the proficient and advanced levels;
- Determine what constitutes “adequate yearly progress” on the new assessments and hold schools and districts accountable for meeting such targets.
The record of states in implementing the new law show that the 1994 statute poses a substantial challenge. Although nearly every state has adopted content standards, as the law requires, reviews of such standards show that their rigor and usefulness vary widely (American Federation of Teachers, 1998; Council for Basic Education, 1998; Fordham Foundation, 1998). In addition, only 21 states and Puerto Rico adopted performance standards by the law's deadline of spring 1998; the rest received waivers to allow them more time.
Although the law's requirements for assessments and accountability do not take effect until 2000–2001, the plans that states have developed for such measures suggest that they may fall short of the law's intent. For example, many states have failed to indicate that they will include students with disabilities and English-language learners in their assessments, despite the law's requirements that they do so. Others have created accountability mechanisms that do not necessarily encourage schools to focus attention on poor and disadvantaged students (Citizens' Commission on Civil Rights, 1998; Chun and Goertz, 1999).
In part because of the uneven progress in implementing the statute, several commentators have begun the debate over the 1999 reauthorization of the Elementary and Secondary Education Act by suggesting that it is time to rethink Title I's purpose and to rewrite the law in dramatic ways. Some critics, contending that the law has failed in its attempt to raise the academic performance of poor children, have argued that the federal government should scrap most of its rules and send money to states with few strings attached, while holding states accountable for results (Finn et al., 1999; Ravitch, 1999). Others maintain that under Title I the federal government has been too lax and has allowed states to support reforms that were ineffective or even harmful; they argue that Title I funds should be directed at efforts that have been shown to improve schooling for disadvantaged children (Orfield, 1999). Still others note that the law is not yet fully in place, but that its principles are sound and could show promise if implemented effectively (Independent Review Panel, 1999).
The purpose of this report is not to recommend a plan for the reauthorization of Title I. The Committee on Title I Testing and Assessment was not asked
to critique the law nor to evaluate its implementation. The committee was asked to bring to bear our analysis of the evidence and our experience in schools, school districts, and states to help states and districts implement the statute in a way that will be effective for the disadvantaged students Title I is intended to benefit.
The Committee's Approach
The Committee on Title I Testing and Assessment was charged with assessing research on the use of testing and assessment for accountability purposes, examining the experience of states and districts in this domain, and developing a “decision framework that incorporates technical quality, effects on teaching and learning, costs and benefits, fairness, and other criteria for evaluating assessment strategies.” Our goal was to produce a practical guide for states and districts to use in developing the systems they were creating under the Title I law.
The committee went about its task in a number of ways. First, we reviewed available evidence from research on assessment, accountability, and standards-based reform. However, we recognized that in many areas the evidentiary base was slim. Standards-based reform is a new idea, and few places have put all the pieces in place, and even fewer have put them in place long enough to enable scholars to observe their effects. Therefore, we supplemented our review of the research with evidence from our own observations and reports from the field. Many committee members are practitioners, whose daily work is to develop and implement assessments and accountability mechanisms. Other members are in classrooms regularly, helping and observing teachers and administrators as they implement standards-based reform. The knowledge gleaned from these observations helped inform our work.
In addition, the committee also sought testimony from educators in leading-edge states and districts, who described for the committee their efforts. This testimony helped the committee understand not only the effects of assessment and accountability, but also the practical challenges involved in putting in place a system constrained by cost and political demands.
As we studied the research, examined our own experiences, and listened to the testimony, the committee kept in mind three underlying principles. First, the committee agreed that the purpose of assessment and accountability is to contribute to and support high levels of student learning. We recognized that there are many ways to respond to the law's requirements, and some evidence suggests that at least some states fell well short of the law's goals even as they complied with its mandates (Citizens' Commission on Civil Rights, 1998). Yet, in the committee's view, the potential educational power of assessment and accountability far outweighs the bureaucratic purposes of such instruments, particularly for those who have been historically poorly served by the education
system. We therefore used as a yardstick a simple measure: whether a state's decision would help improve student learning and reduce the achievement gap. Simple compliance with the law was not enough.
Second, the committee agreed that the education improvement system, of which assessment and accountability are key components, should be conceived of and implemented as just that—a system. That is, the system should consist of a number of components, at various levels (classroom, school, school district, and state), each of which plays a role in measuring and contributing to student learning, yet which are interrelated, not separate from one another. Applying this principle, one might view one component—for example, a state testing program—that by itself falls short, as appropriate in a system that also includes components that complement it.
Third, the committee agreed that a hallmark of the state and district systems should be continuous improvement at all levels—for students, for teachers and administrators, and for the system itself. Improvement for students is obvious; as noted above, it is the reason for the system. Improvement for teachers and administrators is also a necessary part of the system. Although the role of professional development was not formally part of the committee's charge, the committee found it impossible to discuss assessment without addressing the role of enhancing the knowledge and skills of teachers. High student performance depends on high-quality instruction, and building the capacity of teachers and administrators is at least as much a necessary condition of educational improvement as establishing standards or putting in place accountability mechanisms.
Improvement for the system itself is also essential. Assessments and accountability schemes change constantly, despite the best-laid plans of educators and blue-ribbon panels. Legislators, responding to teachers, parents, and other constituents, frequently mandate adjustments, from adding multiple-choice components to requiring individual student reports. And administrators change programs as data come in that show the effects of the system on students and schools. Although the Title I statute calls the assessments that are to be put in place by 2000–2001 “final,” these assessments are likely to undergo numerous rounds of revision, even if the structure of the education improvement system remains in place.
Moreover, as noted above, the effects of the components of a standards-based system are not completely certain. Many of the systems the committee studied are relatively new or still in some cases under development, and their full impact on students, particularly disadvantaged students, remains to be seen. Therefore, states and districts need to monitor their improvement systems continually, to ensure that they are achieving their desired goals.
For these and other reasons, the committee developed criteria not for the one best system—which does not exist—but for systems that are continually changing and adapting to new knowledge and new circumstances. In this way, the committee's framework is appropriate for states and districts just starting out
on redesigning their education improvement system, as well as for states and districts that have had redesigned systems in place for several years.
The committee also recognized that each state and district must develop its own system to meet local circumstances. The committee did not intend to propose a blueprint that all states and districts should adopt. Rather, our guidelines are intended to be used as yardsticks against which states and districts can measure their own judgments. Moreover, we recognize that building effective education improvement systems is hard work, particularly in the charged political environments in which states and school districts operate. We would never presume that policy makers or administrators could simply implement complex systems with a wave of the hand, much less carry through with the even harder work of building the capacity of schools to educate all students to high levels.
Organization of this Report
To carry out its charge to provide guidance to state and district officials responsible for making decisions about appropriate assessment and accountability systems to meet the requirements of the Title I law, the committee has conceived of this report as a guide. That is, it was designed to be useful as well as informative. To that end, we have organized the report so that our readers can walk through the various components of the system and consider a set of questions that state and district leaders should ask themselves as they develop their systems. We identify what we consider the key criteria for each component. And we include examples of states and districts that have applied these criteria in different ways.
This approach is intended to accomplish two goals. First, we present what our review of the research and our experience show are the basic principles beneath an effective system, allowing state and district officials to measure their own approaches against our criteria. Second, we present examples to show that there are many ways of applying these criteria; we do not want to suggest at any point that there is one right way to do this. In addition, we do not want to suggest that these examples represent ideal solutions to the challenges states and districts face. Some of these examples do not completely meet our criteria, and we indicate this in introducing them. Some examples remain controversial and deserve continuing study; a future edition of this guide might include a different set of examples.
Before turning to the criteria, however, we need to examine the entire system. In Chapter 2, we consider and critique the theory of action behind the Title I law and the various attempts at standards-based reform. We then expand on the theory of action to reflect our analysis of effective reform.
assessments for young children and for special populations, as well as reporting and disaggregating assessment results. In Chapter 5, we consider systems for monitoring the conditions of instruction at the school level and professional development at the district level. In Chapter 6, we examine ways to measure adequate progress of schools toward standards, and in Chapter 7 we discuss accountability.