In 2007, the District of Columbia passed a law (Public Education Reform Amendment Act [PERAA]) that gave control of its public schools to its mayor. The law’s purpose was to allow leaders flexibility so they could make bold changes to improve a school system that had been performing poorly for decades. The law also called for an independent evaluation of how well the public schools fared under new governance, to be carried out by a committee of the National Research Council of the National Academy of Sciences. The D.C. Council asked the committee to assess
- whether the law’s expectations have been met and whether the changes have led to improved coordination, efficiency, and accountability;
- the extent to which the actions school leaders took were consistent with research and best practices; and
- changes in the conditions for learning in the schools and outcomes for students 7 years after the governance change.
These questions called for a detailed analysis of PERAA’s provisions and the goals it was intended to achieve, a review of the changes the city has made in response, an assessment of actions taken by education leaders empowered by the law, and a review of progress in both the conditions for learning in the schools and the outcomes for students since the law was passed.
It was necessary to seek an exceptionally wide range of evidence to answer the questions in the committee’s charge, using publicly available
data and information; information provided by city agencies; papers commissioned by the committee; interviews; and other reports and analyses, including independent reports prepared for this project by the Education Consortium on Research and Evaluation (DC-EdCORE). Although the committee was able to obtain evidence on many issues, our work was hampered by difficulty in obtaining some of the information we sought from the city.
WERE PERAA’S EXPECTATIONS MET, AND DID ITS CHANGES BRING ABOUT IMPROVED COORDINATION, EFFICIENCY, AND ACCOUNTABILITY?
We conclude that the city has executed most of what was called for by PERAA. For example, the District of Columbia Public Schools (DCPS) and the Public Charter School Board (PCSB) are carrying out their functions and have used the flexibility and authority they gained under the law to pursue improvements that show promise. In addition, as specified by the law, three new agencies created by PERAA, the Deputy Mayor for Education (DME), the Office of the State Superintendent of Education (OSSE), and the State Board of Education (SBOE) are operating as called for in the law.
There are several areas for which the results so far do not match PERAA’s expectations, however. The law called for a body that could coordinate across the city agencies concerned with the well-being of children, adolescents, and families: this body was created but was subsequently defunded. The law also called for a data warehouse that would support interagency coordination by allowing data sharing across agencies and other functions: despite progress in data collections efforts, this data infrastructure is not in place. [Conclusion 3-1]
PERAA was also intended to promote coordination among the education agencies, efficiency, and accountability. However, coordination among all of the education agencies is more the product of collegial approaches than institutional structures and incentives. The missions and lines of authority among the three oversight bodies (DME, OSSE, and the SBOE) are not clearly delineated. At present, none of the three agencies is clearly recognized as having the primary responsibility for monitoring and overseeing the quality of public education for all students. OSSE, by far the largest of the three, is not consistently functioning as an effective state education authority, and it has not yet earned the full confidence of officials in other agencies who rely on it. [Conclusion 3-1] DME and SBOE have neither the resources to monitor effectively nor the meaningful authority to oversee the approximately 83,000 students enrolled in DCPS and the charter schools. [Conclusions 3-2 and 3-4]
The issue of monitoring is a complex one in D.C. for two reasons. First, D.C. functions as both a state and a city. PERAA created OSSE to
perform the specific state functions associated with federal compliance and contracting, but it also gave the agency additional responsibilities, not all of which were well defined. Second, the city’s public schools were operated by a single district (DCPS) until the first charter schools were established in the mid-1990s. Today, nearly half the city’s public school students are enrolled in charter schools, and there are 62 districts (local education agencies): DCPS and the 61 chartered entities. The governance structure outlined out in PERAA did not address the changing balance between traditional public schools and charters and how that might affect the governance challenge.
PERAA called for an ombudsman to help meet the goal of greater transparency and accountability for public education in D.C. That position was created, abolished, and then reestablished, but it has yet to play a significant role. In addition, the budgeting process for education expenditures is neither simpler nor more transparent than it was before PERAA. [Conclusion 3-3]
The current governance structure represents a reasonable response to the provisions of PERAA but leaves two issues for the city to consider: whether the current oversight structure provides sufficient monitoring of the educational opportunities provided to students attending DCPS and charter schools throughout the city, and how best to oversee the education of all students attending any publicly funded school. [Conclusion 3-5]
WERE THE ACTIONS SCHOOL LEADERS TOOK CONSISTENT WITH RESEARCH AND BEST PRACTICES?
We focused on one of the most prominent actions taken by DCPS under its new leadership, the decision to emphasize improving teacher quality using a new evaluation system, IMPACT. It was not possible to examine similar strategies for the charter schools because no programmatic strategies apply across all of them. We examined IMPACT’s design and implementation plan and reviewed data on changes in the teacher workforce. Based on the information available to us, IMPACT—with its multiple measures of teacher performance, feedback and supports provided to teachers, and opportunities for professional development—generally reflects the guidance available in research literature for teacher evaluation systems of its type. [Conclusions 4-1 and 4-2]
We have several concerns that we believe it would be advisable for DCPS to address. The city articulated a number of goals for IMPACT but has not yet developed a plan for evaluating progress toward meeting them. DCPS should monitor how well IMPACT serves its intended purposes, particularly the goal of improving teaching in schools serving lower-achieving students. The city placed a high priority on improving the quality of the teacher workforce, under the premise that improving teacher quality would
lead to improved conditions and outcomes for all students. The evidence available to date shows that most DCPS teachers receive high effectiveness ratings; however, the highest-rated teachers are not distributed equitably across the wards, with fewer of them serving the most disadvantaged students. DCPS has more work to do in ensuring that it has a team of highly rated teachers in every school; we have no systematic information about the teachers in charter schools. [Conclusions 4-3, 4-4, and 4-5]
IMPACT provides important information about DCPS educators but it is generally not used by the charter schools and little systematic information about charter school teachers is available. The city would benefit from maintaining a wide range of data about teachers in both DCPS and the charter schools. [Conclusion 4-6]
WHAT IS KNOWN ABOUT LEARNING CONDITIONS 7 YEARS AFTER THE GOVERNANCE CHANGE?
The conditions that should be in place to promote learning encompass not only academic offerings and resources, but also a healthy and productive school climate and supports for the challenges faced by many student groups. We examined a set of topics that reflect the broad scope of issues that should be monitored to ensure that all students have an equitable opportunity to learn.
The limited evidence available to us shows evidence of efforts to improve learning conditions, but also suggests that there are differences across student groups and wards in access to educational opportunity and the quality of the educational experience. The committee could find very little information about learning conditions in charter schools because many types of information are not collected systematically for this sector. We found slightly more information about DCPS but still saw many gaps in the information needed. [Conclusion 5-1]
Of significant concern is the fact that the governance structure with respect to learning opportunities is diffuse: no one entity has both the responsibility and the authority for monitoring the provision of education and supports for students, particularly those at risk for school failure, across both the DCPS and charter schools. We believe that a single entity should be responsible for this essential function systemwide: to meet this responsibility, the entity in charge will need to maintain and make publicly accessible data about students with particular needs, including those with disabilities, English-language learners, and students in poverty; school climate, including discipline, attendance, safety, and facilities; and academic supports for learning. [Conclusions 5-2 and 5-3]
WHAT IS KNOWN ABOUT STUDENT OUTCOMES 7 YEARS AFTER THE GOVERNANCE CHANGE?
In order to understand outcomes for students it is important to look not only at the most readily available information—test data and graduation rates—but also other indicators of outcomes and attainment, including indicators of school behavior and postsecondary attainment. The committee did not have the data needed to examine most of this information. We found that, in general, scores from both the District of Columbia Comprehensive Assessment System (DC CAS) and the National Assessment of Educational Progress (NAEP) increased between 2007 and 2014 across most student groups. The increase is larger for math than it is for reading; however, indicators of proficiency in both subjects remain low. Graduation rates have fluctuated from year to year, with no discernable pattern, but they, too, remain disturbingly low. [Conclusions 6-1 and 6-4]
Black and Hispanic students, those with disabilities, those eligible for free or reduced-price lunches, and English-language learners are much more likely to be in the lowest performance categories than other students. Some improvement is evident since 2009, but more than half of these students still score below proficient. There is little indication that these performance disparities—in test scores or in graduation rates—are lessening. [Conclusions 6-1, 6-2, and 6-4]
The signs of improvement are positive, but a more complete picture of student outcomes is needed. To better understand outcomes for D.C. students, the city needs improved reporting of test results to provide more detailed information about student performance. It also needs to make data available that will cover a range of outcomes and allow detailed analyses of trends across time and among student groups. [Conclusions 6-3 and 6-5]
We can document changes that occurred over the past 7 years but it is not possible to attribute any of them directly to PERAA. The law gave the city a mechanism with which to address problems, and it has done so. The committee sees reasons for optimism about the future for D.C.’s public schools: DCPS and the PCSB have made choices that show promise, and the city has sustained its focus on its improvement over several leadership changes.
Nevertheless, our evaluation shows that:
- Monitoring and oversight of the needs of students with particular needs, including students with disabilities, English-language learners, low-income students, and others is not adequate.
- DCPS schools in the lowest income sections of the city have fewer teachers with high IMPACT ratings and provide less access to advanced coursework than other DCPS schools; there were no data available on these issues for the charter schools.
- There are stark gaps in academic achievement and graduation rates across student groups.
Our three recommendations are intended to help the city build on the work it has already done to address this fundamental challenge. Our conclusions in each of the broad areas we examined stressed the need for improvement in the way the city collects and uses information about public education. A significant array of data, documentation, and reports concerning the city’s schools is available, but these materials are widely scattered and not structured to support districtwide evaluation. More important, however, is that no one entity is currently responsible for coordinating information from across the education agencies and across all the public schools.
Regardless of the governance structure in place, a reliable source of comprehensive information about the functioning of the public schools will be crucial to improving monitoring and accountability. More accessible data would also reveal progress the city is making in education, and greater accessibility would likely build public trust and patience during the time it takes to pursue lasting change.
RECOMMENDATION 1 The District of Columbia should have a comprehensive data warehouse that makes basic information about the school system available in one place. That information should be readily accessible online to parents, the community, and researchers. That information should include data on the school system as a whole and at more detailed levels. Building such a warehouse will take time, but it can begin with the data collection efforts already in place. An optimal data warehouse would have the following characteristics:
- It would integrate and track data that are relevant to schooling and students across DCPS and the charter schools and eventually across the education, justice, and human service agencies.
- It would provide data about learning conditions in all public schools, DCPS and the charters, and their students covering students with particular needs, including those with disabilities, English-language learners, and students in poverty; school climate, including discipline, attendance, safety, and facilities; and academic supports for learning.
- It would provide data about outcomes for all public school students, in DCPS and the charters, covering graduation rates, per-
formance on tests including college entrance exams, attendance and truancy, course-taking and completion, college enrollment and progression, and career outcomes.
- It would be usable and accessible to researchers, educators, parents, and the public. The format would be structured to allow ready access to data and analysis in ways that can be customized to the needs of different users, including parents and other nonspecialists.
PERAA called for an interagency coordinating body to develop a data warehouse of this type. Our recommendation for a centralized data warehouse is more comprehensive than PERAA’s specifications, and we believe that it should serve a broader purpose—that is, that such a resource should not be used only for coordinating data across city agencies, but also for helping the city effectively monitor all of its public schools and students.
At present, no single entity in D.C. is looking analytically at the way all of the city’s public school students are being educated and making sure that certain basic conditions are provided. We distinguish between a responsibility to ensure that basic conditions are met and interference with the way DCPS or the charter schools make the decisions that are their responsibility about how to fulfill their educational missions.
Because D.C. functions as a state with 62 school districts—DCPS and 61 charter entities—it has a responsibility to collect and maintain the systemwide data needed to measure progress toward meeting the objective of ensuring an equitable education for all public school students. If the city does decide to have a single entity with responsibility across DCPS and the charter schools, it would be reasonable to consider transforming OSSE—although it currently has a number of problems—into that entity.
At the same time, the city would benefit from having access to ongoing, independent evaluations of its progress. D.C. would derive great benefit from having a program of ongoing evaluation, and a comprehensive data warehouse could be the foundation of that program. Such a program would benefit researchers, education policy makers, and city residents.
RECOMMENDATION 2 The District of Columbia should establish institutional arrangements that will support ongoing independent evaluation of its education system. Whatever structure is developed, three conditions should be met:
- The evaluation entity should have sufficient resources to collect and analyze primary data, including at the school level, without being entirely dependent on district-generated test and administrative data.
- Evaluations should be conducted by experts with the qualifications needed for specific tasks. Ideally, the structure will allow the city to benefit from the expertise of external researchers and practitioners who specialize in teaching and learning, curriculum, testing and measurement, and finance and policy.
- All products produced by the entity should undergo rigorous peer review.
We were not asked to make recommendations to the city about its governance structure, but we close with a recommendation regarding priorities for the city as it approaches the 10-year anniversary of PERAA. PERAA provided the city with a structure it could use to make bold changes, but a governance change by itself cannot be expected to bring about the desired changes. Using the flexibility provided by PERAA, the city has made a solid start. The next step is to build on it in addressing the major long-standing challenges in D.C. These challenges are at the heart of the findings from our evaluation because they have persisted in spite of significant progress made in the years since PERAA.
RECOMMENDATION 3 The primary objective of the District of Columbia for its public schools should be to address the serious and persistent disparities in learning opportunities and academic progress across student groups and wards by attending to
- centralized, systemwide monitoring and oversight of all public schools and their students, with particular attention to high-need student groups;
- the fair distribution of educational resources across schools and wards;
- ongoing assessment of how well strategies for improving teacher quality are meeting their goals;
- more effective collaboration among public agencies and with the private sector to encourage cross-sector problem solving for the city’s schools;
- accessible, useful, and transparent data about D.C. public schools, including charters, that are tailored to the diverse groups with a stake in the system; and
- measures to strengthen public trust in education in a diverse, highly mobile city.