6

Paths Forward: Recommendations

In Chapters 1 through 5 of this report we present our rationale and vision for monitoring equity in K–12 education with a set of indicators and more specific measures, which are detailed in Chapters 4 and 5. The committee’s intent is for these indicators to form the core of a national program to monitor educational equity by assembling information that can be reported at the national, state, and local levels and can be disaggregated for important population groups. This targeted set of indicators will shed light on the nature of between-group differences in academic achievement and educational attainment and disparities in access to critical educational resources. In so doing, they will provide a scientific basis for policies to address those inequities. This final chapter focuses on implementation: namely, how to transition from identifying research-based indicators of educational equity in concept to implementation of a functioning system.

READINESS OF INDICATORS FOR OPERATIONAL USE

As detailed in Chapters 4 and 5, we propose indicators for seven domains. The first three domains (A, B, C) reflect transition points in students’ lives across K–12 education: readiness for the transition into kindergarten, steady progress through the grades, and readiness for the transition to postsecondary endeavors (see Figure 2-2 in Chapter 2). The other four domains (D, E, F, G) reflect structures and resources in the K–12 education system that can mitigate or exacerbate disparities: exposure to racial or socioeconomic segregation, access to high-quality early childhood education, access to high-quality curricula and instruction across all achievement

levels, and access to supportive schools. For each of these domains, we propose one or more indicators that reflect factors that are (1) critical to academic success and education attainment and (2) sources of between-group differences. For each indicator, we suggest, mindful of parsimony, constructs to measure, track, and compare between-group differences.

Some of the indicators we propose are ready to implement operationally: measures of the constructs have been developed and subgroup data are available at the school, district, state, and national levels. Attendance (or its converse, absenteeism) is an example: states and districts now have administrative data systems that enable reliable, valid, and accurate calculation of attendance rates (or absentee rates) and that support cross-state comparisons.

Other indicators will require additional work before they can be implemented. For example, while many school systems evaluate readiness skills when children enter kindergarten, the assessments they use differ. They differ in terms of the skills assessed, the methods by which they are assessed, the timing of the assessments, and the way of determining readiness. These assessments can be useful in monitoring between-group differences in readiness at the local level, but further work is needed to implement them on a national basis.

In our view, an equity indicator system must evolve, with structures, processes, and resources for continuous improvement in light of research, experience, and richer consensus. A corollary is that indicators or measures will not initially be identical in sophistication, or perhaps ever. As a practical matter, there will be differences in, among other characteristics, strength of research base; precise measures and data definitions across jurisdictions; availability of data disaggregation by population groups; and complexity arising from independent policy choices when there are no federal standards. For example, except for the federally constructed and administered National Assessment of Educational Progress (NAEP), comparisons of achievement across jurisdictions are difficult because states use different tests for their assessments. For another example, for early childhood education states have different standards for quality, even for licensed providers. Differences in financial accounting systems will complicate comparison of resources. Therefore, an indicator system close to a researcher’s ideal is impossible, although a useful system is possible—especially if designed for continuous improvement, as we discuss below.

Several of our proposed indicators and measures either exist or can be readily constructed from existing data collection systems mandated by federal or state policies; typically, these have the force of law by statute or regulation. Chapters 2, 4, and 5—with additional details in Appendixes A, B, and C—described a vast array of nationwide data elements compiled by the National Center for Education Statistics, other agencies in the U.S.

Department of Education, and the U.S. Census Bureau. Many, though not all, of the committee’s proposed indicators of educational equity are available from existing data collection programs. Not all data sources are collected on an annual basis, however, and data sets vary in the extent to which indicators can be disaggregated by population group, in the unit of analysis (state, district, school, classroom), and in the availability of the data after collection. Many of the challenging key constructs (e.g., measures of socioeconomic status) are not always consistent among sources, and some indicators require modeling and linking of more than one data set.

There has been innovative work to develop indicators in specific domains. One example is the work of the Stanford Education Data Archive in using calibration procedures to put state assessment results on a common scale based on the National Assessment of Educational Progress (see description of the Stanford Education Data Archive in Appendix A). Another example is adjusted-per-pupil spending to measure equitable distribution of resources relative to student need (see description of the Education Law Center and Rutgers University collaboration in Appendix A). Such work indicates the potential for analytically meaningful indicators of educational equity, although the effort required for the two examples cited is considerable and, at present, results in a lag in availability.

There are many publications of key indicators for K–12 education and for child well-being more generally, and most publications link to more detailed underlying data. But none of the publications, including those that focus specifically on between-group disparities, presents a fully developed representation based on a carefully articulated concept of equity that covers all student groups of interest. In addition, the indicators in some reports are based on data sources that cannot support subnational detail, although in many cases the indicators could be developed from the bottom up by states and school districts for individual schools if there was a federal or national (meaning multistate) mechanism for coordination. In addition, data collection is not only a technical issue, but also a matter of public policy so there are political considerations and disagreement over such issues as compliance burdens and federalism: Which level of government has the “right” to impose data collection requirements and constructs? For example, the Every Student Succeeds Act (ESSA) contains bare-bones requirements for data collection but leaves many design and implementation choices to states. ESSA is in some respects a federal framework, but it deemphasizes national uniformity by leaving many important choices to individual states. Relatedly, and relevant to the equity discussion, there is now serious political division over the future scope of the Civil Rights Data Collection program.

IMPLEMENTATION

Clearly, refining and implementing the committee’s proposed system would be easier if there were available data and indicators that simply needed to be brought together or modified slightly to satisfy the goals of an equity indicator system. Our review reveals that this is not the case. Considerable effort will be needed to assemble the necessary data and conduct the necessary analyses and data manipulations to generate comparable indicators for the nation, states, districts, and schools. Substantial effort will also be required to implement, evaluate, and improve a system of educational equity indicators on a continuing, regular basis.

Yet the committee concludes that such a system, including its continuous improvement, is essential for myriad social, economic, and fairness reasons. As we state throughout this report, a problem cannot be addressed if it is invisible. A system of equity indicators with its design based on research and consensus would, we believe, be a major advance in the nation’s capacity to understand and address ubiquitous disparities in education opportunities and outcomes. This is important for each level of government. Indeed, we believe it is an essential step.

Bottom-Up and Top-Down Approaches

We do not anticipate that an indicator system will be built in a day. Rather, we envision that a system would be developed gradually, making use of existing data first, adapting them to the needs of the indicator system, and making use of that beginning until a fuller system is developed. We see two paths forward.

The first path (bottom up) is for a set of “early adopter” states and districts to partner with each other, researchers, stakeholders, and governmental or philanthropic funders to develop the design and implementation details for the indicator system we describe. This effort would provide the early adopters with useful information, but the more important goal would be creation of a prototype for other jurisdictions to adopt. The prototype would build on existing data and indicator systems used by the early adopters and would also be informed by broader analysis of successes and failures in other jurisdictions. This approach would need to grapple with the existing variation in many indicators, both within and across jurisdic-

tions, and offer a way to consolidate indicators or establish comparability with disparate indicators.

The second path (top down) is for the federal government, working with relevant educational intermediaries—particularly national organizations, such as the National Governors Association, the Council of Chief State School Officers, and the Council of the Great City Schools—to develop an initial version of the indicator system we propose. (This is somewhat analogous to development of the Common Core State Standards.) This path would capitalize on data currently available as a first step and would outline a plan and timetable to add other indicators as conceptual, methodological, and data issues are satisfactorily addressed.

We see these paths as complementary rather than mutually exclusive. In the best case, they would operate concurrently and interact so that the on-the-ground experience of the prototypes can inform the national effort, and the national effort can facilitate ways for districts to improve their data and measures.

Oversight and Guidance

This country has a great deal of experience with national indicator systems. In the context of education, NAEP is now more than 40 years old and commands respect throughout the field of education. It is overseen by a governing board (the National Assessment Governing Board [NAGB]) that is charged with its oversight and updates and modifies it so that it continues to be useful.

NAEP provides a model that may be useful in designing the indicators system we propose. We note the involvement of federal, state, and local stakeholders, research firms with survey and test development expertise, and private foundations in the development, evaluation, and improvement of NAEP. We also note the time it took to establish NAEP as a valued national resource, which culminated with congressional action in 1988 to establish the National Assessment Governing Board (NAGB) as the NAEP oversight body and in 2001 to require states to participate in math and reading assessments. NAEP is still evolving, having, for example, recently added a technology and engineering literacy assessment: see Box 6-1 for a chronology of key milestones in NAEP’s development.

We see a need for such collaboration in the development of a national equity indicator system and suggest that interested stakeholders use it as a model. In particular, we see value in a governing body that would serve a role analogous to NAGB’s role in NAEP and that partners with the National Center for Education Statistics. We also note that other data collection efforts, such as the Current Population Survey of the U.S. Census Bureau, have oversight or advisory boards, as do several international edu-

cation endeavors, such as the Program for International Student Assessment (PISA), the Progress in International Reading Literacy Study (PIRLS), and the Trends in International Mathematics and Science Study (TIMSS).

There are many technical issues to address in formulating the system so the indicators are valid and useful, and a panel of experts to advise on these issues would be essential. There is also a need for input and buy-in from a range of stakeholders who can address not only the content of and process for producing an informative and coherent set of education equity indicators, but also guidance on how users can interpret the results.

RECOMMENDATIONS

We note that the system of indicators we propose focuses on the role the education system should play in addressing academic disparities. Although unaddressed in this report, other child-serving agencies play an equally important role in helping children. The effects of adversity on a child or adolescent depends not only on individual resilience and natural variations in child development, but also on the child’s opportunity for experiences, interventions, and supports that may mitigate or even undo the effects of adversity, both material and psychological. That is, learning obstacles born of context are not student deficits barring success, but student needs that can be met with appropriate opportunities. Research is needed to increase understanding of how various interventions or opportunities map onto individual student needs that are rooted in context. Consensus-building is needed to create indicators and measures that should be included, eventually, in a broader equity indicator system. For many student needs, screening and responses can best be provided outside of the school settings. Therefore, an indicator system that encompasses all the domains of opportunity important for equity would need to monitor how well student success is supported by other child-serving agencies and nonprofit organizations.

A concerted effort is needed to create the system of equity indicators. Demonstration projects and early prototypes will help catalyze interest in the system and test its feasibility and usefulness.

Regardless of the path chosen, a system of equity indicators needs input and buy-in from a range of stakeholders. This input is needed to develop a process for producing an informative and coherent set of educational equity indicators, determine the content of the indicators, and ensure that the results will be understood by users. For these purposes, we believe a governing body is needed to provide governance and implementation. We suggest that one analogous to NAGB, which partners with the National Center for Education Statistics for NAEP, could be a useful model.

Acting on these recommendations will keep in the public eye a critical goal for the nation: to ensure that all students receive the supports they need to obtain a high-quality pre-K to grade 12 education. Educating all students is fundamental to the nation’s ability to grow and develop and to afford all of its people the opportunity to live full and rewarding lives.

This page intentionally left blank.