5
Improving the Survey and Its Use
The work of the Office for Civil Rights (OCR) in addressing violations of students’ civil rights and combating discrimination continues to be important, and the E&S survey is a useful, albeit underused, resource in OCR’s efforts to enforce civil rights laws. The information produced by the E&S survey facilitates the identification of disparities in learning opportunities associated with race, ethnicity, language, gender, and disability status. Importantly, the survey is the only available data source that can be used to identify problems not only at the national and state levels, but also in school districts, and in individual schools. The committee finds that the E&S survey continues to play an important role in protecting the rights of minority students, students with disabilities, students with limited English proficiency, and women. The E&S survey is the only nationwide source of much of the information it collects. The survey’s capacity to identify interdistrict, intradistrict, and even intraschool disparities in learning opportunities makes it a unique resource that provides some, though not all, of the information needed for spotting potentially actionable discrimination and violations of civil rights.
The survey currently is not being extensively used by OCR in enforcement, except as one of a number of sources of information used to identify school districts for compliance reviews. The number of compliance reviews initiated by OCR varies considerably from year to year; in 2002, no new compliance reviews were initiated. The survey’s most important use in recent years has been to provide parents and others with information on disparities in access to high-quality learning opportunities.
The committee also finds that the
survey and the manner in which it is administered can be improved to: (1) more effectively assess whether students have access to critical learning opportunities; (2) make the data more accessible to interested citizens, as well as to educators and policy makers; and (3) enhance its usefulness as a resource for research that could lead to school improvement.
Despite the many societal and educational changes that have occurred during the past 35 years, major disparities in opportunities to learn and in education outcomes persist, especially those associated with race and ethnicity—the original focus of the E&S survey. The timely collection and analysis of classroom-, schooland district-level data that can help identify educational policies and practices that may have inequitable, if not discriminatory, effects on students is no less important today than it was when the E&S survey was first administered.
LEARNING OPPORTUNITIES— THE KEY
In one way or another, virtually all civil rights related to education involve the assurance that students will have the necessary opportunities to learn. The definition of “necessary” is contested, but the idea of equal opportunities to learn for persons of different backgrounds is the basis of contemporary understandings of rights. Legislation and court rulings addressing the needs of persons who do not speak English or who have disabilities have extended the protection of the law and gone beyond the goal of equality to guarantee the provision of needed services. School finance cases also have sought to broaden the definition of individual rights related to education beyond equality of spending per student to take into account the fact that some students, including but not limited to students who do not speak English and students with disabilities, need more services than others if they are to succeed in school (see Rebell, 2002).
Regardless of the race, ethnicity, socioeconomic status, or disability status of the students whose educational rights are at issue, the focus of concern is learning opportunities. If students with certain characteristics have differential access to learning opportunities in particular situations, it may signal discrimination. However, despite the centrality of learning opportunities to the protection of civil rights, there is no accepted or even widely discussed model that identifies the full range of opportunities to learn, their relative importance to student learning, and their interrelationships. Research that links data on patterns of access to
learning opportunities and resources, such as those from the E&S survey, with other datasets that are used to examine the effects of various policies, practices, and resources, potentially could produce a clearer understanding of what kinds of disparities make a difference for students’ learning outcomes (see Appendixes A and D). Not only would the development of such a model be important for the protection of civil rights, but it would also shape education strategies generally.
Children’s learning is influenced by many experiences. Those that are available in school typically account for less than one-third of students’ academic achievement, as measured by most common tests (e.g., see HalpernFelsher et al., 1997; National Research Council, 1999a). Yet the focus of the E&S survey, and thus the focus of this committee’s interests, are the learning opportunities and resources provided in schools or by schools. Trying to identify the full range of influences on student learning that might reasonably be the concern of those who would protect civil rights is daunting. Nonetheless, the E&S survey focuses on learning opportunities that are of critical importance. Continuing refinement of the survey and judicious additions to its scope would make it an even more substantial resource for enhancing the education of students who have been the victims of discrimination or of inadequate access to learning opportunities and resources.
PROMISING NEW USES OF E&S SURVEY DATA
Integrating E&S Survey Data with Other Information Resources
As discussed in Chapter 4, planning is under way to integrate the E&S survey into a consolidated data system that would coordinate the collection and management of data related to the administration of various programs administered by the U.S. Department of Education. The effort to develop this system, known as the PerformanceBased Data Management Initiative (PBDMI), is being conducted in partnership with state departments of education. Besides integrating the administrative data systems of the department, other goals of PBDMI include facilitating the integration of state and local education data with the department’s administrative database to make the data more usable and accessible for all.
Integrating the E&S survey with the PBDMI would help to ensure that the definitions of items that appear on the
E&S survey are consistent with those on other department surveys. Departmental literature on PBDMI also promises that it will reduce the data collection burden on schools by eliminating redundancy among the various surveys currently administered. It also promises that PBDMI will facilitate efforts to ensure accountability.
When the PBDMI data system is implemented, it will be important to ensure that the attributes that make the E&S survey an essential resource for monitoring equality of access to learning opportunities and resources is retained. Written descriptions of PBDMI from the Department of Education do not currently mention the collection and integration of data on access to learning opportunities and resources as a feature of the new data system. It is essential that information on access to opportunities to learn be an integral part of the PBDMI system. The new data system must continue to provide information pertinent to students’ civil rights under Title VI, Title IX, and Section 504. These data must continue to be disaggregated by race and ethnicity, traceable to specific schools and school districts, and accessible not only to departmental officials, but also to local educators and to the public. If some or all of the E&S survey were incorporated into the PBDMI, it could become less burdensome and therefore more cost-effective to routinely collect E&S data from all schools.
The Survey as a Resource for School Improvement Efforts
Large disparities in education outcomes among students of different backgrounds persist, and OCR continues to find that violations of students’ civil rights are not uncommon. Whether or not the disparities identified by the E&S survey are caused by violations of students’ legal rights, the disparities evidence the failure of schools to provide equal access to highquality education for students from all backgrounds. This information is relevant not just to civil rights enforcement, but also to broader based efforts to achieve equity and excellence. Providing educators, parents, and the general public with easy access information that suggests that their own schools may be failing to ensure equal access to learning opportunities can lead to greater public involvement in school improvement efforts.
As society and public school students become more diverse, issues of equity will become important in places that previously have not had diverse student populations. The successful implementation of school reforms in a postdesegregation era will require that
policies and practices be closely monitored so that they do not disadvantage minority students. The E&S survey provides a resource that can help to ensure that no child will be left behind. Without such a resource, education policy makers and practitioners may be unable to determine if reforms differently affect educational outcomes for racial and ethnic minority, language minority, or disabled students or even whether outcomes may vary by gender.
Until the summer of 2002, OCR had not published or otherwise made available to the public E&S survey data for more than 20 years—except by request. The recent placement of E&S data on the Internet is a very important innovation whose implications for public use of the data are not yet known. The software used to examine the data for exploratory purposes requires no technical training to use, and it allows individuals to make very specific queries about the various items addressed by the survey. For example, one could examine the number and percentage of sixth graders at a specific school who passed or failed a districtor stateadministered test required for grade promotion, with the information disaggregated by race, ethnicity, and sex. Data for different schools can be compared, as can data for different states. Although some of the data from the survey are suppressed to protect students’ privacy, the data available over the Internet provide extensive information concerning disparities in access to learning opportunities.1 Making these data easily available to the public is consistent with current federal and state policies emphasizing the public reporting of school-level data as a means of promoting accountability.
Educational Equity Self-Assessments
One way to do this could be in the form of an Educational Equity SelfAssessment (EESA). An EESA is similar in concept to the Racial Justice Report Card developed by the Applied Research Center of Oakland, California (Gordon, Della Piana, and Keleher, 2002). Using E&S survey data, possibly in combination with data from other sources, OCR could develop an EESA that educators, parent-teacher organizations, and others could use to evaluate individual schools. The EESA would be a computer-based, interactive program with access from the Internet or delivered on CD-ROM or DVD. School administrators and others in a school community could examine their school’s performance on equity measures of
issues, such as school discipline; graduation rates; student retention; testing outcomes; access to advanced placement and gifted and talented programs; tendencies toward within-school segregation, particularly as related to tracking; and racial disparities in special education. The data for a given school could be compared with district, state, or national averages or to equity standards suggested by OCR. Significant deviations from the suggested “equity standards” could produce an interactive diagnostic exercise that would include descriptions of one or more researchbased strategies or programs that could help the school to reduce the observed disparities and to achieve the suggested standards.
Public Reporting and Accountability
Another possible new use of E&S survey data is to incorporate it (and other information pertaining to access to learning resources) with the kinds of disaggregated achievement data that must be publicly reported under the No Child Left Behind Act. Under the act, schools are required to report achievement outcomes disaggregated by students’ race and poverty status and for language minority students. Schools are held accountable for ensuring that students from each group make adequate progress toward achieving statedefined standards for learning.
Data related to access to learning opportunities and resources, such as those available from the E&S survey, should be an integral part of school data systems and should provide information that is helpful to diagnosing causes of disparities, whether or not they are related to discrimination. The inclusion by schools in routine public reporting of information on access to learning opportunities and resources, along with data on students’ progress toward the achievement of standards, could be helpful in providing guidance about causes of disparities in outcomes as well as promising strategies to address them.
New Research Opportunities
Although the E&S survey is the most important source of information available on a wide range of topics related to access to learning opportunities and resources, data from the survey infrequently have been used in academic research. As discussed in Chapter 2, the survey is the sole national data source for a number of topics, including the application of disciplinary practices, classroom assignment data, and gender equity in sports. For many other issues, it is the only source for disaggregated data that can be linked to specific schools and districts or be projected to state and national levels.
The researchers who most frequently have made use of the data are graduate students working with education advo
cacy organizations and journalists (Peter McCabe, Office for Civil Rights, personal communication, 2002). These individuals typically conduct basic analyses—or, prior to web access, requested OCR staff to conduct basis analyses—documenting disparities that point to schools’ failure to adequately serve students of diverse backgrounds. This kind of basic research is important and provides citizens with information that is essential to inform and motivate their involvement in school improvement efforts. However, using E&S data to conduct complex analyses to document the causes of disparities and identify the kinds of resources and educational practices that would be effective for overcoming them has been rare.
Several changes must take place if the data are to be more widely used in academic research. First, OCR must make the data more easily available to researchers. Neither OCR nor any other unit of the Department of Education routinely analyzes the data or publishes even basic findings from the survey in departmental publications. Outside the advocacy community, very few people are aware of the E&S survey—including many highly skilled and experienced education researchers. Some of the researchers contacted by the committee who were aware of the survey in the past have had difficulty gaining access to usable datafiles. One researcher who works for an advocacy organization and who has used E&S survey data for more than 30 years told the committee, “These data are used to enforce civil rights laws. They are not intended for academic researchers” (Paul Smith, director of research for the Children’s Defense Fund, personal communication, 2002).
Yet most of the researchers who worked with E&S survey data under the auspices of this committee found them to be of great interest. They could be an important resource on access to opportunities to learn if OCR could implement procedures to improve the quality of the data and make them available in a format that is amenable to research (see Chapter 4), although there currently does not appear to be any protocol for this. The Beyond 20/20 software that OCR provides to facilitate public access to E&S data over the Internet is well suited for displaying descriptive statistics or for doing cross-tabulations that can identify various kinds of disparities. However, for researchers, the software cannot be used to conduct more elaborate studies that might uncover the underlying causes for observed disparities.2
CONCLUSIONS
Disparities in educational outcomes and in learning opportunity among different types of students continue to be an important social problem. In that context, the committee has three principal conclusions about the E&S survey:
-
The E&S survey, or an equivalent research instrument, continues to be needed to gather disaggregated data related to the equality of access to learning opportunities and resources that are traceable to individual classrooms, schools, and districts.
-
The survey, although useful for civil rights enforcement, informing educational policy, and the conduct of research, is greatly underused.
-
The survey can be made more useful by improvements to the content, the manner in which the survey is administered, and access to the valuable data it provides.
RECOMMENDATIONS
The committee offers recommendations in four categories: survey administration, improving data quality, increasing access to the data, and disseminating survey findings. We end with an overall conclusion about the role of the E&S survey.
Survey Administration
-
The mandated and certified collection of data related to possible violations of students’ educational civil rights should be sustained.
-
The survey should be supported by line-item funding in the department’s budget to ensure its ongoing support at a level that is consistent with its continued quality.
-
Because of the survey’s importance, the department should consider undertaking a thorough study of the survey aimed at ensuring that it deals appropriately and in sufficient depth with the problems of discovering possible restrictions on students’ learning opportunities and, if possible, reducing the reporting burden on schools and school systems.
-
The E&S survey content and protocols should be coordinated with those of other department surveys to ensure consistency of definitions and the complementarity of the data and to eliminate redundant questions.
-
The various stakeholders in the E&S survey—such as OCR enforcement staff, student advocates, state and local educators, and researchers—should discuss and explore the advantages and disadvantages of less frequent but more comprehensive surveys. With respect to a comprehensive survey, all schools should be surveyed, at a minimum, every 10 years, as was done in 2000.
Improving Data Quality
-
Survey items should be revised to provide more useful and complete information on five topics:
-
the qualifications and experience of teachers;
-
the assignment of students to different types of classrooms and educational settings;
-
the consequences for students of high-stakes testing;
-
high school completion; and
-
interscholastic athletics.
-
OCR should ensure that respondents understand how to complete the survey accurately and thoroughly.
-
OCR should carefully scrutinize the data that are collected for thoroughness and reliability.
Increasing Access to the Data
There are several steps that OCR should take to increase access to the E&S survey data:
-
train staff to make more effective use of the survey data;
-
continue to improve the software provided for public access to E&S survey data over the Internet;
-
sponsor or support programs to train advocates, researchers, and educators to use the data for various purposes;
-
make well-edited data available to researchers and others in a usable format, and provide a data manual and technical assistance;
-
consider developing a small grants program to encourage research on the topic of access to learning opportunities using E&S survey data; and
-
archive and preserve data from all surveys in a common format and make them accessible to researchers and other interested parties on disk or over the Internet, both for historical purposes and to enable researchers to track longitudinal trends.
Disseminating Survey Findings
Three steps should be taken by OCR to improve dissemination of E&S survey data:
-
conduct or sponsor the conduct of basic tabulations of the data;
-
include findings from analyses of the data in OCR’s regular reports to Congress; and
-
publicize the basic findings from the survey in widely disseminated government publications.
CONCLUSION
If the E&S survey were to be recreated today, with a core objective of providing information on equality of access to the kinds of learning opportunities and resources that would be useful in shaping education policy and practices and for informing efforts to protect students’ civil rights, the issues addressed by the current survey would be an essential part of the framework. However, much more information also would be needed.
Knowledge of how the learning environment, peers, learning resources, teacher preparation, and curriculum affect student learning is constantly evolving, as is knowledge of how to measure variables associated with each of these categories of learning opportunities and resources. To admit that there is still much that is unknown about how to measure learning opportunities and resources should be a spur for more work. The committee urges the Department of Education not only to continue collecting the kind of information that currently is on the E&S survey, but also to constantly reassess its quality and utility. Finally, we urge the department to recommit itself to using this information not only to protect students’ legally defined civil rights, but also to ensure that all students who are being held accountable for achieving high standards have equal access to the opportunities and resources needed to do so.