National Academies Press: OpenBook

Measuring Access to Learning Opportunities (2003)

Chapter: 5. Improving the Survey and Its Use

« Previous: 4. Strengthening the E&S Survey Data
Suggested Citation:"5. Improving the Survey and Its Use." National Research Council. 2003. Measuring Access to Learning Opportunities. Washington, DC: The National Academies Press. doi: 10.17226/10673.
×

5
Improving the Survey and Its Use

The work of the Office for Civil Rights (OCR) in addressing violations of students’ civil rights and combating discrimination continues to be important, and the E&S survey is a useful, albeit underused, resource in OCR’s efforts to enforce civil rights laws. The information produced by the E&S survey facilitates the identification of disparities in learning opportunities associated with race, ethnicity, language, gender, and disability status. Importantly, the survey is the only available data source that can be used to identify problems not only at the national and state levels, but also in school districts, and in individual schools. The committee finds that the E&S survey continues to play an important role in protecting the rights of minority students, students with disabilities, students with limited English proficiency, and women. The E&S survey is the only nationwide source of much of the information it collects. The survey’s capacity to identify interdistrict, intradistrict, and even intraschool disparities in learning opportunities makes it a unique resource that provides some, though not all, of the information needed for spotting potentially actionable discrimination and violations of civil rights.

The survey currently is not being extensively used by OCR in enforcement, except as one of a number of sources of information used to identify school districts for compliance reviews. The number of compliance reviews initiated by OCR varies considerably from year to year; in 2002, no new compliance reviews were initiated. The survey’s most important use in recent years has been to provide parents and others with information on disparities in access to high-quality learning opportunities.

The committee also finds that the

Suggested Citation:"5. Improving the Survey and Its Use." National Research Council. 2003. Measuring Access to Learning Opportunities. Washington, DC: The National Academies Press. doi: 10.17226/10673.
×

survey and the manner in which it is administered can be improved to: (1) more effectively assess whether students have access to critical learning opportunities; (2) make the data more accessible to interested citizens, as well as to educators and policy makers; and (3) enhance its usefulness as a resource for research that could lead to school improvement.

Despite the many societal and educational changes that have occurred during the past 35 years, major disparities in opportunities to learn and in education outcomes persist, especially those associated with race and ethnicity—the original focus of the E&S survey. The timely collection and analysis of classroom-, schooland district-level data that can help identify educational policies and practices that may have inequitable, if not discriminatory, effects on students is no less important today than it was when the E&S survey was first administered.

LEARNING OPPORTUNITIES— THE KEY

In one way or another, virtually all civil rights related to education involve the assurance that students will have the necessary opportunities to learn. The definition of “necessary” is contested, but the idea of equal opportunities to learn for persons of different backgrounds is the basis of contemporary understandings of rights. Legislation and court rulings addressing the needs of persons who do not speak English or who have disabilities have extended the protection of the law and gone beyond the goal of equality to guarantee the provision of needed services. School finance cases also have sought to broaden the definition of individual rights related to education beyond equality of spending per student to take into account the fact that some students, including but not limited to students who do not speak English and students with disabilities, need more services than others if they are to succeed in school (see Rebell, 2002).

Regardless of the race, ethnicity, socioeconomic status, or disability status of the students whose educational rights are at issue, the focus of concern is learning opportunities. If students with certain characteristics have differential access to learning opportunities in particular situations, it may signal discrimination. However, despite the centrality of learning opportunities to the protection of civil rights, there is no accepted or even widely discussed model that identifies the full range of opportunities to learn, their relative importance to student learning, and their interrelationships. Research that links data on patterns of access to

Suggested Citation:"5. Improving the Survey and Its Use." National Research Council. 2003. Measuring Access to Learning Opportunities. Washington, DC: The National Academies Press. doi: 10.17226/10673.
×

learning opportunities and resources, such as those from the E&S survey, with other datasets that are used to examine the effects of various policies, practices, and resources, potentially could produce a clearer understanding of what kinds of disparities make a difference for students’ learning outcomes (see Appendixes A and D). Not only would the development of such a model be important for the protection of civil rights, but it would also shape education strategies generally.

Children’s learning is influenced by many experiences. Those that are available in school typically account for less than one-third of students’ academic achievement, as measured by most common tests (e.g., see HalpernFelsher et al., 1997; National Research Council, 1999a). Yet the focus of the E&S survey, and thus the focus of this committee’s interests, are the learning opportunities and resources provided in schools or by schools. Trying to identify the full range of influences on student learning that might reasonably be the concern of those who would protect civil rights is daunting. Nonetheless, the E&S survey focuses on learning opportunities that are of critical importance. Continuing refinement of the survey and judicious additions to its scope would make it an even more substantial resource for enhancing the education of students who have been the victims of discrimination or of inadequate access to learning opportunities and resources.

PROMISING NEW USES OF E&S SURVEY DATA

Integrating E&S Survey Data with Other Information Resources

As discussed in Chapter 4, planning is under way to integrate the E&S survey into a consolidated data system that would coordinate the collection and management of data related to the administration of various programs administered by the U.S. Department of Education. The effort to develop this system, known as the PerformanceBased Data Management Initiative (PBDMI), is being conducted in partnership with state departments of education. Besides integrating the administrative data systems of the department, other goals of PBDMI include facilitating the integration of state and local education data with the department’s administrative database to make the data more usable and accessible for all.

Integrating the E&S survey with the PBDMI would help to ensure that the definitions of items that appear on the

Suggested Citation:"5. Improving the Survey and Its Use." National Research Council. 2003. Measuring Access to Learning Opportunities. Washington, DC: The National Academies Press. doi: 10.17226/10673.
×

E&S survey are consistent with those on other department surveys. Departmental literature on PBDMI also promises that it will reduce the data collection burden on schools by eliminating redundancy among the various surveys currently administered. It also promises that PBDMI will facilitate efforts to ensure accountability.

When the PBDMI data system is implemented, it will be important to ensure that the attributes that make the E&S survey an essential resource for monitoring equality of access to learning opportunities and resources is retained. Written descriptions of PBDMI from the Department of Education do not currently mention the collection and integration of data on access to learning opportunities and resources as a feature of the new data system. It is essential that information on access to opportunities to learn be an integral part of the PBDMI system. The new data system must continue to provide information pertinent to students’ civil rights under Title VI, Title IX, and Section 504. These data must continue to be disaggregated by race and ethnicity, traceable to specific schools and school districts, and accessible not only to departmental officials, but also to local educators and to the public. If some or all of the E&S survey were incorporated into the PBDMI, it could become less burdensome and therefore more cost-effective to routinely collect E&S data from all schools.

The Survey as a Resource for School Improvement Efforts

Large disparities in education outcomes among students of different backgrounds persist, and OCR continues to find that violations of students’ civil rights are not uncommon. Whether or not the disparities identified by the E&S survey are caused by violations of students’ legal rights, the disparities evidence the failure of schools to provide equal access to highquality education for students from all backgrounds. This information is relevant not just to civil rights enforcement, but also to broader based efforts to achieve equity and excellence. Providing educators, parents, and the general public with easy access information that suggests that their own schools may be failing to ensure equal access to learning opportunities can lead to greater public involvement in school improvement efforts.

As society and public school students become more diverse, issues of equity will become important in places that previously have not had diverse student populations. The successful implementation of school reforms in a postdesegregation era will require that

Suggested Citation:"5. Improving the Survey and Its Use." National Research Council. 2003. Measuring Access to Learning Opportunities. Washington, DC: The National Academies Press. doi: 10.17226/10673.
×

policies and practices be closely monitored so that they do not disadvantage minority students. The E&S survey provides a resource that can help to ensure that no child will be left behind. Without such a resource, education policy makers and practitioners may be unable to determine if reforms differently affect educational outcomes for racial and ethnic minority, language minority, or disabled students or even whether outcomes may vary by gender.

Until the summer of 2002, OCR had not published or otherwise made available to the public E&S survey data for more than 20 years—except by request. The recent placement of E&S data on the Internet is a very important innovation whose implications for public use of the data are not yet known. The software used to examine the data for exploratory purposes requires no technical training to use, and it allows individuals to make very specific queries about the various items addressed by the survey. For example, one could examine the number and percentage of sixth graders at a specific school who passed or failed a districtor stateadministered test required for grade promotion, with the information disaggregated by race, ethnicity, and sex. Data for different schools can be compared, as can data for different states. Although some of the data from the survey are suppressed to protect students’ privacy, the data available over the Internet provide extensive information concerning disparities in access to learning opportunities.1 Making these data easily available to the public is consistent with current federal and state policies emphasizing the public reporting of school-level data as a means of promoting accountability.

Educational Equity Self-Assessments

One way to do this could be in the form of an Educational Equity SelfAssessment (EESA). An EESA is similar in concept to the Racial Justice Report Card developed by the Applied Research Center of Oakland, California (Gordon, Della Piana, and Keleher, 2002). Using E&S survey data, possibly in combination with data from other sources, OCR could develop an EESA that educators, parent-teacher organizations, and others could use to evaluate individual schools. The EESA would be a computer-based, interactive program with access from the Internet or delivered on CD-ROM or DVD. School administrators and others in a school community could examine their school’s performance on equity measures of

1  

Researchers can obtain access to the suppressed data on the Internet by signing an agreement that protects the confidentiality of data pertaining to individually identifiable students.

Suggested Citation:"5. Improving the Survey and Its Use." National Research Council. 2003. Measuring Access to Learning Opportunities. Washington, DC: The National Academies Press. doi: 10.17226/10673.
×

issues, such as school discipline; graduation rates; student retention; testing outcomes; access to advanced placement and gifted and talented programs; tendencies toward within-school segregation, particularly as related to tracking; and racial disparities in special education. The data for a given school could be compared with district, state, or national averages or to equity standards suggested by OCR. Significant deviations from the suggested “equity standards” could produce an interactive diagnostic exercise that would include descriptions of one or more researchbased strategies or programs that could help the school to reduce the observed disparities and to achieve the suggested standards.

Public Reporting and Accountability

Another possible new use of E&S survey data is to incorporate it (and other information pertaining to access to learning resources) with the kinds of disaggregated achievement data that must be publicly reported under the No Child Left Behind Act. Under the act, schools are required to report achievement outcomes disaggregated by students’ race and poverty status and for language minority students. Schools are held accountable for ensuring that students from each group make adequate progress toward achieving statedefined standards for learning.

Data related to access to learning opportunities and resources, such as those available from the E&S survey, should be an integral part of school data systems and should provide information that is helpful to diagnosing causes of disparities, whether or not they are related to discrimination. The inclusion by schools in routine public reporting of information on access to learning opportunities and resources, along with data on students’ progress toward the achievement of standards, could be helpful in providing guidance about causes of disparities in outcomes as well as promising strategies to address them.

New Research Opportunities

Although the E&S survey is the most important source of information available on a wide range of topics related to access to learning opportunities and resources, data from the survey infrequently have been used in academic research. As discussed in Chapter 2, the survey is the sole national data source for a number of topics, including the application of disciplinary practices, classroom assignment data, and gender equity in sports. For many other issues, it is the only source for disaggregated data that can be linked to specific schools and districts or be projected to state and national levels.

The researchers who most frequently have made use of the data are graduate students working with education advo

Suggested Citation:"5. Improving the Survey and Its Use." National Research Council. 2003. Measuring Access to Learning Opportunities. Washington, DC: The National Academies Press. doi: 10.17226/10673.
×

cacy organizations and journalists (Peter McCabe, Office for Civil Rights, personal communication, 2002). These individuals typically conduct basic analyses—or, prior to web access, requested OCR staff to conduct basis analyses—documenting disparities that point to schools’ failure to adequately serve students of diverse backgrounds. This kind of basic research is important and provides citizens with information that is essential to inform and motivate their involvement in school improvement efforts. However, using E&S data to conduct complex analyses to document the causes of disparities and identify the kinds of resources and educational practices that would be effective for overcoming them has been rare.

Several changes must take place if the data are to be more widely used in academic research. First, OCR must make the data more easily available to researchers. Neither OCR nor any other unit of the Department of Education routinely analyzes the data or publishes even basic findings from the survey in departmental publications. Outside the advocacy community, very few people are aware of the E&S survey—including many highly skilled and experienced education researchers. Some of the researchers contacted by the committee who were aware of the survey in the past have had difficulty gaining access to usable datafiles. One researcher who works for an advocacy organization and who has used E&S survey data for more than 30 years told the committee, “These data are used to enforce civil rights laws. They are not intended for academic researchers” (Paul Smith, director of research for the Children’s Defense Fund, personal communication, 2002).

Yet most of the researchers who worked with E&S survey data under the auspices of this committee found them to be of great interest. They could be an important resource on access to opportunities to learn if OCR could implement procedures to improve the quality of the data and make them available in a format that is amenable to research (see Chapter 4), although there currently does not appear to be any protocol for this. The Beyond 20/20 software that OCR provides to facilitate public access to E&S data over the Internet is well suited for displaying descriptive statistics or for doing cross-tabulations that can identify various kinds of disparities. However, for researchers, the software cannot be used to conduct more elaborate studies that might uncover the underlying causes for observed disparities.2

2  

After several months of discussions with the committee and internal work, OCR was able to provide the committee with files containing data from the 2000 survey in a format that lent itself to complex studies.

Suggested Citation:"5. Improving the Survey and Its Use." National Research Council. 2003. Measuring Access to Learning Opportunities. Washington, DC: The National Academies Press. doi: 10.17226/10673.
×

CONCLUSIONS

Disparities in educational outcomes and in learning opportunity among different types of students continue to be an important social problem. In that context, the committee has three principal conclusions about the E&S survey:

  • The E&S survey, or an equivalent research instrument, continues to be needed to gather disaggregated data related to the equality of access to learning opportunities and resources that are traceable to individual classrooms, schools, and districts.

  • The survey, although useful for civil rights enforcement, informing educational policy, and the conduct of research, is greatly underused.

  • The survey can be made more useful by improvements to the content, the manner in which the survey is administered, and access to the valuable data it provides.

RECOMMENDATIONS

The committee offers recommendations in four categories: survey administration, improving data quality, increasing access to the data, and disseminating survey findings. We end with an overall conclusion about the role of the E&S survey.

Survey Administration

  • The mandated and certified collection of data related to possible violations of students’ educational civil rights should be sustained.

  • The survey should be supported by line-item funding in the department’s budget to ensure its ongoing support at a level that is consistent with its continued quality.

  • Because of the survey’s importance, the department should consider undertaking a thorough study of the survey aimed at ensuring that it deals appropriately and in sufficient depth with the problems of discovering possible restrictions on students’ learning opportunities and, if possible, reducing the reporting burden on schools and school systems.

  • The E&S survey content and protocols should be coordinated with those of other department surveys to ensure consistency of definitions and the complementarity of the data and to eliminate redundant questions.

Suggested Citation:"5. Improving the Survey and Its Use." National Research Council. 2003. Measuring Access to Learning Opportunities. Washington, DC: The National Academies Press. doi: 10.17226/10673.
×
  • The various stakeholders in the E&S survey—such as OCR enforcement staff, student advocates, state and local educators, and researchers—should discuss and explore the advantages and disadvantages of less frequent but more comprehensive surveys. With respect to a comprehensive survey, all schools should be surveyed, at a minimum, every 10 years, as was done in 2000.

Improving Data Quality

  • Survey items should be revised to provide more useful and complete information on five topics:

  1. the qualifications and experience of teachers;

  2. the assignment of students to different types of classrooms and educational settings;

  3. the consequences for students of high-stakes testing;

  4. high school completion; and

  5. interscholastic athletics.

  • OCR should ensure that respondents understand how to complete the survey accurately and thoroughly.

  • OCR should carefully scrutinize the data that are collected for thoroughness and reliability.

Increasing Access to the Data

There are several steps that OCR should take to increase access to the E&S survey data:

  • train staff to make more effective use of the survey data;

  • continue to improve the software provided for public access to E&S survey data over the Internet;

  • sponsor or support programs to train advocates, researchers, and educators to use the data for various purposes;

  • make well-edited data available to researchers and others in a usable format, and provide a data manual and technical assistance;

  • consider developing a small grants program to encourage research on the topic of access to learning opportunities using E&S survey data; and

  • archive and preserve data from all surveys in a common format and make them accessible to researchers and other interested parties on disk or over the Internet, both for historical purposes and to enable researchers to track longitudinal trends.

Disseminating Survey Findings

Three steps should be taken by OCR to improve dissemination of E&S survey data:

Suggested Citation:"5. Improving the Survey and Its Use." National Research Council. 2003. Measuring Access to Learning Opportunities. Washington, DC: The National Academies Press. doi: 10.17226/10673.
×
  • conduct or sponsor the conduct of basic tabulations of the data;

  • include findings from analyses of the data in OCR’s regular reports to Congress; and

  • publicize the basic findings from the survey in widely disseminated government publications.

CONCLUSION

If the E&S survey were to be recreated today, with a core objective of providing information on equality of access to the kinds of learning opportunities and resources that would be useful in shaping education policy and practices and for informing efforts to protect students’ civil rights, the issues addressed by the current survey would be an essential part of the framework. However, much more information also would be needed.

Knowledge of how the learning environment, peers, learning resources, teacher preparation, and curriculum affect student learning is constantly evolving, as is knowledge of how to measure variables associated with each of these categories of learning opportunities and resources. To admit that there is still much that is unknown about how to measure learning opportunities and resources should be a spur for more work. The committee urges the Department of Education not only to continue collecting the kind of information that currently is on the E&S survey, but also to constantly reassess its quality and utility. Finally, we urge the department to recommit itself to using this information not only to protect students’ legally defined civil rights, but also to ensure that all students who are being held accountable for achieving high standards have equal access to the opportunities and resources needed to do so.

Suggested Citation:"5. Improving the Survey and Its Use." National Research Council. 2003. Measuring Access to Learning Opportunities. Washington, DC: The National Academies Press. doi: 10.17226/10673.
×
Page 60
Suggested Citation:"5. Improving the Survey and Its Use." National Research Council. 2003. Measuring Access to Learning Opportunities. Washington, DC: The National Academies Press. doi: 10.17226/10673.
×
Page 61
Suggested Citation:"5. Improving the Survey and Its Use." National Research Council. 2003. Measuring Access to Learning Opportunities. Washington, DC: The National Academies Press. doi: 10.17226/10673.
×
Page 62
Suggested Citation:"5. Improving the Survey and Its Use." National Research Council. 2003. Measuring Access to Learning Opportunities. Washington, DC: The National Academies Press. doi: 10.17226/10673.
×
Page 63
Suggested Citation:"5. Improving the Survey and Its Use." National Research Council. 2003. Measuring Access to Learning Opportunities. Washington, DC: The National Academies Press. doi: 10.17226/10673.
×
Page 64
Suggested Citation:"5. Improving the Survey and Its Use." National Research Council. 2003. Measuring Access to Learning Opportunities. Washington, DC: The National Academies Press. doi: 10.17226/10673.
×
Page 65
Suggested Citation:"5. Improving the Survey and Its Use." National Research Council. 2003. Measuring Access to Learning Opportunities. Washington, DC: The National Academies Press. doi: 10.17226/10673.
×
Page 66
Suggested Citation:"5. Improving the Survey and Its Use." National Research Council. 2003. Measuring Access to Learning Opportunities. Washington, DC: The National Academies Press. doi: 10.17226/10673.
×
Page 67
Suggested Citation:"5. Improving the Survey and Its Use." National Research Council. 2003. Measuring Access to Learning Opportunities. Washington, DC: The National Academies Press. doi: 10.17226/10673.
×
Page 68
Suggested Citation:"5. Improving the Survey and Its Use." National Research Council. 2003. Measuring Access to Learning Opportunities. Washington, DC: The National Academies Press. doi: 10.17226/10673.
×
Page 69
Next: References and Bibliography »
Measuring Access to Learning Opportunities Get This Book
×
Buy Paperback | $40.00 Buy Ebook | $31.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Since 1968 the Elementary and Secondary School Civil Rights Compliance Report (known as the E&S survey) has been used to gather information about possible disparities in access to learning opportunities and violations of students’ civil rights. Thirty-five years after the initiation of the E&S survey, large disparities remain both in educational outcomes and in access to learning opportunities and resources. These disparities may reflect violations of students’ civil rights, the failure of education policies and practices to provide students from all backgrounds with a similar educational experience, or both. They may also reflect the failure of schools to fully compensate for disparities and current differences in parents’ education, income, and family structure.

The Committee on Improving Measures of Access to Equal Educational Opportunities concludes that the E&S survey continues to play an essential role in documenting these disparities and in providing information that is useful both in guiding efforts to protect students’ civil rights and for informing educational policy and practice. The committee also concludes that the survey’s usefulness and access to the survey data could be improved.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!