Appendix A
Synopses of Papers Prepared for the Committee

This appendix contains synopses of five papers commissioned by the committee and presented at its second meeting, August 12, 2002, at Woods Hole, Massachusetts. The committee’s purpose in commissioning these papers was to obtain information about the value of E&S survey data for research on current educational policies and practices. The committee considered the findings of these papers, as well as the researchers’ accounts of their experiences in working with the E&S data, in its deliberations.

Given the short amount of time available to the researchers, the papers should be considered preliminary analyses; they address the following topics:

  • the effects of school disciplinary policies,

  • gender equity in sports,

  • the impact of high-stakes testing,

  • the degree of segregation of English-language learners, and

  • the availability of services for English-language learners.

The committee encouraged the researchers to continue the analyses begun under the auspices of the committee and to independently publish papers based on their analyses, as appropriate.

ACHIEVEMENT DISRUPTED

Philip Babcock

University of California, San Diego

This paper investigates correlations between test scores and school expulsion rates to determine to what degree these correlations might be driven by policy. The investigation is framed as part of a larger inquiry into the underlying causes of peer effects.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 77
Appendix A Synopses of Papers Prepared for the Committee This appendix contains synopses of five papers commissioned by the committee and presented at its second meeting, August 12, 2002, at Woods Hole, Massachusetts. The committee’s purpose in commissioning these papers was to obtain information about the value of E&S survey data for research on current educational policies and practices. The committee considered the findings of these papers, as well as the researchers’ accounts of their experiences in working with the E&S data, in its deliberations. Given the short amount of time available to the researchers, the papers should be considered preliminary analyses; they address the following topics: the effects of school disciplinary policies, gender equity in sports, the impact of high-stakes testing, the degree of segregation of English-language learners, and the availability of services for English-language learners. The committee encouraged the researchers to continue the analyses begun under the auspices of the committee and to independently publish papers based on their analyses, as appropriate. ACHIEVEMENT DISRUPTED Philip Babcock University of California, San Diego This paper investigates correlations between test scores and school expulsion rates to determine to what degree these correlations might be driven by policy. The investigation is framed as part of a larger inquiry into the underlying causes of peer effects.

OCR for page 77
The analysis begins with a basic description of the data, e.g., histograms of expulsions and suspensions, sample covariances between test scores and demographic measures. The main body of the paper explores implications of a specific theory of peer effects. While studies suggest that peer effects have an impact on measures of student performance, the specific mechanism by which they operate remains unclear. One explanation is that disruptive behavior by a student impedes the learning of every other student in a classroom. If so, then systematic efforts made by teachers and educators to reduce disruptive behavior ought to lead to higher test scores, other things being equal. The paper examines an implication of this basic theory. Taking suspension rates and expulsion rates as proxies for discipline, one could attempt to determine the effect of discipline on test scores by means of a simple ordinary least squares regression. The immediate difficulty with such an approach is that there is apt to be correlation between the error term and the regressors. One would expect higher rates of expulsion at schools whose students come from troubled or dysfunctional social environments and have behavior problems because of unobserved factors. These students might perform poorly on tests because of the same unobserved traits that affect the rate of expulsion. The paper attempts to distinguish between the two sources of correlation, endogenous and exogenous, by constructing a panel of test scores, suspension rates, and expulsion rates for ninth graders in California, school by school, using data from 1998 and 2000. The analysis includes both fixed-effect and between-effects estimates of the coefficients on expulsion and suspension, the assumption being that neighborhood effects for a given school changed little in two years. The analysis indicates that when one does not control for school-specific unobserved effects, higher rates of expulsion are associated with lower math scores. The paper argues that this is evidence of endogeneity: schools whose students have been unobservably disadvantaged by their local environment exhibit more behavior problems and also lower math scores. In the fixed-effects regression, however, the correlation is positive. Holding constant the school, an increase in expulsion rates between 1998 and 2000 was associated with an increase in math scores. The strictness of discipline policy, then, might have a positive effect on test scores. The result is merely suggestive, however: the analysis was not robust to heteroskedastic specifica

OCR for page 77
tions or to reasonable alterations of the dataset. The paper concludes with a discussion of the database of the Office for Civil Rights (OCR), including some suggestions for additions that would be helpful in this area of research. UNEVEN PLAYING FIELDS: STATE VARIATIONS IN BOY’S AND GIRL’S ACCESS TO AND PARTICIPATION IN HIGH SCHOOL INTERSCHOLASTIC SPORTS Jomills Henry Braddock II, Jan SokolKatz, and Anthony Greene University of Miami This year marks the 30th anniversary of Title IX. Yet despite considerable progress and the need for further improvement, Title IX is facing increased opposition (including numerous legal challenges) and scrutiny (Secretary of Education’s Commission on Opportunities in Athletics). Using aggregate data from OCR, combined with demographic and contextual data available from the National Center for Educational Statistics, this paper (1) analyzes state-level disparities in boy’s and girl’s access to both singleand mixed-gender interscholastic athletic programs and their patterns of participation in singleand mixed-gender school sports and (2) examines demographic and contextual correlates of variation among states in relative access and participation of boys and girls in interscholastic athletics. While unity, or even virtual parity, has been achieved in only a handful of states, we find wide variation among the states in how equitably girls and boys have access to both singleand mixedgender interscholastic sports and teams. We also find that states vary widely in patterns of participation in singleand mixed-gender sports activities. Regression analyses suggest that variation in gender disparities in single-gender athletic participation opportunities (number of sports and teams offered to male and female students) among states can be predicted by a combination of contextual characteristics (median household income) and school demographics (percentage of white enrollments). And variation in gender disparities in single-gender athletic participation rates (ratio of male to female students participating in single-gender sports) among states can be strongly predicted by participation opportunities (number of sports and teams offered to male and female students) and educational investment (per pupil expenditures). This analysis also suggests that the impact on gender differences in participation rates of school demographics (percentage of white enrollments) is largely indirect and mediated by partici

OCR for page 77
pation opportunities (number of sports and teams offered to male and female students). With regard to gender disparities in mixed-gender athletic participation opportunities (number of sports and teams offered to both male and female students) among states, our model is not adequate to explain state variations. However, gender disparities in mixed-gender athletic participation rates among states can be modestly predicted by a combination of contextual characteristics (median household income) and educational investment (per pupil expenditures). Our analysis suggests that monitoring gender equity in athletic access and participation is crucial. Because the OCR compliance reports are the most reliable sources of data on this equity issue, it will be important to both continue and strengthen data collection efforts. Specifically, additional information is required to better understand the relationship between enrollment racial composition and access to and participation in single-gender athletics. Recent questions that have been raised about whether black females are benefiting from Title IX could be informed with better information about characteristics of student participants. Better information regarding specific sports offered is also important. This would help clarify general understanding of such issues as the types of sports available to girls and boys as both singleand mixed-gender activities. Next steps should include multilevel analyses taking into account variations at the school, district, and state levels. PREVALENCE OF HIGH-STAKES TESTING IN U.S. PUBLIC ELEMENTARY AND SECONDARY SCHOOLS: CONSEQUENCES FOR MINORITY CHILDREN Robert G. Croninger and Karen Douglas University of Maryland Recent policy initiatives have placed increasing importance on the implementation of high-stakes tests for reforming public elementary and secondary schools. Provisions in the 2001 Elementary and Secondary Education Act call for increased testing to determine if students and teachers are meeting high academic standards, as do new statelevel policies that seek to align state testing practices with rigorous content and performance standards for students and teachers. Forty-six states have developed or are in the process developing testing policies aligned to gradelevel content and performance standards (see National Research Council, 1999). Of these states, 18 require that students pass some form of an exit examination before receiving a high school diploma (see Council of Chief

OCR for page 77
State School Officers 2000), while 3 states require schools to use state standards and assessments in making promotional decisions for elementary school or middle school students. Four additional states plan to make promotion contingent on high-stakes tests by 2003 (see Editorial Projects in Education, 2001). Although high-stakes testing provides new opportunities for holding individuals and schools accountable for the quality of educational opportunities that they provide students (see Weckstein, 1999), it also raises important equity issues about the actual consequences of testing for specific populations of students. If minority students and the schools that they attend disproportionately bear the burden of high-stakes testing both in terms of the requirements to pass high-stakes tests and the sanctions imposed for failure to do so, then the new wave of testing polices and practices may not promote more equitable educational opportunities. On the contrary, it may promote unfair (National Research Council, 1999) and perhaps illegal testing practices (Debra P. v. Turlington, 474 F. Supp. 244, M.D. Fla. 1979) that further deny minority students access to valuable educational opportunities (Howe, 1997). As policy makers at all levels of government call for new and more demanding testing practices, it is increasingly important that we examine the consequences of such practices against not only standards of academic rigor but also standards of educational equity. We know surprisingly little about either the prevalence or consequences of high-stakes testing given the attention it has received in recent education policies. While a number of surveys have been conducted to characterize high-stakes testing policies at the state level (e.g., see American Federation of Teachers, 2001; Council of Chief State School Officers, 2000; Editorial Projects in Education, 2001), only the 2000 OCR E&S survey provides data about the prevalence and consequences of highstakes testing as practiced by individual districts and schools. Although the states have taken the lead in implementing high-stakes testing, there is ample evidence that individual districts (e.g., Chicago Public Schools) have implemented high-stakes testing policies independent of state legislators and education officials (National Research Council, 1999). Even in states that require the use of high-stakes tests for promotional decisions or the certification of high school graduation, there may be considerable variability in the implementation of policies or in the consequences of policies for students. Moreover, because the 2000 E&S survey asked schools to report the pass and failure rates of students by race,

OCR for page 77
gender, and disability status, it may be possible to use these data to investigate not only the prevalence of high-stakes testing practices but also their immediate consequences for specific student populations. These types of data are unavailable elsewhere. We propose to explore the feasibility of using the 2000 E&S survey data to analyze the prevalence and consequences of high-stakes testing practices in U.S. elementary and secondary schools. Specifically, we propose to (1) evaluate the research utility of the current format for E&S survey data by attempting to download data about testing (Tables 12a and 12b) using the Beyond 20/20 interface; (2) link these data to the 2000 Common Core of Data (CCD); and (3) examine the validity of E&S survey data by comparing these data with other data sources (e.g., OCR and CCD enrollment data; OCR and Council of Chief State School Officers reports of testing data by states). The analytic method used will depend on the variability between schools, districts, and states in testing practices and their consequences. The research questions that we propose to address in the study include: What is the prevalence of highstakes testing as a requirement for grade promotion? How do these practices vary by state, districts, and schools? Are there differences by race in who is and is not held to test-based promotional requirements? What are the immediate consequences of using high-stakes testing to make promotion decisions for students? Are there differences by race in who passes or fails these tests? What is the prevalence of highstakes testing as a requirement for high school graduation? How do these practices vary by state, districts, and schools? Are there differences by race in who is and is not required to pass an exit examination before receiving a high school diploma? What are the immediate consequences of using high-stakes testing as a graduation requirement for students? Are there differences by race in who passes or fails these tests? References American Federation of Teachers. (2001). Making standards matters 2001. Washington, DC: Author. Council of Chief State School Officers. (2000). Executive summary of state student assessment programs. Washington, DC: Author. Editorial Projects in Education. (2001, January 11 ). Standards related policies. Education Week, 68–87. Howe, K.R. (1997). Understanding equal educa-tional opportunity. Social justice, democracy, and schooling. New York: Teachers College Press.

OCR for page 77
National Research Council. (1999). High stakes: Testing for tracking, promotion, and graduation. Committee on Appropriate Test Use, J.P. Heubert and R.M. Hauser (Eds.). Board on Testing and Assessment, Center for Education. Washington, DC: National Academy Press. Weckstein, P. (1999). School reform and enforceable rights to quality education. In J.A. Heubert (Ed.), Law and school reform. Six strategies for promoting educational equity (pp. 306–389). New Haven: Yale. THE INTERSECTION OF ENGLISH LANGUAGE LEARNING, RACE, AND POVERTY Catherine Horn Harvard University The school-age population in the United States is becoming increasingly diverse. As examples, in 1999, 20 percent of the school-age children had at least one foreign-born parent, including 5 percent of elementary and secondary students who were themselves foreign born (Jamieson, Curry, and Martinez, 2001). The number of schoolage students who are Hispanic rose from 13 percent in 1993 to 17 percent nationally in 2000 (U.S. Census Bureau, 2001). Also in 2000, almost 1 in 10 public school students was an Englishlanguage learner, and Spanish continues to be the predominant language background of the students receiving English-language learner services (Kindler, 2002). Two of the reported variables in the E&S survey broken out by race/ ethnicity are the numbers of students needing and enrolled in programs for English-language learners. Coupled with data from the National Center for Educational Statistics’ CCD, this information begins to shed light on the ways in which English proficiency, race, and income intersect. To that end, this paper looks closely at the following broad research question: In what ways do concentrations of English-language learners interact with poverty and race? Studying the ways in which these three important characteristics intertwine is paramount to better understanding the influence of each on its own. For example, on an aggregate level, we know that while 18 percent of white, non-Hispanic students and 7 percent of black students have foreignborn parents, 88 and 65 percent of Asians and Hispanics, respectively, have at least one parent who was born outside the United States; 25 and 18 percent respectively were themselves foreign born (Jamieson, Curry, and Martinez, 2001). Of course, while these numbers do not exactly reflect the pool of students needing English-language services, they are certainly an indication that certain racial and ethnic groups may have a disproportionate need for them. Through the use of data included in the E&S survey and CCD, we can

OCR for page 77
explore the racial and ethnic and socioeconomic conditions of the schools in which large proportions of Englishlanguage learners are present, in comparison with schools with smaller proportions. The paper first explores the interrelationship of English proficiency, race, and income by presenting descriptive tables, including, but not limited too, the following: by deciles, concentrations of English-language learners by concentrations of poverty; by deciles, concentrations of English-language learners by concentrations of nonwhite school demographics; concentrations of English-language learners by location (e.g., rural, suburban, urban); and districts with the highest percentage of their total student population needing or receiving Englishlanguage learner services. The work then turns its focus toward those schools in which concentrations of English-language learners are the highest. Using a modified exposure index, the paper presents a series of findings displaying schools’ racial distributions for the average student in a school with a high concentration of those students. So, for example, these data show the percentage of white students in schools with high concentrations of English-language learners attended by the typical black or Hispanic student. These findings are compared with similar exposure indices, by race, for schools in general. To the extent possible, this paper also explores the racial distributions for the average student in a school with high concentrations of poverty. The paper concludes with a discussion of the policy implications of the findings. From a civil rights perspective, such information should be useful, for example, in considering the confounding impacts of legal decisions to end court-mandated desegregation. The paper also includes a brief discussion about the viability and limitations of the E&S survey data for continued research around the issues of Englishlanguage learners, race, and poverty and suggestions for better data collection to understand these issues. References Jamieson, A., Curry, A., and Martinez, G. (2001). School enrollment in the United States—Social and economic characteristics of students, October, 1999 . Current Population Reports. Washington, DC: U.S. Census Bureau. Available: http://www.census.gov/population/www/socdemo/school.html. Kindler, A. (May, 2002). Survey of the states’ limited English proficient students and available educational programs and services 1999–2000 summary report. Washington, DC: George Washington University, The National Clearing-house for English-Language Acquisition and

OCR for page 77
Language Instruction Educational Programs. Available: http://www.ncela.gwu.edu/ncbepubs/seareports/99-00/sea9900.pdf. U.S. Census Bureau. (2001). Table A-1: School enrollment of the population 3 to 34 years old, by level and control of school, race, and Hispanic origin: October 1955 to 2000. Washington, DC: Author. Available: http://www.census.gov/population/www/socdemo/school.html. LINKING THE OFFICE FOR CIVIL RIGHTS’ ELEMENTARY AND SECONDARY SCHOOL SURVEY WITH NATIONALLY REPRESENTATIVE EDUCATIONAL DATASETS: POSSIBILITIES AND PROBLEMS Douglas D. Ready and Valerie E. Lee University of Michigan We began investigating the E&S survey data with two objectives in mind. First, we sought to determine the feasibility of linking the E&S to data from the Early Childhood Longitudinal Study (ECLS-K), also collected by the U.S. Department of Education.1 Second, if we could indeed link the datasets, we intended to perform analyses possible only through the combined E&S survey and ECLS-K datasets. Our hope was that E&S survey data would provide school-level measures that were useful to such analyses and unavailable on ECLS-K. Conversely, because the E&S survey includes no student-level outcomes, we hoped to augment those data with student-level social and academic measures available on ECLS-K After considerable effort, we were successful in attaining our first goal; it is indeed possible to create a combined E&S survey and ECLS-K datafile. The standard NCES 12-digit school identification codes, which are included on both E&S survey and ECLS-K data (restricted file only), make this linkage possible.2 Using these codes to match E&S survey to ECLS-K schools, we ultimately linked 687 public schools.3 Our next task was to identify important questions that could be answered 1  Both of the authors of this paper are engaged in a multiyear study using ECLS-K data that is funded by the U.S. Department of Education. 2  We accomplished this by saving the E&S survey data as a comma-separated file, opening the file in SPSS, and then merging the file with the ECLS-K data using the NCES school ID common to both files. 3  We used the 2000 E&S survey data. ECLS-K kindergarten data were collected during the 1998–1999 school year with first grade data collected the following school year. The school sample includes only public schools (as E&S contains only public schools) that have both kindergarten and first grade and that enroll at least five ECLS-K students. Twenty ECLS-K schools that matched these criteria were not located in the E&S survey dataset. The student sample includes children with fall-K, spring-K, and spring-first test scores; full data on race, socioeconomic status, and gender; and were in the same school for kindergarten and first grade. The resulting average within-school ECLS-K sample size is 15.03.

OCR for page 77
only through analyses using this newly created dataset. Such analyses are by nature multilevel, requiring the use of hierarchical linear modeling. Our goal was to investigate how school-level characteristics (provided by the E&S survey) influence student outcomes (provided by ECLS-K), and whether those characteristics influence the relationship between student characteristics and student outcomes. For example, previous research suggests that the proportion of schools’ minority enrollment is related to student achievement, even after accounting for the average social class and prior achievement of the students they enroll. Although research on racial segregation is personally and professionally important to us, we realized that ECLS-K already contains information about each school’s racial composition. The E&S survey also contains important information about teacher certification, but again, this information is included in ECLS-K. Because ECLS-K studies children, classrooms, and schools in the early elementary grades (at present, kindergarten and first grade), the E&S survey high school measures were not salient. The E&S survey also contains several measures pertaining to student suspension, expulsion, and corporal punishment, which are again not particularly relevant in schools attended by kindergartners and first graders. We settled on a series of research questions involving access to limitedEnglish-proficiency (LEP) programs in public elementary schools that offer kindergarten. We were interested in whether first grade LEP students suffered academically by attending schools in which access to LEP services was limited or restricted. The E&S survey contains a measure indicating the number of students in each school needing LEP services and another indicating the number that actually receive LEP services. Our next step would have been to create a measure for each school indicating the extent to which LEP students were being denied access to LEP services—the proportion eligible for but not receiving LEP services. However, initial analyses revealed the correlation between the two measures was: 96. This is welcome news for LEP students and for researchers interested in equity; students who need LEP services in U.S. public schools generally receive them, at least according to the school staff who filled out the survey. For us, however, the fact that these two measures are essentially identical precluded their use in our investigation because of the virtual lack of variability between measures. Because of the lack of informative school-level measures and the dearth of variables unique to the E&S survey, we decided we could not conduct empirical

OCR for page 77
analyses using the combined E&S/ECLS-K datafile. We therefore altered both our approach and our questions substantially. Instead of restricting our focus to one or two empirical questions, we broadened our efforts to investigate the general utility of the E&S survey data. In the first section of the full report, we document the extent to which the E&S survey includes data in common with three other large, widely used datasets collected by the U.S. Department of Education: the CCD, the National Educational Longitudinal Study (NELS:88), and the ECLS-K. Of course, the timing of all data collections must coincide. For example, students in NELS:88 began high school in 1988 and graduated (most of them) in 1992. Our findings are displayed in a series of tables that indicate which E&S survey measures are redundant and available on other datasets and which data are unique to E&S survey. In a concluding section of the report we share our views regarding the value of the E&S survey from the standpoint we know best: as quantitative researchers interested in studying educational equity using large, nationally representative databases.