Allocating Federal Funds for State Programs for English Language Learners
and
NATIONAL RESEARCH COUNCIL
OF THE NATIONAL ACADEMIES
THE NATIONAL ACADEMIES PRESS
Washington, D.C.
www.nap.edu
THE NATIONAL ACADEMIES PRESS
500 Fifth Street, N.W.
Washington, DC 20001
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.
This study was supported by U.S. Department of Education Contract Number ED-08-CO-0119 to the National Academy of Sciences. Support of the work of the Committee on National Statistics is provided by a consortium of federal agencies through a grant from the National Science Foundation (award number SES-0453930). Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the organizations or agencies that provided support for the project.
International Standard Book Number-13: 978-0-309-18658-2
International Standard Book Number-10: 0-309-18658-7
Additional copies of this report are available from the
National Academies Press,
500 Fifth Street, N.W., Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area); Internet, http://www.nap.edu
Copyright 2011 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
Suggested citation: National Research Council. (2011). Allocating Federal Funds for State Programs for English Language Learners. Panel to Review Alternative Data Sources for the Limited-English Proficiency Allocation Formula under Title III, Part A, Elementary and Secondary Education Act. Committee on National Statistics and Board on Testing and Assessment. Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.
THE NATIONAL ACADEMIES
Advisers to the Nation on Science, Engineering, and Medicine
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council.
PANEL TO REVIEW ALTERNATIVE DATA SOURCES FOR THE LIMITED-ENGLISH PROFICIENCY ALLOCATION FORMULA UNDER TITLE III, PART A, ELEMENTARY AND SECONDARY EDUCATION ACT
Alan Zaslavsky (Chair),
Department of Health Care Policy, Harvard Medical School
Jamal Abedi,
School of Education, University of California, Davis
Frank D. Bean,
Center for Research on Immigration, Population and Public Policy, University of California, Irvine
David Francis,
Texas Institute for Measurement, Evaluation and Statistics, University of Houston
Edward Haertel,
School of Education, Stanford University
David Hubble,
WESTAT, Inc., Rockville, MD
Rebecca Kopriva,
Wisconsin Center for Education Research and School of Education, University of Wisconsin, Madison
Robert Linquanti,
WestEd, Oakland, CA
Helen Malagon,
Office of Superintendent of Public Education, State of Washington Migrant and Bilingual Education Programs
Catherine Neff,
Office of Federal Programs, South Carolina Department of Education
P. David Pearson,
Graduate School of Education, University of California, Berkeley
Charlene Rivera,
Graduate School of Education and Center for Equity and Excellence in Education, George Washington University
Judith A. Koenig, Costudy Director
Thomas Plewes, Costudy Director
Esha Sinha, Research Associate
Michael J. Siri, Program Associate
COMMITTEE ON NATIONAL STATISTICS 2010
Lawrence D. Brown (Chair),
Department of Statistics, Wharton School, University of Pennsylvania
John M. Abowd,
School of Industrial and Labor Relations, Cornell University
Alicia Carriquiry,
Department of Statistics, Iowa State University
William DuMouchel,
Oracle Health Sciences, Waltham, MA
V. Joseph Hotz,
Department of Economics, Duke University
Michael Hout,
Department of Sociology, University of California, Berkeley
Karen Kafadar,
Department of Statistics, Indiana University
Sallie Keller,
Science and Technology Policy Institute, Washington, DC
Lisa Lynch,
Heller School for Social Policy and Management, Brandeis University
Sally Morton,
Department of Biostatistics, University of Pittsburgh
Joseph Newhouse,
Division of Health Policy Research and Education, Harvard University
Samuel H. Preston,
Population Studies Center, University of Pennsylvania
Hal Stern,
Department of Statistics, University of California, Irvine
Roger Tourangeau,
Joint Program in Survey Methodology, University of Maryland, and Survey Research Center, University of Michigan
Alan Zaslavsky,
Department of Health Care Policy, Harvard Medical School
Constance F. Citro, Director
BOARD ON TESTING AND ASSESSMENT
Edward Haertel (Chair),
School of Education, Stanford University
Lyle Bachman,
Department of Applied Linguistics, University of California, Los Angeles
Stephen Dunbar,
College of Education, University of Iowa
David J. Francis,
Texas Institute for Measurement, Evaluation, and Statistics, University of Houston
Michael Kane,
National Conference of Bar Examiners, Madison, WI
Kevin Lang,
Department of Economics, Boston University
Michael Nettles,
Educational Testing Service, Princeton, NJ
Diana C. Pullin,
Lynch School of Education, Boston College
Brian Stecher,
RAND, Santa Monica, CA
Mark Wilson,
Graduate School of Education, University of California, Berkeley
Rebecca Zwick,
Graduate School of Education, University of California, Santa Barbara
Stuart Elliott, Director
Acknowledgments
This report is the product of the work by many people, each of whom contributed expertise to the enterprise. I first acknowledge with great appreciation the efforts of my fellow panel members, who brought expertise as data users and state data providers and in education policy, demography, statistical estimation methods, census and American Community Survey (ACS) methodology, administrative data systems, and testing and assessment. All of this expertise was critical to the multifaceted issues that had to be addressed in this evaluation of the admissible data sources for allocation of funds under Title III, Part A, of the Elementary and Secondary Education Act. The panel members freely contributed their time to accomplishing the myriad of tasks associated with assembling information and preparing this report and cheerfully accepted the duties of facilitating sessions at the five committee meetings. Their contributions during the period in which the report was in final preparation and after the outside review, when sections of the report had to be turned around on a very truncated schedule, are especially appreciated. These efforts manifested the panel members’ deep dedication to the well-being and effective education of English language learners and immigrant children.
The panel held public sessions at its first and second meetings. The sessions were organized as formal workshops, with presentations by internal and outside presenters followed by free-flowing discussion. In developing these public sessions, the committee greatly benefitted from the support of the staff of the U.S. Department of Education and the U.S. Census Bureau.
The panel was set on the right path at its first open meeting by Thomas M. Corwin, director, Division of Elementary, Secondary and Vocational Analysis, Budget Service, U.S. Department of Education, and Richard Smith, the department’s acting assistant deputy secretary and director, Office of English Language Acquisi-
tion. They elaborated on the department’s charge to the committee and established the background for the study for the information of the panel.
This report was preceded by a significant report by the U.S. Government Accountability Office in 2006 that described the allowable data for the allocation of formula-based grants to states to support the education of students with limited English proficiency. It described the effects of using the ACS and the states’ assessment data, and it recommended a series of steps for the U.S. Department of Education to improve the quality and use of the data. The report’s authors, Harriet C. Ganson, assistant director for education, workforce and income security, and Nagla’a El-Hodiri, senior economist, discussed the findings of that report with the panel at its April 2009 workshop.
The panel was assisted in developing a full understanding of the ACS at its first meeting by Susan Schecter, chief of the American Community Survey Office of the U.S. Census Bureau. She updated the committee on the current status of the survey and discussed the plans for development of estimates for small areas and populations.
At its second meeting, the panel heard from experts on state testing, state practices, and programs for English language learners, as well as the U.S. Department of Education official responsible for the development and maintenance of the departmental databases that hold the state data. We thank a group of Title III directors for their contribution at the workshop: Celina Arias-Romero (California), committee member Helen Malagon (Washington), Joanne Marino (North Carolina), Barbara Medina (Colorado), Elizabeth Minjarez (Texas), committee member Catherine Neff (South Carolina), Redro Ruiz (New York), and Maria Santos (New York City). The administrative data are strongly affected by state practices, and the members of this group documented the state practices that reflect the individual policies and needs as perceived by the state agencies that conduct these programs.
Ross Santy, director of the Performance Information Management Service in the Office of Planning, Evaluation and Policy Development, U.S. Department of Education, provided a summary of the types of data that are maintained on state programs, and traced the recent efforts to build a data system that increasingly standardizes and validates data maintained and submitted by state education agencies. His unit provided the committee with a number of special tabulations of data submitted by state education agencies to the U.S. Department of Education that greatly assisted our work.
Robert Kominski, assistant chief, Housing and Household Economic Statistics Division, U.S. Census Bureau, discussed the evolution and status of the questions on the ACS that provide the basic information for derivation of estimates of the number of limited English proficient students by state. The committee also extends special thanks to David Raglin and Sandra Clark of the Special Studies staff of the American Community Survey Office of the U.S. Census Bureau for facilitating the provision of extensive special tabulations of ACS data that permitted the panel to assess the adequacy of the ACS for allocation purposes.
We also acknowledge the guidance and support provided to this study by the
two contracting officer technical representatives who were assigned to shepherd this project by the U.S. Department of Education. On numerous occasions, Pat Butler and Sandra Furey stepped in to assist in setting up meetings with the appropriate departmental experts, securing data for comparison purposes, and facilitating necessary contractual modifications to enable the successful completion of this study. Britt Jung of the department’s budget office provided advice and guidance throughout the study and assisted in the provision of special funding for the purchase of ACS tabulations from the Census Bureau that permitted much of the analysis contained in this report. William Sonnenberg of the National Center for Education Statistics provided significant assistance to the panel about data used in the current and past allocation formula for the distribution of Title III funds by the U.S. Department of Education.
The committee gratefully acknowledges the dedicated effort provided by the staff of the Committee on National Statistics (CNSTAT) and the Board on Testing and Assessment (BOTA) of the Division of Behavioral and Social Sciences and Education of the National Research Council (NRC). Tom Plewes and Judy Koenig brought deep experience and broad organizational skill to their service as costudy directors, and their work was a key factor in ensuring the efficient completion of this report. Esha Sinha of the CNSTAT staff supported the analysis by developing the research databases of the data provided in various formats by the U.S. Department of Education and the U.S. Census Bureau, and she managed, through extraordinary efforts, to conduct analyses for the committee. Michael Siri, also of the CNSTAT staff, provided smooth administrative management of the five meetings held in several venues. The task of contacting state directors of English language learner programs to ascertain the latest information on their screening and assessment tests was ably and cheerfully handled by Rose Neugroschel of BOTA. We also want to acknowledge the active participation and guidance provided by Constance Citro, director of CNSTAT, and Stuart Elliott, director of BOTA, whose advice and leadership were instrumental in moving this study from the planning to report preparation stages.
This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the Report Review Committee of the NRC. The purpose of this independent review is to provide candid and critical comments that assist the institution in making its report as sound as possible, and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process.
The panel thanks the following individuals for their review of this report: Jonathan G. Dings, Planning and Assessment, Boulder Valley School District; Gerunda B. Hughes, Office of Institutional Assessment and Evaluation, Howard University; Joseph B. (Jay) Kadane, Department of Statistics, Carnegie Mellon University; Neal Kingston, Center for Educational Testing and Evaluation, University of Kansas; Jan Lanier, Division of College and Career Readiness, Tennessee Department of Education; Lilia G. Sánchez, English Learner and Curriculum Support Division,
California Department of Education; and Robin Scarcella, Academic English/ESL Program, University of California, Irvine. Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations, nor did they see the final draft of the report before its release. The review of this report was overseen by Allen L. Schirm, Human Services Research, Mathematica Policy Research, Inc., and Kenneth W. Wachter, Demography and Statistics, University of California, Berkeley. Appointed by the NRC, they were responsible for making certain that the independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of the report rests entirely with the authoring committee and the NRC.
Alan Zaslavsky, Chair
Panel to Review Alternative Data Sources for the Limited-English Proficiency Allocation Formula under Title III, Part A, Elementary and Secondary Education Act
Tables, Figures, and Boxes
TABLES
1-1 |
Summary Definitions of Selected Variables Used in This Report, |
|||
1-2 |
Operational Definitions of the ELL Population, |
|||
2-1 |
ACS Sample Sizes: Initial Addresses and Final Interviews, by Type of Unit, |
|||
2-2 |
English Language Learning Children and Youth Aged 5-21, by State, 2005-2008, |
|||
2-3 |
Average Number of ELL Children and Youth Aged 5-21, by State, |
|||
2-4 |
Percentage Share of ELL Children and Youth Aged 5-21, by State, |
|||
2-5 |
Standard Errors of Percentage Shares of ELL Children and Youth Aged 5-21, by State (in percentage), |
|||
2-6 |
Ratio of ELL Students Aged 5-18 in Public Schools to All Students Aged 5-18 in Public Schools (in percentage), |
|||
2-7 |
Standard Errors of Ratio of ELL Students Aged 5-18 in Public School to All Students Aged 5-18 in Public School (in percentage), |
|||
2-8 |
Coefficients of Variation of Estimates of ELL Students, by State Size, |
|||
2-9 |
Absolute Difference in Percentage Share of States Across Years (in percentage), |
|||
2-10 |
Difference in Percentage Share of ELL Students of States by Varying Age Groups, Enrollment Status, and Type of School (in percentage), |
|||
2-11 |
Difference in Percentage Share of ELL Students of States by Varying ELL Criterion, |
|||
2-12 |
Coverage Rates for Housing Units, Group Quarters, and Total Population (in percentage), |
2-13 |
Allocation Rates for Language Questions in ACS, for United States (in percentage), |
|||
3-1 |
English Language Proficiency Assessments, by State, 2009-2010 School Year, |
|||
3-2 |
Tests Reviewed by the Panel, |
|||
4-1 |
Tests Used by the States for Initial Classification of English Language Learners for the 2009-2010 School Year, |
|||
4-2 |
Numbers and Shares of All ELL Students by State: School Years 2006-2007, 2007-2008, and 2008-2009, |
|||
4-3 |
Rates of All ELL Students by State: School Years 2006-2007, 2007-2008, and 2008-2009, |
|||
4-4 |
Numbers and Shares of ELL Students Reported Tested, Not Proficient for 2007-2008 and 2008-2009 School Years, |
|||
4-5 |
Rates of Tested, Not Proficient Students by State, 2007-2008 and 2008-2009 School Years (in percentage), |
|||
5-1 |
Differences Between the ACS Estimates and State-Provided Counts of ELL Students, |
|||
5-2 |
Shares of ELL Students Based on ACS and State-Provided Counts (in percentage), |
|||
5-3 |
Ratio of State Shares Based on ACS Estimate to Shares Based on State-Provided Counts, |
|||
5-4 |
Total Absolute Difference Between Shares Based on ACS Estimates and Shares Based on State-Provided Counts, |
|||
5-5 |
Rate of ELL Students by State Based on ACS Estimates and State-Provided Counts (in percentage), |
|||
5-6 |
Ratio of Rates Based on ACS Estimates to Rates Based on State-Provided Counts, |
|||
5-7 |
Analysis of Using ACS 3-Year Estimate and Other Variables to Predict State-Provided Rate of All ELL Students, |
|||
5-8 |
Analysis of Using ACS 3-Year Estimate and Other Variables to Predict State-Provided Rate of Tested, Not Proficient ELL Students, |
|||
5-9 |
Descriptive Summaries of LEA-Level Data on Rate of ELL Students, by State, |
|||
5-10 |
Results of Within-State Regressions, |
|||
5-11 |
Comparison of Volatility in ACS Estimates and State-Provided Counts (in percentage), |
|||
6-1 |
Allocation Rates for Nonresponse on Immigrant Items in the ACS, 2005-2008 (in percentage), |
|||
6-2 |
Number of Immigrant Children and Youth Aged 3-21, by State, |
6-3 |
Average Number of Immigrant Children and Youth Aged 3-21, by State, |
|||
6-4 |
Share of Immigrant Children and Youth Aged 3-21, by State (in percentage), |
|||
6-5 |
Standard Errors of Shares of Immigrant Children and Youth Aged 3-21, by State (in percentage), |
|||
6-6 |
Ratio of Immigrant Children Aged 5-18 Enrolled in Public School to All Children Aged 5-18 Enrolled in Public School (in percentage), |
|||
6-7 |
Difference in the Percentage Share of Immigrants Aged 3-21 of States by Age Group, Enrollment Status, and Type of School, |
|||
6-8 |
Absolute Difference and Absolute Relative Difference in Ratio of Immigrant Children and Youth (in percentage), |
|||
6-9 |
Key Features of ACS and State-Collected Data on Immigrant Children and Youth, |
|||
6-10 |
Comparison of State Student Immigrant Counts and American Community Survey Estimates of Recent Immigrant Students, |
|||
6-11 |
Comparison of Volatility in ACS Estimates of Youth Aged 5-18 and Enrolled in Public School and State Counts of Recent Immigrants (in percentage), |
|||
6-12 |
Rates of Immigrant Children by Eligible School District, |
|||
6-13 |
Relationship at the School District Level Between ACS Estimates and State-Provided Estimates of the Rate of Immigrant Children Among Public School Enrollees, in Eligible Districts as Described in Text, for States with at Least 10 Eligible Districts, |
|||
7-1 |
Comparison of ACS and State-Provided Data on Desired Characteristics for an Allocation Formula, |
|||
A-1 |
English Language Proficiency Tests Reviewed and the States That Use Them, |
FIGURES
4-1 |
ELL classification and reclassification procedures, |
|||
4-2 |
Number of criteria used by states for ELL reclassification, |
|||
5-1 |
Comparison of ACS 3-year rate and state-provided rate of all ELL students for the 2008-2009 school year, |
|||
5-2 |
Comparison of ACS 3-year rate and state-provided rate of tested, not proficient students for the 2008-2009 school year, |
|||
6-1 |
Rules for determining immigrant education program student in Illinois, 2009-2010 school year, |
6-2 |
Immigrant ratio from state counts (2007-2008 academic year) and ACS 3-year estimates (2006-2008), |
BOXES
1-1 |
Limited English Proficiency (LEP) Student: Definition, |
|||
1-2 |
State Allotments, |
|||
1-3 |
Legislative Mandate for Estimating the Number of LEP Students, |
|||
2-1 |
Question on Language Use from the ACS, |
|||
2-2 |
History of the Census Language Questions, |
|||
2-3 |
Item Nonresponse Rate Comparisons: 2000 Census, 2000 C2SS, and 2005 ACS (in percentage), |
|||
6-1 |
ACS Questions on Birth, Citizenship, and Year of Entry into the United States, |
Acronyms and Abbreviations
ACCESS Assessing Comprehension and Communication in English State to State
ACS American Community Survey
AERA American Educational Research Association
AIR American Institutes for Research
AMAO annual measurable achievement objective
APA American Psychological Association
AYP adequate yearly progress
AZELLA Arizona English Language Learner Assessment
BOTA Board on Testing and Assessment
C2SS Census 2000 Supplementary Survey
CAPI computer-assisted personal interviewing
CATI computer-assisted telephone interviewing
CCD Common Core of Data
CCSSO Council of Chief State School Officers
CDE California Department of Education
CELA Colorado English Language Assessment
CELDT California English Language Development Test
CELLA Comprehensive English Language Learning Assessment
CMA for ELA California Modified Assessment for English-Language Arts
CNSTAT Committee on National Statistics
CRESST National Center for Research on Evaluation, Standards and Student Testing
CSAP Colorado Student Assessment Program
C-SAVE Center for the Study of Assessment Validity and Evaluation
CSPR Consolidated State Performance Report
CST for ELA California Standards Test for English-Language Arts
CV coefficients of variation
DoEd U.S. Department of Education
EDEN Education Data Exchange Network
ELDA English Language Development Assessment
ELL English language learner
ELP English language proficiency
ELPAS English Language Proficiency Assessment Standards
ELPS English Language Proficiency Survey
EPAS English Proficiency for All Students
ESEA Elementary and Secondary School Act
ESL English as a second language
ESOL English for speakers of other languages
ESS EDEN Submission System
ETS Educational Testing Service
GAO U.S. Government Accountability Office
HLS home language survey
IELA Idaho English Language Assessment
IEP Immigrant Education Program
IPT IDEA Proficiency Test
KELPA Kansas English Language Proficiency Assessment
LAB-R Language Assessment Battery-Revised
LAS Language Assessment Scales
LAS Links Language Assessment Scales Links
LEA local education agency
LEP limited English proficiency
LEP-SCASS Limited English Proficient State Collaborative on Assessment and Student Standards
LIEP Language Instruction Educational Program
LPTS Language Proficiency Test Series
MAC II Maculaitis Assessment of Competencies Test of Language Proficiency
MAD mean absolute difference
MAF master address file
MARD mean absolute relative difference
MELA-O Massachusetts English Language Assessment-Oral
MEPA-R/W Massachusetts English Proficiency Assessment-Reading and Writing
MI-ELPA Michigan English Language Proficiency Assessment
MN-SOLOM Minnesota Modified Student Oral Language Observation Matrix
MontCAS Montana Comprehensive Assessment System
MWAC Mountain West Assessment Consortium
NCELA National Clearinghouse for English Language Acquisition
NCES National Center for Education Statistics
NCLB No Child Left Behind Act
NCME National Council on Measurement in Education
NGA Center National Governors Association’s Center for Best Practices
NRC National Research Council
NV-ELPA Nevada State English Language Proficiency Assessment
NYSED New York State Education Department
NYSESLAT New York State English as a Second Language Achievement Test
OCR Office of Civil Rights, U.S. Department of Education
OELA Office of English Language Acquisition, U.S. Department of Education
OR-ELPA Oregon English Language Proficiency Assessment
OTELA Ohio Test of English Language Acquisition
PEP Population Estimates Program
PUMA public-use microdata area
PUMS public-use microdata samples
RFEP reclassified as fluent English proficient
RMSE root mean square residual error
RPTE Reading Proficiency Tests in English
SAIPE Small Area Income and Poverty Estimates Program
SEA state education agency
SELP Stanford English Language Proficiency Test
TEA Texas Education Agency
TEAE Test of Emerging Academic English
TELPA Tennessee English Language Placement Assessment
TELPAS Texas English Language Proficiency Assessment Systems
TESOL Teachers of English to Speakers of Other Languages