As the United States continues to be a nation of immigrants and their children, the nation’s school systems face increased enrollments of students whose primary language is not English. With the 2001 reauthorization of the Elementary and Secondary Education Act (ESEA) through the No Child Left Behind Act (NCLB), the allocation of federal funds for programs to assist these students to be proficient in English became formula-based: 80 percent on the basis of the population of children with limited English proficiency1 and 20 percent on the basis of the population of recently immigrated children and youth.
Title III of ESEA directs the U.S. Department of Education (DoEd) to allocate funds based on the data that will yield the most accurate, up-to-date numbers, and further directs the secretary to choose the more accurate source of data between the two allowable data sources: the number of students assessed for English proficiency as reported to the federal government by each state education agency or data from the American Community Survey (ACS). The department determined that the ACS estimates are more accurate, and since 2005, those data have been the basis for the federal distribution of Title III funds.
Subsequently, an analysis of the two data sources by the U.S. Government Accountability Office (2006b) raised concerns about the fact that the data sources would allocate quite different amounts to the states and suggested that the DoEd develop and implement a transparent methodology for determining the relative accuracy of the two allowable data sources. In addition, while shortcomings were
noted in the data provided by the states, the ACS estimates were shown to fluctuate between years, causing concern among the states about the unpredictability and unevenness of program funding.
In this context, the DoEd commissioned the National Research Council’s Committee on National Statistics and Board on Testing and Assessment to convene this panel to address the accuracy of the estimates from the two data sources and the factors that influence the estimates and to consider means of increasing the accuracy of the data sources or alternative data sources that could be used for allocation purposes.
DESIRABLE CHARACTERISTICS FOR TITLE III ALLOCATION FORMULA DATA
The panel developed a set of criteria for evaluating which of the allowable data sources the department should use for allocating federal Title III funds: conceptual fit, level of geographic fit, timeliness, quality, cost, fairness, stability, insensitivity to policy and methodological differences, transparency, and comparability.
Conceptual Fit. A data element used in an allocation formula should meet the conceptual objectives of the program for which the allocation is aimed. In the case of allocating Title III education funds to states, a data element with a good conceptual fit is one that meets the definition provided in the legislation—the number of LEP and immigrant children and youth in a state. In a larger sense, however, considering the overall objective of the allocation of federal funds, a conceptually fitting data element would provide subnational governments a level of federal funding that is proportional to their need and circumstances. The ACS defines need as the numbers of children and youth who are eligible on the basis of their skill in speaking the English language. The state-provided counts are more conceptually fit in that they define need as those identified by schools through questionnaires and assessments, which are increasingly standardized.
Level of Geographic Detail. The state is the key level of detail for which the data should be available. Both the ACS and state-provided estimates are available for both states and local education agencies (LEAs).
Timeliness. The elapsed time between the reference period for the estimates and the period for which the allocations are being made should be as short as possible. Both the ACS and the state-provided estimates are available approximately 9 months following the reference period.
Quality. The ACS data meet statistical reliability standards and are of acceptable precision. The state-provided counts of English language learners, based on administrative data, are not subject to sampling error, but there may be some different interpretation of the instructions for data collection. The state-provided counts of immigrant children and youth rely on LEA judgments and fall short of the quality of ACS estimates.
Cost. Both ACS estimates and state-provided counts are available at minimal extra cost.
Fairness. The U.S. Census Bureau, which administers the ACS, ensures the data are free from manipulation. State-provided counts have improved such that data are similarly free from manipulation; however, states’ discretion over various policies may affect perceptions of fairness.
Stability. The state-provided counts are relatively stable from year to year. The 1-year ACS estimates for smaller states are subject to noticeable variation. The 3-year ACS estimates are more stable than the 1-year estimates and than the state counts.
Insensitivity to Policies and Methodological Differences. The ACS estimates are not sensitive to administrative practices or policy differences, but they may be sensitive to differences in the demographic composition of respondents. The state-provided counts are somewhat sensitive to state decisions regarding identification, testing, and program entry and exit policies.
Transparency. The ACS data are collected by professional staff using highly standardized, well-documented methods. The state-provided counts are collected by varying methods and depend on LEA implementation so that documentation is not readily available.
Comparability. The ACS data are comparable across geographic and demographic dimensions. The state-provided counts conform to basic federal definitions but are not comparable in their constructs due to differences in state tests and the states’ classification and reclassification criteria.
In sum, the panel concludes that the ACS data are slightly better on some characteristics, while the state-provided data have advantages on others, which is not reflected in the current formula.
Data Sources for Title III Allocation Formula
RECOMMENDATION 1 As soon as technically possible, the U.S. Department of Education should begin to incorporate state-provided counts of English language learner students into Title III formula allocation calculations. Initially, the state-provided data should be given a weight of 25 percent of the allocation, with the remaining 75 percent weight given to the American Community Survey data.
RECOMMENDATION 2 In the portion of the allocation that is based on state-provided data, the U.S. Department of Education should use the state-provided count of the number of students who are determined not
to be English proficient on the basis of the state’s English language proficiency test.
RECOMMENDATION 3 The U.S. Department of Education should continue to use the American Community Survey estimate as the basis for allocating the 20 percent of the Title III funds that are to be based on the population of recently immigrated children and youth (relative to national counts of these populations).
RECOMMENDATION 4 The U.S. Census Bureau should conduct research on the accuracy of the American Community Survey language item for assessing population prevalence of English language learner children and youth, including the strength of its association with more comprehensive English language proficiency measures. With the objective of evaluating and improving the item, researchers should examine the effects on responses of situational, cultural, demographic, and socioeconomic factors, placement of the item in the questionnaire, and the ability of adult responders to make English language proficiency distinctions.
Weighting of Data Sources
RECOMMENDATION 5 When the quality and cross-state comparability of state-provided data have reached an acceptable standard, the weight given to the state-provided counts should be adjusted upward to the point at which the American Community Survey estimates and the state-provided counts contribute equally to the 80 percent portion of the allocation formula. State-provided counts should continue to be based on the number of students who are determined not to be English proficient on the basis of the state’s English language proficiency test, in a way that is comparable across states.