Small-Area Estimates of School-Age Children in Poverty: Evaluation of Current Methodology (2000)

For Title I fund allocations to be made in spring 1999 for the 1999-2000 school year, the Census Bureau was charged to produce updated estimates of the number of poor school-age children at the school district level. Three sets of school district estimates were required: (1) estimates of related school-age children (aged 5-17) who were in poor families in the preceding calendar year;¹ (2) estimates of all school-age children; and (3) estimates of the total population of the district. The first two sets of estimates were needed to implement the allocation formulas for basic and concentration grants; the third set of estimates was needed to determine which school districts have fewer than 20,000 people.²

This chapter considers estimates of poor school-age children for school districts. It reviews the difficulties that confront attempts to develop such estimates; describes the procedure that the Census Bureau used to develop district-level estimates of school-age children in July 1996 who were in poor families in 1995; and assesses the limited evaluations that are possible of these estimates. Finally, the chapter discusses the implications of the evaluations for the use of updated school district estimates for Title I allocations. Chapter 8 describes the procedure

and evaluations for estimates of the number of all school-age children and of the total population in July 1996 for school districts.

Developing estimates of the number of poor school-age children (or other characteristics) for school districts presents difficult problems. These problems include the small population size of most districts and several other features of their boundaries and scope: school district boundaries in many instances cross county lines; they can and often do change over time; and some school districts cover specific grade levels, such as kindergarten-8 or 9-12. Because of these problems, there are no data sources now available for developing updated school district estimates of poor school-age children by using the type of model-based approach that was used for county estimates. These problems also compromise the quality of the estimates for school districts that can be made by aggregating data for blocks from the decennial census. We briefly review each of these issues in turn.

Table 7-1 shows the distribution of total school districts, school districts coterminous with counties, and total counties by population size from the 1990 census. Of 15,226 districts, 49 percent had fewer than 5,000 people, and fully 82 percent had fewer than 20,000 people, while only 9 percent had 40,000 or more people; the median population size was about 5,250. By comparison, of 3,141 counties, 10 percent had fewer than 5,000 people, and 32 percent had 40,000 or more people; the median population size was about 23,000. Small districts, while numerous, accounted for small proportions of school-age children: districts with fewer than 5,000 people included only 6 percent of all school-age children, and districts with fewer than 20,000 people included only 27 percent of all school-age children; in contrast, districts with more than 40,000 people included 58 percent of all school-age children. Such uses as Title I fund allocations, however, require estimates for all school districts, no matter how small. Yet it is not possible to obtain direct estimates for school districts from national surveys, such as the March CPS. Many school districts will have no sampled households in national surveys, and the estimates for all but the largest districts with sampled households will be very unreliable (i.e., exhibit high sampling variability).³ Even longform census data, as discussed below, are unreliable for many school districts.

School district boundaries are, in general, determined by state regulations and practices. In seven states and the District of Columbia, school districts are coterminous with counties; these states included 370 districts in 1990 (2% of the total).⁴ In another 17 states, school district boundaries coincide with other political units, such as townships. The boundaries of most, but not all, of the school districts in these states respect county lines. These states included 3,344 districts in 1990 (22% of the total), of which 190 crossed county lines. In the remaining 26 states, school district boundaries are unique to districts and often cross county lines. These states included 11,563 districts in 1990 (76% of the total), of which 3,931 crossed county lines. In all, 4,121 school districts (27% of the total) crossed county lines.

It is relatively easy to develop updated estimates of poor school-age children for districts that are coterminous with counties because county boundaries are generally stable over time, counties are relatively large areas, and data sources are available for counties (e.g., the data used to estimate the county model).

Overall, in 1990, there were 928 districts that comprised an entire county or, in the case of a few districts (e.g., New York City), two or more entire counties. (The 928 districts include the districts in the seven states and the District of Columbia in which all school districts are counties together with selected districts in other states.) These districts accounted for 6 percent of all districts and 22 percent of all school-age children in 1990. Their median population size in 1990 was about 18,500 (Table 7-1, col. 3)–close to the median population size for all counties (Table 7-1, col. 5).

Most of the remaining districts, whether or not they cross county lines, present more or less serious problems for updating: they are small, with a median population size of less than 5,000; their boundaries can and often do change; and few data are available for estimating poverty. These districts accounted for 94 percent of districts and 78 percent of all school-age children in 1990.

In 1990, 11,284 school districts (74% of the total) served all grades –pre-kindergarten, kindergarten, or 1st grade through 12th grade. The remaining 3,942 districts (26% of the total) served a subset of grades, such as elementary grades, high school grades, or middle school grades. Developing updated estimates of poor school-age children for districts that serve specific grades is difficult because a method must be devised to allocate the limited available data on school-age poverty to the age range that is appropriate to the grade range of the school district.

The Census Bureau's county model can readily provide updated estimates of the number of poor school-age children for the small subset of school districts that comprise entire counties. However, as noted above, a model similar to the county model cannot be developed for the remaining 94 percent of school districts, principally because of the lack of administrative data with which to form the predictor variables in a regression model. For example, states do not generally geocode the addresses of Food Stamp Program participants to school districts, so there are no counts of food stamp participants for school districts. Similarly, a substantial proportion of addresses on federal income tax returns cannot be geocoded to census blocks, so it is not possible to estimate the number of poor children reported by families on tax returns for school districts. Finally, data from school districts on students who are approved to receive free meals under the National School Lunch Program (requested from the states by the National Center for Education Statistics in its Common Core of Data program) are far from complete and are of uncertain quality and applicability (see below, “School Lunch Data ”). In the future, it may be possible to develop appropriate

data sources for a model-based approach to estimating poor school-age children for school districts (see Chapter 9; see also National Research Council, 2000: Ch.5), but such data are not now available.

In the absence of data with which to develop a school district model similar to the county model, the Census Bureau used a simple within-county shares approach to estimate poor school-age children by school districts for 1995. The approach involved seven steps:

1. A survey was conducted in which officials in every state were asked to provide school district boundaries for the 1995-1996 school year.

2. Each 1990 census block was assigned to a school district, as defined for 1995-1996.⁵

3. The 1990 census data were aggregated for the blocks (or fractions of blocks) in each school district or part of a school district that lay wholly within a county.

4. The 1990 census data for each school district or school district part were tabulated to form a ratio estimate of the number of poor school-age children: the ratio estimate was obtained by applying the proportion of poor school-age children from the census long-form sample data to the short-form complete-count estimate of all school-age children. The ratio estimate was used because it reduced somewhat the high variability in the census estimates for school districts in comparison with estimates formed by simply inflating the long-form number of poor school-age children by the sampling weight.

5. For the school districts or school district parts in a county, the share (proportion) for each school district or school district part of the 1990 census county total of poor school-age children was calculated from the ratio estimates. (For districts that are coterminous with a county, the share was 100%.)

6. The 1990 census shares from step (5) were applied to the updated 1995 county estimates of poor school-age children produced by the county model (see Chapter 4) to obtain 1995 estimates of poor school-age children for school districts or school district parts.

7. The 1995 school district estimates of poor school-age children were

the estimates from step (6) for school districts wholly within a county and the sum of the estimates of school district parts for school districts that crossed county lines.

As an example of the within-county shares procedure, take a county with 1,600 poor school-age children in 1989 (1990 census data) of whom 1,200 (75%) resided in school district A, 240 in school district B (15%), and 160 in school district C (10%). If the 1995 county model estimated that the county had only 1,200 poor school-age children, then the estimates of poor school-age children in 1995 for school districts A, B, and C are 900, 180, and 120, respectively. The estimation method assumes that all three school districts in the county experienced the same proportionate decrease in the number of poor school-age children–25 percent–as the county as a whole. If this assumption is incorrect (e.g., because the decrease in poverty in the county was concentrated in one of the districts, perhaps because of changes in the housing stock), then the estimates for the three school districts will be incorrect.

For the 1997-1998 school year, 18 states used a similar procedure for allocating their Title I county funds to school districts, in that they made within-county allocations on the basis of 1990 census school district shares of poor school-age children, either solely or in combination with estimates of the other categories of formula-eligible children (e.g., foster children). Another nine states used 1990 census data together with other data sources, such as school lunch data, to allocate Title I county funds to school districts (according to the U.S. Department of Education).

The Census Bureau's 1995 school district estimates of poor school-age children are not the only input to the Title I allocation formula. To make direct allocations to school districts for the 1999-2000 school year, the Department of Education also had to obtain several other data elements for school districts, most of which have not been previously available at the district level: counts of the other categories of formula-eligible children (children in foster homes, in local institutions for neglected and delinquent children, and in families with income above the poverty line who receive welfare assistance);⁶ and the dollar amounts of Title I allocations that school districts received for the 1998-1999 school year (to use in the hold-harmless computations). The Census Bureau's estimates of poor school-age children also had to be adjusted to reflect school district boundary changes between 1995-1996 and 1998-1999; the department left it to the states to make appropriate adjustments.

To evaluate the Census Bureau's 1995 estimates of poor school-age children for school districts, the panel and the Census Bureau first assessed the 1990 census estimates that are used to form school district shares of poor school-age children within counties. The 1990 census estimates are subject to high sampling variability, which is a problem for the Bureau's shares procedure. This high variability is also a problem for evaluations that use the 1990 census estimates as the standard of comparison.

Opportunities to evaluate the school district estimates are constrained by the limitations of available data. The panel and the Census Bureau used a 1980-1990 school district census file to evaluate a few variations of the Bureau's shares procedure for a subset of districts. The panel also evaluated the use of National School Lunch Program data as an alternative method for constructing updated school district estimates of poor school-age children in New York State.

The two inputs to the Census Bureau's within-county shares procedure for school district estimates of the number of poor school-age children are the county model estimates for the target year, which have been extensively evaluated (see Chapter 6), and the 1990 census estimates for determining school district shares, which are discussed in this section. The income data that are used to determine poverty status in the census are collected on the long-form questionnaire, which was administered to an average of about one-sixth of households in 1990. The long-form sample size is orders of magnitude larger than the sample size of such household surveys as the CPS, but for small areas, the long-form estimates can exhibit high sampling variability.

Table 7-2 shows the mean and median coefficient of variation (in percent) for the estimated number of poor school-age children from the 1990 census longform sample, obtained as a simple inflation estimate, for school districts distributed into groups categorized by number of school-age children, with each group containing approximately the same number of districts. The mean coefficient of variation is 32 percent for all school districts, varying from 64 percent for districts in the smallest size category (1-185 students) to 14 percent for districts in the largest size category (3,770 or more students). ⁷ This degree of variability is high. For example, if a typical school district has about 200 poor school-age children, the long-form sample might give estimates anywhere from about 70 to about 330 poor school-age children. (This range is from 200 minus twice the coefficient of variation of 32% for the typical district to 200 plus twice that

coefficient of variation.) By comparison, a common design goal for estimates that are published from a survey is a coefficient of variation of 10 percent or less.

Table 7-2 also shows the mean and median coefficient of variation for school district estimates of poor school-age children that were constructed by ratio estimation. In this approach, the proportion of poor school-age children is computed from the long-form sample data and that proportion is then applied to the estimated total number of school-age children from the short-form or complete-count census data, which are not subject to sampling variability. This procedure somewhat reduces the variability of the estimates: the mean coefficient of variation of the ratio-adjusted estimates is 30 percent, compared with 32 percent for the long-form estimates, a reduction of 7 percent.

The Census Bureau used the ratio-adjusted 1990 census estimates of poor school-age children to construct the 1995 school district estimates but, given time constraints, did not conduct research on ways to further reduce the variability of the census estimates. One possible line of research is to use other short-form data

(such as race and ethnicity, tenure, family type) as auxiliary information in the estimation of poor school-age children. Another line of research is to smooth the 1990 census school district estimates with the 1990 census county estimates, which would reduce the variability for smaller size districts (see Chapter 9).

The Census Bureau constructed a file of 1980 and 1990 census data for selected school districts, which was used to compare three sets of estimates of poor school-age children in 1989 with estimates from the 1990 census. In each instance, the 1980 census data that are used in the estimation are solely from the long form, while the 1990 census data are ratio adjusted. Three methods were used for the estimates:

1. One method used county model estimates to construct school district estimates: method (1) applied the 1980 census shares of poor school-age children for school districts (or parts of school districts) within counties in 1979 to the Census Bureau's 1989 estimates of poor school-age children from its county model, with the county estimates controlled to the national estimate of poor school-age children in 1989 (from the 1990 census). This within-county shares procedure is analogous to that used by the Census Bureau to produce the 1995 school district estimates from 1990 census within-county shares applied to 1995 county model estimates, except that the 1980 census data are not ratio adjusted. (Also, the 1980 census estimates for 1979 are 10 years out of date for 1989 estimates, while the 1990 census estimates for 1989 are 6 years out of date for 1995 estimates.)

2. A second method used 1990 census county estimates to construct school district estimates: method (2) applied the 1980 census shares of poor school-age children for school districts (or parts of school districts) within counties to the 1990 census county estimates of poor school-age children. This procedure eliminates the error in method (1) that is due to the county model.

3. The third method was a national stable shares procedure: method (3) applied the 1980 census shares of poor school-age children for school districts within the nation as a whole to the national estimate of poor school-age children in 1989 from the 1990 census. This procedure assumes no change whatsoever in the relative shares of poor school-age children among school districts from the previous census, not even the change that occurs in methods (1) and (2) because of changes in the relative shares of poor school-age children among counties.

For several reasons, these comparisons provide only limited information with which to evaluate the Census Bureau's within-county shares model for school district estimates. First, the alternative models are not very different from the Census Bureau's model. Second, the 1990 census estimates that are the

standard of comparison are subject to high sampling variability even after ratio estimation. Finally, the evaluation file, of necessity, contains only a subset of school districts.

The 1980-1990 evaluation file was constructed from school district data sets that were prepared after each census. It was not possible to retabulate the individual block records from the 1980 census to match the 1990 census school district boundaries; instead, the goal was to identify a set of school districts in the data set for each year that could reasonably be assumed to have retained the same boundaries and grade ranges. The 1980 and 1990 census school district files were matched, using their identification numbers and other characteristics, and the following kinds of 1990 districts were dropped from the evaluation file:

• 928 districts or district parts for which the district or part was coterminous with a county and, hence, for which the county model would provide estimates;

• 4,108 districts that were not “unified,” that is, that covered a limited grade range, such as kindergarten-8 or 9-12;

• 416 districts that were newly formed and had no counterpart in 1980;

• 12 districts in counties that changed boundaries between 1980 and 1990; and

• 609 districts that crossed county lines and for which one or more of the county pieces had no counterpart in 1980.

The resulting evaluation file contains 9,243 districts, which are 61 percent of the 15,226 school districts that were included in the 1990 census school district file and 56 percent of school-age children. The subset of school districts in the evaluation file closely resembles the entire set of 1990 school districts in terms of the distribution of total population and total number of school-age children in 1990. For example, the subset of districts in the evaluation file includes 47 percent with fewer than 5,000 people and 8 percent with more than 40,000 people; the corresponding figures for the entire set of 1990 school districts are 49 percent and 9 percent, respectively.

A key assumption for using the evaluation file is that the 9,243 districts in the file, which had the same identification numbers in both 1980 and 1990, are the

same districts and that their boundaries have not changed.⁸ This assumption could be incorrect in some instances. For example, if a school district follows township boundaries and the township annexed land from another town between 1980 and 1990, it is likely that the school district identification number was the same in both 1980 and 1990 even though the boundaries changed.

To investigate this assumption, the Census Bureau looked at unified school districts, not coterminous with counties, that had the same identification numbers in 1990 and in the 1995-1996 school district boundary survey. For 6 percent of these districts, which accounted for 2 percent of school-age children, the total number of school-age children originally tabulated in the 1990 census differed by 5 percent or more from the number retabulated according to the 1995-1996 boundaries. For the remaining 94 percent of districts, the two tabulations were exactly the same or differed by less than 5 percent, indicating that the same identification number is a reasonably good indicator of stability in school district boundaries.

Table 7-3 provides summary statistics for the three sets of school district estimates of poor school-age children in 1989 in comparison with the 1990 census estimates. The statistics provided are the average absolute difference between the estimates from a model or method and the census, as a percentage of the average number of poor school-age children in the census, and the average proportional absolute difference between each set of estimates and the 1990 census estimates. For comparison purposes, the last row of the table provides the same statistics for county estimates of poor school-age children in 1989 from the Census Bureau's county model.

The first measure in Table 7-3 assesses the absolute difference between estimates from a method and the 1990 census in terms of numbers of poor children, while the second measure assesses the absolute difference in terms of proportional errors for school districts. From a national perspective, it can be argued that the absolute differences in terms of numbers are more important for effective Title I allocations because, with direct allocation, Title I funds are primarily distributed in proportion to the number of children in a school district. Therefore, the amount of funds that are misallocated depends primarily on the number of children rather than on the percentages by district (see Chapter 6). However, from the district perspective, the proportional error for a district's allocation is also important.

Ideally, a method will perform well on both types of measures, but, as discussed below, all three shares methods perform much worse on the average proportional absolute difference measure overall than on the average absolute difference measure. The reason for this consistent finding is that there are many small school districts that tend to have much larger-than-average proportional errors, which are reflected in the average proportional absolute difference mea-

sure. However, the much larger proportional errors for small districts do not represent many poor school-age children and so do not contribute as much to the absolute difference measure.

As seen in the last row of Table 7-3, the average absolute difference of the county model estimates from the 1990 census county estimates is 10.7 percent of the 1990 census county average number of poor school-age children; the average proportional absolute difference is 16.4 percent. The school district estimates show much larger differences. The average absolute difference for the Census Bureau's within-county shares method (1), which applies 1980 census school district shares of poor school-age children within counties to the county model estimates for 1989, is 22.2 percent of the 1990 census school district average number of poor school-age children (2.1 times the corresponding figure for the

county model estimates); the average proportional difference is 60 percent (3.7 times the corresponding figure for the county model estimates).

Method (1) reduces the average absolute difference measure by 23 percent (22.2/28.7) and the average proportional absolute difference measure by 16 percent (60.0/71.7) compared with the national stable shares method (3), which assumes no change in school district shares of all poor school-age children in the nation between the 1980 and 1990 censuses. Method (2), which applies 1980 census school district shares within counties to the 1990 census county estimates of poor school-age children, performs somewhat better: it reduces the average absolute difference measure by 37 percent (18.0/28.7) and the average proportional absolute difference measure by 23 percent (55.4/71.7) when compared with the national stable shares method (3). However, method (2) is of theoretical interest only. In a noncensus year, such as 1995, model-based county estimates have to be used for adjusting school district shares from the census, and there will be errors in these estimates.

The Census Bureau also explored a fourth method in which a set of estimates was constructed by applying the 1980 census shares of poor school-age children for school districts within each state to the 1990 census state estimates of poor school-age children. This method produced average absolute and average proportional absolute differences between those of methods (2) and (3). It also is of theoretical interest only because it cannot be used in a noncensus year. However, it illustrates that using state estimates to control school district shares (which could be done with the Census Bureau's state model estimates) is better than using a single national control, but worse than using county controls.

There are several reasons for the large differences between the estimates of poor school-age children for districts produced by method (1) and the comparison ratio-adjusted estimates from the 1990 census: the sampling variability in the 1980 census estimates of school district shares, which is high for many districts; the inability of the within-county shares method to capture within-county changes in school district shares of poor school-age children from the 1980 census to the 1990 census; the errors in the county model itself (although these are not a large component); and the sampling variability that remains in the 1990 census comparison estimates even after ratio estimation. Because of the sizable sampling variability in the 1990 census estimates, the difference measures in Table 7-3 are overestimates of the differences from the true numbers of poor school-age children in 1989. It would be useful to remove this effect.

As an extension of this analysis, the Census Bureau has produced graphs of three quantities: a measure of the difference between the school district estimates from the census estimates, which is the root mean square difference; the estimated sampling variability of the census estimates; and the resulting calculated root mean square error of the school district estimates and the census estimates adjusted for the sampling variability in the latter (Bell et al., 2000). The graphs indicate that for school districts with small population sizes and small proportions of poor school-age children the sampling variability in the census estimates ac-

counts for a sizable proportion of the root mean square difference. Extensions of this type of analysis to other categorizations of school districts would be useful.

Considering school districts by size, method (1) performs reasonably well on both the average absolute difference measure and the average proportional absolute difference measure for districts with 40,000 or more people in 1990 (data not shown). For these districts, the estimates are not markedly worse than the county estimates. Districts with 40,000 or more people are only 8 percent of the total number of school districts in the 1980-1990 evaluation file, but they contain 55 percent of the poor school-age children in the file.

Method (1) performs less well for school districts with 10,000 to 39,999 people in the 1990 census and performs very poorly for districts with fewer than 5,000 people in the 1990 census. Thus, while the average absolute difference measure for districts with 40,000 or more people in 1990 is 17 percent, it is 24 percent for districts with 20,000 to 39,999 people, 26 percent for districts with 10,000 to 19,999 people, 30 percent for districts with 5,000-9,999 people, and 43 percent for districts with 5,000 or fewer people. Districts with 5,000 or fewer people in 1990 contain only 8 percent of the poor school-age children in the 1980-1990 evaluation file, but they are 47 percent of total districts.

The much larger differences between the estimates from method (1) and the 1990 census estimates for smaller school districts relative to larger districts are due in part to the greater variability in the 1990 census estimates for smaller districts. As noted above, the panel believes there are ways to further reduce the variability in the 1990 census estimates beyond the reduction achieved by using simple ratio estimates instead of simple inflation estimates. A reduction in the variability of the 1990 census estimates would permit not only a more accurate assessment of the within-county shares approach, but also an improvement in the 1995 school district estimates that are formed by applying 1990 census within-county school district shares to the 1995 estimates from the county model.

The evaluation also examined the algebraic differences by category of school district. The following categories were used: census geographic division, 1980 population, 1990 population, 1980-1990 population growth, percentage of poor school-age children in 1980, percentage of poor school-age children in 1990, change in the poverty rate for school-age children from 1980 to 1990, percentage of Hispanic population in 1980, percentage of black population in 1980, and percentage of group quarters residents in 1980. The results are summarized below for method (1); detailed results are provided in U.S. Census Bureau (1998b).

The category algebraic difference is the sum, for all school districts in a category, of the algebraic (signed) difference between the estimate of poor school-age children from a model or method and the 1990 census estimate for each

district, divided by the sum of the census estimates for all districts in the category. This measure expresses model-census differences in terms of the numbers of poor children, similar to the overall absolute difference in the first column of Table 7-3. However, the category algebraic difference is expressed as an algebraic measure in which positive differences (overpredictions) within a category offset negative differences (underpredictions). The measure is intended to identify instances of potential bias in the predictions from a model or method. For example, the method may over(under)predict, on average, the number of poor school-age children in larger school districts relative to smaller districts.

The comparison of category algebraic differences for estimates from the Census Bureau's within-county shares method (1) with 1990 census estimates found no strong patterns of over(under)prediction for school districts categorized by percentage of black, percentage of Hispanic, or percentage of group quarters residents in 1980. However, method (1) did somewhat overpredict the number of poor school-age children in districts with no black or Hispanic residents or a very small proportion of group quarters residents in 1980 relative to other districts. Method (1) also somewhat overpredicted the number of poor school-age children in districts with fewer than 5,000 people in 1980 and 1990 relative to other districts. These findings may be related, in that districts with no black or Hispanic residents or very few group quarters residents are also districts that have very small populations.

For school districts categorized by population growth from 1980 to 1990, method (1) overpredicted the number of poor school-age children in districts that experienced a decline in population of more than 10 percent and underpredicted the number of poor school-age children in districts that experienced an increase in population of more than 10 percent. The same pattern was even greater for districts categorized by change in the poverty rate for school-age children from 1980 to 1990. These findings are not unexpected in that the within-county shares method, by definition, will not reflect large increases or decreases in population or poverty for school districts except to the extent that the district increase or decrease parallels that of the county in which it is located.

For school districts categorized by percentage of poor school-age children, method (1) underpredicted the number of poor school-age children in districts that had a lower school-age poverty rate in 1980 relative to districts with a higher rate. In contrast, method (1) overpredicted the number of poor school-age children in districts that had a lower school-age poverty rate in 1990 relative to districts with a higher rate. These findings are also not unexpected. They are evidence of the so-called “regression to the mean” phenomenon, in which, due to sampling variability, school districts that have low estimates of school-age poverty rates in one year will tend to have higher rates in another year (other things being equal) and vice versa.

Finally, for school districts categorized by census geographic division, method (1) overpredicted the number of poor school-age children in districts in

the Pacific Division and, to a lesser extent, in the Mountain Division relative to districts in other divisions. This finding is consistent with a similar finding for the 1989 county model estimates, which, in turn, was attributed to the state model.

As noted at the beginning of the chapter, there is a lack of administrative data with which to estimate school-age poverty for school districts. Food stamp data are not generally available for districts, and federal tax return data at present cannot be reliably coded to school district in many areas. Another possible source of information on poverty for school districts is data from the National School Lunch Program, which provides free and reduced-price meals to qualifying children.

The Census Bureau decided that it could not use school lunch data in developing updated estimates of poor school-age children for school districts for two major reasons. First, there is no complete and accurate set of school lunch data for all school districts. The National Center for Education Statistics (NCES) obtains school lunch counts as part of its Common Core of Data (CCD) system, in which state educational agencies report a large number of data items for public school systems. ⁹ The school lunch data are not published and have not been a priority of NCES. The center does not follow up with states when there is no information provided for a school district or to evaluate the accuracy of the reports. Hence, the quality of the data is not established, and they are far from complete.

Files of school lunch data for 1990-1995 that NCES provided to the panel contain large numbers of missing and zero values. In some cases, missing data may be due to the fact that a school district no longer exists (e.g., it may have been combined with another district); however, most instances of missing data appear to be due to nonreporting by school districts. Zero values may be valid in many instances, but NCES staff indicated that missing data are sometimes reported as zero, and analysis supported this assessment. Also, while states are asked to report counts of students approved to receive free lunches, it appears that many states report the combined total number of students approved for free or reduced-price lunches, which have different income eligibility limits.

Only 18 states have reports that are more than 90 percent complete (fewer

than 10% of school districts with missing or zero values) in all 6 years of the NCES files. At the other extreme, 10 states have reports that are less than 50 percent complete in all 6 years; most of these states do not report school lunch data at all. Clearly, if school lunch data are to be used to estimate the number of poor school-age children, it would be necessary to make school lunch reporting a priority in the CCD system for follow-up and evaluation.

The second reason for the Census Bureau not to use school lunch data in developing a consistent set of school district estimates nationwide is that counts of students approved for free lunches differ from poor school-age children in at least three respects, and the differences are probably not the same across jurisdictions:

• The eligibility standard to qualify for free lunches is family income that is less than 130 percent of the poverty threshold, which means that students approved for free lunches include near-poor as well as poor children. Children in families with incomes as high as 185 percent of poverty can receive reduced-price lunches.

• Participation in the school lunch program is voluntary and may be affected by such factors as perceived stigma (it is believed that high school students are less likely to participate than elementary school students for this reason) and the extent of outreach by school officials to encourage families to sign up for the program.

• Students approved for free lunches include children enrolled in participating schools in the district, whereas the Census Bureau is charged to produce estimates of poor school-age children who reside in the district. The two populations differ to the extent that poor resident children attend nonparticipating private schools or schools outside their district (nonresident poor children may also attend schools in the district).

If the relationship between students approved for free lunches and poor school-age children varies across jurisdictions, then it would not be possible to use school lunch data to estimate school-age poverty for school districts directly (e.g., by applying a constant factor to the school lunch counts to obtain estimated numbers of poor school-age children). If school district estimates are obtained by suballocating or distributing county-level estimates, then school lunch data could be used in modeling the suballocation if the relationship between students approved for free lunches and poor school-age children is constant across school districts within counties. However, variations in the relationship within counties would be a problem for such modeling.

There are two other reasons that such modeling could be problematic if school lunch data appeared suitable to use in models for some but not all states and counties. First, there would be practical difficulties for the Census Bureau to collect the data and develop and evaluate different estimation procedures for

different sets of school districts, even when it might be possible to improve the accuracy of the estimates in some cases. Second, if the use of different estimation procedures produced estimates with different biases across school districts, there could be a problem of equity for concentration grants. The reason is that, under direct allocations, the concentration grant allocation to one area can affect the allocations to other areas. This was not the case under the two-stage allocation process, in which states that used school lunch data (or another data source) to allocate concentration grant funds to school districts were constrained by the county allocations determined by the Department of Education.

Yet the number of students approved for free lunches is an indicator of low income that relates specifically to the population of school-age children and that could be updated annually. Moreover, it is not subject to the sampling error that is such a serious problem for the Census Bureau's estimation procedure that suballocates county-model estimates on the basis of sample data from the census long form. Further, school lunch data carry considerable face validity with local officials.¹⁰

Thus, if school lunch data were available and determined to relate in a reasonably consistent manner to school-age poverty across jurisdictions, the Census Bureau could consider using such data to modify its current estimation process (see National Research Council, 2000:Ch.5). For example, it could follow the practice of the states that previously used school lunch data, solely or together or with census data, to distribute the Department of Education's Title I allocations for counties to school districts under the two-stage procedure. In effect, these states used a shares approach for school district estimates that is similar to the Census Bureau's method, except that the district shares within counties were computed on the basis of contemporaneous school lunch data instead of 1990 census estimates of poor school-age children.

The panel undertook a limited evaluation of a school lunch-based shares approach in one state–New York–for which it was able to obtain complete free and reduced-price school lunch data for almost all public schools for 1989-1990 and assign them to school districts and counties.¹¹ There are 623 New York State school districts in the 1980-1990 evaluation file, or 7 percent of the total number

of districts in the file. The New York State districts in the evaluation file are somewhat larger than average, with a median population size in 1990 of about 9,000 compared with a median population size of about 5,250 for all districts.

The analysis compared three sets of estimates of poor school-age children in 1989 for school districts in New York State with estimates from the 1990 census. The methods used to develop the three sets differ only in the estimation of within-county school district shares: the Census Bureau's method (2), in which 1980 census within-county school district shares of poor school-age children were applied to 1989 county estimates from the 1990 census; a method in which 1989-1990 within-county school district shares of students approved to receive free lunches were applied to 1989 county estimates from the 1990 census; and a method in which 1989-1990 within-county school district shares of students approved to receive free or reduced-price lunches were applied to 1989 county estimates from the 1990 census.

Table 7-4 provides summary statistics for the three sets of school district estimates of poor school-age children in 1989 for New York State compared with the 1990 census estimates for these districts. The table includes the average absolute difference between the estimates from a method and the census, expressed as a percent of the average number of poor school-age children in the census, and the average proportional absolute difference between each set of estimates and the 1990 census estimates. For comparison purposes, the last row of the table provides the same statistics for estimates of poor school-age children for all U.S. school districts in the evaluation file from method (2), which applies within-county school district shares to 1990 census county estimates.

The average absolute difference of the estimates for all school districts from the 1990 census estimates using method (2) is 18 percent; the average proportional absolute difference is 55 percent. The corresponding figures for estimates for New York State school districts only are 24 percent and 53 percent, respectively, for a method analogous to method (2); 22 percent and 49 percent, respectively, for a method based on free lunch counts; and 24 percent and 52 percent, respectively, for a method based on free and reduced-price lunch counts.

The absolute differences in all three methods of estimating poor school-age children in 1989 for New York State school districts are similar and large in magnitude. Even though the school lunch data pertain to the same year as the 1990 census comparison estimates, neither set of school lunch-based shares estimates is much more accurate than the 1980 census-based shares estimates. However, looking at both absolute differences and category algebraic differences, the use of free lunch counts as the basis for estimates is marginally more accurate than the other two methods that were evaluated. This finding suggests that it could be worthwhile to conduct a similar analysis for other states to determine if there is enough consistency across jurisdictions in the relationship of school lunch data to school-age poverty to warrant further consideration of the use of

school lunch data for school district estimates.¹² If these data were to be used, a major effort would be needed to improve the reporting of the data to NCES for use by the Census Bureau for estimation purposes.

It was difficult for the panel to draw firm conclusions from the evaluations of the Census Bureau's updated school district estimates of poor school-age children regarding their use for Title I allocations. On the positive side, the estimates are reasonably good for two groups of districts that contain many poor school-age children: districts that are coterminous with a county or more than one county, for which the county model provides estimates, and other districts with a total population of 40,000 or more, for which the Census Bureau's within-county shares method produces estimates that are only somewhat less reliable than the county model estimates.¹³ These two groups together (adjusting for the overlap among them) comprise only a small fraction of districts, 13 percent of the total as of 1990, but they contain a large fraction of poor school-age children, 62 percent of the total. On the negative side, the school district estimates are highly variable for the remaining 87 percent of districts, which contain 38 percent of poor school-age children.

In terms of the mandate to the panel, the estimates might be judged to be “inappropriate or unreliable” for direct allocations of Title I funds to school districts. However, such a conclusion implies a definition of “inappropriate or unreliable” that does not take into account the allocation procedures that might otherwise be used. Given that some set of estimates will be used to make Title I allocations, the panel believed that “inappropriate or unreliable” should be defined in a relative sense. Applying a relative definition, one could argue that, in the context of currently available information, a direct allocation procedure that uses the Census Bureau's school district estimates is at least as good as and perhaps preferable to the alternative, which would be to return to the two-stage process in which the states distributed the county allocations from the Department of Education to school districts by using a variety of data sources.

As described in Chapter 2, the states used several types of data for sub-allocations of Title I funds when the two-stage procedure was in effect. For the 1997-1998 school year, 18 states relied on 1990 census data, either solely or together with estimates of the other categories of formula-eligible children, to distribute the county allocations to school districts. For these states, the Census Bureau's 1990 census shares-based estimates are likely to be somewhat more accurate than the corresponding estimates that the states were producing because the Bureau had access to 1990 census block data and so could more accurately retabulate the census data to reflect changes in school district boundaries; the states had access only to public use census files for 1989-1990 school district

boundaries.¹⁴ In addition, the ratio-adjustment procedure employed by the Census Bureau to estimate census shares somewhat reduces their variability. For the six states in this group that used 1990 census data to make direct allocations of basic grants to school districts without regard to the county allocation amounts, the use of the Bureau's 1990 census shares-based estimates has the advantage that they reflect the updated county estimates from the Bureau's county model.

Twenty-five states used data sources other than the census, or in combination with the census, to suballocate county Title I funds to school districts. (Three of these states made direct allocations of basic grants to districts.) It was not possible to evaluate the accuracy of such sources as school lunch data across states. The analysis that was conducted for New York (and the preliminary analysis for Indiana, see above) suggests that there are only marginal gains in accuracy from use of school lunch data. Moreover, it is not likely that the use of a shares approach based on school lunch data would produce results that are as consistent across states as the use of a shares approach based on census data: in some states, school lunch shares might be better than census shares; in other states, they might be worse. This inconsistency could be a problem for direct allocation of concentration grants.

Overall, the panel found four reasons to support use of the Census Bureau's school district estimates of poor school-age children for direct allocation of Title I allocation funds: the congressional mandate for direct allocations; the use of a uniform procedure to derive the Census Bureau's estimates; the somewhat greater accuracy of the Census Bureau's estimates of 1990 census shares compared with what the states could likely produce; and the absence of strong evidence that there are other, better data sources available for estimation. For the rest of the panel's assessment, it considered more carefully the features of the basic grant and concentration grant allocation formulas and how they might interact with the provision in the 1994 legislation that states may redistribute the aggregate allocations for districts with fewer than 20,000 people by some other method that the Department of Education approves.

Under the two-stage allocation process, basic grants were allocated to school districts essentially as shares of the county total amounts. Whatever the data source used by a state to form the within-county shares (e.g., census data, school lunch data, combination of two or more data sources), the county totals remained as specified by the Department of Education. The exception is that the department allowed nine states in which school district boundaries bear little correspon-

dence to county boundaries to redistribute the total basic grant allocation for the state without regard to the county allocations. For other states, the county totals, which, in turn, reflected (approximately) the Census Bureau's updated estimates from its county model, were maintained.¹⁵

Direct allocation of basic grants to school districts by using the Census Bureau's within-county shares estimates has the same property of essentially respecting the county totals because the Census Bureau 's estimation procedure controls the school district estimates to county estimates derived separately from its county model. The correspondence between the county totals from the two-stage allocation process and those from the sums of direct allocations to the districts in each county will not be exact for several reasons. One, the hold-harmless provisions applied at the county level will give a somewhat different result from applying the hold-harmless provisions to districts and aggregating the resulting amounts to counties. Also, in contrast to counties, a proportion of school districts (about 10-12%) do not receive basic grants: although there was no eligibility threshold for counties to qualify for basic grants, school districts must have at least 10 formula-eligible children, and the number of eligible children must exceed 2 percent of the total number of school-age children in the district. Nonetheless, for basic grants, the county totals under direct allocations to school districts are likely to be fairly similar to what the allocations would be under the two-stage procedure.

However, when states choose the option in the legislation to redistribute the aggregate of the direct allocation amounts for school districts with fewer than 20,000 people by using some other data source (such as school lunch data), then the sum of the amounts for the districts in a county may not be similar to what the county amount would be under the two-stage process. The panel was concerned about this possible outcome: the county allocations that were made under the two-stage process reflected (approximately) the Census Bureau's county estimates from its county model, and these estimates are the only small-area estimates of poor school-age children that have been thoroughly evaluated and determined to be reasonably reliable.¹⁶ Direct allocations that use the Census Bureau's within-county shares estimates for school districts also reflect (approximately) the Bureau's county estimates, but state plans to redistribute the direct allocation amounts for school districts with fewer than 20,000 people by using some other data source may not have this desirable property.

Analysis with 1989 school lunch data for New York State districts with fewer than 20,000 people (476 districts, see Appendix D, Table D-9) did not find evidence of this problem. The average absolute and average proportional absolute differences from 1990 census school district estimates of poor school-age children were about the same for estimates that were developed by using free lunch counts with and without county controls. However, this analysis pertains to only one alternate data source in only one state. In the absence of a complete analysis of alternate data sources, the panel believed it to be desirable, to the extent possible, that the basic grant allocations reflect the county model estimates in all states, including those that choose the option of redistributing the aggregate of the direct allocations for school districts under 20,000 population by using another data source. The Department of Education can achieve this outcome by approving state reallocation plans that, in general, propose to aggregate the direct allocation amounts for districts under 20,000 population within counties and redistribute the county totals among the districts under 20,000 population in each county.

Concentration grants, in contrast to basic grants, were never allocated as shares of the county totals under the two-stage procedure because only a fraction (less than half) of jurisdictions was eligible.¹⁷ Under the two-stage process, concentration grants were allocated to those counties that had more than 6,500 or more than 15 percent of formula-eligible school-age children. In turn, states allocated county concentration grants to those districts in eligible counties that exceeded the threshold number or percentage of formula-eligible children: most districts that qualified for concentration grants presumably did so on the basis of exceeding the percentage threshold; few presumably did so on the basis of having more than 6,500 formula-eligible children.

Tabulations of 1990 census data in the evaluation file identified 30 percent of school districts, containing 60 percent of poor school-age children, as eligible for concentration grants under the two-stage allocation process.¹⁸ Eligible districts under the two-stage process were 65 percent of the total districts in eligible counties. (In states that used another data source, such as free lunch counts, to distribute county concentration amounts to districts, a higher percentage of school districts in eligible counties were likely classified as eligible for concentration grants; see below.)

The census tabulations showed that an additional 9 percent of school districts, containing 14 percent of poor school-age children, would be eligible for concentration grants except that they are located in a county that is not eligible. Under the two-stage procedure, states could reserve up to 2 percent of their concentration grant funds to allocate to eligible districts that were not in eligible counties, but these amounts were probably not adequate for the children in those districts.

The panel noted that the use of fixed thresholds for concentration grants places great demands on the quality of the estimates of those thresholds. An error of only one poor school-age child can make the difference between receiving a grant and not receiving a grant. For school districts that receive concentration grants to which they would not be entitled if true estimates of poor school-age children were available, these errors will be perpetuated through the hold-harmless provisions, particularly if the hold-harmless rate is retained at 100 percent. (There are also fixed thresholds for school districts to receive basic grants, although they are low, as noted above.) ¹⁹

One of the reasons for the legislation mandating direct allocations to school districts was to target concentration grants to all eligible school districts, including those in ineligible counties. To assess the appropriateness and reliability of the Census Bureau's updated school district estimates of poor school-age children for direct allocation of concentration grants, the panel first examined the rate of agreement between the Census Bureau's within-county shares method (1) and the 1990 census in classifying school districts into one of two poverty rate categories for school-age children in 1989 that correspond to the concentration grant threshold: 0 to 15 percent and 15 percent or higher; see Table 7-5. The tabulations were prepared from the 1980-1990 evaluation file for districts that did not cross county lines.

The method (1) school district estimates and the 1990 census ratio-adjusted estimates for 1989 assigned the same poverty rate category (0 to 15% or 15% or higher) to 76 percent of school districts and 87 percent of poor school-age children. By comparison, the county model estimates and the 1990 census county estimates for 1989 assigned the same poverty rate category to 88 percent of counties and 92 percent of poor school-age children. The rate of agreement between the method (1) school district estimates and the 1990 census ratio-adjusted estimates was least for school districts with fewer than 5,000 people: 64

percent agreement for districts and 65 percent agreement for poor school-age children. ²⁰ The rate of agreement was highest for school districts with 40,000 or more people: 92 percent for both districts and poor school-age children, slightly

better than the rate of agreement for counties. For districts and poor school-age children for which the method (1) and 1990 census estimates were not in agreement, method (1) classified a higher percentage in the under 15 percent school-age poverty rate category.

To focus on the issue of concentration grant eligibility for school districts with direct allocations versus the two-stage process, the panel examined the correspondence between the method (1) estimates and the 1990 census estimates for cross-classifications of 1989 school district and county school-age poverty rate categories; see Table 7-6 and Table 7-7. Method (1) estimated that 32 percent of districts, containing 59 percent of poor school-age children would be eligible for a concentration grant under the two-stage process (cell f, Table 7-6 and Table 7-7), and that another 10 percent of districts, containing 12 percent of poor school-age children would be eligible for a concentration grant under direct allocations (cell o). These aggregate percentages are similar to those for the 1990 census, noted above (see cells h and q in Table 7-6 and Table 7-7), but method (1) and the 1990 census classified a number of districts differently.

Of the districts and poor school-age children that the 1990 census estimated would be eligible for concentration grants under the two-stage process, method (1) agreed for 86 percent of districts and 96 percent of poor school-age children (cell e divided by cell h). The other 14 percent of districts and 4 percent of poor school-age children would be ineligible for concentration grants under the two-stage process according to method (1). There are also districts and poor school-age children that would be eligible under the two-stage process according to method (1) but ineligible according to the 1990 census: they comprise 18 percent of the districts and 3 percent of the poor school-age children that are eligible according to method (1) (cell d divided by cell f).

Of the additional districts and poor school-age children that the 1990 census estimated would be eligible for concentration grants under direct allocations (i.e., those in counties with school-age poverty rates under 15%), method (1) agreed for 53 percent of districts and 76 percent of poor school-age children (cell n divided by cell q). The other 47 percent of the additional districts and 24 percent of the additional poor school-age children would be ineligible according to method (1). There are also additional districts and poor school-age children that would be eligible according to method (1) but ineligible according to the 1990 census: they comprise 49 percent of the additional districts and 10 percent of the additional poor school-age children that are eligible according to method (1) (cell m divided by cell o).

Overall, the classification differences between the 1990 census estimates and the method (1) estimates are relatively large for the additional districts that would be eligible under direct allocations (i.e., districts with 15% or more poor school-age children in counties with less than 15% poor school-age children). However, the classification differences are relatively small for the additional poor school-age children that would be eligible under direct allocations. In particular, the

percentage of poor school-age children in the additional districts that would be eligible for concentration grants according to the within-county shares method (1) estimates but would not be eligible according to the 1990 census estimates is relatively small (10%).

It should be kept in mind that these evaluations are limited in at least three ways. First, they apply only to a subset of school districts in the evaluation file, which are, themselves, a subset of total districts. Second, like all of the evaluations of the Census Bureau 's school district estimates, they are based on a single comparison point. Third, the 1990 census estimates that are the standard of comparison are subject to high sampling variability for smaller school districts even with ratio adjustment.

Understanding the limits of the evaluations and the alternatives available, the panel concluded, on balance, that the use of the Census Bureau's school district estimates for direct allocations of concentration grants would be an improvement over the two-stage process. As intended by the 1994 legislation, many of the eligible districts that could not receive concentration grants with a two-stage allocation would receive such grants with direct allocations.

The option for states to redistribute concentration grant direct allocations for school districts with fewer than 20,000 people raised several issues for the panel to consider. States could propose to use another method, not only to redistribute the allocations among the districts that the Department of Education determined to be eligible for concentration grants on the basis of the Census Bureau's estimates, but also to redetermine eligibility. Under the previous two-stage procedure, the states that distributed county concentration grant allocations to districts on the basis of some other data source than the census used the alternate data source for both eligibility and amounts.

The use of free lunch or free and reduced-price lunch data in place of estimates of poor school-age children to redetermine eligibility as well as to redistribute allocation amounts likely has the effect that more districts receive concentration grants than would be the case with the use of the Census Bureau's school-age poverty estimates. The reason is that the income eligibility thresholds for free or reduced-price school lunches are higher than the poverty threshold. Consequently, more children fall below 130 percent of poverty (the threshold for free lunches) or below 185 percent of poverty (the threshold for reduced-price lunches) than fall below 100 percent of poverty.²¹ (About 20% of school-age children nationally are in families with incomes below 100% of the poverty threshold, while about 26% are in families with incomes below 130% of the

poverty threshold and about 38% are in families with incomes below 185% of the poverty threshold.)²² For this same reason, it is likely that proportionately more districts received concentration grants under the two-stage process in states that used school lunch data to determine eligibility than in states that used 1990 census data. In either case, the effect is to spread concentration grant dollars more thinly.

Analysis with 1989 school lunch data for New York State school districts with fewer than 20,000 people (476 districts; see Appendix D, Table D-5, Table D-6, Table D-7 through Table D-8) provides evidence of the effect of using estimates that reflect higher poverty thresholds. Under the two-stage process, 136 such districts in New York State would be eligible for concentration grants by using free and reduced-price lunch data and 112 would be eligible by using free lunch data, whereas only 76 districts would be eligible according to the method (2) estimates (or the 1990 census). Under direct allocations, the effect is much more pronounced: 294 districts with fewer than 20,000 people would be eligible for concentration grants by using free and reduced-price lunch data, and 214 districts would be eligible by using free lunch data, whereas only 109 districts would be eligible according to the method (2) estimates (115 districts according to the 1990 census).

As noted above, the panel concluded that any redistribution of basic grant direct allocations for districts with fewer than 20,000 people should be performed for such districts within each county to the extent possible, thereby reflecting (approximately) the county estimates of poor school-age children. For concentration grants, the panel reached the same conclusion, although it should be noted that there may be a problem with this approach when different data are used for reallocation. For example, if a county has two school districts and only one district is eligible for a concentration grant according to the Census Bureau's estimates of poor school-age children, but both districts are eligible by using school lunch data, then the first district will lose some of its dollars to the second district. Presumably, similar situations occurred under the two-stage allocation process, in which school district concentration grants were allotted from county totals.²³ However, such situations may be somewhat more likely to occur under direct allocations, which provide concentration grants to eligible districts in counties that do not meet the concentration grant threshold.

One approach that could ameliorate this effect is to adjust school lunch data for school districts in a county to equal the Census Bureau 's estimate of total poor school-age children for the county. The use of adjusted school lunch data to determine school-age poverty rates would be less likely to result in a much larger number of school districts qualifying for concentration grants than the use of the

Census Bureau's estimates of school-age poverty rates. Analysis conducted for New York State confirmed this outcome (see Appendix D, Table D-7, Table D-8): 127 school districts with fewer than 20,000 people would be eligible for concentration grants under direct allocations by using adjusted free and reduced-price lunch data versus 294 districts that would be eligible by using unadjusted data. The corresponding figures are 124 districts and 214 districts by using adjusted and unadjusted free lunch data. By comparison, 109 districts would be eligible by using the method (2) estimates.

Overall, by applying a relative standard for evaluation, the panel found reasons to support the use of the Census Bureau's updated estimates of poor school-age children for direct allocation to school districts. Also, the panel concluded that, in general, it is desirable for both basic grant and concentration grant allocations to reflect the county model estimates in all states, including those that choose the option of redistributing the direct allocations for school districts under 20,000 population by using another data source. However, the panel recognized that there are uncertainties about the operation of the formulas: for example, the extent to which the sum of direct school district allocations for counties approximates the allocations that would result for counties under the two-stage process and the extent to which there may be significant reallocations of concentration grant dollars from poorer to less poor districts with county controls. For this reason, the panel believed it to be critically important for the Department of Education to undertake a thorough study of the direct allocation process, both the methods used by the states and the results. Simulations of the allocations that would likely have been made under the two-stage process would be very helpful to inform the study.