Small-Area Estimates of School-Age Children in Poverty: Evaluation of Current Methodology (2000)

This chapter describes the data sources used for the Census Bureau 's SAIPE Program model-based estimates of poor school-age children for counties for 1993 and 1995. Data sources used for estimates of poor school-age children for school districts for 1995 are discussed in Chapter 7.

The data sources reviewed below are used not only to produce the initial county estimates from the county regression model, but also to produce the state estimates to which the initial county estimates are controlled (see Chapter 4). These sources include the March Current Population Survey (CPS), which provides the dependent variable in the state and county regression models, and the 1990 decennial census, Food Stamp Program administrative records, and federal income tax return administrative records, which provide predictor variables for the state and county models. The state and county regression models also use population estimates from the Census Bureau's postcensal population estimates program, which are described in Chapter 8.

The CPS income estimates that are used to form the dependent variable in the state regression model pertain to the estimation year–data from the March 1994 CPS for income year 1993 for the 1993 SAIPE state estimates; data from the March 1996 CPS for income year 1995 for the 1995 SAIPE state estimates. The county regression model uses an average of 3 years of CPS data, centered on the estimation year –data from the March 1993, 1994, and 1995 CPS for income years 1992, 1993, and 1994 for the 1993 SAIPE county estimates; data from the March 1995, 1996, and 1997 CPS for income years 1994, 1995, and 1996 for the 1995 SAIPE county estimates. The food stamp and IRS data used in the models

pertain approximately to the estimation year; the 1990 census data are for income year 1989.

Prior to the introduction of the SAIPE estimates, the census was the sole basis of poverty estimates for Title I allocations. The SAIPE county and state estimates used in recent allocations derive from CPS-based models that reflect a somewhat different standard of measurement than the census. The discussion therefore reviews differences between decennial census and CPS estimates of poverty.

Traditionally, the decennial census has been the source of estimates of poor school-age children for counties, with each census being used for Title I allocations until data from the next census became available. Many states also used census data for suballocations to school districts (see Chapter 2). The 1980 census data, covering income and poverty for 1979, were used for Title I county allocations for the 1983-1984 through 1993-1994 school years (and, in part, for the 1982-1983 school year). The 1990 census data, covering income and poverty for 1989, were used for county allocations for the 1994-1995 through 1996-1997 school years, and were averaged with the 1993 SAIPE county estimates for allocations for the 1997-1998 school year.

In the 1990 census, income data–the basis for measuring poverty–were collected in the long-form sample survey. The long form includes the small number of items that are asked of every household on the short form and other questions that are unique to the long form. The long-form sample in 1990 was about 15.7 million households, or about 1 in 6 households spread systematically across the country, except that very small counties and places (with estimated 1988 populations under 2,500) were sampled at a 1-in-2 rate, and very populous census tracts (or equivalent areas) were sampled at a 1-in-8 rate.

Data in the census are collected mainly by self-enumeration, in which respondents fill out questionnaires received in the mail. In 1990 approximately 70 percent of households that received the long-form questionnaire returned their questionnaires with some or all of the requested information; return rates were somewhat higher (75%) for households that received the short-form questionnaire (National Research Council, 1995b:189-190). Data from the balance of the population were obtained by enumerators who interviewed a household member or, failing that, a neighbor or landlord. The enumerators were mainly inexperienced temporary workers who were given limited training.

The income data in the 1990 census are based on seven questions on various components of income, such as wages and salaries and Social Security benefits. The long form also included a total income question, which was intended to permit respondents to enter a single amount if they could not provide amounts by source. Nonresponse rates are higher for income than for most other items in the

census. When household income information is missing, the Census Bureau uses statistical techniques to impute it on the basis of nearby households with similar characteristics. For the 1990 census, on average, 19 percent of aggregate household income was imputed (National Research Council, 1995b:387).

All censuses are subject to undercount–that is, failure to count everyone. There is also overcount, when persons are counted more than once or when ineligible persons are counted. For 1990, the net undercount (gross overcount minus gross undercount) was estimated at 1.8 percent for the total population, but there were substantial differences among population groups categorized by race, ethnicity, and age. Minorities were more heavily undercounted than others. By age, almost two-thirds of the estimated omitted population consisted of two groups: children under age 10 and men aged 25-39 (Robinson et al., 1993:13). The undercount was higher in large cities than in other areas and was disproportionately concentrated in the inner areas of those cities. These are also the areas where poverty is high. There are no direct estimates of the undercount for poor school-age children. However, it seems likely that the undercount for poor school-age children is larger than the undercount of all school-age children.

Decennial census data on income are estimates, and as such they are subject to sampling error because the data are collected from only a sample of households. Although sampling errors are relatively small for large geographic areas, such as states, the sampling errors for smaller geographic areas can be large relative to the estimate.

Table 3-1 provides information on the amount of error due to sampling variability in the estimated numbers of poor school-age children by county from the 1990 census. For example, for 63 counties, the margin of error due to sampling variability is less than 5 percent of the estimated number of poor school-age children.¹ The estimates for these counties are thus fairly precise. Moreover, these counties, although a small percentage (2%) of all 3,141 counties in 1990, are large ones: they contained 37 percent of the nation 's poor school-age children estimated by the 1990 census. However, for 1,405 counties, the margin of error due to sampling variability is 25 percent or more of the estimated number of poor school-age children. Although these counties contained only 6.4 percent of the poor school-age children in the nation estimated by the 1990 census, the imprecision in their estimates is of concern for Title I allocations.

Because the census is taken only once every 10 years, the data do not reflect current socioeconomic conditions and demographic distributions in the population. Concerns about using outdated decennial census poverty estimates for Title I allocations were reinforced by changes observed between the 1980 and 1990 censuses. Nationally, the number of poor school-age children rose by 5 percent over the 10-year period, from 7.7 million to 8.1 million. At the state level, there was considerable variability: 24 states and the District of Columbia experienced declines in the number of poor school-age children of up to 34 percent; 15 states saw increases of up to 25 percent; 8 states had increases ranging between 25 and 50 percent; and 3 states had increases between 50 and 67 percent (Moskowitz et al., 1993:71).

When considering the use of 1990 census data for allocations to be made later in that decade and into the next decade, there were similar concerns about the use of out-of-date information. Income data collected in the 1990 census are referenced to 1989; they do not reflect either the recession that began in 1990 or the recovery that began in 1991 and, consequently, do not reflect changes in the

proportion and geographic distribution of people below the poverty level that resulted from the rise and subsequent decline in unemployment and related economic and demographic changes. The belief that census data would not accurately reflect changes in need over time and across areas was a prime impetus for developing and using SAIPE estimates that reflected more up-to-date information. Although the level of poverty for an area at one point in time may not be a good measure of the area's poverty level at another point in time, there is nonetheless a relationship between the two measures. The county and state regression models take advantage of this relationship by including 1990 census poverty levels as predictor variables for estimating poverty later in the decade.

The CPS is designed primarily to provide monthly estimates of labor force participation, employment, and unemployment. Every March, the CPS collects additional data on income for the prior calendar year from which poverty rates can be determined. The CPS is therefore a more timely source of data on poverty than the census. Indeed, the annual March Income Supplement to the CPS provides the official national measure of poverty.² The March Income Supplement also serves as a basis for some federal fund allocations (U.S. Office of Management and Budget, 1993).

For the period from 1990 to 1994, the CPS sample included about 60,000 households each month that were eligible for interview; starting in 1996, this sample size was reduced to about 50,000 households each month. Of eligible occupied households, about 94-95 percent provide an interview. To obtain more reliable income data for the Hispanic population, all November CPS households with one or more Hispanic persons are reinterviewed in March if they still include a Hispanic person. This procedure adds about 2,500 Hispanic households to the sample in March.³

The CPS sample design, which is a multistage probability sample design, is revised about once every 10 years on the basis of the results of the latest census.

From 1986 to 1994, the CPS sample design included 727 sample areas consisting of about 1,300 counties. These areas were chosen on the basis of 1980 census data to represent the noninstitutional population in all 3,141 counties (in 1990) and independent cities in the 50 states and the District of Columbia. A design based on the 1990 census was phased in between April 1994 and July 1995: it included 792 sample areas consisting of about 1,300 counties, chosen to represent all 3,143 counties (in 1994) and independent cities in the 50 states and the District of Columbia. In January 1996, the number of sample areas was reduced from 792 to 754.

In general, larger states have larger CPS sample sizes. The largest states, however, have CPS sample sizes that are smaller than their proportionate share of the U.S. population, and the smallest states have proportionately larger sample sizes. For example, California, with 12.2 percent of the U.S. population, has 9.9 percent of the CPS sample; Wyoming, with 0.18 percent of the U.S. population, has 1.3 percent of the CPS sample. This sample design means that estimates of numbers and proportions poor in large states are generally more precise than those in smaller states. The largest states, however, have larger relative errors due to sampling variability than would be expected if the CPS sample were allocated to the states in proportion to their population; the reverse holds true for smaller states.⁴

The CPS is carried out by permanent, experienced, and well-trained interviewers, who interview each household the first month it is in the sample in person, with subsequent interviews by telephone. ⁵ For the March Income Supplement, the CPS asks household respondents about their money income received during the previous year, using a detailed set of questions for identifying about 28 different sources. About 20 percent of aggregate household income (about the same percentage as in the census) is imputed–that is, the data are missing and therefore constructed from information from similar households (National Research Council, 1993: Table 3-6).

Like other household surveys, the CPS exhibits population undercoverage at higher rates than the census itself. The coverage ratios for the CPS show the magnitude of the population undercoverage relative to population control totals that update the previous census and are produced by the Census Bureau's population estimates program. Coverage ratios are defined as the estimated survey

population before ratio-adjustment to census-based population controls divided by the census-based population controls. (Beginning with the March 1994 CPS, the population controls reflect an adjustment for the undercount in the census.) For March 1994, the ratio of the CPS estimated population to the adjusted population control total (all ages) was 92 percent; for the age group 0-14 years and the age group 15-19 years, the ratios were 94 percent and 88 percent, respectively (U.S. Census Bureau, 1996: Table D-2).

CPS undercoverage is corrected by ratio adjustments to the survey weights that bring the CPS estimates of population in line with updated national population controls by age, race, sex, and Hispanic origin. However, the ratio adjustments do not correct for other characteristics on which the undercovered population might be expected to differ from the covered population. For example, the ratio adjustments reweight equally the sample households within an age-race-sex-Hispanic origin category, when research suggests it is likely that lower income households within a category are more poorly covered than higher income households (see National Research Council, 1985:App.5.1).

The CPS sample size is not large enough to produce detailed information on the changes that occur over time in the geographical distribution of the population in poverty, but the survey can provide some useful indicators. It can illustrate how large changes can occur over short periods of time and how different areas can experience substantially different rates of change. As an example, consider the changes in the distribution in the number of poor people of all ages between 1990 and 1994 (income in 1989 and 1993). The CPS sample is sufficiently large to estimate such changes for 11 states, although the estimates are subject to large sampling errors; see Table 3-2.⁶ Overall, the estimated total number of poor people in the country increased by 24.5 percent, but with a wide range across states: 52 percent for Florida and 44 percent for California, but only 7 percent for Illinois and only 4 percent for Texas. Statistical sampling error affects the precision of these estimates, but it is still clear that there were changes over the period and that they differed among states.

The CPS data, when grouped by selected categories of counties and averaged over 3 years to improve precision, show similar changes in the estimated number of poor school-age children, which increased for the nation as a whole by 19.6 percent between 1989 and 1993; see Table 3-3. The increase is evident for counties in all regions of the country, in metropolitan and nonmetropolitan areas, and in all population size categories, but there is substantial variation in the size of the increase. The largest increases are for counties with a population size in the category of 1 million or more (33.1%), other (noncentral) counties in metro-

politan areas (30.1%), and counties in the West region (30.1%). The smallest increases are for counties with a population size in the category of 10,000-49,999 (3.3%, not statistically significantly different from zero), counties in nonmetro politan areas (10.8%), and counties in the Midwest region (13.3%).⁷

Although the CPS provides more current data than the decennial census, its much smaller sample size limits its ability to produce estimates for smaller areas. For all but a few very large counties, the CPS sample size is too small to produce reliable estimates. In fact, there is no CPS sample in over one-half of U.S. counties; only about 1,300 counties of 3,143 counties (in 1994) are represented in the sample. And for those counties for which CPS sample data are available, the

estimates of poverty and of the population aged 5-17 are, as a rule, extremely imprecise because of small sample sizes. However, as discussed in Chapter 4, a model-based approach that combines CPS estimates with administrative data in a statistical model can be used to yield estimates for counties that are more up to date than census estimates and have acceptable prediction errors. The Census Bureau's county-level model increases the CPS sample size for counties by combining 3 years of estimates. ⁸

The census and the CPS differ in other ways besides sample size. Even for a census year, the decennial census and the CPS produce different results with regard to children in poverty. Table 3-4 shows the differences between the 1990 census (1989 income) estimates of the number of poor school-age children and the 1989 CPS estimates for the nation as a whole and for various subcategories of counties. Table 3-5 provides a similar comparison of poverty rates. The CPS estimates in the two tables are averages of income data for 1988, 1989, and 1990; averaging is used to improve precision given the small CPS sample size in smaller areas.

Overall, for the U.S. population, the CPS provides an estimate of the number of poor school-age children that is 6.5 percent higher than the decennial census.⁹ For most groups of counties, the CPS estimate is also higher than the census estimate, and there is a suggestion of a pattern in which the ratio of the CPS estimate to the census estimate of poor school-age children in 1989 may increase as a function of county size. The panel conducted an analysis to determine whether there were statistically significant differences among the CPS-census

ratios for counties grouped by population size and other characteristics, but did not find such differences. However, this work was very preliminary.¹⁰

Though not fully researched and understood, differences between census and CPS estimates of poverty may result from the different ways the income data are obtained. The census and CPS use the same official poverty thresholds to deter-

mine poverty status,¹¹ income is counted in both as annual money income received in the previous calendar year, and both are intended to measure the same kinds of income. However, the CPS questionnaire asks respondents to provide income amounts for many more detailed categories than does the census questionnaire. For example, the 1990 census asked respondents to provide a com-

bined income amount for Supplemental Security Income (SSI), Aid to Families with Dependent Children (AFDC), and other public assistance or public welfare payments; the CPS asks separately for SSI, AFDC, and other public assistance or public welfare (including the source). Methodological research suggests that more detailed questions elicit more complete income reports (see National Research Council, 1995a:402-405); however, the extent to which questionnaire differences affect the responses in the CPS and the census is not known.¹²

The CPS and the census also use somewhat different rules for defining the universe to which poverty applies. For example, the CPS includes students living in college dormitories as family members in their parental households; the census considers the dormitory the place of residence and excludes residents of college dormitories from the poverty universe. The result is that somewhat more families with college students may be estimated as living in poverty in the CPS than in the census because a college student in a family increases its size and therefore its poverty threshold but likely does not add appreciably to its income.

The way the data are collected may also result in differences. In the CPS, data are collected through personal contacts (mostly by telephone) made by trained field representatives. In contrast, the census primarily relies on respondents to complete and return a questionnaire by mail. These and other differences imply that CPS-based estimates of poor school-age children represent a somewhat different standard of measurement from decennial census estimates. Consequently, moving from the decennial census to the CPS as the basis for estimates of poor school-age children may have had an effect on the time series of allocations for the Title I program.

In addition to predictor variables that are formed from the 1990 census and demographic population estimates, the SAIPE state and county regression models include predictor variables that are formed from administrative records data. Criteria for selecting administrative data sources for this purpose were that the data relate to poverty and that they be available for all states and counties on a consistent basis–that is, obtained using the same definitions and procedures and bearing a similar relationship to poverty across areas. The Census Bureau examined a variety of administrative records and selected two sources as most nearly meeting these criteria: administrative data on recipients of food stamps and federal income tax return reports of child exemptions in families with re-

ported income below the poverty threshold. Neither of these two data sources gives the number of school-age children in poverty as measured by the March CPS, but this is not a problem for model-based estimation: it is necessary only that the variables chosen to be used in the model can provide good predictions of that number.

The total number of recipients of food stamps is available monthly for states and annually for counties. Eligibility requirements for the program are generally uniform across all states, with some exceptions for Alaska and Hawaii. Two key eligibility requirements are that households must have gross income before deductions that is below 130 percent of the applicable poverty guideline and net countable income that is below 100 percent of the applicable guideline.¹³ The gross and net income limits for eligibility and the ceilings on allowable deductions are higher in Alaska and Hawaii than in the other states due to their higher cost of living.

The Census Bureau obtains monthly totals of food stamp recipients for states from the U.S. Department of Agriculture. After releasing the revised 1993 SAIPE state and county estimates but before preparing the 1995 estimates, the Census Bureau conducted research to determine how best to use these data for input to the state regression model. Based on that research, the Bureau decided to use the monthly counts averaged over a 12-month period centered on January 1 of the calendar year subsequent to the income reference year to form the predictor variable in the model. (Previously the data used were for July of the reference year.) The Census Bureau further refined the food stamp counts in three ways: it subtracted counts by state of the numbers of people who received food stamps due to specific natural disasters from the counts of the total number of recipients; it used the results of time-series analysis of monthly food stamp data from October 1979 through September 1997 to smooth outliers; and it adjusted the counts of food stamp recipients in Alaska and Hawaii downward to reflect the higher eligibility thresholds for those states.

For counties, the Census Bureau obtains counts of food stamp recipients from USDA and, in some instances, from state agencies. However, the information obtained for each county is not always the same: in most counties, the counts of food stamp recipients pertain to July; for some counties, they are an average of the monthly counts for the year. In developing the 1995 county regression model, the Census Bureau raked the county food stamp numbers to the adjusted state food stamp numbers.

Although the Food Stamp Program is generally administered uniformly across all states, estimated participation rates–the proportion of eligible households that apply for and receive benefits–are not the same. Differences in participation rates, which may stem from differences in outreach efforts, the stigma associated with participation, or other factors, could affect the comparability of food stamp counts across areas in terms of how well they relate to poverty.

The Census Bureau uses information from federal income tax returns to construct family units and determine the number of child exemptions in families that report incomes below the applicable poverty thresholds on their returns. Individual tax returns are assigned to counties on the basis of their address information. If the address is a post office box or rural route number and not a citystyle address, such as 104 Main St., the ZIP code is used to assign the address to a county. There are three major advantages of data from tax returns: (1) coverage of a very large proportion of the population, (2) coverage of a very large proportion of the income received by families, and (3) the availability of data on an annual basis.

After releasing the revised 1993 state and county estimates and before developing the 1995 estimates, the Census Bureau discovered and corrected an error in processing the 1989 IRS data. The corrected data were used to reestimate the decennial census equation that provides the residual predictor variable in the 1995 state model (see Chapter 4). The corrected data were also used to reestimate the 1989 state and county models for evaluation purposes. In both the state and county models, child exemptions reported by families on tax returns were redefined to include children away from home in addition to children at home. This change may increase the number of IRS poor child exemptions in households with children away from home both because of the additional children and because poverty thresholds are higher for larger size families.

The number of child exemptions reported on tax returns for families with incomes below the poverty threshold, like the number of food stamp recipients, is an imperfect measure of poverty for school-age children. Not all people file tax returns, especially those with very low incomes or income mostly from nontaxable sources. In addition, “income” as defined on tax returns does not include all the sources of income that are used in the official measure of poverty, and tax filing units are not totally consistent with the Census Bureau's definition of families. Moreover, the address on a tax return does not always correspond to a filer's residential address. Also, from evaluation, the Census Bureau has found some differences between states in the completeness of the tax files that it obtains from IRS that may affect use of the data in models (Cardiff, 1998). These differences occur because the Census Bureau receives an early version of the data for each tax filing year from the IRS. Nonetheless, tax information, like counts of

food stamp recipients, is a useful variable to develop predictions of poverty for school-age children.

The CPS provides more timely data than the decennial census; however, SAIPE estimates of poor school-age children for counties that are derived from the CPS will not be current. Thus, the Census Bureau released SAIPE county estimates of poor school-age children in early 1997 for income year 1993 and county and school district estimates in early 1999 for income year 1995, for use for Title I allocations for the next two school years (1999-2000 and 2000-2001 in the case of the 1995 school district estimates).¹⁴

The reason for the lag between the income reference year and the year of release of estimates is that most data used in the county model are not available until 2 years after the period to which they refer. The time lag is also caused by the decision to use 3 years of CPS data in the Census Bureau's model to improve the precision of the estimates. The lag means that the estimates will not capture any changes in the extent and distribution of poverty among school-age children that may have occurred since the year to which they apply.

Published CPS data indicate that poverty among school-age children for the nation as a whole increased from 17.4 percent in 1989 to 20.1 percent in 1993 and then declined to 15.5 percent in 1999 (U.S. Census Bureau, 1990:Table 18; 1995a:Table 8; 2000:Table 2).¹⁵ Data are not available for 2000, and no data are readily available with which to estimate the changes in the distribution of poverty among school-age children across states and counties that may have occurred since the last release of estimates.

The panel was asked to evaluate the accuracy of the updated county-level estimates that the Census Bureau was able to produce with available data. The panel addressed the question of the accuracy of the estimates for the estimation year (1993, 1995), not the question of how well the estimates for 1993 (1995) predict poverty among school-age children in 1997 (1999). It should be a priority for research and development by the Census Bureau to determine ways to reduce the lag between the time period of the estimates and the year of their release (see Chapter 9).