National Academies Press: OpenBook
« Previous: 3. ALTERNATIVE COUNTY MODELS
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

4
Evaluations

The development of model-based estimates for small areas is a major research and development effort for which extensive evaluation is required. For updated estimates of poor school-age children for counties, a thorough assessment of all aspects of the estimation procedure is necessary so that policy makers can have confidence in using the estimates for allocating federal Title I education funds to counties. That assessment includes both an evaluation of a given model and comparisons with alternative models. Because there are no absolute criteria for what are acceptable evaluation results, a way to determine if the performance of a model can be improved is to examine alternative models. Such comparisons may indicate changes that would be helpful for a model; they may also suggest that an alternative model is preferable.

The Census Bureau's county estimates of poor school-age children are produced by using a county regression model, a state regression model, and county population estimates developed with demographic analysis techniques (see Chapter 2). A comprehensive evaluation for each of these components of the estimation procedure should include ''internal'' and "external" evaluations.

An internal evaluation is primarily an investigation of the validity of the underlying assumptions and features of a model. For a regression model, an internal validation is typically based on an examination of the residuals from the regression—the differences between the predicted and reported values of the dependent variable for each observation. In an external evaluation, the estimates from a model are compared with target or "true" values that were not used to develop the model. Ideally, internal evaluation of regression model output should precede external evaluation. If the assumptions required by a regression model

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

are not supported to a reasonable extent, then even a positive external evaluation would not justify the choice of the model. Changes made to a model to address concerns raised by an internal evaluation would likely improve its performance in an external evaluation. Both internal and external evaluations should be carded out for alternative models.

When the original 1993 county estimates of poor school-age children were provided to the panel, the Census Bureau had not had time to complete a full evaluation of them. Subsequently, the panel developed a set of evaluation criteria, and the panel and the Census Bureau conducted a series of internal and external evaluations. The focus of the evaluation effort was on alternative county models, particularly the assumptions underlying the regression equations and how the estimates of poor school-age children in 1989 from each model compared with 1990 census estimates. The state model and the county population estimates were examined as well, both directly and as they contribute to the county estimates of poor school-age children. The evaluations, which are described in this chapter, include:

  1. internal evaluation of the regression output for alternative county models estimated for 1993 and 1989;

  2. comparison of estimates of poor school-age children for 1989 from alternative county models with 1990 census estimates, a form of external evaluation;

  3. consideration of differences between the CPS and census measurement of income and poverty as a factor that could explain differences between model-based estimates and census estimates for 1989;

  4. examination of the original 1993 county estimates to identify possibly anomalous estimates that were then reviewed with knowledgeable local people, another form of external evaluation;

  5. evaluation of the state model, including examination of regression output, external evaluation in comparison with 1990 census estimates, and consideration of the state raking factors by which county model estimates are adjusted to make them consistent with the state model estimates; and

  6. evaluation of county population estimates for children aged 5-17 (see also Appendix B).

The internal evaluation of regression output and the comparison of model-based estimates of poor school-age children for 1989 with 1990 census estimates—evaluations (1) and (2) above—were carded out for the four single-equation county models that were considered serious candidates to produce revised 1993 county estimates of poor school-age children (see Chapter 3 and Appendices C and D):

  1. log number model (under 21), the original model that the Census Bu-

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

reau used to produce the original 1993 county estimates of poor school-age children;

  1. log number model (under 18), the revised model that the Census Bureau used to produce the revised 1993 county estimates of poor school-age children;

  2. log rate model (under 21); and

  3. log rate model (under 18).

In addition, the 1990 census comparisons (2) were performed for some other estimation procedures that rely much more heavily than do the four candidate models on estimates from the 1980 census (see below, "Comparisons with 1990 Census County Estimates").

The internal evaluation of regression output (1) and the comparison of estimates of poor school-age children for 1989 with 1990 census estimates (2) examined residuals and model differences from the census, respectively, for categories of counties. The following characteristics were used for categorizing counties: census division; metropolitan status of county; population size in 1990; population growth from 1980 to 1990; percent poor school-age children in 1980; percent Hispanic population in 1990; percent black population in 1990; for rural counties, persistent poverty from 1960 to 1990; for rural counties, economic type; percent group quarters residents in 1990; number of households in the CPS sample (or whether the county had sampled households); and (for 1990 census comparisons only) percent change in the proportion of poor school-age children from 1980 to 1990 (see details in Table 4-3, below).

INTERNAL EVALUATION: COUNTY MODEL REGRESSION OUTPUT

The first test of a regression model is that it perform well when evaluated internally, that is, for the set of observations for which it is estimated. The panel and the Census Bureau examined the underlying assumptions of the four candidate models through evaluation of the regression model output for 1989 and 1993.1 Although such an evaluation is not likely to provide conclusive evidence with which to rank the performance of alternative models, particularly when they use different transformations of the dependent variable, examination of the regression output is helpful to determine which models perform reasonably well.

1  

The evaluation of the county regression output pertains to the regression models themselves, that is, before the predictions are combined with the direct CPS estimates in a "shrinkage" procedure or raked to the estimates from the state model (see Chapter 2). For these models, the regression output comprises the model predictions for counties with at least one household with poor school-age children in the CPS sample. For the two log number models, the predictions are the log number of poor school-age children; for the two log rate models, the predictions are the log proportion of poor school-age children.

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

The assumptions investigated fall into two groups: assumptions concerning the functional form of the regression model and assumptions concerning the error distribution. Because properties of the error distribution affect the ability to fit a model, studies of these two types of assumptions are not entirely separable.2

The assumptions examined in the first group are linearity of the relationship between the dependent variable and the predictor variables; constancy of the assumed linear relationship over different time periods; and whether any of the included predictor variables are not needed in the model and, conversely, whether other potential predictor variables are needed in the model. The assumptions examined in the second group are normality (primarily symmetry and moderate tail length) of the distribution of the standardized residuals;3 whether the standardized residuals have homogeneous variances, that is, whether the variability of the standardized residuals is constant across counties and does not depend on the values of the predictor variables; and absence of outliers. Each assumption is discussed in terms of the methods used for evaluation and the results of the evaluation for the four candidate models.

Linearity

Linearity of the relationships between the dependent variable and the predictor variables was assessed graphically, by observing whether there was evidence of curvature in the plots of standardized residuals against the predictor variables in the model. In addition, plots of standardized residuals against CPS sample size and against the predicted values from the regression model were also examined for curvature.

The only evidence of nonlinearity is for the log number (under 21) model (a) for 1989. For that year, the standardized residuals appear to have a very modest curvature when plotted against the predicted values.

Constancy over Time

Constancy over time of the assumed linear relationship of the dependent and predictor variables was assessed through comparison of the regression coefficients on the predictor variables for 1989 and 1993. While major changes in

2  

These assumptions were also examined for the analogous 1990 census regressions. However, since the census equations only affected the weights for the weighted least squares regression and the extent of "shrinkage" in combining model estimates and direct estimates for counties with households in the CPS sample, analyses of the 1990 census regressions are not discussed here.

3  

The standardization of the residuals involved estimating the predicted standard errors of the residuals, given the predictor variables, and dividing the observed residuals by the predicted standard errors. The predicted standard error of the residual for a county is a function of the estimated model error variance and the estimated sampling error variance (see Belsley, Kuh, and Welsch, 1980).

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

economic conditions are expected to cause some changes in the coefficients, a relatively stable regression equation would be desirable.4

Table 4-1 shows the regression coefficients for the predictor variables for the four candidate models for 1989 and 1993. In the log number models (a, b) for 1989 and 1993, the coefficients for the three "poverty level" predictor variables—child exemptions reported by families in poverty on tax returns (column 1), food stamp recipients (column 2), and poor school-age children from the previous census (column 5)—are similar. There are substantial differences across the two time periods in the estimated coefficients for the other two variables—population (under age 21 or under age 18, column 3) and total number of child exemptions on tax returns (column 4). However, the sum of the these two coefficients is generally close to 0 in each model in each year. Because these two variables are highly positively correlated, the predictions from equations with a similar sum for the two coefficients will be similar.

The sum of all coefficients in each equation for models (a) and (b) ranges from 1.04 to 1.07 and is significantly greater than 1. A sum equal to 1 would mean that county population size itself has no effect on the estimated number of poor school-age children. Because the sum is greater than 1, the estimated number of poor school-age children is a larger percentage of the population in the larger counties. While this result is difficult to explain as a function of county size, it may be that size reflects the effects of variables not included in the models.

In the log rate models (c, d), the coefficients for the three "poverty rate" predictor variables—ratio of child exemptions reported by families in poverty on tax returns to total child exemptions (column 1), ratio of food stamp recipients to the total population (column 2), and ratio of poor school-age children from the previous census (column 4)—are all positive and about the same size.5 The coefficients for the ratio of total child tax exemptions to the population (under age 21 or under age 18, column 3) are negative, and there are substantial differences across the two time periods in the estimated coefficients. The sign of the related variable (total number of child tax exemptions) is generally negative in the log number equations. As in the log number equations, the coefficients in the log rate equations for population under 21 differ from the coefficients for population under 18.

4  

Because the county model is refit for each prediction year, constancy over time is not as important as it would be if the estimated regression coefficients from the model for one year were used for predictions for subsequent years. Nonetheless, it is disturbing for the regression coefficients to exhibit large, unexplained changes over time.

5  

The coefficients are also similar to the coefficients for the corresponding variables—number of child exemptions reported by families in poverty on tax returns, number of food stamp recipients and number of poor school-age children from the previous census—in the log number equations.

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

TABLE 4-1 Estimates of Regression Coefficients for Four Candidate County Models for 1989 and 1993

 

 

Predictor Variablesa

Model

Counties (Number)

1

2

3

4

5

(a) Log Number (under 21)

1989

1,028

0.52 (.07)

0.30 (.05)

0.76 (.22)

-0.81 (.22)

0.27 (.07)

1993

1,184

0.31 (.08)

0.30 (.07)

0.03 (.21)

0.03 (.21)

0.40 (.09)

(b) Log Number (under 18)

1989

1,028

0.50 (.06)

0.23 (.05)

1.79 (.27)

-1.80 (.27)

0.32 (.07)

1993

1,184

0.38 (.08)

0.27 (.07)

0.65 (.24)

-0.59 (.24)

0.34 (.09)

 

 

Predictor Variablesb

(c) Log Rate (under 21)

1989

1,028

0.32 (.07)

0.29 (.04)

-0.73 (.19)

0.40 (.07)

 

1993

1,184

0.23 (.08)

0.31 (.06)

-0.07 (.18)

0.41 (.09)

 

(d) Log Rate (under 18)

1989

1,028

0.29 (.07)

0.26 (.04)

-1.13 (.24)

0.43 (.07)

 

1993

1,184

0.26 (.08)

0.30 (.06)

-0.42 (.20)

0.38 (.09)

 

NOTES: All predictor variables are on the logarithmic scale for numbers and rates. Standard errors of the estimated regression coefficients are in parentheses. The four models were estimated for each year with maximum likelihood. The original 1994 population estimates were used for the 1993 models; 1990 census population estimates were used for the 1989 models.

a predictor variables: (1) number of child exemptions reported by families in poverty on tax returns; (2) number of people receiving food stamps; (3) population (under age 21 or under age 18); (4) total number of child exemptions on tax returns; (5) number of poor school-age children from previous (1980 or 1990) census.

b Predictor variables: (1) ratio of child exemptions reported by families in poverty on tax returns to total child exemptions; (2) ratio of people receiving food stamps to total population; (3) ratio of total child exemptions on tax returns to population (under age 21 or under age 18); (4) ratio of poor school-age children from previous (1980 or 1990) census.

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

Inclusion or Exclusion of Predictor Variables

The possibility that one or more predictor variables should be excluded from a model was assessed by looking for insignificant t-statistics for the estimated values of individual regression coefficients.6 The need to include a predictor variable, or possibly to model some categories of counties separately, was assessed by looking for nonrandom patterns, indicative of possible model bias, in the distributions of standardized residuals displayed for the various categories of counties.7

The only predictor variables with nonsignificant t-statistics are the population under age 21 (column 3 in Table 4-1) and total child exemptions on IRS income tax returns (column 4) for the log number (under 21) model (a) in 1993, and the ratio of child tax exemptions to the population under age 21 (column 3) for the log rate (under 21) model (c) in 1993. All other regression coefficients are significantly different from 0 at the 5 percent level. Application of Akaike's information criterion (AIC) confirmed the superiority of using the population under age 18 as a predictor variable in preference to the population under age 21 in the log number model. (The test was not performed for the log rate model.)

For most ways of categorizing counties, the standardized residuals do not exhibit systematic patterns. The exceptions are that all four models in 1989 tend to overpredict poor school-age children in counties with a high percentage of Hispanic residents (i.e., the model estimates are somewhat higher than the CPS direct estimates for these counties relative to other counties) and that the log number (under 21 and under 18) models (a, b) tend to overpredict poor school-age children in counties that are in metropolitan areas but are not the central county in the area.

Normality

The normality of the standardized residuals was evaluated through use of Q-Q plots, which match the observed distribution of the residuals with the theoretical distribution, and other displays of the distribution. All four models exhibit some skewness in their standardized residuals, with the log rate models (c, d) showing somewhat more skewness than the log number models (a, b). For none of the models does the skewness appear sufficiently marked to be a problem.

6  

Although the performance of a predictive regression model is best assessed in terms of the joint impact of the predictor variables, examining the individual predictor variables can suggest ways in which a model might be improved.

7  

The distributional displays examined for this and other model assumptions were box plots.

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

Homogeneous Variances

The homogeneity of the variance of the standardized residuals was assessed using a variety of statistics and graphical displays (see Appendix C). Examination of them clearly demonstrates some variability in the size of the absolute standardized residuals as a function of the predicted value (number or proportion of poor school-age children) and the CPS sample size for all four models. With regard to CPS sample size, one would expect the standardized residual variance to remain constant over the distribution of CPS sample size; however, it increases with increasing CPS sample size.

The heterogeneity of the variance of the residuals should be investigated because it suggests that there may be a problem with the model specification or in the assumptions that were used to calculate the standardized residuals. However, adjusting a model to remove this type of heterogeneity is likely to have only a small effect on the estimated regression coefficients or the model estimates. The effect on estimates of poor school-age children would stem from: a shift in the weights assigned to each county in fitting the regression model, which would very likely result in only a modest change in the estimated regression coefficients; and a change in the weight given to the direct estimates, which could have an appreciable effect only on the estimates for counties with large CPS sample sizes.

Outliers

The existence of outliers was evaluated through examination of plots of the distributions of the standardized residuals and plots of standardized residuals against the predictor variables and through analysis of patterns in the distribution of the 30 largest absolute standardized residuals for the various categories of counties. However, it is difficult to evaluate the evidence for outliers that results from a least squares model fit, which has the property that it may miss influential outliers. In addition, since the four models arc so similar and make use of the identical data, it is unlikely that an observation that was a marked outlier for one model would not also be a marked outlier for the other models.

An examination of the distributions of the standardized residuals indicates that none of the four models is especially affected by outliers, although the 1993 estimates have more outliers than the 1989 estimates, and nonrural counties and metropolitan counties that are not central counties have somewhat more outliers than other categories of counties. This analysis is only a start. It would be useful, using other statistics and various graphical techniques, to identify the counties that are not well fit by robustly estimated versions of these models in order to determine any characteristics that outlier counties have in common.

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

Summary

The panel concludes that the analysis of the regression output for the four candidate county models for 1989 and 1993 largely supports the assumptions of the models: there is little evidence of important problems with the assumptions. The analysis does not strongly support one model over another, although it does support use of the population under age 18 instead of the population under age 21 as a predictor variable in the log number model.

All of the models exhibit a few common problems. First, they all behave somewhat differently for larger urban counties and counties with large percentages of Hispanic residents than for other counties. The differences are not pronounced, but research should be conducted to determine possible ways to modify the models to eliminate or reduce this problem. Second, all models show evidence of some variance heterogeneity with respect to both CPS sample size and poverty rate. This problem can likely be eliminated or reduced by research currently ongoing at the Census Bureau to develop direct estimates of the county-level sampling variances (see Chapter 6).

EXTERNAL EVALUATION: COMPARISONS WITH 1990 CENSUS COUNTY ESTIMATES

For external evaluation, the panel and the Census Bureau compared the estimated number and proportion of poor school-age children for 1989 for the four candidate models with 1990 census estimates.8 The evaluation examined the overall difference between the estimates from a model and the census and the differences for groups of counties categorized by various characteristics.

Evaluation by comparison with the 1990 census is not ideal because the census estimates are not true values. They are affected by sampling variability and population undercount; also, the census measurement of poverty differs from the CPS measurement in ways that are not fully understood (see National Research Council, 1997:Ch. 2, App. B; see also the Census Bureau's web site: http://www.census.gov/hhes/www/saipe93/inputs/cencpsdf.html). In addition, there is only one census-based validation opportunity: because of the lack of IRS and food stamp program data for counties for 1979, it is not possible to evaluate model-based estimates by comparison to the 1980 census. Reliance on a single

8  

The county estimates reflect the effects of the state model and the county population estimates as well as the county regression model, but the differences in model performance vis-à-vis the census in the evaluation are due to the particular form of the county model.

The models for which the 1990 census comparisons were performed were estimated with the method of moments. Maximum likelihood was used to estimate the log number (under 18) model (b) for the revised 1993 county estimates of poor school-age children. The differences in the estimates from the two techniques are small.

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

validation using the 1990 census is a problem because a model may perform better or worse in any one validation than it would on average over multiple validations. For this reason, if it were possible to compare model estimates with census or other estimates for 1993 instead of 1989, the results might turn out differently. Nonetheless, in the absence of other means of external validation, the panel and the Census Bureau relied heavily on the 1990 census comparisons to understand the performance of alternative models.

Evaluation by comparison with the 1990 census is intended to assess the accuracy of model estimates for the prediction year (i.e., 1989). The evaluation does not address the issue that model-based estimates are likely to be used for Title I allocations several years later. It would be useful to conduct research to reduce the time lag between the prediction year for model-based estimates and the year for Title I allocations to the extent possible (see Chapter 6).

The 1990 census estimates that are used in the comparisons are ratio adjusted by a constant factor to make the census national estimate of poor school-age children equal the 1989 CPS national estimate. This adjustment removes the difference of about 5 percent between the CPS and census estimates of total poor school-age children for 1989. Consequently, the differences between a model and the 1990 census in estimating poor school-age children for groups of counties can be interpreted as differences in shares. This feature is useful because the Title I allocation formula distributes funding as shares (percentages) of a fixed total dollar amount.

In addition to the four candidate models, the 1990 census comparisons were performed for four estimation procedures that rely much more heavily on 1980 census estimates. Given the substantial changes in the number and proportion of poor school-age children between the 1980 and 1990 censuses (see National Research Council, 1997:8-9), one would expect these procedures to perform less well than the candidate models in predicting poverty for school-age children in 1989.9 In a period of less pronounced change, one or more of them might perform relatively well. The census comparisons were done for the following procedures:

  1. Stable shares procedure, in which the county estimates of poor school-age children for 1989 are the 1980 census estimates for 1979 after ratio adjustment to make the 1980 census national estimate equal the CPS national estimate for 1989. This simple procedure assumes no change over the decade in each county's share of the total number of poor school-age children nationwide: this is the same assumption that underlies previous practice for Title I allocations, in

9  

Although the interval was only 4 years instead of 10, substantial changes in the number and proportion of poor school-age children also occurred between 1989 and 1993 (see National Research Council, 1997:10-13).

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

which estimates from the decennial census were used in the formulas each year until the results from the next census became available. 10

  1. Stable shares within state procedure, in which the county estimates of poor school-age children for 1989 are the 1980 census estimates for 1979 after raking the estimates for the counties in each state to the estimates from the Census Bureau's state model for 1989. (The national raking employed in the state model also adjusts the total to equal the CPS national estimate for 1989.) This procedure assumes no change over the decade in each county's share of the total number of poor school-age children in its state.

  2. Stable rates within state procedure (with conversion), in which the county estimates of poor school-age children for 1989 are developed by converting 1980 census estimates of the proportions of poor school-age children for 1979 to estimated numbers by use of 1990 county population estimates of total school-age children 5-17 and then raking the estimated numbers to the Census Bureau's state model estimates for 1989.

  3. Averaging procedure, in which the county estimates of poor school-age children for 1989 are developed from an average of estimates from the 1980 census and the log number (under 21) model (a) for 1989. 11

The rest of this section first discusses overall absolute differences from the 1990 census estimates for the four candidate models and the four procedures that rely more heavily on the 1980 census. It then discusses differences for categories of counties for the four candidate models and two of the procedures: the stable shares procedure and the averaging procedure. Differences for categories of counties for the other two procedures, which are intermediate in their reliance on 1980 census estimates, are provided in Appendix D.

10  

However, the estimates from the 1990 census that were previously used for Title I allocations were not adjusted to the current CPS national estimate of poor school-age children, which could affect the allocations for some counties. For example, some counties might meet the threshold test for a concentration grant if the census estimates were adjusted to the current CPS national estimate but not if the estimates were unadjusted.

11  

More precisely, the estimates are developed by averaging the proportions of poor school-age children from the 1980 census and the log number (under 21) model (a) for 1989, converting the estimates to numbers by use of 1990 county population estimates of total school-age children, and making an overall ratio adjustment to the CPS national estimate for 1989.

This procedure is analogous to the panel's recommendation for averaging 1990 census and 1993 model-based estimates for use in Title I allocations for the 1997-1998 school year. However, the panel's recommendation did not include raking the average estimates to the CPS national estimate of poor school-age children in 1993 (see National Research Council, 1997:38).

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

Absolute Differences Between Model and Census County Estimates

Table 4-2 presents measures of the overall absolute difference between the model-based county estimates and the 1990 census county estimates of poor school-age children in 1989 for the four candidate models and the four procedures that rely more heavily on the 1980 census. If the 1990 census estimates are reasonably accurate, a good model will produce estimates that differ little from the census estimates, and the absolute differences will be less than for other reasonable models. Also, a good model will perform significantly better than a simple procedure that relics heavily on the previous census.

Column 1 of Table 4-2 is the average absolute difference for county estimates of the number of poor school-age children in 1989, measured as the sum for all counties of the absolute difference (ignoring the direction of the difference) between the model estimate and the 1990 census estimate for each county, divided by the total number of counties. Column 2 of Table 4-2 is the average proportional absolute difference for county estimates of the number of poor school-age children, measured as the sum for all counties of the absolute difference between the model estimate and the 1990 census estimate as a proportion of the census estimate for each county, divided by the total number of counties and expressed as a percentage. Column 3 is the average proportional absolute difference for county estimates of the proportion of poor school-age children.

The measure in column I assesses the difference between a model and the 1990 census in terms of numbers of poor children; the measures in columns 2 and 3 assess the difference in terms of percentage errors for counties. To illustrate the difference between absolute and proportional absolute differences, consider two counties, one with an estimated 10,000 poor school-age children from the census and an estimated 9,600 poor school-age children from the model and the other with an estimated 1,000 poor school-age children from the census and an estimated 1,400 poor school-age children from the model. The absolute difference in the number of poor school-age children is the same for both counties (400), but the proportional absolute difference is only 4 percent for the first county and 40 percent for the second.

From a national perspective, it can be argued that absolute differences are more important for effective Title I allocations because Title I funds are primarily distributed in proportion to the number of children in a county; therefore, the amount of funds that are misallocated depends primarily on the number of children rather than the percentages by county. For example, an error of 5 percent in the number of school-age children in poverty in a large county could correspond to tens of thousands of children and have more impact on the allocation of funds than errors of 5 percent in several smaller counties. However, from the county perspective, proportional errors are also important. Ideally, a model will perform well on both types of measures.

The panel draws several conclusions from Table 4-2:

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

TABLE 4-2 Comparison of Model Estimates and Other Procedures with 1990 Census County Estimates of the Number and Proportion of Poor Related Children Aged 5-17 in 1989

 

Average Absolute Difference:

Average Proportional Absolute Difference, in Percent

Model

1 Number of Poor Children 5-17a

2 Number of Poor Children 5-17b

3 Proportion of Poor Children 5-17c

Candidate Models

(a) Log number (under 21)

272

15.4

16.4

(b) Log number (under 18)

268

16.4

17.7

(c) Log rate (under 21)

275

17.5

17.1

(d) Log rate (under 18)

283

18.8

18.6

Procedures that Rely More Heavily on the 1980 Census

(i) Stable shares

570

30.1

N.A.

(ii) Stable shares within state

380

27.1

N.A.

(iii) Stable rates within state, with conversion

381

26.2

N.A.

(iv) Average of 1980 census and 1989 log number (under 21) model (a)

286

19.0

N.A.

NOTES: The census estimates are controlled to the CPS national estimate for 1989. See text for definitions of models and measures; N.A.: not available.

a The formula where there are n counties (i), is .

b The formula is .

c The formula is .

SOURCE: Data from the Bureau of the Census.

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
  • The performance of the four candidate models is similar, which is not surprising, given that they are variations of the same basic formulation. Thus, the range of the average absolute difference in the estimated number of poor school-age children (column 1) is from 268 children (model b) to 283 children (model d). The average county had about 2,500 poor school-age children for 1989, so that the average absolute difference ranges from 10.7 to 11.3 percent. The range of the average proportional absolute difference in the estimated number of poor school-age children (column 2) is somewhat larger, from 15.4 percent (model a) to 18.8 percent (model d).

  • The log number models (a, b) have somewhat lower average absolute differences for estimates of numbers of poor school-age children than do the log rate models (c, d). This is expected because the estimates from the log rate models must be converted to numbers by use of population estimates of total school-age children, which themselves contain error (see below, ''Use of Postcensal Population Estimates''). It was expected for the same reason that the log number models would have higher average absolute differences for estimates of proportions of poor school-age children than would the log rate models because population estimates must be used to convert the estimated numbers from the log number models to estimated proportions. However, model (a) shows lower and model (b) shows not appreciably higher average proportional absolute differences for estimates of poverty rates compared with the better log rate model (c)—see column 3 of Table 4-1.

  • The four candidate models substantially outperform the three procedures (i-iii) that rely solely or largely on 1980 census data. For example, the largest average absolute difference for the four candidate models is 283 poor school-age children (11% of the average number) for the log rate (under 18) model (d), while the smallest average absolute difference for procedures (i-iii) is 380 poor school-age children (15% of the average number) for the procedure that assumes stable poverty shares within state (ii). The differences are even somewhat larger for the average proportional absolute difference for estimates of the number of poor school-age children: 18.8 percent for the worst candidate model, model (d), compared with 26.2 percent for the best procedure of these three, the procedure that assumes stable poverty rates within state with conversion (iii).

  • The four candidate models also perform better than the procedure (iv) that averages 1980 census estimates with estimates from the log number (under 21) model (a) for 1989, although the differences are not large.

Category Differences in Numbers of Poor School-Age Children

Table 4-3 shows the difference in the number of poor school-age children from the 1990 census for categories of counties for each of the four candidate models and two of the procedures that rely more heavily on the 1980 census—the stable shares procedure (i) and the averaging procedure (iv). The measure shown

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

TABLE 4-3 Comparison of Model Estimates with 1990 Census County Estimates of the Number of Poor School-Age Children in 1989: Algebraic Difference by Category of County (in percent)

 

Model

Other Procedures

Category

Log Number Under 21 (a)

Log Number Under 18 (b)

Log Rate Under 21 (c)

Log Rate Under 18 (d)

Stable Shares (i)

Average of Census and (a) (iv)

Number of Countiesa

Census Divisionb

New England

-2.9

-2.9

-2.9

-2.9

35.9

7.8

67

Middle Atlantic

-2.8

-2.8

-2.8

-2.8

27.1

4.4

150

East North Central

-0.2

-0.2

-0.2

-0.2

-2.8

-5.6

437

West North Central

1.7

1.7

1.7

1.7

-1.8

-2.1

618

South Atlantic

0.5

0.5

0.5

0.5

14.8

8.1

591

East South Central

-4.5

-4.5

-4.5

-4.5

14.1

2.1

364

West South Central

-2.7

-2.7

-2.7

-2.7

-18.1

-6.3

470

Mountain

4.3

4.3

4.3

4.3

-23.2

-3.1

281

Pacific

6.5

6.5

6.5

6.5

-21.3

0.2

163

Metropolitan Status

Central county of metropolitan area

2.4

1.6

-0.1

-0.5

-1.6

0.4

493

Other metropolitan

-6.6

-5.0

5.1

6.3

3.2

3.4

254

Nonmetropolitan

-4.2

-2.8

-0.3

0.4

3.3

-1.4

2394

1990 Population Size

under 7,500

-9.0

-2.3

-1.9

2.3

16.5

1.3

525

7,500-14,999

-4.4

0.5

2.5

5.5

10.9

2.2

630

15,000-24,999

-5.1

-2.6

0.3

1.9

6.2

-0.6

524

25,000-49,999

-4.2

-2.9

0.6

1.3

2.4

-1.3

620

50,000-99,999

-3.5

-5.1

-1.2

-2.3

-2.5

-3.3

384

100,0000-249,999

-1.8

-4.4

-1.8

-3.5

-4.9

-3.3

259

250,000 or more

3.3

3.2

0.5

0.5

-0.6

1.8

199

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

 

Model

Other Procedures

Category

Log Number Under 21 (a)

Log Number Under 18 (b)

Log Rate Under 21 (c)

Log Rate Under 18 (d)

Stable Shares (i)

Average of Census and (a) (iv)

Number of Countiesa

1980 to 1990 Population Growth

Decrease of more than 10.0%

-1.9

0.6

-3.4

-1.9

9.1

-3.4

444

Decrease of 0.1-10.0%

-0.6

-0.5

-1.9

-1.8

7.5

-2.7

972

0.0-4.9%

-2.8

-2.8

-3.2

-3.1

11.0

-0.2

547

5.0-14.9%

0.0

-1.0

0.2

-0.6

6.1

2.1

620

15.0-24.9%

7.7

5.8

5.5

4.6

-12.8

2.4

260

25.0% or more

-4.0

-1.4

1.7

3.1

-21.2

1.0

292

Percent Poor School-Age Children, 1980

Less than 9.4%

-4.0

-4.5

0.0

0.2

2.4

-1.1

516

9.4-11.6%

-0.5

-1.0

-1.6

-1.8

-9.9

-3.6

524

11.7-14.1%

3.6

2.3

1.8

1.0

-4.2

0.2

530

14.2-17.2%

0.9

1.2

-1.2

-1.4

-5.0

-1.8

523

17.3-22.3%

1.8

1.7

0.3

-0.1

10.7

4.2

519

22.4-53.0%

-2.2

0.8

1.3

2.8

12.3

4.1

523

Percent Hispanic, 1990

0.0-0.9%

-3.4

-3.3

-1.6

-1.5

10.7

0.2

1770

1.0-4.9%

0.5

0.1

0.4

0.1

0.2

-0.4

847

5.0-9.9%

-1.4

-0.6

-1.1

-0.8

6.7

1.7

193

10.0-24.9%

2.2

1.8

0.7

0.5

-5.7

0.1

181

25.0-98.0%

3.9

4.6

2.2

2.7

-16.8

-0.4

150

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

 

Model

Other Procedures

Category

Log Number Under 21 (a)

Log Number Under 18 (b)

Log Rate Under 21 (c)

Log Rate Under 18 (d)

Stable Shares (i)

Average of Census and (a) (iv)

Number of Countiesa

Percent Black, 1990

0.0-0.9%

-1.2

0.3

3.9

4.9

-3.7

-0.5

1446

1.0-4.9%

-0.7

-2.0

1.3

0.5

-6.3

-2.9

615

5.0-9.9%

-2.9

-2.5

-0.7

-0.6

-8.4

-1.8

294

10.0-24.9%

2.0

1.2

-1.0

-1.3

-2.6

0.2

381

25.0-87.0%

1.0

1.7

-1.8

-1.4

16.5

3.7

405

Persistent Rural Poverty, 1960-1990c

Rural, not poor

-4.0

-3.7

-1.2

-1.0

0.1

-3.4

1740

Rural, poor

-5.0

-2.1

0.7

2.1

9.8

1.2

535

Not classified

1.7

1.2

0.3

0.0

-1.2

0.7

866

Economic Type, Rural Countiesc

Farming

-5.5

-2.5

-1.6

0.7

13.2

1.1

556

Mining

-10.7

-5.1

-6.3

-3.6

-8.9

-10.6

146

Manufacturing

-6.2

-5.9

-1.7

-1.0

12.1

-0.2

506

Government

2.1

-1.3

6.3

3.2

-0.9

0.0

243

Services

-3.9

-3.0

-1.8

-1.2

-5.8

-4.3

323

Nonspecialized

-3.7

-1.0

-0.1

1.4

2.2

-1.5

484

Not classified

1.7

1.2

0.3

0.0

-1.2

0.7

883

Percent Group Quarters Residents, 1990

Less than 1.0%

-6.7

-2.7

2.0

4.7

-1.4

0.3

545

1.0-4.9%

0.3

0.7

-0.3

0.1

-0.4

0.1

2187

5.0-9.9%

2.3

-4.4

0.5

-5.2

7.8

-0.8

299

10.0-41.0%

14.2

-3.2

7.4

-7.5

1.8

-2.2

110

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

 

Model

Other Procedures

Category

Log Number Under 21 (a)

Log Number Under 18 (b)

Log Rate Under 21 (c)

Log Rate Under 18 (d)

Stable Shares (i)

Average of Census and (a) (iv)

Number of Countiesa

Status in CPS, 1989-1991

In CPS sample

1.4

1.0

-0.2

-0.5

-0.6

0.5

1028

In CPS, no poor children 5-17

-2.6

-1.9

7.3

7.8

10.0

5.9

246

Not in CPS sample

-4.1

-2.8

-0.1

0.6

0.6

-2.3

1867

Change in Poverty Rate for School-Age Children, 1980-1990

Decrease of more than 3.0%

7.5

10.4

16.2

18.1

51.6

30.0

536

Decrease of 0.1-3.0%

2.1

1.9

3.1

2.9

29.2

12.1

649

0.0-0.9%

-2.6

-0.8

-0.4

0.5

4.3

3.1

272

1.0-3.4%

3.8

2.2

3.4

2.6

-5.1

0.2

621

3.5-6.4%

-1.2

-2.4

-3.8

-4.3

-14.3

-8.3

532

6.5-38.0%

-7.2

-5.2

-8.7

-7.8

-25.2

-14.5

523

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

NOTES: The census estimates are controlled to the CPS national estimate for 1989. The algebraic difference by category is the sum for all counties in a category of the algebraic (signed) difference between the model estimate of poor school-age children and the 1990 census estimate for each county, divided by the sum of the census estimates for all counties in the category. See text for definitions of models.

a 3,141 counties are assigned to a category for most characteristics; 3,135 counties are assigned to a category for 1980-1990 population growth and 1980 percent poor school-age children; 3,133 counties are assigned to a category for 1980-1990 percent change in poverty rate for school-age children.

b Census division states:

New England: Maine, New Hampshire, Vermont, Massachusetts, Rhode Island, Connecticut

Middle Atlantic: New York, New Jersey, Pennsylvania

East North Central: Ohio, Indiana, Illinois, Michigan, Wisconsin

West North Central: Missouri, Minnesota, Iowa, North Dakota, South Dakota, Nebraska, Kansas

South Atlantic: Delaware, Maryland, District of Columbia, Virginia, West Virginia, North Carolina, South Carolina, Georgia, Florida

East South Central: Kentucky, Tennessee, Alabama, Mississippi

West South Central: Arkansas, Louisiana, Oklahoma, Texas

Mountain: Montana, Idaho, Wyoming, Colorado, New Mexico, Arizona, Utah, Nevada

Pacific: Washington, Oregon, California, Alaska, Hawaii

c The Economic Research Service, U.S. Department of Agriculture, classifies rural counties by 1960-1990 poverty status and economic type. Counties not classified are urban counties and rural counties for which a classification could not be made.

SOURCE: Data from the Bureau of the Census.

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

is the algebraic difference by category, which is the sum for all counties in a category of the algebraic (signed) difference between the model estimate of poor school-age children and the 1990 census estimate for each county, divided by the sum of the census estimates for all counties.12 Counties are grouped into five or six categories for each of 11 characteristics—those that were considered in the assessment of the county model regression output discussed above. 13

The measure in Table 4-3 expresses model-census differences for groups of counties in terms of numbers of poor children, similar to the overall average absolute difference in column 1 of Table 4-2. However, the category difference is expressed as an algebraic measure in which positive differences (overpredictions) within a category offset negative differences (underpredictions). The measure is intended to identify instances of potential bias in a model's predictions. For example, the model may over (under) predict, on average, the number of poor school-age children in larger counties relative to smaller counties. 14

If the census estimates are a reasonably accurate standard for comparison, sizable category differences between model and census estimates that are not explained by differences in the CPS and census measurement of poverty or another reason would be disturbing. They would indicate that the errors in the model estimates are not random errors (which occur in any set of estimates), but occur in part because the model systematically over (under) predicts poverty in certain types of counties. Indeed, bias, in terms of over (under) prediction for different types of counties, is arguably more important than the overall absolute difference in evaluating a model that is used repeatedly because there is the risk

12  

The formula for counties (i) in each category (j) is Σi(Ymodelij-Ycensusij)/ΣiYcensusij.

13  

In addition to the algebraic difference for each category for the four candidate models and four procedures, Appendix D shows for each of them the average proportional algebraic difference—that is, the category difference expressed in terms of percentage errors for counties instead of poor children (see Tables D-1 and D-2). Differences between the two measures can help identify particular types of counties within a category for which a model performs less well than others.

14  

However, an apparent bias identified in a single validation may be a one-time discrepancy that will not occur in other years for which a model is estimated. The panel and the Census Bureau considered another type of external validation to try to identify systematic or persistent prediction biases, in which estimates from the four candidate models of poor school-age children for categories of counties for 1989 and 1993 would be compared to weighted CPS direct estimates for those categories for the two periods, using 3 years of CPS data to form the weighted estimates in each case. (This analysis is not the same as the analysis of regression output described above, in which the residual from the model estimate for each county with sampled households in the CPS is compared to the direct estimate on the log scale.) However, there was not enough time to complete the analysis.

Comparisons with weighted CPS direct estimates have the advantage that they can be performed for multiple years. They have the disadvantage that the sample size for CPS estimates, even aggregated for 3 years, is small for some categories of counties, which makes the comparisons more uncertain than the 1990 census comparisons. Also, in analyzing the CPS comparisons, one must bear in mind that the model estimates are raked to the state estimates, which are developed from a single year of the CPS.

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

that the bias will operate for the same areas on each occasion.15 Although one would not want to use a model that had a large overall absolute difference from the standard of comparison, a model that performed somewhat worse in overall terms but exhibited fewer and less severe biases than another model would be preferable to it.

The panel draws several general conclusions from Table 4-3 about the performance of alternative county models in predicting numbers of poor school-age children for categories of counties:

  • The performance of the four candidate models is similar. However, the log number (under 18) model (b) performs somewhat better than the log rate (under 21) model (c), which in turn performs better than the other two, the log number (under 21) model (a) and the log rate (under 18) model (d).

Performance in this instance is evaluated principally in terms of the spread among the differences for categories of counties (the spread between the largest positive and negative category differences for a characteristic). A better performing model has a narrower spread for a greater number of characteristics than other models. As an example (see Table 4-3), the spread among the category differences for counties classified by percent group quarters residents is 5.1 percentage points for model (b), 7.7 percentage points for model (c), 12.2 percentage points for model (d), and 20.9 percentage points for model (a).

Also entering into the panel's judgment is consideration of the magnitude and pattern of differences: a better performing model has smaller differences from the census and exhibits fewer obvious patterns across categories than other models. Continuing with the same example from Table 4-3, there is no pattern to the category differences for counties classified by percent group quarters residents for model (b), whereas model (a) exhibits a strong monotonic pattern in which the number of poor school-age children is overpredicted for counties with higher percentages of group quarters residents relative to counties with lower percentages. Also, the magnitude of the category differences for counties classified by percent group quarters residents is small for model (b)—no difference is larger than 5 percent in either direction. In contrast, the category differences for model (a) are as high as 14 percent for one of the categories.

  • There are characteristics for which some or all models exhibit poor performance in terms of the spread between the largest and smallest category differences, the pattern of the differences across categories, or the magnitude of the differences (see below, "Category Differences for Specific Characteristics"). There are also some characteristics for which all four models perform well: percent poor school-age children in 1980; percent black population in 1990; and whether a rural county was persistently poor from 1960 to 1990.

15  

A search for potential biases is also important to identify possible approaches to model improvement.

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
  • The four candidate models perform better on most characteristics than the four procedures that rely more heavily on the 1980 census. This is generally true, as discussed below, even for characteristics on which the candidate models perform poorly. However, the averaging procedure (iv), which averages 1980 census estimates and estimates from model (a), performs reasonably well for many characteristics. In contrast, the stable shares procedure (i), which simply ratio adjusts the 1980 census estimates to the CPS national estimate for 1989, performs substantially worse than all of the models and other procedures on almost every characteristic.

Category Differences for Specific Characteristics

Category differences from the 1990 census estimates are discussed below for characteristics for which Table 4-3 shows that some or all four candidate models exhibit poor performance in comparison with the census in estimating the number of poor school-age children: percent change from 1980 to 1990 in the poverty rate for school-age children; percent population growth from 1980 to 1990; 1990 population size; percent Hispanic population in 1990; percent group quarters residents in 1990; and census division.

Percent Change from 1980 to 1990 in Poverty Rate for School-Age Children

All four candidate models show a pronounced pattern of overpredicting the number of poor school-age children in counties that experienced the greatest decline in the poverty rate for school-age children from 1980 to 1990 and, conversely, underpredicting the number of poor school-age children in counties that experienced the greatest increase in the poverty rate for school-age children in that period. The category differences are smaller for the log number models (a, b) than for the log rate models (c, d): the spread between the largest positive and largest negative differences is 15-16 percentage points for models (a) and (b) and 25-26 percentage points for models (c) and (d).

One would not expect any of the candidate models to perform particularly well in predicting the number of poor school-age children for the counties at the extremes of the distribution of change in the poverty rate from 1980 to 1990. This variable is closely related to the variable that the models are trying to estimate, and the process of fitting a regression line to all of the data will generally not result in good predictions for the extreme values of the distribution. In other words, one would expect the models to perform less well for counties that experienced the largest changes (increase or decrease) in the poverty rate for school-age children.

Despite the large differences for some categories of this characteristic, however, the four candidate models perform substantially better than the procedures

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

that rely more heavily on the 1980 census—see Table 4-3. (See also Figure 4-1, which shows the category differences for percent change in the school-age poverty rate from 1980 to 1990 for the log number (under 21) model (a), the log number (under 18) model (b), the stable shares procedure (i), and the averaging procedure (iv).) The stable shares procedure performs very poorly: because it assumes the same proportional distribution of poor school-age children in 1989 as in 1979 (from the 1980 census), by definition it Will miss any change in poverty rates that occurred over time. The procedure (iv) that averages the estimates from the 1980 census and the log number model (under 21) for 1989 performs better than the stable shares procedure but not nearly as well as the four candidate models (two not shown).

Percent Population Growth from 1980 to 1990

All four candidate models tend to overpredict the number of poor school-age children in counties that experienced larger population increases from 1980 to 1990 relative to counties that experienced smaller increases or declines in population. The exception to a generally monotonic pattern is that the four models underpredict the number of poor school-age children for counties that experienced population increases of 25 percent or more relative to counties that experienced increases of 15-25 percent. The log number (under 21) model (a) has the largest spread in category differences for this characteristic of the four candidate models—12 percentage points between the largest positive and negative differences.

The stable shares estimation procedure (i) performs very poorly on this characteristic. In contrast to the four candidate models, it overpredicts the number of poor school-age children in counties that experienced declines or smaller increases in population from 1980 to 1990 relative to counties that experienced larger population increases. The spread between the largest positive and negative category differences for the stable shares procedure is 32 percentage points. The averaging procedure (iv) exhibits small differences for population growth categories (see Figure 4-2).

1990 Population Size

The four candidate models vary in their performance for counties classified by population size. The log number (under 21) model (a) tends to overpredict the number of poor school-age children in larger size counties relative to smaller size counties. The log number (under 18) model (b) and the log rate (under 21) model (c) do not show a particular pattern to the category differences for this characteristic, and the category differences are not large. The four candidate models perform better than the stable shares model (i), which relies solely on 1980 census data. However, the model (iv) that averages 1980 census estimates with

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

FIGURE 4-1 Change in poverty rate for school-age children, 1980-1990: Category differences from the 1990 census.

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

FIGURE 4-2 Population growth, 1980-1990: Category differences from the 1990 census.

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

estimates from the log number (under 21) model (a) for 1989 performs reasonably well in predicting numbers of poor school-age children for county population size categories (see Figure 4-3).

Percent Hispanic Population in 1990

All four candidate models tend to overpredict the number of poor school-age children in counties with larger percentages of Hispanics relative to counties with smaller percentages, but the spread between the largest positive and negative differences is small. When the category differences are measured in proportionate terms for counties instead of in terms of numbers of poor school-age children, the models tend to underpredict the number of poor school-age children in counties with larger percentages of Hispanics (see Appendix D). The different patterns of the two category difference measures suggest that the models may perform differently for small counties with many Hispanics (primarily rural border counties) and large counties (cities).

The stable shares procedure (i), which relies solely on the 1980 census estimates, performs poorly on this characteristic. However, the averaging procedure (iv) performs reasonably well (see Figure 4-4).

Percent Group Quarters Residents in 1990

The four candidate models vary in their performance for counties classified by percentage of group quarters residents. The log number (under 21) model (a) substantially overpredicts the number of poor school-age children in counties with larger proportions of group quarters residents relative to other counties. The log rate (under 21) model (c) shows a similar but less pronounced pattern of category differences. The log rate (under 18) model (d) shows the opposite pattern, in which it underpredicts the number of poor school-age children in counties with larger proportions of group quarters residents relative to other counties. In contrast, the category differences for the log number (under 18) model (b) are small and do not show a pronounced pattern across categories of this characteristic.

When the evident bias in predicting the number of poor school-age children in counties relative to their percentage of group quarters residents was discovered in the first round of evaluations of model (a), the Census Bureau developed model (b) to ameliorate the problem, with the desired result. The reasoning was as follows. In model (a), the two predictor variables—total child exemptions (assumed to be under age 21) from IRS tax records and the population estimate of the under 21 age group—are used together to estimate the number of people under age 21 in families that do not file tax returns. These families are assumed to be poorer, on average, than families that file tax returns. As can be seen from

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

FIGURE 4-3 Population size, 1990: Category differences from the 1990 census.

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

FIGURE 4-4 Percent Hispanic population, 1990: Category differences from the 1990 census.

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

Table 4-1, the regression coefficients for these two variables are of similar magnitude but of opposite sign.

However, in counties with large percentages of group quarters residents under age 21, primarily college students and military personnel, the relationship between the IRS variable and the population estimate may be distorted. To the extent that college students and military personnel under age 21 live in a county that is not the same as the county in which their parents reside or file tax returns on their own behalf, they will not be recorded as child exemptions in their county of residence. Consequently, there will be an overestimate of the number of people under age 21 in families that do not file returns in these counties and a corresponding overestimate, through the model, of the number of school-age children in poverty.

Model (b) replaces the population estimate for the under 21 age group as a predictor variable with the population estimate for the under 18 age group. This change not only eliminates the pattern of overpredicting the number of poor school-age children as a function of the percentage of group quarters residents that is so pronounced in model (a), but it also causes model (b) to perform better than model (a) on a number of other characteristics (e.g., population size). For reasons that are not clear, the under 18 formulation does not improve the performance of the log rate model; in fact, the log rate (under 18) model (d) generally performs worse than the log rate (under 21) model (c).

Interestingly, the procedures that rely more heavily on the 1980 census (i-iv)—even the stable shares procedure—perform reasonably well in predicting the number of poor school-age children for counties categorized by percent group quarters residents (see Figure 4-5).

Census Division

All four candidate models show differences from the census for counties categorized by census division. In particular, the four models overpredict the number of poor school-age children in counties in the West (in the Mountain Division and, particularly, in the Pacific Division) relative to counties in other areas. The spread between the largest positive and negative differences is 11 percentage points.

Because the county estimates from the four candidate models are raked to the state estimates from the Census Bureau's state model, category differences on this characteristic must be attributable to the state model.16 The state model

16  

The category differences are the same for all four candidate models because they axe raked to the same set of state estimates; see Table 4-3. The average proportional category differences shown in Appendix D vary somewhat because they are calculated relative to each county's 1990 census estimated number of poor school-age children before being summed (see Table D-2).

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

FIGURE 4-5 Percent group quarters residents, 1990: Category differences from the 1990 census.

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

needs to be more fully investigated to determine if category differences by area of the country are random occurrences in a single year or whether they persist across years, and why (see below, ''State Model''). Yet the state raking procedure, which is done for the four candidate models and for the procedures that assume stable shares within state and stable rates within state (ii, iii), results in substantially better performance on this characteristic than the stable shares procedure (i). The averaging procedure (iv), which partly reflects the effects of the state raking, also performs better than the stable shares procedure (see Figure 4-6).

Differences in Proportions of Poor School-Age Children

Examining the data for category differences in estimates of proportions of poor school-age children similar to those in Table 4-3, the panel reaches the same conclusions. Comparisons were performed only for the four candidate models, not for the other procedures.

First, the performance of the four candidate models is similar. Second, the two models that performed best in estimating the number of poor school-age children—log number (under 18) model (b) and log rate (under 21) model (c)—also perform best in estimating the proportion of poor school-age children. However, model (c) performs slightly better than model (b) in estimating proportions, while model (b) performs slightly better than model (c) in estimating numbers of poor school-age children. This reversal is expected because the use of population estimates for children aged 5-17, which themselves contain errors, to convert estimated numbers to estimated proportions from the log number models puts these models at a disadvantage for comparisons of proportions. Conversely, the use of population estimates for children aged 5-17 to convert estimated proportions to estimated numbers from the log rate models puts these models at a disadvantage for comparisons of numbers (see below, "Use of Postcensal Population Estimates").

Poverty rates (proportions poor) of school-age children enter the Title I allocation formulas as thresholds, so the panel and the Census Bureau examined the correspondence between each of the four candidate models and the 1990 census in classifying counties and school-age children into three poverty rate categories: 0 to 15 percent; 15 to 30 percent; and 30 percent or higher. A poverty rate of 15 percent or higher is an eligibility threshold for concentration grants; 15 percent and 30 percent poverty rates are thresholds for hold-harmless provisions of the allocation formulas (see Table 4-4); no comparisons were performed for the other procedures.

When there are two poverty rate categories, 0 to 15 percent and 15 percent or higher, each of the four candidate models performs equally well, assigning about 87 percent of the counties, which include about 92 percent of the poor school-age children, to the same category as the 1990 census (column 5, top half and bottom half of Table 4-4). When there are three poverty rate categories, 0 to 15 percent,

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

FIGURE 4-6 Census division: Category differences from the 1990 census.

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

TABLE 4-4 Agreement Between Model Estimates for 1989 and 1990 Census County Estimates for Proportions of School-Age Children in Poverty in 1989 (in percent)

 

Counties

 

Model and Census Estimate in Same Poverty Rate Category

Percent in Agreement

 

Under 15%

15% or More

15-30%

30% or More

Under 15% and 15% or More

Under 15%, 15-30%, and 30% or More

Model

 

(1)

(2)

(3)

(4)

(5)

(6)

(a) Log Number (Under 21)

30.1

57.0

39.2

11.4

87.1

80.7

 

(b) Log Number (Under 18)

30.5

57.1

38.4

11.9

87.6

80.8

 

(c) Log Rate (Under 21)

28.8

58.6

40.1

12.9

87.4

81.8

 

(d) Log Rate (Under 18)

28.4

58.6

39.5

13.0

87.0

80.9

 

 

School-Age Children

 

Model and Census Estimate in Same Poverty Rate Category

Percent in Agreement

Model

Under 15%

15% or More

15-30%

30% or More

Under 15% and 15% or More

Under 30%, 15-30%, and 30 or More

(a) Log Number (Under 21)

40.7

51.0

39.9

7.3

91.7

87.9

 

(b) Log Number (Under 18)

40.9

50.7

38.5

7.3

91.6

86.7

 

(c) Log Rate (Under 21)

40.1

51.5

41.0

7.6

91.6

88.7

 

(d) Log Rate (Under 18)

40.3

51.1

40.3

7.5

91.4

88.1

 

NOTE: Census estimates are controlled to the CPS national estimate for 1989.

SOURCE: Data from the Bureau of the Census.

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

15 to 30 percent, and 30 percent or higher, each of the four candidate models assigns about 81 percent of the counties, which include about 88 percent of the poor school-age children, to the same category as the 1990 census (column 6, top half and bottom half of Table 4-4).

Summary

Keeping in mind the limitations of a single census-based validation opportunity, the panel concludes that the four candidate models perform substantially better in predicting the number and proportion of poor school-age children for counties for 1989 than the simple stable shares procedure (i), which relies solely on estimates from the previous (1980) census and the current (1989) CPS national total. Using the state model to rake the 1980 census county estimates for consistency with updated estimates of poor school-age children in each state, as is done in procedures (ii) and (iii), is an improvement over procedure (i). However, the four candidate models, which use a county regression model together with the state model, perform much better than procedures (ii) and (iii). Finally, the four candidate models perform better in many respects than procedure (iv), which averages the 1980 census estimates and the 1989 estimates from the log number (under 21) model (a), although this averaging procedure shows good performance on some characteristics. Overall, the comparisons with the procedures that rely more heavily on the 1980 census provide significant evidence in favor of a model-based approach for updated estimates of poor school-age children and against using estimates that derive solely or largely from the previous census.

The panel further concludes that, while the performance of the four candidate models in comparison with the 1990 census is broadly similar, when consideration is given to measures of overall absolute difference and differences for categories of counties, for estimates of numbers and estimates of proportions of poor school-age children, the log number (under 18) model (b) and the log rate (under 21) model (c) perform better than the other two. Comparing models (b) and (c), model (b) performs somewhat better, and the Census Bureau used this model to prepare the revised county estimates of poor school-age children in 1993. The comparisons also identify areas of performance of model (b) that deserve further examination in an ongoing research program to continue to improve model-based estimates of poverty for small geographic areas (see Chapter 6).

CPS-CENSUS DIFFERENCES

A possible explanation of some of the category differences identified in the 1990 census comparisons just described may be, not that a model is in error, but that measurement of poverty differs systematically between the census and the CPS because of the many differences in data collection procedures (see National

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

Research Council, 1997:App. B). The Census Bureau performed chi-square tests to determine if there were significant differences between estimates from the March 1990 CPS and the 1990 census of the number of school-age children and the number and proportion poor in this age group in 1989 for county groupings (Fay, 1997).17 More specifically, the tests determined if the ratios of the CPS and census estimates for categories of a characteristic, such as county population size, were significantly different from each other. The characteristics tested were those examined in the 1990 census comparisons.

The tests generally show inconclusive results. However, there is some evidence that, when compared with the 1990 census, the March 1990 CPS estimates higher numbers and proportions of poor school-age children in metropolitan counties and larger size counties relative to medium-size counties. (CPS estimates for small-size counties have low reliability because of the relatively small proportion of the population in such counties and the small number of these counties in the CPS sample.) Also, while not significant, a pattern is evident in which the March CPS, when compared with the 1990 census, tends to estimate higher numbers and proportions of poor school-age children in counties with higher percentages of Hispanic population. These results for population size and percent Hispanic population parallel the results from the 1990 census comparisons described above. They suggest that at least some portions of the category differences for the candidate models for these two characteristics arise from differences in the CPS measurement of poverty and are not due to model error as such. Whether similar CPS-census differences would be present for 1993 is, of course, not known.

EXTERNAL EVALUATION: LOCAL ASSESSMENT OF 1993 COUNTY ESTIMATES

The panel performed another type of external evaluation of the original 1993 county estimates of poor school-age children—the use of local knowledge.18 Using the original 1993 model estimates for all 3,143 counties in the United States, the analysis first sought to identify groups of counties for which the 1993 estimates seemed unusually high or low in relation to prior levels and trends (e.g., from 1980 to 1990) in the number and proportion of poor school-age children and known social and economic trends for these groups of counties. Then, local informants—including staff and members of local councils of government, eco-

17  

The March 1990 CPS estimates for the categories involved are direct estimates produced using the CPS weights.

18  

This evaluation was carried out at the University of Wisconsin-Madison by Dr. Paul Voss, a member of the panel, with the assistance of Richard Gibson and Kathleen Morgen (see Voss, Gibson, and Morgen, 1997).

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

nomic development authorities, welfare agencies, state demographic units, state data centers, and other agencies—were contacted to obtain their assessment of the reasonableness of the implied trends in poverty for school-age children given their knowledge of local socioeconomic conditions19

County Analysis

Changes in the number and proportion of poor school-age children implied by the 1993 estimates were examined for counties categorized by several characteristics, including: population size and metropolitan status; population change; percent immigrant population; college-dominated counties; reservation and Native American counties; for nonmetropolitan counties, whether predominantly agricultural; and several classifications by geographic location (e.g., state and the regions identified by the U.S. Department of Agriculture).

The analysis identified a number of categories of counties for which further investigation of the reasonableness of the 1993 estimates seemed warranted:

  • Large metropolitan central city counties had a high implied percentage change in the number of school-age children in poverty between 1989 and 1993-42 percent. This change declined systematically with decreasing size for metropolitan counties and continued to decline to the most remote, rural nonmetropolitan counties, for which the implied change in the number of school-age children in poverty was-6 percent.

  • Counties with higher levels of international immigration had higher implied increases in the number and proportion of poor school-age children.

  • Counties with higher percentages of Native Americans had lower implied increases in the number and proportion of poor school-age children. There was no particular pattern for counties with reservations.

  • Farm counties had an implied decline in the number and proportion of poor school-age children, while nonfarm metropolitan counties had an implied increase.

  • When the country was divided into the 26 regions identified by the U.S. Department of Agriculture, several regions were identified on the extremes of change in the number and proportion of poor school-age children. High implied increases were found in the Northern Metropolitan Belt, the Florida Peninsula, the Southwest, Northern New England, Mohawk New York and Pennsylvania, Lower Great Lakes Industrial, Southern Piedmont, and the Northern Pacific Coast. Small implied increases were found in the Central Corn Belt, the Southern Appalachian Coal Region, the Coastal Plain Cotton Region, the Northern Great Plains,

19  

The discussion refers to "implied" trends because the Census Bureau's county model is not designed to directly estimate change over time.

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

and the Rockies, Mormon, Columbia River Region. The single region with an implied decrease in the number and proportion of poor school-age children was the Mississippi Delta.

Some of these implied changes are apparently related to the general effect of population size, discussed above. However, the findings in this regional analysis, in particular, suggested which states and counties to follow up in discussions with local officials.

Local Input

When counties that share certain characteristics appeared also to share a common pattern of change in the number and proportion of poor school-age children, a variety of individuals with local knowledge were contacted. Initially, 70 individuals associated with state data centers or state data center affiliate units were contacted; they provided a series of responses and referrals to other state and local officials. In addition, 26 states that appeared to have a sizable number of counties that shared a common implied trend in poverty for school-age children were targeted for intensive contact.

The nature of responses varied considerably. In some states, the original 1993 county estimates released by the Census Bureau had not been examined, and there appeared to be little interest in discussing them. In other states, the estimates had been looked at, but the general admonitions about standard errors that accompanied their release had dampened interest in studying them in detail. In contrast, several states had carded out in-depth analyses of the estimates. Of the 26 states targeted for intensive follow up, 8 provided detailed explanations (supported by examples) of trends suggested by the original 1993 county estimates, and 7 more states provided in-depth responses supported by their own analyses.

Almost every state agency contacted expressed specific doubts about the original 1993 estimates for one or more counties—too high here, too low there. In general, however, there was no consensus that the trends implied by the original 1993 county estimates were wrong, even in states for which large numbers of counties experienced apparent declines in the number and proportion of poor school-age children. Of the 26 states, 21 provided explanations as to why the original 1993 estimates appeared to show poverty trends in a specific direction or why the direction of change is too difficult to know. The most common explanations included comments about the size of the county, its rural agricultural nature, the fact that it is a diverse metropolitan county, immigration from abroad, and economic growth or economic decline. Occasionally, reference was made to a military base, an Indian reservation, or a university as an explanation for an apparent trend in poverty for school-age children. In three states, concern was

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

expressed about the role of food stamp program data in the estimation model, as these data were deemed to be unreliable.

In summary, a high level of concern was expressed by individuals with local knowledge about the statistical reliability of the original 1993 county estimates, which is largely due to the Census Bureau's own cautions in this regard, coupled with specific county estimates that seem on the basis of local knowledge to be highly doubtful. These concerns notwithstanding, no categories of counties were identified that experienced apparent trends in the number and proportion of poor school-age children between 1989 and 1993 that were not accepted by local informants. Although the trends for a few counties were not accepted locally, the analysis found no strong indicators of potential bias for groups of counties sharing common characteristics in the county model.

STATE MODEL

The state model plays an important role in the production of county estimates of poor school-age children. Evaluations conducted of the state model include an internal evaluation of the regression output for 1989 and 1993 and an external evaluation through comparing 1989 estimates from the model with 1990 census estimates of the proportion of poor school-age children by state. The results in each case, which are summarized below, support the use of the model. However, the state model evaluations have been more limited than the county model evaluations, as alternative state model formulations have not been evaluated explicitly. Further evaluation of the state model would be useful (see Chapter 6), particularly to examine the relationship between the state and county models and what factors may underlie the variations in the state estimates from the state model and the state estimates formed by summing the estimates from the county models (see below, "State Raking Factors").

State Model Regression Output

The state regression model is a poverty rate model with the variables not transformed (see Chapter 2). The analysis of the regression output for the state model for 1989 and 1993 examined the same assumptions that were examined for the four candidate county models. The analysis is somewhat less informative for the state model than for the county models for two reasons. First, few explicit alternatives were developed for the state regression model. A log rate model was developed, and comparisons of that model with the rate model demonstrated that the log rate formulation had no particular advantage. Work was also started on a multivariate state model.20 However, no formal analysis of regression output

20  

A multivariate formulation could be advantageous not only for the state model, but also for the county model, as an extension of the bivariate formulation for which initial development work was carried out (see Chapter 6).

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

was performed for either of these alternatives or for other alternatives that were explored early on in the development of the model. Second, there are more than 3,000 counties but only 50 states, and states vary much less than counties with respect to poverty rates and other characteristics. Hence, comparisons for categories of states are less informative than comparisons for categories of counties, and some categories of states do not contain enough states for analysis.

Nonetheless, examination of the regression output for the state model helps assess the validity of its assumptions. Overall, the analysis finds strong support for the assumptions underlying the state model (see below); there is no evidence of significant problems with the model formulation (although there may be other models that fit just as well).

Linearity Plots of standardized residuals against the four predictor variables in the state model—the proportion of child exemptions reported by families in poverty on tax returns, the proportion of people receiving food stamps, the proportion of people under age 65 who did not file a tax return, and a residual from the analogous regression equation using the previous census as the dependent variable—support the assumption of linearity. Furthermore, the standardized residuals, when plotted against the model predicted values, provide no evidence of the need for any transformation of the variables. This result helps justify the decision not to use the log transformation of the proportion of poor school-age children as the dependent variable.

Constancy over Time Table 4-5 shows the regression coefficients for the predictor variables for the state model for 1989 and 1993. The coefficients for all four poverty rate predictor variables are positive in both years. Generally, the coefficients are similar for 1989 and 1993, with the exception that the coefficient of the residual from the previous census (column 4) is large and significant for 1993 but fairly small and not significant for 1989.

Inclusion or Exclusion of Predictor Variables The standardized residuals for the state regression model were grouped into four categories for each of the following characteristics: census region; 1990 population size; 1980 to 1990 population growth; percent black population in 1990; percent Hispanic population in 1990; percent group quarters residents in 1990; and percent poor school-age children in 1979 (from the 1980 census). The distributions of the standardized residuals for each category were then displayed using box plots. For none of these box plots was there an obvious pattern to the standardized residuals across categories. The model slightly overpredicts the proportion of poor school-age children for large states in 1993 (i.e., the model estimates are somewhat higher than the CPS direct estimates for large states relative to other categories), but this pattern is not evident in 1989. The model also slightly overpredicts the proportion of poor school-age children for states with a moderate percentage of Hispan-

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

TABLE 4-5 Estimates of Regression Coefficients for the State Model for 1989 and 1993

 

Predictor Variablesa

Year

1

2

3

4

1989

0.53

(.10)

0.57

(.21)

0.33

(.10)

0.37>

(.32)

1993

0.31

(.11)

0.98

(.22)

0.52

(.13)

1.36

(.39)

NOTES: All predictor variables are in terms of rates. Standard errors of the estimated regression coefficients are in parentheses.

a Predictor variables: (1) ratio of child exemptions reported by families in poverty on tax returns to total child exemptions; (2) ratio of people receiving food stamps to total population; (3) ratio of people under age 65 who did not file an income tax return to total population under age 65; (4) residual from a regression of poverty rates for school-age children from the prior decennial census (1980 or 1990) on the other three predictor variables.

ics in 1989 and slightly underpredicts the proportion of poor school-age children for states in the West census region in 1993, but in neither case is the pattern observed for the other year. Therefore, there is no strong reason to suggest that these variables need to be incorporated in the state regression model.

Normality, Homogeneous Variances, and Outliers The distribution of the standardized residuals from the state regression model appears to follow a normal distribution. Also, although there is less information available for the state model than for the county regression models, the residual plots and the box plots of the distributions of the standardized residuals against the categories of states show little evidence of any heterogenous variance. Finally, there is no evidence of outliers from examination of the residual plots or displays of the distributions of the standardized residuals from the state regression model.

1990 Census Comparisons

Fay and Train (1997) compare 1989 estimates of the proportion of poor school-age children from the state model with 1990 census estimates. They find that the differences between the model and census estimates are much smaller than the differences between the 1989 CPS direct estimates and the 1990 census

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

estimates and considerably smaller than the differences between the 1980 census estimates and the 1990 census estimates. These findings, which are presented graphically in Fay and Train (1997), support the use of a model-based approach to producing updated state estimates of poor school-age children instead of relying on estimates from the previous census or from the CPS alone. Similarly, a formal hypothesis test performed for the state model (Fay, 1996) supports the conclusion that the model-based estimates for 1993 are preferable to estimates from the 1990 census.21 Comparable evaluations have not been performed for alternative state models or for categories of states.

State Raking Factors

The final stage in producing updated estimates of the number of poor school-age children for counties is to rake the estimates from the county model for consistency with the estimates from the state model. The raking procedure is clearly beneficial to the county estimates. Thus, the 1990 census comparisons for the two procedures (ii and iii) that raked the 1980 census estimates to the estimates from the state model for 1989 showed better performance than the stable shares procedure (i), which did not entail raking. Also, an evaluation that was performed of the original log number (under 21) model (a) found a smaller overall average absolute difference from the 1990 census when the county model estimates were raked to the state model estimates for 1989 than when the county model was used without raking (National Research Council, 1997:31).

On the assumption that a county model is performing well, one would expect the state raking factors to be tightly distributed around 1.0—that is, one would expect relatively minor differences between the estimates for states formed by summing the county estimates before raking and the estimates from the state model. However, the raking factors vary considerably across states. For example, the log number (under 18) model (b), which shows somewhat less variation than the other three candidate models, has raking factors that range from 0.86 to 1.31 in 1989 (two-thirds falling between 0.96 and 1.17) and from 0.84 to 1.29 in 1993 (two-thirds falling between 0.98 and 1.15). This degree of variation suggests that there may be state effects not captured in the county model, which, in turn, could possibly affect the behavior of the model in estimating poor school-age children for counties within states. Also, the state model uses 1 year of CPS data, while the candidate models use 3 years: this difference could contribute to the variation in raking factors and also to the fact that they average greater than 1.

Implementation of a fixed state effects formulation of the county model in which state indicator variables are included as predictor variables in the regres-

21  

The test assumes that the objective is to predict poverty rates that reflect the CPS measurement of poverty and not the decennial census measurement.

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

sion (see Chapter 3) widened rather than narrowed the range of the state raking factors. Technical reasons having to do with the transformation of the predicted log values of poor school-age children to estimated numbers probably explain the increased variation in the state raking factors with a fixed state effects model. However, further investigation of the state raking factors and how to account for state effects in the county model should be topics for research in the near term (see Chapter 6). The investigation should include consideration of whether there is any feature of the state model that might explain the variation in the raking factors.

USE OF POSTCENSAL POPULATION ESTIMATES

The process for producing updated estimates of school-age children in poverty at the county level and the use of those estimates in the Title I allocation formulas require population totals by age in noncensus years for two purposes: as a variable in the county regression equation (population under age 18 or under age 21, depending on the model), and as the basis for computing the estimated number or proportion of poor school-age children, depending on the model specification (log number or log rate). Population totals by age are also required for the state model.

The Census Bureau's log number (under 18) model (b) produces estimates of the number of poor school-age children in each county. Because the Title I allocations require both numbers and proportions, the Census Bureau provides the Department of Education with population estimates for the 5-17 age group to use as denominators for calculating the proportion of poor school-age children.22

The Census Bureau currently develops county age estimates within the framework of total population estimates for counties and age estimates for states (see Appendix B). Briefly, in a process that begins anew with each decennial census, total population estimates for counties are developed by updating the population estimates for the preceding year with data on births, deaths, net immigration from abroad, and net internal migration. (Net internal migration is estimated from a year-to-year match of federal income tax returns for people under age 65 and from the change in Medicare enrollment records for people aged 65 and over.) Estimates are developed separately for the population over and under

22  

The population estimates of school-age children that accompany the 1993 county model estimates pertain to July 1994. In addition, the Census Bureau makes available on its web site estimated proportions of poor school-age children in which the denominators are estimates of related children aged 5-17 in each county. These estimates are developed by adjusting the estimates from the Census Bureau's population estimates program for the noninstitutionalized population aged 5-17 on the basis of the ratio of related children aged 5-17 to noninstitutionalized children aged 5-17 for each county in the 1990 census.

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

age 65 in households and in group quarters. County total population estimates are aggregated to form state total population estimates.

State estimates by single years of age are developed by similar demographic methods, in which the preceding year's estimate for each cohort (single year of age) is updated with data on births, deaths, and migration. (For people under age 65, net internal migration is estimated from school enrollment data.) The state estimates for single ages are then raked to equal the state population totals. Finally, county estimates by age are developed by ratio-adjusting the 1990 census county age estimates to both the updated county total population estimates and the updated state age estimates in an iterative proportional fitting (raking) procedure. This procedure assumes that the age distribution of each county within a state changes in the same manner as that state's age distribution.

The Census Bureau has an active program to develop and review the performance of its demographically based population estimates, including evaluating the estimates at 10-year intervals by comparing them with the decennial census as a measure of the true values. These comparisons provide an indication of the differences, but they are not complete measures of accuracy and precision because the standard (i.e., the decennial census) itself is flawed, notably from net population undercount, which varies by age group across time and place (see Robinson et al., 1993).

The Census Bureau's methods and data for producing postcensal population estimates have generally improved over time, but three patterns of differences, which are practically inevitable, continue to affect the county and state estimates (see Davis, 1994). First, the proportional differences of the estimates in comparison with the census are larger on average for small areas than for large ones. Second, the proportional differences tend to be larger for areas in which the population is changing rapidly than for areas that are more stable. Third, the proportional differences for age groups tend to be higher than those for the total population.

The Census Bureau recently completed an evaluation of the county estimates of total population and children aged 5-17 by comparison with the 1990 census for all counties and for categories of counties similar to the categories used in the 1990 census model evaluations described above (see Appendix B). The procedure to develop updated estimates for counties by age for 1990 was to ratio adjust the 1980 census county age estimates to 1990 county total population estimates and 1990 state age estimates. The overall average proportional absolute difference in the 1990 county estimates of the population aged 5-17 was 6.3 percent, unweighted by county population size, and 4.9 percent, weighted by size. By comparison, the overall average absolute difference in the 1990 county estimates of the total population was 3.6 percent unweighted and 2.3 percent weighted.

Population size markedly affects the accuracy of the estimates for children aged 5-17. For counties with more than 1 million people in 1990, the average proportional absolute difference in the estimate for this age group was 5.2 per-

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

cent, but it was 12.4 percent for counties with fewer than 2,500 people. This relationship is expected given the likelihood that errors in the input data will be disproportionately greater for smaller counties than for larger counties.

In terms of bias measured by the average proportional algebraic difference for categories of counties, the population estimates procedure, in comparison with the 1990 census, tends to overestimate children aged 5-17 in larger counties relative to smaller counties and in metropolitan counties relative to nonmetropolitan counties. The estimation procedure also tends to underestimate children aged 5-17 in counties with larger percentages of group quarters residents, to overestimate children aged 5-17 in counties with larger percentages of blacks, and to underestimate children aged 5-17 in counties with larger percentages of Hispanics relative to other counties. However, the differences are small for each characteristic.

The issue in the context of Title I allocations is the extent to which differences from the census in the population estimates for children aged 5-17 affect the estimates of the proportion of poor school-age children from log number models (a, b), or how they affect the estimates of the number of poor school-age children from log rate models (c, d). In the aggregate, the use of population estimates to convert estimated numbers from log number models to estimated proportions adds about 1 percentage point to the overall average proportional absolute difference between the model estimates for 1989 and the 1990 census estimates (compare column 3 with column 2 of Table 4-2 for the two log number models). The use of population estimates to convert estimated proportions from log rate models to estimated numbers has even less effect overall (compare column 2 with column 3 of Table 4-2 for the two log rate models).

In addition, although a rigorous analysis was not done, there seems to be little systematic contribution of errors in the population estimates to category differences in the model estimates of poor school-age children from the 1990 census estimates (see Appendix D). For the three single-equation rate models that were examined for 1989 in the first round of evaluations, including the log rate (under 21) model (c), the use of population estimates instead of 1990 census estimates (''true values'') to convert estimated proportions to estimated numbers of poor school-age children worsened the performance of the models for some characteristics (e.g., by increasing the spread between the largest negative and positive category differences compared with the census), improved their performance for other characteristics, and made essentially no difference for other characteristics. None of the category differences between the model estimates of poor school-age children developed with population estimates and those developed with 1990 census estimates was large.

The evaluations of the effects of the population estimates on estimates of poor school-age children relate to a 10-year period: the population estimates for 1990 were developed on the basis of 1980 census data updated with other sources. The 1994 population estimates that are used to convert estimated numbers to

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×

estimated proportions of poor school-age children in 1993 from the log number (under 18) model (b) were developed on the basis of 1990 census data. Because of the 4-year instead of 10-year period for updating, it is likely that errors in the 1994 population estimates are smaller than errors in the 1990 population estimates and that they have even smaller effects on the estimates of the number and proportion of poor school-age children.

Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 33
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 34
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 35
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 36
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 37
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 38
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 39
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 40
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 41
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 42
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 43
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 44
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 45
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 46
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 47
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 48
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 49
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 50
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 51
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 52
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 53
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 54
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 55
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 56
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 57
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 58
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 59
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 60
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 61
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 62
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 63
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 64
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 65
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 66
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 67
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 68
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 69
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 70
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 71
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 72
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 73
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 74
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 75
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 76
Suggested Citation:"4. EVALUATIONS." National Research Council. 1998. Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations. Washington, DC: The National Academies Press. doi: 10.17226/6122.
×
Page 77
Next: 5. RECOMMENDATION FOR TITLE I ALLOCATIONS FOR THE 1998-1999 SCHOOL YEAR »
Small-Area Estimates of School-Age Children in Poverty: Interim Report 2, Evaluation of Revised 1993 County Estimates for Title I Allocations Get This Book
×
Buy Paperback | $58.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF
  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!