Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 93
4
Data Analysis and Results
T
o determine the suitability of the American Community Survey
(ACS) as a source of claiming percentages for reimbursement under
an ACS Eligibility Option (AEO) for universal free school meals,
the panel implemented the technical approach described in Chapter 3
and conducted extensive analyses of the ACS direct and model-based
estimates produced by the U.S. Census Bureau. This chapter describes the
principal results of these analyses and presents the panel's main conclu-
sions. Additional results from our analyses are reported in Appendix F.1
The chapter begins with an analysis of the differences between ACS
and administrative estimates, including consideration of the many rea-
sons why such differences might arise. The potential sources of differ-
ences include errors in each set of estimates. ACS estimates are subject not
only to sampling error but also to nonsampling error from, for example,
households not responding at all to the survey or responding incorrectly
by misreporting their incomes or whether they received benefits from the
Supplemental Nutrition Assistance Program (SNAP, formerly the Food
Stamp Program). Although not subject to sampling error, administrative
estimates reflect the effects of certification error, as discussed in Chapter 2,
as well as data entry, tabulation, and transmission error. Some differences
between the estimates are undoubtedly attributable to the use of survey
1To simplify an already complex set of analyses, the panel focused on school lunches. For
a district considering actual implementation of the AEO, it will be important to consider
breakfasts separately from lunches, given the different reimbursement rates for the two
programs.
93
OCR for page 94
94 USING ACS DATA TO EXPAND ACCESS TO THE SCHOOL MEALS PROGRAMS
versus administrative procedures, while others arise because the proce-
dures are intended to obtain different data. For example, the ACS collects
data on income received in the past 12 months on a rolling basis. Thus
households interviewed in January report on income received during the
period from the previous January through December, while households
interviewed in December report on income received during the period
from the previous December through November. In contrast, school meals
program applications obtain data on current monthly income, which
will typically be income for the month in which the application is being
completed or the previous month--probably July, August, or September
for most students. Even if the data obtained by the ACS and by program
applications are fully accurate, eligibility based on annual income can
be different from eligibility and certification based on monthly income.
Yet another difference is that the ACS records where students live, while
school meals program certification data are based on where students
attend school. In areas with school choice options, such as charter and
magnet schools or open enrollment policies, some students may not
attend their neighborhood school or even any school in the district in
which they reside. This phenomenon will be captured in the administra-
tive data but not the ACS data.
The second section of the chapter presents the panel's analysis of the
precision, intertemporal stability, and timeliness, as well as the general
relative performance, of the alternative estimates from the ACS, includ-
ing the 1-year, 3-year, 5-year, and model-based estimates. As discussed
in Chapter 3, stability in reimbursement is important to districts because
it facilitates budgeting and other planning activities involved in operat-
ing the school meals programs. Nonetheless, some instability in reim-
bursement occurs naturally under traditional operating procedures as a
result of changes in certification percentages and participation rates from
year to year due to ups and downs in the economy, outreach efforts by
school authorities, and other factors. However, basing reimbursements
on ACS estimates will introduce additional instability due to sampling
variability and other sources of error that cause estimates to fluctuate.
Because they are based on larger samples and average the data collected
in different years, 5-year ACS estimates will tend to be more precise and
stable than 3-year estimates, which will be more precise and stable than
1-year estimates. However, the precision and stability carry a cost: the
5-year estimates and, to a lesser degree, the 3-year estimates will be less
timely and less responsive to real changes in socioeconomic conditions.
The panel's analyses explored these trade-offs.
The panel also explored the role of participation--that is, the pur-
chase or free receipt of meals by students. Participation is important
because it is the basis for reimbursing districts for the meals they serve
OCR for page 95
DATA ANALYSIS AND RESULTS 95
under traditional operating procedures or Provisions 2 and 3. The ACS,
however, does not collect data on participation. It provides estimates of
eligibility, specifically the numbers and percentages of students eligible
for free, reduced-price, and full-price meals. Unless students in the three
eligibility categories participate at the same rate, which, generally, they
do not, the distributions across the categories of students and of meals
served will not be the same and may differ substantially. Thus claiming
percentages based entirely on the percentages of eligible students in each
category will differ from claiming percentages based on the percentages
of meals served in each category. In fact, with students eligible for free
and reduced-price meals participating at higher rates than students pay-
ing full price, claiming percentages based solely on ACS estimates of
eligible students--with no accounting for differences in participation--
could cause districts to be substantially under reimbursed should they
adopt the AEO. This effect could be at least partially mitigated, how-
ever, by the changes in participation that might occur under the AEO
with free meals being offered to all students, substantially lowering the
monetary cost of meals for those students formerly paying full price
and increasing their participation rates relative to other students. In the
third section of the chapter, the panel analyzes the role of participation
and the potential effect of offering free meals to all students under the
AEO. In Chapter 5, we propose an approach to implementing the AEO
that incorporates into the AEO claiming percentages not only the ACS
eligibility estimates but also the participation rates of students when all
are offered free meals.
DIFFERENCES BETWEEN ACS AND
ADMINISTRATIVE ESTIMATES
The panel compared ACS estimates of students eligible for school
meals by category (free, reduced price, full price) with administrative
data on students certified for each category. The administrative data are
from the Common Core of Data (CCD) for most of our analyses at the
district level. School districts report data for the CCD to state agencies,
which submit the data to the National Center for Education Statistics. For
our school-level analyses, the case study districts provided the adminis-
trative data directly to us at our request. As described in Chapter 3, the
administrative data are subject to error; thus, they are not a gold stan-
dard. However, they were the best standard available to us. Although we
generally characterize average differences between ACS and administra-
tive estimates as measures of systematic error in the ACS estimates, the
limitations of the administrative data should be kept in mind. Later in
this chapter, we explore the potential effects of certification error in the
OCR for page 96
96 USING ACS DATA TO EXPAND ACCESS TO THE SCHOOL MEALS PROGRAMS
administrative estimates on the differences between ACS and administra-
tive estimates.
The analyses presented in this chapter focus on those districts for
which the AEO is most relevant: the districts described as "very high
FRPL [free or reduced-price lunch]" and "high FRPL" in Chapter 3. A very
high FRPL district had at least 75 percent of its students eligible for free
or reduced-price meals according to the CCD in one or more school years
from 2004-2005 to 2009-2010. Although a high FRPL district never reached
that threshold, it did have at least 50 percent of its students eligible for
free or reduced-price meals during one or more of those years. For some
of our analyses of these districts, we present separate results for large,
medium, and small districts. The large districts have populations of at
least 65,000, and thus have 1-year ACS estimates as well as 3- and 5-year
estimates. The medium districts have populations of 20,000 to 65,000 and
have 3- and 5-year but not 1-year ACS estimates. The remaining districts,
with populations under 20,000, have only 5-year ACS estimates and are
designated as small. Although all districts included in our analyses have
model-based estimates, we focus in this section on the 1-, 3-, and 5-year
direct estimates from the ACS.
Systematic Differences Between ACS and Administrative Estimates
The panel's analyses revealed that ACS estimates differ systematically
from administrative estimates for districts that might be most interested
in the AEO. Figure 4-1 plots ACS and CCD estimates of the percentage
of students eligible for free meals in very high FRPL districts. The ACS
estimates are 5-year estimates for 2005-2009, and the CCD estimates are
for school year (SY) 2009-2010. Because the purpose of using ACS data is
to provide current estimates, we compare the most recent ACS estimates
with the most recent estimates from the CCD. Thus, the ACS 5-year esti-
mates for 2005-2009 are compared with the CDD estimates for SY 2009-
2010.2 Some of the observed average difference between these two sets
of estimates maybe attributable solely to their different reference periods
and the fact that the economy was worsening, resulting in an upward
trend in the percentage of students eligible for free or reduced-price
meals. From 2005 to 2009 according to the CCD, the percentage of stu-
dents eligible for free or reduced-price meals in very high FRPL districts
rose from 76.3 percent to 79.7 percent. For high FRPL and all districts,
2We follow the same principle with 3-year estimates, comparing the estimates for 2005-
2007, 2006-2008, and 2007-2009 with CCD estimates for school years 2007-2008, 2008-2009,
and 2009-2010, respectively.
OCR for page 97
DATA ANALYSIS AND RESULTS 97
100
Small
Medium
Large
80
ACS 5-Year Estimate
60
40
20
0
0 20 40 60 80 100
CCD Estimate
FIGURE 4-1 Comparison of ACS 5-year (2005-2009) and CCD (2009-2010) esti-
mates for very high FRPL districts: Percentage of students eligible for free meals.
NOTE: ACS = American Community Survey; CCD = Common Core of Data; FRPL
FIG4-1.eps
= free or reduced-price lunch.
SOURCE: Prepared by the panel.
this percentage increased from 52.9 percent to 59.8 percent and from 43.2
percent to 47.5 percent, respectively.3,4
3For
very high FRPL districts, the percentages of students eligible for free or reduced-price
meals were 76.3, 75.4, 75.3, 77.6, and 79.7 for 2005, 2006, 2007, 2008, and 2009, respectively,
according to the CCD. For the high FRPL districts, the corresponding percentages were
52.9, 53.1, 54.2, 56.3, and 59.8, and for all districts, they were 43.2, 43.3, 43.8, 45.3, and 47.5.
4Although the use of older data is a potentially serious limitation of the 5-year estimates
relative to the 1-year and even the 3-year estimates, we also compared the 5-year ACS esti-
mates with 5-year averages of CCD estimates to assess their differences when they include,
OCR for page 98
98 USING ACS DATA TO EXPAND ACCESS TO THE SCHOOL MEALS PROGRAMS
In Figure 4-1, the overwhelming majority of districts fall below the
45 ° line of equality between the estimates, indicating that the ACS iden-
tifies a smaller percentage of students as eligible for free meals relative
to the CCD. For many of these very high FRPL districts, the percentage
of students eligible for free meals according to the ACS is substantially
lower than the percentage based on the administrative data on certified
students.
In contrast, a different pattern pertains to the estimates of students eli-
gible for reduced-price meals. According to Figure 4-2,5 the ACS estimate
exceeds the CCD estimate for a majority of districts, but the difference
often is just a few percentage points. Many districts are clustered around
the line of equality between the ACS and administrative estimates for the
reduced-price category.
The scatter plots in Figures 4-1 and 4-2 suggest that for the typical
very high FRPL district, the ACS substantially underestimates the per-
centage eligible for free meals and slightly overestimates the percentage
eligible for reduced-price meals. The net effect of these patterns is that on
average, the ACS estimate is substantially less than the CCD estimate for
the percentage of students eligible for free or reduced-price meals and
for the blended reimbursement rate (BRR) based on eligible students, as
shown in Figures 4-3 and 4-4, respectively.
Tables 4-1, 4-2, 4-3, and 4-4 provide numerical estimates of the aver-
age differences between ACS and CCD eligibility percentages and BRRs. 6
The first column in the top panel of Table 4-1 pertains to 5-year estimates
for all very high FRPL districts and corresponds to the results presented
in principle, the same trend within the reference period of the estimates. The results of that
comparison are qualitatively the same as the results of our comparisons of 5-year ACS esti-
mates with 1-year CCD estimates. Although statistically significant for all types of estimates
and large for percentage free, percentage free or reduced price, and BRR, the differences,
of course, are smaller than those from our main comparisons because the differences based
on CCD 5-year averages ignore the loss of timeliness due to the use of older data by the
ACS 5-year estimates. Further details can be found in Appen dix F, which also presents a
comparison of 3-year ACS estimates with 3-year averages of CCD estimates.
5In this figure, the 5-year ACS estimates have a relatively large number of sampling
zeros because the percentage eligible for reduced-price meals is relatively small, and some
districts are small areas. One possible reason for zeros in the CCD data is that missing data
are recorded as zeros.
6For reasons given above, Tables 4-1, 4-2, and 4-4 compare the ACS 5- and 3-year estimates
with the CCD estimates for the most recent school year in the reference period of the ACS
estimates. Accordingly, the ACS 5-year estimates for 2005-2009 and 3-year estimates for
2007-2009, for example, are compared with the CCD estimates for SY 2009-2010. Appendix F
presents tables that compare the ACS 5- and 3-year estimates with 5- and 3-year averages of
CCD estimates for the same time periods. Such comparisons reflect differences when data
are aligned in time but do not reflect the loss of timeliness that would result from using the
multiyear estimates in the AEO.
OCR for page 99
DATA ANALYSIS AND RESULTS 99
60
50
40
ACS 5-Year Estimate
30
20
10
Small
Medium
Large
0
0 10 20 30 40 50 60
CCD Estimate
FIGURE 4-2 Comparison of ACS 5-year (2005-2009) and CCD (2009-2010) esti-
mates for very high FRPL districts: Percentage of students eligible for reduced-
price meals.
FIG4-2.eps
NOTES: This figure excludes two outliers. Both are small districts. One has a
CCD estimate of 2 percent and an ACS estimate of 78 percent, and the other has
a CCD estimate of 80 percent and an ACS estimate of 6 percent. ACS = American
Community Survey; CCD = Common Core of Data; FRPL = free or reduced-price
lunch.
SOURCE: Prepared by the panel.
in Figures 4-1 through 4-4. The last three columns in the top panel of Table
4-1 provide separate estimates for large, medium, and small districts, and
the bottom panel provides estimates of average ACS-CCD differences for
high FRPL districts. Tables 4-2 and 4-3 display average ACS-CCD differ-
ences for 3-year and 1-year ACS estimates, respectively. Table 4-2 includes
OCR for page 100
100 USING ACS DATA TO EXPAND ACCESS TO THE SCHOOL MEALS PROGRAMS
100
Small
Medium
Large
80
ACS 5-Year Estimate
60
40
20
0
0 20 40 60 80 100
CCD Estimate
FIGURE 4-3 Comparison of ACS 5-year (2005-2009) and CCD (2009-2010) es-
timates for very high FRPL districts: Percentage of students eligible for free or
reduced-price meals.
FIG4-3.eps
NOTE: ACS = American Community Survey; CCD = Common Core of Data; FRPL
= free or reduced-price lunch.
SOURCE: Prepared by the panel.
only large and medium districts because small districts do not have 3-year
ACS estimates. Similarly, Table 4-3 includes only large districts because
they are the only districts with 1-year ACS estimates. Table 4-2 provides
results for each of the three available sets of 3-year estimates (2005-2007,
2006-2008, and 2007-2009), and Table 4-3 provides results for each of the
five available sets of 1-year estimates. All differences in each of these
OCR for page 101
DATA ANALYSIS AND RESULTS 101
2.5 Small
Medium
Large
2.0
ACS 5-Year Estimate
1.5
1.0
0.5
0.5 1.0 1.5 2.0 2.5
CCD Estimate
FIGURE 4-4 Comparison of ACS 5-year (2005-2009) and CCD (2009-2010) esti-
FIG4-4.eps
mates for very high FRPL districts: BRR.
NOTES: This figure excludes three outliers, all of which are small districts with
ACS BRRs of $0.26. Their CCD BRRs are $1.50, $2.10, and $2.10. ACS = American
Community Survey; BRR = blended reimbursement rate; CCD = Common Core
of Data; FRPL = free or reduced-price lunch.
SOURCE: Prepared by the panel.
tables are statistically significant, that is, significantly different from zero.7
Table 4-4 summarizes the results in the other tables by averaging across
the three sets of 3-year estimates and the five sets of 1-year estimates.
7Statistical significance is determined by comparing the ratio of the average difference to
its estimated standard error with critical values from a standard normal distribution.
OCR for page 102
102 USING ACS DATA TO EXPAND ACCESS TO THE SCHOOL MEALS PROGRAMS
TABLE 4-1 Average Differences Between ACS 5-Year Estimates for
2005-2009 and CCD Estimates for 2009-2010
All Large Medium Small
Estimand Districts Districts Districts Districts
Very High FRPL Districts (1,641) (122) (227) (1,292)
Percentage free 21.7 19.5 20.4 22.2
Percentage reduced price 4.0 4.5 5.0 3.8
Percentage free or reduced price 17.8 15.0 15.4 18.4
BRR, $ 0.43 0.37 0.38 0.44
High FRPL Districts (4,214) (304) (710) (3,200)
Percentage free 10.8 13.6 12.1 10.3
Percentage reduced price 2.3 2.7 2.8 2.1
Percentage free or reduced price 8.5 11.0 9.3 8.1
BRR, $ 0.21 0.27 0.23 0.20
NOTES: All average differences are statistically significant (different from zero) at the 0.01
level. ACS = American Community Survey; BRR = blended reimbursement rate; CCD =
Common Core of Data; FRPL = free or reduced-price lunch.
SOURCE: Prepared by the panel.
For very high FRPL districts, several consistent patterns emerge from
these tables of average ACS-CCD differences:
· The average ACS estimate of the percentage of students eligible
for free meals is typically 15 to 22 percentage points lower than
the average CCD estimate.
· The average ACS estimate of the percentage of students eligible
for reduced-price meals is typically about 3 to 4 percentage points
higher than the average CCD estimate.
· The ACS's overestimation of the percentage eligible for reduced-
price meals is not sufficient to compensate for the underestima-
tion of the percentage eligible for free meals. Thus, the average
ACS estimate of the percentage eligible for either free or reduced-
price meals is typically 12 to 18 percentage points lower than the
average CCD estimate.
· For a BRR based on the distribution of students across categories,
the average ACS estimate is usually about $0.30 to $0.40 lower
than the average CCD estimate of roughly $2.10.
Qualitatively similar patterns are observed for average high FRPL
districts: overestimation of the percentage reduced-price-eligible, but
underestimation of the percentage free-eligible, the percentage free or
reduced-price-eligible, and the BRR. Also, all of the differences are statis-
OCR for page 103
TABLE 4-2 Average Differences Between ACS 3-Year Estimates and CCD Estimates for Last School Year in
ACS Reference Period
Large and Medium Districts Large Districts Medium Districts
2005- 2006- 2007- 2005- 2006- 2007- 2005- 2006- 2007-
Estimand 2007 2008 2009 2007 2008 2009 2007 2008 2009
Very High FRPL Districts (337) (350) (349) (121) (123) (122) (216) (227) (227)
Percentage free 17.1 18.6 20.1 15.5 18.2 19.5 17.9 18.8 20.4
Percentage reduced price 3.6 3.1 3.9 3.7 2.9 3.7 3.5 3.2 4.1
Percentage free or reduced price 13.5 15.5 16.2 11.8 15.3 15.7 14.5 15.6 16.4
BRR, $ 0.33 0.37 0.39 0.29 0.37 0.38 0.35 0.38 0.40
High FRPL Districts (972) (1,012) (1,014) (298) (303) (304) (674) (709) (710)
Percentage free 8.4 10.2 12.7 9.8 11.2 13.6 7.8 9.8 12.3
Percentage reduced price 1.9 1.7 2.3 1.9 1.7 2.2 1.9 1.7 2.3
Percentage free or reduced price 6.5 8.5 10.5 7.9 9.5 11.4 5.9 8.1 10.1
BRR, $ 0.16 0.21 0.25 0.19 0.23 0.27 0.15 0.20 0.24
NOTES: All average differences are statistically significant (different from zero) at the 0.01 level. ACS = American Community Survey; BRR =
blended reimbursement rate; CCD = Common Core of Data; FRPL = free or reduced-price lunch.
SOURCE: Prepared by the panel.
103
OCR for page 140
140 USING ACS DATA TO EXPAND ACCESS TO THE SCHOOL MEALS PROGRAMS
TABLE 4-15 BRRs Based on Certified Students Versus BRRs Based
on Meals Served: Illustration with Case Study District Schools
Claiming Percentages
Participation Rates (%) Certified Students
Reduced Full Reduced Full
School Free Price Price Free Price Price
1 98 94 23 65 7 27
2 96 9 85 63 11 26
3 85 64 35 65 9 26
4 96 91 71 59 16 25
5 96 79 44 68 8 24
6 91 82 59 57 20 23
7 63 55 18 67 10 23
8 74 71 71 68 9 23
9 93 94 83 64 14 22
10 45 31 6 72 7 21
11 57 44 16 74 6 20
12 96 86 66 76 6 18
13 89 95 27 75 6 18
14 89 87 74 68 15 17
15 77 67 33 77 9 15
16 99 93 33 80 7 14
17 97 98 55 83 4 13
18 90 89 82 83 5 12
19 82 67 35 78 11 11
20 96 90 90 84 5 10
21 62 41 28 77 13 10
22 70 47 22 82 8 10
23 95 93 60 88 3 9
24 92 92 68 84 8 8
25 86 80 40 87 7 6
26 95 95 95 89 6 5
27 94 67 60 88 7 5
28 90 84 78 86 10 4
29 84 77 78 89 7 4
30 90 83 60 87 9 4
NOTE: BRRs = blended reimbursement rates.
SOURCE: Prepared by the panel.
OCR for page 141
DATA ANALYSIS AND RESULTS 141
Claiming Percentages
Meals Served BRRs
Certified
Reduced Full Students Meals Difference Percentage
Free Price Price ($) Served ($) ($) Difference
83 9 8 1.92 2.36 0.44 19
72 1 27 1.94 1.97 0.03 1
79 8 13 1.95 2.26 0.31 14
64 17 20 1.95 2.06 0.11 5
80 8 13 2.01 2.27 0.26 11
64 20 17 1.97 2.12 0.15 7
82 10 8 2.01 2.36 0.35 15
69 9 22 2.02 2.03 0.02 1
66 15 20 2.03 2.07 0.04 2
90 6 3 2.08 2.49 0.41 17
87 6 7 2.10 2.41 0.31 13
81 5 14 2.14 2.25 0.11 5
86 8 6 2.14 2.41 0.27 11
70 15 14 2.14 2.20 0.06 3
85 8 7 2.22 2.39 0.18 7
88 7 5 2.24 2.44 0.20 8
88 4 8 2.27 2.39 0.12 5
84 5 11 2.28 2.31 0.02 1
85 10 5 2.29 2.43 0.14 6
85 5 10 2.33 2.34 0.01 1
85 10 5 2.31 2.44 0.13 5
91 6 3 2.34 2.49 0.15 6
91 3 6 2.37 2.44 0.07 3
86 8 6 2.37 2.42 0.05 2
90 7 3 2.42 2.50 0.07 3
89 6 5 2.45 2.45 0.00 0
91 6 3 2.45 2.50 0.04 2
87 9 4 2.45 2.46 0.02 1
90 7 4 2.47 2.48 0.01 0
89 8 3 2.47 2.50 0.03 1
OCR for page 142
142 USING ACS DATA TO EXPAND ACCESS TO THE SCHOOL MEALS PROGRAMS
TABLE 4-16 Illustration of Potential Participation Effects of
Universal Free Meals Under the AEO in Case Study Districts
Participation Rates
Actual, Pre-AEO (%) Illustrative, Post-AEO (%)
Reduced Full Reduced Full
District Free Price Price Free Price Price
Austin, Texas 86 72 34 91 88 44
Chatham County, Georgia 75 72 48 80 77 58
Norfolk, Virginia 77 71 43 82 79 53
Omaha, Nebraska 92 84 61 97 94 71
Pajaro Valley, California 68 52 23 73 70 33
Claiming Percentages (based on meals served)
Actual, Pre-AEO (%) Illustrative, Post-AEO (%)
Reduced Full Reduced Full
District Free Price Price Free Price Price
Austin, Texas 73 8 19 69 9 22
Chatham County, Georgia 67 10 23 65 9 26
Norfolk, Virginia 59 12 29 56 12 32
Omaha, Nebraska 58 12 30 56 12 32
Pajaro Valley, California 77 9 14 72 10 18
BRRs
Actual, Illustrative,
Pre-AEO Post-AEO Difference Percentage
District ($) ($) ($) Difference
Austin, Texas 2.12 2.05 0.07 3
Chatham County, Georgia 2.01 1.96 0.05 3
Norfolk, Virginia 1.87 1.81 0.07 4
Omaha, Nebraska 1.84 1.80 0.04 2
Pajaro Valley, California 2.22 2.13 0.09 4
NOTE: AEO = American Community Survey (ACS) Eligibility Option; BRRs = blended
reimbursement rates.
SOURCE: Prepared by the panel.
bursement rate per meal. However, another potentially important con-
sideration is that any changes in participation rates under the AEO could
impact the total reimbursement received by a district by affecting not
only the BRR but also the total number of meals served. Thus, a district
would have to assess the cost implications of a change in the scale of food
service operations. A large increase in the total number of meals served
might require, for example, that more staff be hired or that the kitchen
facilities be expanded.
OCR for page 143
DATA ANALYSIS AND RESULTS 143
SUMMARY AND CONCLUSIONS
The panel's evaluations of the ACS-based eligibility estimates encom-
passed a wide range of issues. The main results of our analyses include
the following:
· ACS estimates are systematically different from administrative
estimates for high and very high FRPL districts.
· BRRs based on ACS estimates of eligible students are substan-
tially less than BRRs based on CCD estimates of certified students
for high and very high FRPL districts, on average.
· Average ACS-CCD differences are larger for very high FRPL dis-
tricts than for high FRPL districts.
· There are several potentially important sources of systematic
differences between ACS and administrative estimates, and the
effects of these sources are likely to vary across districts.
· A statistical model can explain a substantial fraction--but far from
all--of the variability across districts in the differences between
ACS and administrative estimates.
· Relative to the inter temporal changes in BRRs normally expe-
rienced by a district, as reflected in the administrative data on
certified students, the typical large district would likely experience
less variability if it used 3- or 5-year ACS estimates but greater
variability if it used 1-year ACS estimates.39 The typical medium
district would experience about the same variability as normal if
it used 3-year ACS estimates and less variability than normal if it
used 5-year ACS estimates. The typical small district would experi-
ence somewhat less than normal variability if it used 5-year ACS
estimates.
· For districts with enrollments of 400 or higher, ACS 5-year esti-
mates would probably be as stable or more so than the districts'
administrative estimates. The 5-year estimates might be less sta-
ble than administrative estimates for smaller districts.
· For small very high FRPL districts, average differences between
model-based ACS estimates and CCD BRR estimates are sub-
stantially larger than average differences between ACS 5-year
estimates and CCD estimates.
· Based on overall accuracy and consideration of error due to both
variability and bias, the 5-year estimates would likely be more
accurate than the 3-year estimates for medium districts. For large
39As noted previously, a typical school in a category has an enrollment at about the median
enrollment for schools in the category.
OCR for page 144
144 USING ACS DATA TO EXPAND ACCESS TO THE SCHOOL MEALS PROGRAMS
TABLE 4-17 Illustration of Potential Participation Effects of
Universal Free Meals Under the AEO in Case Study District Schools
Claiming Percentages
Participation Rates Based on Meals Served
Illustrative, Post-AEO
Actual, Pre-AEO (%) (%) Actual, Pre-AEO
Reduced Full Reduced Full Reduced Full
School Free Price Price Free Price Price Free Price Price
1 98 94 23 100 97 33 83 9 8
2 96 9 85 100 97 94 72 1 27
3 85 64 35 90 87 45 79 8 13
4 96 91 71 100 97 81 64 17 20
5 96 79 44 100 97 54 80 8 13
6 91 82 59 96 93 69 64 20 17
7 63 55 18 68 65 28 82 10 8
8 74 71 71 79 76 73 69 9 22
9 93 94 83 98 95 92 66 15 20
10 45 31 6 50 47 16 90 6 3
11 57 44 16 62 59 26 87 6 7
12 96 86 66 100 97 76 81 5 14
13 89 95 27 94 91 37 86 8 6
14 89 87 74 94 91 84 70 15 14
15 77 67 33 82 79 43 85 8 7
16 99 93 33 100 97 43 88 7 5
17 97 98 55 100 97 65 88 4 8
18 90 89 82 95 92 89 84 5 11
19 82 67 35 87 84 45 85 10 5
20 96 90 90 100 97 94 85 5 10
21 62 41 28 67 64 38 85 10 5
22 70 47 22 75 72 32 91 6 3
23 95 93 60 100 97 70 91 3 6
24 92 92 68 97 94 78 86 8 6
25 86 80 40 91 88 50 90 7 3
26 95 95 95 100 97 94 89 6 5
27 94 67 60 99 96 70 91 6 3
28 90 84 78 95 92 88 87 9 4
29 84 77 78 89 86 83 90 7 4
30 90 83 60 95 92 70 89 8 3
NOTE: AEO = American Community Survey (ACS) Eligibility Option; BRR = blended
reimbursement rate.
SOURCE: Prepared by the panel.
OCR for page 145
DATA ANALYSIS AND RESULTS 145
Illustrative, Post-AEO BRR
Reduced Full Actual Pre- Illustrative Percentage
Free Price Price AEO ($) Post-AEO ($) Difference ($) Difference
80 9 11 2.36 2.29 0.07 3
64 11 25 1.97 1.96 0.00 0
75 10 15 2.26 2.20 0.05 2
62 17 21 2.06 2.03 0.03 1
77 9 14 2.27 2.22 0.04 2
61 21 18 2.12 2.09 0.03 1
78 11 11 2.36 2.29 0.07 3
69 9 22 2.03 2.05 0.02 1
65 14 21 2.07 2.05 0.02 1
84 8 8 2.49 2.38 0.11 4
84 7 10 2.41 2.34 0.07 3
80 6 15 2.25 2.22 0.03 1
85 7 8 2.41 2.37 0.04 2
70 15 15 2.20 2.18 0.02 1
83 9 8 2.39 2.36 0.03 1
87 7 6 2.44 2.41 0.03 1
87 4 9 2.39 2.36 0.02 1
84 5 12 2.31 2.30 0.01 0
83 11 6 2.43 2.41 0.03 1
85 5 10 2.34 2.34 0.00 0
81 13 6 2.44 2.40 0.04 1
87 8 4 2.49 2.46 0.03 1
91 3 6 2.44 2.43 0.01 1
86 7 7 2.42 2.40 0.01 0
90 7 3 2.50 2.48 0.01 1
90 6 5 2.45 2.46 0.01 0
89 7 3 2.50 2.48 0.01 1
87 9 4 2.46 2.46 0.01 0
80 7 4 2.48 2.48 0.00 0
88 9 3 2.50 2.49 0.01 0
OCR for page 146
TABLE 4-18 BRRs for Case Study Districts Based on Certified Students Versus Meals Served Under
Traditional Operating Procedures and the AEO
Blended Reimbursement Rates Difference from Certified Students BRR
Actual Meals Served, Illustrative Meals Served,
Meals Served Pre-AEO Post-AEO
Certified Actual, Pre- Illustrative, Difference Percentage Difference Percentage
District Students ($) AEO ($) Post-AEO ($) ($) Difference ($) Difference
Austin, Texas 1.71 2.12 2.05 0.41 24 0.34 20
Chatham County, Georgia 1.80 2.01 1.96 0.21 11 0.15 9
Norfolk, Virginia 1.59 1.87 1.81 0.29 18 0.22 14
Omaha, Nebraska 1.64 1.84 1.80 0.20 12 0.16 10
Pajaro Valley, California 1.81 2.22 2.13 0.41 23 0.33 18
NOTE: AEO = American Community Survey (ACS) Eligibility Option; BRR = blended reimbursement rate.
SOURCE: Prepared by the panel.
146
OCR for page 147
DATA ANALYSIS AND RESULTS 147
districts, both 3- and 5-year estimates would likely be more accu-
rate than 1-year estimates. However, whether the 3- or 5-year
estimates would be more accurate is less clear. Although some
results suggest that the 3-year estimates appear to strike the most
effective compromise between precision and stability on the one
hand and responsiveness to change on the other, the panel was
unable to perform some analyses because the sets of estimates
available to us were too limited.
· BRRs based on the distribution of certified students can be sub-
stantially less than BRRs based on the distribution of meals
served, although changes in participation after adoption of the
AEO could reduce these differences.
Based on the panel's empirical analyses, as well as consultations with
experts and reviews of relevant documents, the panel reached the follow-
ing conclusions:
Conclusion 4-1: A one-size-fits-all approach for benchmarking ACS
estimates of students eligible for school meals to administrative
estimates to minimize the differences caused by such factors as
underreporting of SNAP participation is not possible at present.
Further research will be required to determine whether a techni-
cally sound and operationally feasible set of procedures for estimat-
ing the necessary adjustments to the ACS estimates can be devel-
oped. Furthermore, even if such procedures were identified and
used, additional adjustments based on a district's own data might
improve the benchmarking of the ACS estimates to administrative
estimates.
Conclusion 4-2: Medium districts generally should prefer the 5-year
ACS estimates to the 3-year estimates, and large districts generally
should prefer either the 3- or 5-year estimates to the 1-year esti-
mates. However, it is not clear whether large districts should prefer
the 3- or 5-year estimates.
Conclusion 4-3: Although all districts should thoroughly assess
their estimates and the potential implications of adopting the
AEO, as discussed in detail in Chapter 5, districts with enrollments
below 400 should consider especially carefully whether reimburse-
ments might fluctuate too much if they were based on ACS 5-year
estimates.40
40Many districts fall in this category--about 30 percent of the very high FRPL districts.
OCR for page 148
148 USING ACS DATA TO EXPAND ACCESS TO THE SCHOOL MEALS PROGRAMS
TABLE 4-19 BRRs for Case Study District Schools Based on
Certified Students Versus Meals Served Under Traditional Operating
Procedures and the AEO
BRR
Meals Served
Certified Students Actual, Pre-AEO Illustrative, Post-AEO
School ($) ($) ($)
1 1.92 2.36 2.29
2 1.94 1.97 1.96
3 1.95 2.26 2.20
4 1.95 2.06 2.03
5 2.01 2.27 2.22
6 1.97 2.12 2.09
7 2.01 2.36 2.29
8 2.02 2.03 2.05
9 2.03 2.07 2.05
10 2.08 2.49 2.38
11 2.10 2.41 2.34
12 2.14 2.25 2.22
13 2.14 2.41 2.37
14 2.14 2.20 2.18
15 2.22 2.39 2.36
16 2.24 2.44 2.41
17 2.27 2.39 2.36
18 2.28 2.31 2.30
19 2.29 2.43 2.41
20 2.33 2.34 2.34
21 2.31 2.44 2.40
22 2.34 2.49 2.46
23 2.37 2.44 2.43
24 2.37 2.42 2.40
25 2.42 2.50 2.48
26 2.45 2.45 2.46
27 2.45 2.50 2.48
28 2.45 2.46 2.46
29 2.47 2.48 2.48
30 2.47 2.50 2.49
NOTE: AEO = American Community Survey (ACS) Eligibility Option; BRR = blended
reimbursement rate.
SOURCE: Prepared by the panel.
OCR for page 149
DATA ANALYSIS AND RESULTS 149
Difference from Certified Students BRR
Actual Meals Served, Pre-AEO Illustrative Meals Served, Post-AEO
Percentage Percentage
Difference ($) Difference Difference ($) Difference
0.44 23 0.37 19
0.03 1 0.03 1
0.31 16 0.25 13
0.11 6 0.08 4
0.26 13 0.22 11
0.15 8 0.12 6
0.35 17 0.28 14
0.02 1 0.03 2
0.04 2 0.02 1
0.41 20 0.30 15
0.31 15 0.24 11
0.11 5 0.09 4
0.27 13 0.24 11
0.06 3 0.04 2
0.18 8 0.14 6
0.20 9 0.17 8
0.12 5 0.10 4
0.02 1 0.02 1
0.14 6 0.11 5
0.01 1 0.01 1
0.13 6 0.09 4
0.15 7 0.12 5
0.07 3 0.06 2
0.05 2 0.04 1
0.07 3 0.06 3
0.00 0 0.01 0
0.04 2 0.03 1
0.02 1 0.01 0
0.01 0 0.01 0
0.03 1 0.02 1
OCR for page 150
150 USING ACS DATA TO EXPAND ACCESS TO THE SCHOOL MEALS PROGRAMS
Conclusion 4-4: To develop accurate claiming percentages for use
in implementing the AEO, it will be necessary to estimate not only
the distribution of eligible students across the free, reduced-price,
and full-price categories but also their expected participation rates
with all meals being served free of charge.
As documented in this chapter, the panel's analyses demonstrate that
the ACS eligibility estimates, on average, are substantially and systemati-
cally different from administrative estimates for high and very high FRPL
districts. For all but the smallest districts, however, reimbursements based
on ACS estimates might be equally stable over time and often more so
than reimbursements based on administrative estimates, and this feature
of the AEO might be attractive to districts along with its other benefits.
Although a one-size-fits-all approach for benchmarking ACS estimates
to administrative estimates is not feasible at present, a tailored approach
to using ACS estimates could possibly allow more districts to offer free
meals to all students under the AEO. In the next chapter, we propose an
approach that FNS might consider for implementing the AEO and that
some districts might find attractive if they wished to adopt the AEO in all
or some of their schools.
