6

Recommendations for Future Work

Chapter 5 describes an approach to using estimates from the American Community Survey (ACS) as part of an ACS Eligibility Option (AEO) for the National School Lunch Program (NSLP) and School Breakfast Program (SBP). In conducting our analyses, the panel encountered issues related to data quality and availability that the Food and Nutrition Service (FNS) or other federal agencies should address regardless of whether FNS chooses to implement the AEO (see Recommendations 1-5 below). Additionally, we identified research activities that could improve ACS estimates for all uses, particularly those that require estimating whether low-income individuals are eligible for benefits from various assistance programs, including the school meals programs (see Recommendations 6 and 7 below). Finally, FNS should pursue recommendations 8-16 if it chooses to implement the AEO.

IMPROVING DATA QUALITY AND AVAILABILITY

The panel identified four areas of improvement for data related to the school meals programs: (1) regularly producing and disseminating ACS estimates of schoolchildren eligible for free and reduced-price meals according to our specifications, (2) improving the comparability and quality of relevant administrative records on enrollment and certification, (3) improving the comparability and quality of relevant administrative records on participation, and (4) obtaining information on the costs of operating the traditional certification process for use in evaluating alter-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 203
6 Recommendations for Future Work C hapter 5 describes an approach to using estimates from the Ameri- can Community Survey (ACS) as part of an ACS Eligibility Option (AEO) for the National School Lunch Program (NSLP) and School Breakfast Program (SBP). In conducting our analyses, the panel encoun- tered issues related to data quality and availability that the Food and Nutrition Service (FNS) or other federal agencies should address regard- less of whether FNS chooses to implement the AEO (see Recommen- dations 1-5 below). Additionally, we identified research activities that could improve ACS estimates for all uses, particularly those that require estimating whether low-income individuals are eligible for benefits from various assistance programs, including the school meals programs (see Recommendations 6 and 7 below). Finally, FNS should pursue recom- mendations 8-16 if it chooses to implement the AEO. IMPROVING DATA QUALITY AND AVAILABILITY The panel identified four areas of improvement for data related to the school meals programs: (1) regularly producing and disseminating ACS estimates of schoolchildren eligible for free and reduced-price meals according to our specifications, (2) improving the comparability and qual- ity of relevant administrative records on enrollment and certification, (3) improving the comparability and quality of relevant administrative records on participation, and (4) obtaining information on the costs of operating the traditional certification process for use in evaluating alter- 203

OCR for page 203
204 USING ACS DATA TO EXPAND ACCESS TO THE SCHOOL MEALS PROGRAMS native provisions. We also identified the need for FNS to remove a bar- rier to the adoption of special provisions for the school meals programs by working with the Department of Education and states to promote the use of data sources other than NSLP counts of free and reduced-price certification for determining eligibility for or otherwise administering a variety of other programs. The first area for data improvement pertains to the availability of school district-level ACS estimates for percentages of children eligible for free, reduced-price, and full-price meals under the school meals programs. The National Center for Education Statistics (NCES) already receives such estimates annually from the Census Bureau. However, specifications for these tabulations differ from those used by the panel. While research may improve our specifications in the future, we consider them to be the best available at present for producing estimates that most closely replicate program eligibility rules. Recommendation 1: The Food and Nutrition Service should work with the National Center for Education Statistics and the U.S. Census Bureau to produce improved annual school district-level estimates (and their standard errors) for total enrollment and per- centages of students eligible for free, reduced-price, and full-price meals under the school meals programs from the American Com- munity Survey. These direct American Community Survey esti- mates should be based on the approach documented by the panel in its specifications to the Census Bureau (see Appendix D)1 and should be produced for 1, 3, and 5 years of data. The estimates should be publicly available on an annual schedule. The second area for data improvement concerns the comparability and quality of various administrative data sources, in particular the two sources with school district-level data on enrollment and students certi- fied for free and reduced-price meals--the NCES Common Core of Data (CCD) and form FNS-742.2 These data sources are not easily linked, and once linked, they exhibit troubling discrepancies. The panel was fortunate to have received from FNS a copy of a May 21, 2010, report entitled VSR-CCD Linkfile.3 This report documents 1The approach should be augmented as appropriate based on further research. 2While we refer generically to "school district-level data," these two data sources actually obtain reports for entities that are not always the same. Form FNS-742 obtains reports for school food authorities (SFAs), while the CCD obtains reports for local education agencies (LEAs). See footnote 3 below for additional detail. 3VSR-CCD Linkfile, a report by Mathematica Policy Research, was delivered to FNS on May 21, 2010, and provided to the panel by FNS. The project director was Nancy Cole. The

OCR for page 203
RECOMMENDATIONS FOR FUTURE WORK 205 how to link data from form FNS-742 across years and with data from the CCD, and an accompanying data file provides the information needed to link data for several years from the two sources. We used the report and file in developing the database that supported all of our comparative analyses. FNS and NCES should build on this work to better align the two data collection systems. In particular, FNS should consider requiring school food authorities (SFAs) to make use of their NCES ID when sub- mitting data. The NCES IDs would need to be adapted to cases in which SFAs and local education agencies (LEAs) are not the same. This would greatly facilitate appropriate linkage between the two systems. CCD and FNS-742 data on enrollments and numbers of students certi- fied for free and reduced-price meals differ substantially for some school districts. There is officially a 1-month time lag between the reporting periods for the two data sets that might explain some of the differences.4 In some districts, however, the data are identical. In these cases, either the districts are not observing the reporting date distinctions or the data are the same at the beginning and the end of October. NCES should consider changing its reporting date for certification data from the school meals programs to align with the date used for form FNS-742. Another reason for differences between CCD and FNS-742 data is that even in districts where the SFA and LEA are the same, one or more schools in the LEA may not participate in the school meals programs, and one or more schools in the SFA may not be regular schools in the LEA. These dif- ferences should be noted in the data sets. Additionally, both sources could be improved by incorporating specific indicators for data that are missing versus true zeros, and both systems would benefit from improved editing to address data entry errors. Recommendation 2: The Food and Nutrition Service and the National Center for Education Statistics should work together with their respondents--states and school districts--to understand the differences between reporting in the Common Core of Data and on form FNS-742, and implement improvements to make the two data sources as consistent as possible while clearly documenting neces- sary and legitimate distinctions between them. report notes reasons for differences between Verification Summary Report (VSR) data (from form FNS-742) and CCD data. It explains that while there was usually a one-to-one match, the primary exceptions occurred when SFA operations were centralized for multiple school districts. Places where this was common include Montana, New Hampshire, and New York City. In 2008-2009, there were 14,717 (unduplicated) SFAs in the VSR file, 95.5 percent of which matched with the CCD data. 4CCD data are to be reported as of October 1, or the closest school day to October 1, while the FNS-742 data are to be reported as of the last operating day in October.

OCR for page 203
206 USING ACS DATA TO EXPAND ACCESS TO THE SCHOOL MEALS PROGRAMS A third area for improvement concerns data on participation in the school meals programs. Participation data collected on form FNS-10 are available in files with nationwide coverage only at the state level, and it is difficult even at the state level to obtain consistently defined estimates of participation and estimates of enrollment and certification for the same groups of schools (public, private, and residential child care institutions) for the same time period (annual, monthly). Furthermore, the only data available for school districts that are operating under Provision 2 or 3 are enrollment and certification data collected on form FNS-742, and these data are not available for those districts' base year(s). To enhance the availability of data at a lower level of aggregation than the state, FNS should pursue obtaining data on the number of meals served by eligibil- ity category and on average daily participation for the month of October that are linkable to certification data by category at the school district level for all school districts in the nation. In addition, form FNS-742 should be augmented to collect data on whether a district provides universal free meals; which provision or option (if any) is used to provide free meals; whether universal free meals are provided at breakfast, lunch, or both; whether universal free meals are provided in all schools or only a subset of schools; and whether the district is operating a base year. Items cur- rently collected for districts operating under Provision 2 or 3, not in a base year--number of schools, enrollment, and certification counts--should also be collected during a base year (for all special provisions including the AEO). For districts that provide universal free meals in only a subset of schools, form FNS-10 should obtain meal counts by category separately for schools providing all meals free and those operating under traditional procedures. The additional data would support studies of participation, including changes in participation experienced by a district when provid- ing free meals during a base year. The data would also support analysis of differences between the distributions of certified students and meals served across the free, reduced-price, and full-price categories. Recommendation 3: The Food and Nutrition Service should make the following enhancements to its form FNS-10 and FNS-742 data sets: Improve the FNS-10 and FNS-742 data sets to ensure that con- sistent state-level data on enrollment, certification, and partici- pation for the same groups of schools, the same time periods, and the two programs (National School Lunch Program and School Breakfast Program) can be readily obtained. Expand the collection of FNS-10 data items to include meals served by category (free, reduced price, full price) and average

OCR for page 203
RECOMMENDATIONS FOR FUTURE WORK 207 daily participation, at least for the month of October and for lunch and breakfast separately; whether the district provides universal free meals and the provision or option used (if any); and whether a provision or option has been adopted in all schools or only a subset of schools.5 Such data should be col- lected for all school districts in the nation and maintained in a form that is directly linkable to the FNS-742 data concerning enrollment and certification. Collect more complete information on form FNS-742 on the use of special provisions and other options that entail offering universal free meals. Consider implementing a data collection process that offers a web-based interface for entering source data and that supports appropriate entry, approval, and view access for each level of reporting (school, district, state, and nation). A fourth area for data improvement concerns administrative cost information for evaluating the savings that could accrue from the elimi- nation of administrative tasks under a universal free meals option. The panel attempted to collect administrative cost information from our five case study districts and made use of cost data from the U.S. Government Accountability Office (2002). However, we determined that the data were too inconsistent to be useful for even a crude cost analysis of a universal free meals option. Hence, we recommend that FNS consider the develop- ment of a mechanism for periodic collection of data on administrative costs in the NSLP and SBP. We recognize that a barrier is the diversity of accounting systems used by states and school districts, and as a result, collecting such data in a standardized format may require burdensome changes in school district accounting procedures. FNS should first col- laborate with selected school districts and states on a feasibility study to develop meaningful working definitions of the costs that need to be tracked and then illustrate the application of these definitions by collect- ing cost data in a randomly selected sample of districts. Recommendation 4: The Food and Nutrition Service should study the feasibility of developing a program for periodic collection of data on administrative costs in the school meals programs. It should first collaborate with selected school districts and states to develop meaningful working definitions of the costs that need to be tracked 5Fordistricts that provide universal free meals in only a subset of schools, form FNS-10 should collect meal counts separately for the schools that provide universal free meals (whether in a base year or not) and those that operate under traditional procedures.

OCR for page 203
208 USING ACS DATA TO EXPAND ACCESS TO THE SCHOOL MEALS PROGRAMS and then illustrate the application of the definitions and procedures for collecting cost data in a randomly selected sample of districts. Finally, the panel noted that school districts, states, and other fed- eral organizations use information on certification for free and reduced- price meals, in the aggregate and for individual students, to administer other programs and confer benefits. For example, school meals certifi- cation is used individually to qualify students for free band uniforms and textbooks and for other benefits at the local level, individually (but aggregated to the classroom level) to qualify teachers for bonuses where authorized by state law, in the aggregate to qualify schools as eligible to receive funds under Title I of the Elementary and Secondary Education Act (ESEA) of 1965 (as amended), and individually for reporting educa- tional progress for economically disadvantaged students in Title I schools under the ESEA (as amended by the No Child Left Behind Act of 2001). The challenge is that special provisions and options, which would include the AEO, eliminate the annual certification of students and tabulation of free and reduced-price certification numbers in most years, rendering these data unavailable for other uses. In May 2011, the U.S. Department of Education issued a memoran- dum stating that Community Eligibility Option (CEO) schools, like those operating under Provision 2 or 3, could use other data for various pro- visions of the ESEA (e.g., CEO schools could deem all of their students economically disadvantaged for individual purposes and use the reim- bursement claiming percentage for free meals for aggregate purposes).6 However, either participants in the panel's workshop and Provision 2/3 survey respondents did not know that other data sources could be used, or these options are not widely acknowledged as being applicable for other state and local uses of the data. Regardless of how attractive any of the special provisions may be to local operators of the school meals pro- grams who would like to benefit more low-income children by expand- ing access to free school meals, the need to provide data for other critical programs may discourage them from adopting a special provision.7 The Department of Education memorandum applies only to reporting of data under the ESEA, and does not apply to requirements to qualify individual students for income-conditioned benefits offered by schools or states. One option would be for states or school districts to allow schools 6Memorandum from Carl Harris, deputy assistant secretary of education, to state commis- sioners of education, dated May 20, 2011; refer back to Chapter 2 for details. 7The panel was told that Denver had been operating under Provision 2 for a time but discontinued it in part because teachers could not receive state-authorized bonuses based on the proportion of economically disadvantaged students in their classes.

OCR for page 203
RECOMMENDATIONS FOR FUTURE WORK 209 operating under the CEO, Provision 2 or 3, or the AEO to deem all of their students economically disadvantaged, the approach used in the Depart- ment of Education memorandum. Alternatively, it may be possible to identify students who are on state and local lists (e.g., participants in the Supplemental Nutrition Assistance Program [SNAP, formerly the Food Stamp Program] and foster children) as eligible for other benefits. The decision about which approach to use would depend on how states and districts employ these data for other purposes. Recommendation 5: The Food and Nutrition Service should broadly disseminate to state and local authorities the U.S. Department of Education guidance that permits schools operating under a special provision for school meals to use alternative data for Department of Education purposes for which traditional National School Lunch Program certification data would otherwise be used. The Food and Nutrition Service should also encourage state and local authorities to allow districts that choose to operate under a special school meals provision to use alternative, specified data or methods for determin- ing aggregate and individual measures of economic status for other income-conditioned benefits or reporting.8 IMPROVING ACS ESTIMATES The panel identified two research areas for improving ACS estimates. The first is additional research on the definition of an economic unit for estimating students who are eligible for free or reduced-price school meals. The second is additional research on the quality of the ACS estimates, particularly the effects on quality of (1) underreporting of income and pro- gram participation in the survey and (2) the difference between the income accounting period used in the ACS (annual, for the previous year) and that used administratively for determining eligibility (current or past month). While the panel was comfortable that the definition of an economic unit we specified for ACS estimates of students eligible for the school meals programs was balanced and reasonable given the short time period of this study, the topic would benefit from additional research. For example, one could use the Current Population Survey Annual Social and Economic Supplement (CPS ASEC) or the Survey of Income and Program Participa- tion (SIPP) to evaluate our methodology for using available ACS variables to define an economic unit. The CPS ASEC and SIPP collect more detailed information than the ACS on subfamilies and relationships for individuals who are unrelated to the householder, as well as more detailed informa- 8See Recommendation 8 on the AEO specifically.

OCR for page 203
210 USING ACS DATA TO EXPAND ACCESS TO THE SCHOOL MEALS PROGRAMS tion on receipt of benefits from SNAP, Temporary Assistance for Needy Families (TANF), and the NSLP. Both surveys suffer from underreporting of such benefits (as does the ACS--see below). However, with person-level data on SNAP participation, the CPS ASEC or the SIPP (or both) could be used, for example, to assess the impact of assuming that if one person in the household is participating in SNAP, all students in the household are participating and are therefore eligible for free meals. With detailed information on how members of the household (even those unrelated to the householder) are related to one another, one could directly assess the definition of an economic unit adopted by the panel. With longitudinal data from the SIPP, it might be possible to assess the extent to which the economic units at survey time differ from those at other points in the year during which income was accrued. Another source of information with which to evaluate our definition of an economic unit might be based on simulations with ACS data using the Minnesota Population Center's Inte- grated Public Use Microdata Series (IPUMS) methodology of imputing household relationships instead of our assumptions.9 Recommendation 6: The Food and Nutrition Service should support research to assess the quality of the panel's definition of an eco- nomic unit for use in determining eligibility for the school meals programs from the American Community Survey and suggest alter- natives that would improve that definition. Systematic differences between ACS eligibility estimates and admin- istrative certification estimates also should be evaluated. These include the underreporting of income and program benefits, particularly SNAP benefits, on the ACS (and other household surveys) and the under statement of eligibility based on using annual income (the only measure available in the ACS) rather than monthly income (which is used by school districts to certify students for free or reduced-price school meals). The policy research community and the Census Bureau should continue to investigate these issues and ways to ameliorate their effects on the quality of the ACS estimates. For example, the Census Bureau should do more to evaluate the quality of the ACS data, particularly with respect to the reporting of SNAP participation, and could explore use of the CPS-SNAP direct match study (Meyer and George, 2011) to determine how best to impute participation from administrative data. Recommendation 7: The policy research community and the Census Bureau should continue to investigate causes of and solutions for 9See http://usa.ipums.org/usa/chapter5/chapter5.shtml.

OCR for page 203
RECOMMENDATIONS FOR FUTURE WORK 211 the underreporting of income and benefits, particularly from the Supplemental Nutrition Assistance Program, in the American Com- munity Survey and the differences in estimates of eligibility for the school meals programs based on monthly and annual income. FACILITATING IMPLEMENTATION OF THE ACS ELIGIBILITY OPTION The panel has proposed a way to use ACS data in implementing the AEO. Should FNS decide to implement the AEO, the panel has the fol- lowing recommendations (8 through 16) to facilitate the process. The first parallels Recommendation 5 above, but addresses only Department of Education uses of the data. Recommendation 8: The U.S. Department of Education and the Food and Nutrition Service should agree that school districts that choose to participate in the American Community Survey Eligi- bility Option may use data other than traditional certification of eligibility for school meals for individual and aggregate reporting of economically disadvantaged children under the Elementary and Secondary Education Act of 1965. In particular, the panel suggests that for aggregate reporting, a district should use its most recent benchmarked American Community Survey eligibility estimates, and that for individual reporting, a district should be allowed to designate all students enrolled in an American Community Survey Eligibility Option school as economically disadvantaged. These recommendations parallel the guidance already provided by the Department of Education for students enrolled in a Provision 2, Provision 3, or Community Eligibility Option school. Successful adoption of the AEO by school districts will require train- ing and assistance from FNS. For example, states will need to learn how to use ACS estimates to implement the AEO and carry out other AEO startup activities so they can provide information and assistance to dis- tricts that are candidates for adopting this new provision. It will be useful for state agencies, for example, to regularly identify school districts that are candidates for adopting the AEO, perhaps because of a high percent- age of students certified for free or reduced-price meals under traditional operating procedures, and to provide them with training and technical assistance to facilitate the assessment and implementation of the AEO. As part of this training and assistance, FNS should develop a simple, easy-to-understand document explaining how the AEO would work, including an explanation of how ACS estimates should be interpreted.

OCR for page 203
212 USING ACS DATA TO EXPAND ACCESS TO THE SCHOOL MEALS PROGRAMS FNS also should provide districts with an easy-to-use Web tool (the AEO Calculator) to help them assess the desirability of the AEO and prepare updated benchmarked estimates annually. As described in Chapter 5, districts would enter their data, and the AEO Calculator would compute the necessary statistics and display results in useful ways to help districts determine whether they are good candidates for the AEO. Use of the AEO Calculator also would facilitate communication among agencies about the ACS estimates. Recommendation 9: The Food and Nutrition Service should provide technical assistance to help states and districts understand, evaluate the feasibility of, and implement the school meals American Com- munity Survey Eligibility Option. This assistance should include easy-to-understand documentation and an American Community Survey Eligibility Option Calculator for districts to use in working with American Community Survey estimates of students eligible for the school meals programs. Should FNS decide to implement the AEO and allow it to be used for schools or groups of schools, FNS will need to establish an agreement with the Census Bureau regarding the necessary data requests, including the provision of geographic boundary information for school attendance areas. In particular, the panel believes it is important that FNS serve as the conduit for such requests--providing technical assistance to districts on the estab- lishment of geographic boundaries for school attendance areas, collecting all necessary boundary information from districts, and sending all requests to the Census Bureau on a fixed annual schedule. Data to be provided by the Census Bureau should include all relevant estimates for, initially, at least three consecutive releases of ACS data (e.g., 2005-2009, 2006-2010, and 2007- 2011 if 5-year estimates are provided). Thereafter, new estimates would be provided annually with each subsequent release of ACS data. Recommendation 10: If the American Community Survey Eligibil- ity Option is to be implemented for a subset of schools within a school district, the Food and Nutrition Service and the U.S. Cen- sus Bureau should reach agreement on how and when geographic boundary information for school attendance areas will be provided to the Bureau and how and when the American Community Survey estimates for the schools or group of schools will be delivered. Districts that wish to implement the AEO for a group of schools will need to obtain and evaluate boundary information either individually for the schools or for the group of schools. The boundary information

OCR for page 203
RECOMMENDATIONS FOR FUTURE WORK 213 will need to be in the form of a list of census blocks that make up the individual school attendance areas or the attendance area for the group of schools. Possible sources for such boundary information include (1) the district's local planning office, (2) the School Attendance Boundary Infor- mation System (SABINS),10 (3) the use of remote digitizing software,11 and (4) the use of a geographic information system (GIS) by a local staff per- son. Because census blocks are used to define the boundaries for Census Bureau estimates, districts will need to evaluate the list of census blocks that correspond to the geographic area to be included in the AEO for omission or inclusion of large groups of students (e.g., because a large apartment complex is partly within and partly outside the area). As noted above, FNS should provide technical assistance in this process. Recommendation 11: The Food and Nutrition Service should pro- vide technical assistance to school districts that wish to participate in the American Community Survey Eligibility Option only for a school or group of schools with respect to sources for and prepara- tion of the geographic boundary information for school attendance areas needed to derive American Community Survey eligibility estimates for the school meals programs. The AEO should be evaluated carefully in districts that are early adopters. Early in the implementation of the AEO, these districts could be designated as demonstration sites to which FNS would provide finan- cial assistance to support extra data collection. FNS also could sponsor an independent evaluation of the demonstration projects. Although the panel believes our approach for using ACS estimates for the AEO is prom- ising, the lack of high-quality comprehensive data--particularly regard- ing the likely effects on participation and administrative costs, the effect of school choice policies (e.g., independent charter schools) on the valid- ity of estimates, and the accuracy and stability of the AEO benchmark- ing adjustments over time--made it impossible for us to provide more definitive conclusions and guidance regarding the AEO's desirability and feasibility for all or nearly all districts in the nation. Recommendation 12: The Food and Nutrition Service should desig- nate some of the school districts that are early implementers of the 10In February 2012, the panel learned that SABINS will be funded by NCES, which plans to update it annually and gradually increase its geographic coverage (see http://www. sabinsdata.org/home). 11The panel also learned that NCES plans to ultimately host a remote digitizing service, initially prepared as part of SABINS but was still in testing as of February 2012.

OCR for page 203
214 USING ACS DATA TO EXPAND ACCESS TO THE SCHOOL MEALS PROGRAMS American Community Survey Eligibility Option as demonstration sites and provide them with additional resources for data collec- tion. The Food and Nutrition Service also should commission an independent evaluation of the demonstration projects. The panel's assessment of the effects of school choice, particularly the impact of charter schools and open enrollment, on residence-based eligibility estimates focused on only two school districts in much depth. While we generally believe that large effects of school choice are not wide- spread, the prevalence of charter schools and open enrollment has been growing, and the extent to which school choice has an impact on estimates of eligibility for the school meals programs in schools and school districts should be considered in future research. Recommendation 13: The Food and Nutrition Service should work with the U.S. Department of Education and the education research community to monitor the prevalence of school choice provisions, such as charter schools and open enrollment, and their impact on school meals eligibility percentages for public schools and districts. The findings from such monitoring should be used to evaluate the potential effects of school choice on the accuracy of eligibility per- centages estimated for the American Community Survey Eligibility Option, that is, eligibility percentages based on residence in the catchment area of a school or district rather than on actual enroll- ment in the school or district. Ideally, the ACS would itself provide additional detail on types of schools attended in addition to simply public versus private. Because the ACS is mandatory and is designed to cover many subject areas, however, the ability to add more questions on any one topic is limited. Nonetheless, simply adding a third category of "charter school" together with "regular public school" and "private school" would be helpful to the policy and research communities.12 The accuracy of both ACS and administrative estimates could vary over time and would, it may be hoped, improve. Some of these changes in quality could affect the accuracy and stability of AEO benchmarking adjustments for districts that are considering the AEO, as well as those that have already adopted it. Thus, it is important that FNS track such changes, identify their causes, and assess their implications. 12This information would likely be useful for purposes in addition to the AEO.

OCR for page 203
RECOMMENDATIONS FOR FUTURE WORK 215 Recommendation 14: The Food and Nutrition Service should moni- tor the accuracy of American Community Survey estimates of eligi- bility for the school meals programs, the accuracy of administrative certification estimates, and the accuracy and stability of differences between American Community Survey and administrative esti- mates. Such monitoring should be conducted for a broad sample of districts that are potential candidates for adopting the Ameri- can Community Survey Eligibility Option but are not operating under that option or any other special provision or option. The Food and Nutrition Service also should monitor the accuracy of American Community Survey eligibility estimates for districts that have already adopted the American Community Survey Eligibility Option. The benchmarking approach proposed by the panel for adjusting the ACS eligibility estimates for a consistent difference from administrative certification estimates uses only district-specific ACS and administra- tive data. The predictive modeling approach described in Chapter 4 and documented in Appendix F, which uses data from all districts as well as district-specific data to develop a benchmarking adjustment for each district, has some technical appeal and would benefit from additional research. In particular, FNS might sponsor research to develop and test a global predictive model that might be used for benchmarking either alone or in conjunction with a subsequent benchmarking adjustment based entirely on district-specific data. Recommendation 15: The Food and Nutrition Service should spon- sor research to develop and test a global predictive model for devel- oping American Community Survey estimates of eligibility that are benchmarked to certification data for the school meals programs. The specification and implementation of ACS school district-level model-based eligibility estimates produced by the Census Bureau and their subsequent evaluation by the panel took place in a brief time period. Although we could not recommend the use of the model-based estimates in place of the direct estimates at this time, the current models and esti- mates can be viewed as a proof of concept, and we believe refinement of a model-based approach could lead to more accurate and timely estimates than direct ACS estimates for many if not all school districts. Should implementation of the AEO prove successful, FNS should consider work- ing with the Census Bureau to develop improved ACS model-based esti- mates. Specific suggestions for potential improvements in the models and estimation procedures are provided in Appendix C.

OCR for page 203
216 USING ACS DATA TO EXPAND ACCESS TO THE SCHOOL MEALS PROGRAMS Recommendation 16: The Food and Nutrition Service should work with the U.S. Census Bureau to improve the methodology used in support of this study (see Appendix C) to prepare the American Community Survey small-area model-based estimates of eligibility percentages for free and reduced-price meals.