2
American Community Survey Estimates

This chapter provides background information on the American Community Survey (ACS) estimates of the number of English language learner (ELL) students that are used for computing each state’s share of the national estimate for the allocation of Title III funds. The chapter first provides a summary of the ACS and then assesses the evidence on the quality of the ACS estimates. The third section presents the ACS estimates, and the last section describes the properties of the estimates in terms of their sampling properties, precision, consistency, sensitivity, and coverage.

THE AMERICAN COMMUNITY SURVEY

Characteristics

Although the ACS is a new survey—its first products were released in 2006, after a decade of testing and development by the Census Bureau—it is a very important one. Unlike the long-form sample of the decennial census, which it replaced, it is a significant ongoing undertaking that covers some 2 million households each year. It provides the capacity for the Census Bureau to produce estimates for 1-year, 3-year, and 5-year periods and for successively broader tabulation coverage of geographic areas.

Other characteristics of the ACS enhance its value to users (National Research Council, 2007, p. 2), especially in comparison with the census long form: it is timely, with data products introduced just 8-10 months after collection; frequent, with products updated each year; and of relatively high quality, as measured by the completeness of response to survey questions. Given these characteristics, a great number of uses have already been implemented, and many more have been identified



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 21
2 American Community Survey Estimates This chapter provides background information on the American Community Survey (ACS) estimates of the number of English language learner (ELL) students that are used for computing each state’s share of the national estimate for the alloca - tion of Title III funds. The chapter first provides a summary of the ACS and then assesses the evidence on the quality of the ACS estimates. The third section presents the ACS estimates, and the last section describes the properties of the estimates in terms of their sampling properties, precision, consistency, sensitivity, and coverage. THE AMERICAN COMMUNITY SURVEY Characteristics Although the ACS is a new survey—its first products were released in 2006, after a decade of testing and development by the Census Bureau—it is a very impor- tant one. Unlike the long-form sample of the decennial census, which it replaced, it is a significant ongoing undertaking that covers some 2 million households each year. It provides the capacity for the Census Bureau to produce estimates for 1-year, 3-year, and 5-year periods and for successively broader tabulation coverage of geo - graphic areas. Other characteristics of the ACS enhance its value to users (National Research Council, 2007, p. 2), especially in comparison with the census long form: it is timely, with data products introduced just 8-10 months after collection; frequent, with products updated each year; and of relatively high quality, as measured by the completeness of response to survey questions. Given these characteristics, a great number of uses have already been implemented, and many more have been identified 21

OCR for page 21
22 ALLOCATING FEDERAL FUNDS for the ACS data, including the allocation of federal funds for programs that support activities in states and localities. A recent study by the Brookings Institution found that, in fiscal 2008, “184 federal domestic assistance programs used ACS-related datasets to help guide the distribution of $416 billion, 29 percent of all federal as - sistance. ACS-guided grants accounted for $389.2 billion, 69 percent of all federal grant funding” (Reamer, 2010, p. 1). However, some characteristics of the ACS limit its usefulness for particular applications or levels of detail. Like the census long form, the ACS is a sample survey. Even with the aggregation of data for 5-year estimates, the ACS sample is significantly smaller than the census long-form sample it replaced, and it there - fore has considerably larger margins of error in the sample estimates. In addition to smaller sample size, the ACS sample has greater variation because of greater variation in sample weights because of the subsampling of households for field interviews from among those that do not respond to the mail or telephone con - tacts. Some uncertainty in the ACS estimates is also introduced by the use of postcensal population and housing estimates as controls for the survey over the course of the decade. These estimates are applied at a less detailed level than census controls, and they are indirect estimates rather than a product of a simultaneous census activity (as were the census controls for the long-form sample). However, some of the characteristics of the ACS mitigate these negative aspects. Because of extensive follow-up, the response rates are higher than response rates achieved with the census long form, and because a higher proportion of ACS responses are through the intervention of an interviewer, the overall quality of the responses tends to be higher. The effects of the larger sampling errors fall most heavily on the data for small areas and small population subgroups. Later this is illustrated in Table 2-2, which shows that standard errors are proportionally largest for the smallest states with re - gard to the critical data element used in the allocation of Title III funds. The relative lack of precision for smaller states suggests the need to accumulate data for 3-year and 5-year periods, rather than using 1-year estimates, in order to achieve sufficient precision for some data elements, such as English speaking ability. The issues at - tending the selection of the appropriate ACS period are extensively discussed below. Background It is useful to trace some of the significant events in the evolution of the ACS in order to understand the environment that led to tradeoffs that, in turn, set the objectives for this new survey. After the 1990 census, there were growing concerns, shared by some members of Congress, that the long-form questionnaire had response issues that marginalized its utility. In that census, 29 percent of the households that received the long form failed to mail it back, compared with 24 percent of house - holds that received the short form (National Research Council, 2004, p. 100). Some observers thought that this differential contributed to the poorer coverage of the

OCR for page 21
23 AMERICAN COMMUNITY SURVEY ESTIMATES population in 1990 in comparison with 1980. At the same time, there was increasing interest in obtaining more frequent population estimates for small areas. To counter this problem of declining long-form response rates and to provide more frequent data for small areas, in 1994 the Census Bureau decided to move to - ward a continuous measurement design similar to one that had been proposed years earlier by Leslie Kish (see National Research Council, 1995, p. 71). This continuous measurement survey was named the American Community Survey, and the Cen- sus Bureau set a goal of conducting a short-form-only census in 2010 and to fully implement the ACS by then. It was expected that the ACS could provide estimates for small areas that were about as precise as long-form-sample estimates for small areas by accumulating samples over 5 years. However, very early in the development process, rising costs led to a decision to scale back the originally planned size of 500,000 housing units per month to a sample of 250,000 housing units per month (National Research Council, 1995, p. 127). This decision to reduce the desired sample size had a significant deleterious effect on the ability of the ACS to provide reliable 1-year data for small areas. Design Data Collection Each month, the ACS questionnaire—which is similar in content to the old census long form—is mailed to 250,000 housing units across the nation. The units have been sampled from the Census Bureau’s Master Address File using a probability sample design in which housing units in small areas are oversampled. As with the long form of the census, response to the ACS is required by law. The ACS mail questionnaire uses a matrix layout for questions on sex, age, race, ethnicity, and household relationship. It provides space for information on five household members; information on additional household members is gathered through a follow-up telephone survey. The ACS instructs the household respondent to provide data on all people who, at the time of completing the questionnaire, have been living or staying at the household address for more than 2 months (including usual residents who are away for less than 2 months). Individuals in the ACS samples that reside in group quarters (such as college dormitories and prisons) are counted at the group quarters location, in effect applying a de facto residence rule regardless of how long an individual has lived or expects to live in the group quarters. The residential housing unit addresses in the ACS sample with usable mailing addresses—about 95 percent of each month’s sample of 250,000 addresses—are sent a notification letter 4 days before they receive a questionnaire booklet, and a reminder postcard is sent 3 days after the questionnaire mailing. Whenever a ques - tionnaire is not returned by mail within 3 weeks, a second questionnaire is mailed to the address. If there is no response to the second mailing, and if the Census Bureau is able to obtain a telephone number for the address, trained interviewers conduct tele - phone interviews using computer-assisted telephone interviewing (CATI) software.

OCR for page 21
24 ALLOCATING FEDERAL FUNDS The CATI operation benefits from several quality assurance programs. The software prevents common errors, such as out-of-range responses or skipped ques - tions. Full-time call center staff are carefully trained and provided with periodic training updates. New interviewers receive standard CATI training and a workshop to specifically train them on how to handle refusals. New interviewers are monitored regularly and even qualified interviewers are monitored periodically to make sure they continue conducting interviews in a satisfactory manner. In addition, Census Bureau supervisors at the call centers monitor interviewers’ work to check for other errors, such as keying a different answer from the one the respondent provided or failing to follow procedures for asking questions or probing respondents for answers to questions. The Census Bureau has found its monitoring to be effective in control- ling telephone interviewer errors. Consequently, the ACS, using CATI instruments and procedures, is more accurate than the census long form in that it obtains more complete information than was obtained on the Census long form (National Re - search Council, 2007, p. 161). Interviewers also follow up on a sample of households: those for which no mail or CATI responses have been obtained after 2 months, those for which the postal service returned the questionnaire because it could not be delivered as addressed, and those for which a questionnaire could not be sent because the address was not in the proper street name and number format. The follow-up is in person for 80 percent of the housing units and by telephone for 20 percent. For the in-person interviews, the data are collected though computer-assisted personal interviewing (CAPI). For cost reasons, the personal interview follow-up is conducted on a sample basis: it includes about two-thirds of unusable addresses and between one-third and one-half of usable addresses in each census tract, depending on the expected mail- back and CATI response rate for the census tract. Interviewers also visit group quar - ters in person to collect data from residents, using paper-and-pencil questionnaires. Since it is considered a part of the decennial census, the ACS collects data under legal protections1 with confidentiality requirements. Following the law, the Census Bureau pledges to respondents that their responses will be used only for statistical purposes and not for any kind of administrative or enforcement activity that affects the household members as individuals. This confidentiality protection is one reason for the high response rate to the ACS, even on somewhat sensitive topics. Because ACS data are collected on an on-going basis, data products are avail - able each year and do not pertain to a specific point in time. The 1-year estimates operate on 12 months of data collected during the preceding calendar year. The 3-year estimates are produced using 36 months’ worth of responses, and the 5-year estimates are produced from 60 months’ worth of responses. For the range of sample sizes used in producing ACS estimates for each state, see Table 2-1. The data used to generate the period estimates include all of the mailed back, CATI, and CAPI responses (including additional information obtained by telephone 1 Data Protection and Privacy Policy, available: http://www.census.gov/privacy/data_protection/ federal_law.html [May 2010].

OCR for page 21
25 AMERICAN COMMUNITY SURVEY ESTIMATES for incompletely filled out mail questionnaires). The major data processing steps are coding, editing, and imputation; weighting; and tabulation. Coding, Editing, and Imputation The first data processing step for the ACS is to assign codes for write-in re - sponses for such items as ancestry, industry, and occupation, which is done with automated and clerical coding procedures. Then the raw data, with the codes as - signed to write-in items and various operational data for the responses, are assembled into an “edit-input file.” Computer programs review the records on this file for each household to determine if the data are sufficiently complete to be accepted for further processing and to determine the best set of records to use in instances when more than one questionnaire was obtained for a household. Computer programs then edit the data on the accepted, unduplicated records in various ways. Computer programs also supply values for any missing information that remains after editing, using data from neighboring households with similar characteristics. The goal of editing and imputation is to make the ACS housing and person records complete for all persons and households. Weighting The weighting process is designed to produce estimates of people and housing units that are as complete as possible and that take into account the various aspects of the complex ACS design. The edited, filled-in data records are weighted in a series of steps to produce period estimates that represent the entire population. The basic estimation approach is a series of steps that accounts for the housing units probability of selection, adjusts for nonresponse, and applies a ratio estimation procedure that results in the assignment of two sets of weights: a weight to each sample person record (both household and group quarters persons) and a weight to each sample housing unit record. Ratio estimation takes advantage of auxiliary in - formation (population estimates by sex, age, race, and Hispanic origin, and estimates by total housing units) to increase the precision of the estimates, as well as to correct for differential coverage by geography and demographic detail. This method also produces ACS estimates consistent with the estimates of population characteristics from the Population Estimates Program of the Census Bureau and the estimates of total number of housing units for each county in the United States. Tabulations and Data Releases The final data processing steps are to generate tabulations, profiles, and other data products, such as public-use microdata samples (PUMS). Beginning in summer 2006, the Census Bureau began releasing 1-year estimates from the previous year for

OCR for page 21
26 ALLOCATING FEDERAL FUNDS TABLE 2-1 ACS Sample Sizes: Initial Addresses and Final Interviews, by Type of Unit ACS 2005 ACS 2006 Housing Units Housing Units Group Quarters Initial Initial Initial Addresses Final Addresses Final Sample Final State Selected Interview Selected Interview Selected Interview Alabama 51,050 31,274 51,063 32,647 2,767 1,997 Alaska 9,740 5,759 9,739 5,835 485 337 Arizona 51,685 32,749 52,511 33,718 2,609 1,971 Arkansas 32,648 20,052 32,608 20,825 1,873 1,567 California 266,324 172,287 265,521 178,666 19,583 14,783 Colorado 45,086 29,612 45,053 30,623 2,523 1,974 Connecticut 28,885 20,652 28,651 21,357 2,651 2,266 Delaware 9,722 6,208 9,951 6,411 557 467 District of Columbia 5,941 3,684 5,884 3,672 889 587 Florida 157,536 99,565 159,011 103,089 9,256 6,894 Georgia 77,261 47,171 78,573 49,925 5,805 4,269 Hawaii 12,295 7,627 12,054 7,629 833 598 Idaho 15,165 9,953 15,070 10,378 785 476 Illinois 118,210 80,473 117,521 82,815 7,692 6,076 Indiana 60,872 42,812 60,382 43,302 4,355 3,520 Iowa 38,852 28,729 38,680 29,264 2,592 2,034 Kansas 32,644 22,391 32,338 23,097 2,022 1,580 Kentucky 41,734 27,883 41,834 28,658 2,916 2,214 Louisiana 46,953 27,324 46,815 28,573 3,349 2,487 Maine 24,443 14,842 24,167 15,954 865 582 Maryland 45,975 31,474 45,698 32,435 3,266 2,467 Massachusetts 53,543 37,037 52,988 37,990 5,374 3,950 Michigan 123,933 85,771 123,111 88,400 5,817 4,287 Minnesota 77,962 55,645 77,828 57,762 3,313 2,634 Mississippi 28,396 16,177 28,350 16,829 2,407 1,652 Missouri 64,438 43,493 64,434 44,640 3,962 3,241 Montana 14,248 9,076 14,302 9,482 601 478 Nebraska 25,458 18,002 25,254 18,307 1,252 1,036 Nevada 20,360 12,660 21,334 13,498 815 686 New Hampshire 14,933 9,877 15,078 10,352 858 662 New Jersey 72,896 49,132 72,297 50,641 4,802 3,783 New Mexico 19,901 11,862 19,895 12,397 897 674 New York 183,793 116,910 181,711 121,011 14,249 11,484 North Carolina 83,176 53,038 84,642 55,417 6,225 4,592 North Dakota 11,643 8,066 11,622 8,258 592 502 Ohio 110,366 78,913 109,651 80,011 7,341 5,852 Oklahoma 46,827 28,358 46,478 29,492 2,691 2,184 Oregon 33,884 23,379 33,893 23,785 1,873 1,347

OCR for page 21
27 AMERICAN COMMUNITY SURVEY ESTIMATES ACS 2007 ACS 2008 Housing Units Group Quarters Housing Units Group Quarters Initial Initial Initial Initial Addresses Final Sample Final Addresses Final Sample Final Selected Interview Selected Interview Selected Interview Selected Interview 51,179 32,345 2,699 1,999 51,817 31,973 2,533 2,109 9,751 5,908 465 347 9,749 5,684 901 640 54,928 34,527 2,591 2,062 54,841 34,135 2,735 2,163 31,152 19,422 1,854 1,414 31,571 19,392 1,808 1,376 266,419 176,508 19,498 14,890 265,428 176,249 18,828 15,039 45,155 30,257 2,557 2,009 45,723 30,826 2,459 1,903 28,413 20,762 2,705 2,236 28,158 20,677 2,621 2,203 10,273 6,359 573 447 10,461 6,344 851 699 5,849 3,601 910 582 5,857 3,604 1,043 732 160,855 101,953 9,385 6,685 162,667 102,339 9,284 7,051 79,486 49,623 5,627 4,092 81,535 50,205 5,468 4,349 11,924 7,473 807 457 11,721 7,303 918 590 15,199 10,263 733 446 15,295 10,307 990 641 117,290 81,653 7,233 5,734 117,943 81,731 7,053 5,534 60,320 42,801 4,397 3,256 60,467 42,745 4,253 3,490 38,506 28,584 2,512 2,038 38,901 28,472 2,449 1,965 32,238 22,737 1,927 1,394 32,304 22,409 1,865 1,499 41,916 28,175 2,938 2,277 42,179 28,250 2,843 2,210 46,722 27,905 3,269 2,392 47,083 27,324 3,189 2,254 24,055 15,550 836 539 23,718 15,279 1,010 729 45,627 31,886 3,260 2,284 45,429 31,915 3,088 2,247 52,658 37,141 5,432 4,083 52,596 37,577 5,031 3,963 122,195 86,470 5,835 4,182 121,074 84,987 5,836 4,189 77,808 56,694 3,267 2,601 77,323 56,473 3,182 2,556 28,323 16,369 2,393 1,677 28,934 16,612 2,255 1,773 64,541 43,942 4,011 3,193 64,995 43,767 3,890 3,203 14,259 9,271 587 402 14,294 9,087 979 725 24,841 17,694 1,195 1,016 24,677 17,526 1,192 1,008 21,663 13,403 829 692 22,050 13,540 1,101 946 14,974 10,062 849 680 14,913 10,104 1,098 851 71,804 49,594 4,778 3,696 70,886 49,363 4,820 3,711 20,936 12,588 923 575 21,216 12,792 1,031 801 180,144 118,562 13,610 11,079 178,282 117,120 13,017 10,762 83,367 54,072 6,228 4,672 84,535 54,422 6,071 4,722 11,509 8,083 568 474 11,419 7,841 1,060 836 109,120 78,439 7,261 5,705 108,931 77,738 7,248 5,635 46,598 28,847 2,533 2,089 46,622 28,645 2,560 2,085 33,911 23,489 2,017 1,290 34,068 23,687 2,032 1,437 continued

OCR for page 21
28 ALLOCATING FEDERAL FUNDS TABLE 2-1 Continued ACS 2005 ACS 2006 Housing Units Housing Units Group Quarters Initial Initial Initial Addresses Final Addresses Final Sample Final State Selected Interview Selected Interview Selected Interview Pennsylvania 145,000 101,216 143,856 104,132 10,659 7,888 Rhode Island 8,819 6,110 8,720 6,193 1,001 812 South Carolina 41,029 25,642 41,546 26,804 3,313 2,544 South Dakota 11,678 7,969 11,675 8,234 697 589 Tennessee 54,786 36,339 55,342 37,446 3,646 2,903 Texas 203,497 121,858 205,272 129,186 13,872 10,819 Utah 20,545 14,331 20,813 14,909 987 767 Vermont 12,232 7,677 12,143 8,076 541 382 Virginia 61,445 42,957 61,857 44,699 5,647 4,144 Washington 58,811 40,262 58,784 41,301 3,315 2,282 West Virginia 21,128 13,496 20,880 13,871 1,082 793 Wisconsin 82,755 61,063 82,458 62,489 3,786 2,951 Wyoming 6,031 3,877 6,046 3,877 353 247 United States 2,922,656 1,924,527 2,885,384 1,968,362 189,641 145,311 SOURCE: U.S. Census Bureau, data from: http://www.census.gov/acs/www/methodology/sample_size_ data/ and http://www.census.gov/acs/www/UseData/sse/. areas with 65,000 or more people. By 2008, enough responses had been collected to release the 3-year ACS estimates for 2005-2007. The 3-year estimates cover areas with 20,000 or more people, providing wider tabulation coverage of small geographic areas. By 2010, the first 5-year estimates will have been released, covering 2005-2009. With these estimates, the tabulation coverage of the ACS will have expanded to very small places and neighborhoods, including the areas pertaining to even the smallest local education authorities. Each year, the 1-year, 3-year, and 5-year estimates will be updated to include the most recent data. In addition to the 1-year, 3-year, and 5-year estimates, the Census Bureau has also released ACS 1-year and 3-year PUMS files, and the 5-year PUMS files are scheduled for release early in 2011. PUMS files contain individual and household records, with confidentiality protected through the following means: • deleting names and addresses from the records; • limiting geographic and identification to large areas, known as public-use microdata areas, which are defined to include about 100,000 people; and • limiting the detail that is provided for sensitive variables: for example, as- signing a catchall code to income amounts over a certain threshold, such as $100,000 or more, and not identifying the specific amount.

OCR for page 21
29 AMERICAN COMMUNITY SURVEY ESTIMATES ACS 2007 ACS 2008 Housing Units Group Quarters Housing Units Group Quarters Initial Initial Initial Initial Addresses Final Sample Final Addresses Final Sample Final Selected Interview Selected Interview Selected Interview Selected Interview 142,939 102,116 10,572 7,693 141,995 101,559 10,245 7,443 8,654 6,005 965 699 8,636 5,995 990 704 41,878 26,606 3,415 2,708 42,299 26,991 3,312 2,630 11,612 8,000 696 552 11,610 7,853 1,068 866 55,752 37,279 3,590 2,886 56,490 37,688 3,529 2,829 206,891 127,633 13,024 10,556 211,122 127,639 12,522 10,133 21,082 14,854 969 707 21,234 15,060 1,026 736 12,147 7,984 501 409 11,948 7,802 1,030 781 62,090 44,235 5,783 4,197 62,548 44,223 5,731 4,357 58,642 40,886 3,224 2,352 58,805 40,855 3,095 2,260 20,842 13,632 1,132 900 21,028 13,565 1,118 887 81,905 61,524 3,695 2,861 81,123 60,357 3,716 3,050 6,111 3,893 354 262 6,211 3,924 888 672 2,886,453 1,937,659 187,012 142,468 2,894,711 1,931,955 186,862 145,974 ASSESSMENT OF THE DATA As noted in Chapter 1, Title III of the Elementary and Secondary Education Act requires the U.S. Department of Education (DoEd) to allocate funds to all 50 states, the District of Columbia, and Puerto Rico2 by a formula in which 80 percent is based on the population of children with limited proficiency in English (relative to national counts of this population). The ACS uses a sample of the population to estimate the number of people with limited English proficiency (LEP). The definition of the population of children with limited proficiency in English in the ACS derives from the ACS questionnaire which asks the household respon- dent three questions about the spoken English capability of each household member: see Box 2-1 (also see Chapter 1). The questions are asked of those who are aged 5 years or more. Based on responses to these questions, household members between 5 and 21 years old are categorized as English language learners if the respondent reports that the person speaks a language other than English at home and speaks English less than “very well.” 2 Puerto Rico has a cap; the total amount is not to exceed 0.5 percent of the total amount allotted to all states in a fiscal year.

OCR for page 21
30 ALLOCATING FEDERAL FUNDS BOX 2-1 Question on Language Use from the ACS a. Does this person speak a language other than English at home? Yes, go to b No, skip b and c b. What is this language? (For example: Korean, Italian, Spanish, Vietnamese) c. How well does this person speak English? Very well Well Not well Not at all SOURCE: American Community Survey Questionnaire, Form ACS-1 (INFO)(2010)KFI. Quality of the ACS Language Questions The ACS questions on English speaking ability evolved directly from similar questions on the former census long form. Indeed, the decennial census has collected information on the ability of the population to speak the English language for well over a century, and the question has evolved over time: see Box 2-2. The census ques- tion evolved from a simple English speaking ability question to one which focused on “mother” tongue, and finally in 1980, to the multipart language question that was adopted to fulfill requirements of legislation that sought to identify language limitations which were a source of disadvantage in learning, voting, and access to public services (Kominski, 1989, p. 1). Like other questions on the old census long form, the ones on English speaking ability were incorporated into the ACS during the testing phase and eventually adopted without change. Thus, it is appropriate to review the research used to assess the reasonableness and utility of the language ques- tion as it was asked on the decennial census and to compare the estimates of English speaking ability from the census with the estimates from the ACS. In an article on what “how well” means, Kominski (1989) reported on an independent assessment of English proficiency to validate the multipart question used on the census. Kominski used data from the 1986 National Content Test, a national survey conducted by the Census Bureau to assess new and candidate items for the decennial census. This test included a reinterview survey in which about one-quarter of the original sample was administered follow-up questions. These

OCR for page 21
31 AMERICAN COMMUNITY SURVEY ESTIMATES BOX 2-2 History of the Census Language Questions 2000: Data collected for all ages, retained for persons 5 years and over Does this person speak a language other than English at home? What is this language? How well does this person speak English (very well, well, not well, not at all)? 1990: Data for persons 5 years and over Does this person speak a language other than English at home? What is this language? How well does this person speak English (very well, well, not well, not at all)? 1980: Asked about persons 3 years and older; tabulated for those 5 years and older Does this person speak a language other than English at home? What is this language? How well does this person speak English (very well, well, not well, not at all)? 1970: No age for question, tabulations limited What language, other than English, was spoken in this person’s home when he was a child? (Spanish, French, German, Other (specify)_________, None, English only) 1960: Asked of foreign-born persons What language was spoken in his home before he came to the United States? 1950: No language questions 1940: “Mother tongue (or Native Language)” Language spoken at home in earliest childhood 1930: Asked of foreign-born persons “Mother tongue (or Native Language) of Foreign Born” Language spoken in home before coming to the United States 1920: Place or birth and mother tongue of person and each parent. (May also have used instructions from 1910) 1910: Mother tongue was collected for all foreign-born persons, to be written in with place of birth, also collected for foreign-born parents Ability to speak English “Whether able to speak English; or, if not, give language spoken” (specific instructions on correct languages to write and a list of appropriate Euro- pean languages) 1900: Asked about all persons 10 years and older “Can you speak English” was asked after two questions on literacy, read- ing, and writing. 1890: (All persons 10 years and over) “Able to speak English. If not, the language or dialect spoken.” Asked after two questions on literacy, reading, and writing. 1790- 1880: No known language or English ability questions SOURCE: R. Kominski, U.S. Census Bureau (personal communication, June 15, 2009).

OCR for page 21
48 ALLOCATING FEDERAL FUNDS TABLE 2-9 Absolute Difference in Percentage Share of States Across Years (in percentage) ACS 2006 ACS 2007 ACS 2008 ACS 2006-2008 Compared with Compared with Compared with Compared with ACS 2005 ACS 2006 ACS 2007 ACS 2005-2007 Absolute Absolute Absolute Absolute State Difference Difference Difference Difference Alabama 0.047 0.035 0.080 0.004 Alaska 0.055 0.019 0.056 0.011 Arizona 0.319 0.235 0.156 0.140 Arkansas 0.008 0.027 0.012 0.010 California 1.776 0.446 0.590 0.858 Colorado 0.082 0.075 0.159 0.023 Connecticut 0.027 0.158 0.006 0.061 Delaware 0.028 0.008 0.028 0.021 District of Columbia 0.011 0.013 0.016 0.020 Florida 0.000 0.021 0.300 0.085 Georgia 0.129 0.082 0.361 0.053 Hawaii 0.038 0.051 0.177 0.027 Idaho 0.024 0.009 0.035 0.006 Illinois 0.226 0.153 0.084 0.080 Indiana 0.001 0.080 0.097 0.017 Iowa 0.061 0.073 0.015 0.007 Kansas 0.023 0.006 0.028 0.016 Kentucky 0.018 0.118 0.051 0.003 Louisiana 0.022 0.058 0.069 0.027 Maine 0.027 0.021 0.027 0.004 Maryland 0.154 0.124 0.102 0.029 Massachusetts 0.048 0.126 0.115 0.015 Michigan 0.152 0.024 0.075 0.099 Minnesota 0.150 0.073 0.167 0.076 Mississippi 0.006 0.013 0.006 0.004 Missouri 0.063 0.108 0.065 0.025 Montana 0.005 0.007 0.003 0.007 Nebraska 0.048 0.068 0.019 0.002 Nevada 0.124 0.092 0.131 0.112 New Hampshire 0.048 0.024 0.000 0.020 New Jersey 0.122 0.118 0.178 0.001 New Mexico 0.149 0.172 0.050 0.019 New York 0.631 0.450 0.537 0.214 North Carolina 0.367 0.140 0.192 0.141 North Dakota 0.013 0.013 0.004 0.002 Ohio 0.014 0.065 0.113 0.009 Oklahoma 0.028 0.019 0.025 0.018 Oregon 0.122 0.060 0.110 0.048 Pennsylvania 0.173 0.099 0.092 0.003

OCR for page 21
49 AMERICAN COMMUNITY SURVEY ESTIMATES TABLE 2-9 Continued ACS 2006 ACS 2007 ACS 2008 ACS 2006-2008 Compared with Compared with Compared with Compared with ACS 2005 ACS 2006 ACS 2007 ACS 2005-2007 Absolute Absolute Absolute Absolute State Difference Difference Difference Difference Rhode Island 0.077 0.037 0.020 0.007 South Carolina 0.033 0.006 0.028 0.001 South Dakota 0.022 0.015 0.007 0.006 Tennessee 0.011 0.093 0.042 0.004 Texas 0.282 0.606 0.434 0.506 Utah 0.178 0.037 0.027 0.050 Vermont 0.010 0.002 0.006 0.005 Virginia 0.197 0.386 0.184 0.011 Washington 0.036 0.230 0.009 0.118 West Virginia 0.017 0.008 0.005 0.004 Wisconsin 0.012 0.117 0.167 0.036 Wyoming 0.014 0.007 0.009 0.003 United States 6.230 5.027 5.264 3.066 SOURCE: U.S. Census Bureau Special Tabulations. tion survey, one can examine the sensitivity of the allocations when the criteria are altered slightly. For example, if the goal is to align the ACS data more closely to state counts of ELL children and youth, the group of interest would be those aged 5-18 and enrolled in public school. We examined the effects of changing the criteria in terms of age (5-18 versus 5-21), enrollment status (all enrolled students versus those in public schools only), and English speaking ability (speak English less than very well versus speak English less than well). This analysis was conducted using the 3-year ACS estimates for 2006-2008, with the following steps: 1. We selected as the base definition those aged 5-21 and speaking English less than very well. We calculated the state shares using this definition. 2. We then varied the definition and calculated the revised state shares. 3. We then calculated and summarized differences. Suppose Ax is the state share for state X under the base criteria, and Bx the state’s share under revised criteria (e.g., when the age range is restricted to 5-18). The differ - ence of the two shares is (Bx - Ax). We then took the absolute value of the difference to obtain the absolute difference and summarized these values by their mean across states, as the the mean absolute difference (MAD).

OCR for page 21
50 ALLOCATING FEDERAL FUNDS 4. We also calculated and summarized relative difference. This was cal- culated by dividing the absolute difference by the average of the two shares, (Bx - Ax)/(( Bx + Ax)/2). We then took the mean of these values to calculate the mean absolute relative difference (MARD). The MAD tends to be heavily influenced by differences in large states, the MARD gives comparatively more weight to smaller states. The results of these analyses are presented in Table 2-10. The first row shows the effect of changing the age range to 5-18. The second row shows the effect of restricting students enrolled in any kind of school. The third row shows the results when school enrollment is restricted to those in public school. 8 The fourth row shows the results when both criteria are applied—restricting the population to 5- to 18-year-olds enrolled in public school. In this summary table, we report the statistics for all states and for groups of states classified by their overall share of ELLs under the base allocation as large, medium, small, and “minimum.”9 As noted above, variations in the shares of ELL children and youth of the “minimum” states do not affect their allocations, as they generally fall below the $500,000 threshold. As can be seen in Table 2-11, the variations in age criteria did not influence the allocation of states very much (MAD, 0.06%; MARD, 1.04%). The allocations are more sensitive to restricting estimates to children and youth enrolled in schools (MAD, 0.07%; MARD, 5.46%), and even more so to restricting to those enrolled in public schools (MAD, 0.14%; MARD, 7.92%). Thus, with the latter restriction, states would on the average see a noticeable change (7.92%) in their allocations. This presumably reflects some differences in school enrollment rates among ELL children and youth. The combined restriction by both the age and public school enrollment criteria has a slightly larger effect on allocations (MAD, 0.16%; MARD, 9.58%). For each of the revisions of criteria we considered, the MAD, reflecting the amount of money that would be moved, is largest for the large states (those with the biggest shares of the national population of ELL children and youth). How - ever, the relative impact (measured by the MARD), reflecting the percentage by which a revision would modify a state’s allocation, tends to be larger for the me - dium and small states, for which a small amount of money can be a large percent - age of a state’s allocation. The biggest relative changes are in the “minimum” states, but these would not affect their allocations because they receive a fixed amount. 8 The comparison is only for public schools because state estimates are only available for students in public schools. 9 The large states are Arizona, California, Florida, Georgia, Illinois, New Jersey, New York, Texas, and Washington. The medium states are Colorado, Connecticut, Indiana, Kansas, Maryland, Massachusetts, Michigan, Minnesota, Missouri, North Carolina, New Mexico, Nevada, Pennsylvania, Oregon, Ohio, Oklahoma, South Carolina, Tennessee, Utah, Virginia, and Wisconsin. The small states are Alabama, Alaska, Arkansas, Delaware, Hawaii, Idaho, Iowa, Kentucky, Louisiana, Maine, Mississippi, Nebraska, New Hampshire, Rhode Island, and West Virginia. Minimum allocation states are the District of Co - lumbia, Montana, North Dakota, South Dakota, Vermont, and Wyoming.

OCR for page 21
51 AMERICAN COMMUNITY SURVEY ESTIMATES TABLE 2-10 Difference in Percentage Share of ELL Students of States by Varying Age Groups, Enrollment Status, and Type of School (in percentage) Base Category: Children and Youth Aged 5-21 Who Speak English Less Than “Very Well” Alternatives to Mean Absolute Mean Absolute Difference in Sharea Relative Differenceb Base Category Age Group: 5-18 years old 0.06 1.04 All Large 0.21 0.75 Medium 0.03 0.74 Small 0.01 1.24 Minimum 0.00 2.01 Enrollment Status: Enrolled in School 0.07 5.46 All Large 0.26 4.23 Medium 0.06 5.08 Small 0.02 6.61 Minimum 0.00 5.77 Type of School: Public Schools 0.14 7.92 All Large 0.57 6.57 Medium 0.07 6.61 Small 0.02 9.11 Minimum 0.01 11.52 5-18 Years Old, Public Schools: 0.16 9.58 All Large 0.67 7.45 Medium 0.08 7.49 Small 0.03 10.91 Minimum 0.01 16.74 aThe mean absolute difference in share is calculated by taking an average of absolute difference in share of all states and group of states. bThe mean absolute relative difference in share is calculated by taking an average of absolute relative difference in share of all states and group of states. CONCLUSION 2-3 The 3-year American Community Survey (ACS) esti- mates of English language learner (ELL) children and youth are relatively insensitive to definitional changes in age range, but they are sensitive to changes in enrollment status and type of school. Consequently, adjust- ing the age group used in the ACS definition of ELL children and youth from 5-21 years of age to 5-18 years of age will have little effect on the percentage share of Title III funds going to the states, but changing the

OCR for page 21
52 ALLOCATING FEDERAL FUNDS TABLE 2-11 Difference in Percentage Share of ELL Students of States by Varying ELL Criterion Alternatives to Mean Absolute Mean Absolute Base Category Difference in Share Relative Difference Overall Rate Base Category: Children and Youth Aged 5-21 Who Speak English Less Than Very Well Speaking English Less Than Well 0.17 11.71 38.58 All Large 0.59 5.37 37.82 Medium 0.11 9.71 40.14 Small 0.05 16.82 42.16 Minimum 0.01 15.40 40.57 Base Category: Children and Youth Aged 5-18 Public School Enrolled Who Speak English Less Than Very Well Speaking English Less Than Well 0.27 16.00 30.88 All Large 1.07 7.82 29.77 Medium 0.14 13.21 33.54 Small 0.06 21.44 35.13 Minimum 0.02 24.43 40.27 NOTES: Large States: Arizona, California, Florida, Georgia, Illinois, New Jersey, New York, Texas, Washington. Medium States: Colorado, Connecticut, Indiana, Kansas, Maryland, Massachusetts, Michigan, Minne - sota, Missouri, North Carolina, New Mexico, Nevada, Pennsylvania, Oregon, Ohio, Oklahoma, South Carolina, Tennessee, Utah, Virginia, Wisconsin. Small States: Alabama, Alaska, Arkansas, Delaware, Hawaii, Idaho, Iowa, Kentucky, Louisiana, Maine, Mississippi, Nebraska, New Hampshire, Rhode Island, West Virginia. Minimum States: District of Columbia, Montana, North Dakota, South Dakota, Vermont, Wyoming. enrollment status definition to limit the group to public school children and youth would have a measurable effect on the shares. In this regard, the ACS measure is more closely aligned with the statutory language than are the figures provided by state education authorities. Sensitivity to Variations in Cut Points Another sensitivity analysis considered the allocation effects of alternative ACS proficiency cut points. Currently, an English language learner is defined as one who speaks English “less than very well.” Using special tabulations provided by the Cen - sus Bureau, we examined the impact of changing the proficiency criterion to “less than well,” which has the effect of considering those who speak English “well” as

OCR for page 21
53 AMERICAN COMMUNITY SURVEY ESTIMATES proficient rather than nonproficient. This effect was examined under two different assumptions about the age range and school enrollment criteria corresponding to the base category (5- to 21-year-olds) and last rows (5- to 18-year-olds in public schools) of Table 2-9. The results, presented in Table 2-10, show that ACS estimates are more sensi - tive to this change of cut point (MAD, 0.17%; MARD, 11.71%) than to changes in age range and enrollment status. The impact is even greater with the stricter age and enrollment criteria (MAD, 0.27%; MARD. 16.00%). This result is not surpris - ing given that those speaking English “less than well” constitute only about one- third of those speaking English “less than very well” (39% in the less restrictive age enrollment criteria; 31% with the more restrictive criteria). Given the variation in ethnic composition, country of origin, and recency of immigration of the immigrant populations of the various states, the distribution of ELL children and youth across the nonproficient categories on the ACS is likely to vary as well. In view of the strong sensitivity of the estimates of ELL students to the cut points selected, the continued use of “less than very well” as the cutoff used in ACS to define English language learners is warranted. This determination is consistent with evidence cited earlier in this chapter that even though the language question in ACS is not able to precisely distinguish between the four categories of English speak- ers, it does differentiate between the worst and best speakers of English language. CONCLUSION 2-4 The American Community Survey estimates of Eng- lish language learner (ELL) children and youth are very sensitive to cutoff points in the ELL definition. Changing the criterion from “less than very well” to “less than well” can bring about substantial changes in a state’s share of the total number of ELL children and youth, and, consequently, in the state’s allocation. We return to this topic in Chapter 5, which presents further evidence bearing on the choice of cut point. Reporting of Type of School The ACS asks whether each student attends “public” or “private” school. We know of no assessment of the accuracy of the responses to this question. In particular, charter schools are regarded as public schools for statistical purposes, but because they are often regarded by parents as an alternative to regular district-administered schools, they might be misreported as private. This reporting could affect estimates of public school ELL rates if charter schools have different rates of ELL enrollment than district-administered schools, but it would affect neither estimates of total ELL students nor those of total ELL children.

OCR for page 21
54 ALLOCATING FEDERAL FUNDS Coverage Error The ACS provides yearly survey data on important economic and social characteristics of the U.S. population, but the definition of that population has changed over time in ways that have introduced coverage error. The ACS for 2005 covered the household population, while the 2006, 2007, and 2008 ACS covered not only the household population, but also people who live in college dormitories, armed forces barracks, prisons, nursing homes, correctional institu - tions, and other group quarters.10 The decision to include or exclude housing units of a certain type introduces coverage error. There are two kinds of coverage error: undercoverage (when housing units or people do not have a chance of being selected in the sample) and overcoverage (when housing units or people have more than one chance of being selected in the sample or are included in the sample when they should not have been).11 If the characteristics of undercovered or overcovered housing units or individuals differ from those that are selected, the ACS may not provide an accurate picture of the population. ACS reduces coverage error by controlling specific survey estimates to inde- pendent population controls12 by sex, age, race, and Hispanic origin for population estimates and to independent housing unit controls for housing unit estimates. The Census Bureau calculates coverage rates to measure coverage error in the ACS, and these rates are weighted to reflect the probability of selection into the sample, the subsampling for personal visit follow-up, and nonresponse. As the coverage rate drops below 100 percent, the weights of the people in the survey need greater ad - justment in the final weighting procedure to reach the independent estimate. If the rate is greater than 100 percent, the ACS population estimates are downweighted to match the independent estimates. Independent population estimates are produced by the Census Bureau using independent data on such characteristics as housing, births, deaths, and immigration. The base for these independent estimates is the decennial census. The coverage rates for housing units, group quarters, and the total population for 2005-2008 are shown in Table 2-12. The coverage rate for the total popula - tion for 2008 was 93.8 percent, and that for the Hispanic population was 92.5 percent. On the basis of these data, it can be postulated that coverage error is not a significant concern for the ELL estimates. 10 Residences that are not in ACS but were part of the census long-form sample are circus quarters, crews on merchant ships, domestic violence shelters, recreational vehicles in campground, soup kitchen or mobile food van sites, and street location for the homeless. 11 Overcoverage occurs when units or people have multiple chances of selection; for example, addresses listed more than once on the frame, or people included on a household roster at two different sampled addresses. For details see: Census Bureau, ACS Design and Methodology, Chapter 15, http://www.census. gov/acs/www/Downloads/survey_methodology/acs_design_methodology_ch15.pdf [December 2010]. 12 The use of population controls can introduce another source of error (National Research Council, 2007, pp. 201-208).

OCR for page 21
TABLE 2-12 Coverage Rates for Housing Units, Group Quarters, and Total Population (in percentage*) Groups Housing Quarters Units Population Total Population Native American Hawaiian Indian and Other and Alaska Pacific White Black Native Asian Islander Non- Non- Non- Non- Non- Year Total Total Total Male Female Hispanic Hispanic Hispanic Hispanic Hispanic Hispanic 2008 98.7 80.8 93.8 92.6 95.0 94.7 89.7 96.2 96.9 85.8 92.5 2007 98.5 79.6 94.2 93.2 95.2 95.4 89.1 96.8 95.6 96.1 92.8 2006 98.7 76.2 94.4 93.4 95.3 95.6 89.6 98.0 93.4 93.0 92.9 2005 98.5 N/A 95.1 93.9 96.2 96.3 90.7 97.9 94.5 84.0 93.6 *The Census Bureau does not calculate coverage rates of gender groups cross-tabulated by racial groups (e.g., white non-Hispanic male). SOURCE: Data from http://www.census.gov/acs/www/acs-php/quality_measures_coverage_2008.php [June 2010]. 55

OCR for page 21
56 ALLOCATING FEDERAL FUNDS TABLE 2-13 Allocation Rates for Language Questions in ACS, for United States* (in percentage) Item 2008 2007 2006 2005 Speaks another language at home 3.1 2.2 2.0 1.7 total population 5 years and over Language spoken 5.3 4.4 4.2 4.0 total population 5 years and over who speak another language at home English ability 3.9 3.1 2.8 2.5 total population 5 years and over who speak another language at home *The item allocation rates for year 2005 are for housing units only. The item allocation rates for 2006 to 2008 include housing units and group quarters populations. SOURCE: Census Bureau Quality Measures Page, available: http://www.census.gov/acs/www/UseData/ sse/ita/ita_def.htm [accessed May 2010]. Nonresponse Error The population of interest under Title III is a relatively small subgroup of the population, and the quality of the data for this group is very sensitive to item nonresponse to the questions that are used as criteria for the ELL definition. The Census Bureau does adjust for nonresponse, using methods of imputation that fall into two categories: “assignment,” using the a response to one question that implies the value for a missing response to another question, and “allocation,” using statisti - cal procedures such as within-household or nearest-neighbor matrices populated by donors. Item nonresponse is measured through the calculation of an allocation rate. The formula for allocation rate13 of an item (A) for a particular state (x) in a year (y) is given as follows: ( ) total number of responses allocated for item A in state x for year y) Allocation rate for = * 100 item A (state x, year y) total number of responses to item A required in state x for year y The allocation rate for United States is calculated by summing over the total number of responses allocated and responses required for an item across all states. The overall item allocation rate for the questions determining ELL status for 2005- 2008 is from the Census Bureau.14 As shown in Table 2-13, the number of responses allocated or imputed re- 13 From the Census Bureau, see http://www.census.gov/acs/www/UseData/sse/ita/ita_def.htm [May 2010]. 14 The item allocation rates for 2005 are for housing units only; the item allocation rates for 2006 to 2008 include housing units and group quarters populations.

OCR for page 21
57 AMERICAN COMMUNITY SURVEY ESTIMATES sponses for “Speaks another language at home,” “Language spoken,” and “English ability” items are very low. We note that the amount of imputation over the period from 2005 to 2007 for all items has increased, which relates to the issue of response rate to surveys in general. The amount of imputation is also of concern because it introduces a vari - ability that is not currently factored into the estimates of sampling errors from the ACS (National Research Council, 2007, p. 254). CONCLUSION 2-5 Item nonresponse is a troublesome and growing issue for items used in the calculation of the number of English language learner children and youth.

OCR for page 21