PART I
Using the American Community Survey



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 27
Using the American Community Survey: Benefits and Challenges PART I Using the American Community Survey

OCR for page 27
Using the American Community Survey: Benefits and Challenges This page intentionally left blank.

OCR for page 27
Using the American Community Survey: Benefits and Challenges 2 Essentials for Users This chapter addresses users of the decennial census long-form sample who want to know, in general terms, what benefits—and challenges—the new American Community Survey (ACS) presents to them. The chapter first summarizes the basics that every user should know about the ACS and key ways in which it is similar to and differs from the decennial census long-form sample. It then addresses two central issues: (1) why users should care about the ACS in terms of the benefits it offers and (2) some of the challenges those benefits present for users. Finally, it offers the panel’s assessment of the value of the ACS to users based on the available knowledge about its properties. 2-A ACS DESIGN BASICS To work with data from the ACS, users should be acquainted with the following features of its design and operations: the population or universe covered, rules for assigning people to a place of residence, questionnaire content and reference periods, sample size and design, data collection procedures, data products, and data-processing procedures to generate the products. The key factor to keep in mind is that, unlike the census long-form sample, the ACS is continuous: a fresh sample of addresses is surveyed every month, and data products represent cumulations of monthly data for 1-year, 3-year, and 5-year periods. The discussion below pertains to the ACS

OCR for page 27
Using the American Community Survey: Benefits and Challenges BOX 2-1 The Puerto Rico Community Survey (PRCS) The Puerto Rico Community Survey (PRCS) is identical in most respects to the American Community Survey in the 50 states and the District of Columbia. There were no PRCS test surveys or test sites for Puerto Rico in 2000–2004, so that 2005 PRCS data are the first post-2000 long-form-type data available for Puerto Rico. Following the same basic design as the ACS, the initial sample size of the PRCS is 3,000 housing units each month or 36,000 housing units each year—about 2.4 percent of the total of about 1.5 million residential addresses in Puerto Rico. Initial sampling rates for blocks vary by the estimate of occupied housing units in the governmental jurisdiction or census tract (see Table 2-3, Part A), although the PRCS rates are slightly higher than the ACS rates (see U.S. Census Bureau, 2006:Table 4.1). Data collection in the PRCS uses mailout, CATI, and CAPI, like the ACS. However, because of low mail response, all mailout/CATI nonrespondents are sampled at a 50 percent rate for the CAPI follow-up as of June 2005. Areas for which PRCS products are published include: 78 municipios (county equivalents): 12 will have 1-year estimates; 65 will have 3-year estimates 455 barrios (subdivisions of municipios, similar to minor civil divisions): 5 will have 1-year estimates; 34 will have 3-year estimates (based on 2000 census counts) 225 zonas urbanas (census designated places that are governmental centers of municipios) and communidads (other census designated places): 9 will have 1-year estimates; 20 will have 3-year estimates (based on 2000 census counts) 871 census tracts and 2,477 block groups (5-year estimates only) The PRCS is explained in greater detail by U.S. Census Bureau (2006). in the United States; see Box 2-1 for a brief overview of the Puerto Rico Community Survey (PRCS).1 2-A.1 Population Coverage (Universe) The ACS for 2005 covered the household population. The 2006 ACS covered not only the household population, but also people who live in college dormitories, armed forces barracks, prisons, nursing homes, and other group quarters. The 2006 ACS population coverage was the same as the census long-form-sample coverage, except that the ACS did not in- 1 All of Section 2-A draws heavily on U.S. Census Bureau (2006).

OCR for page 27
Using the American Community Survey: Benefits and Challenges clude people found at soup kitchens or street locations frequented by the homeless and a few other transient situations. Table 2-1 lists the types of residences included in the 2006 ACS. 2-A.2 Residence Rules The ACS instructs the respondent for a household to provide data on all people who, at the time of filling out the questionnaire, are living or staying at the household address for more than 2 months (including usual residents who are away for less than 2 months). In contrast, the long-form sample asked household respondents to report all people who usually lived at the address as of Census Day, April 1, meaning they lived or stayed there most of the time. People whom the ACS samples in group quarters beginning in 2006 are counted at the group quarters location, in effect applying a de facto residence rule regardless of how long an individual has lived or expects to live in the group quarters. The long-form sample also in effect generally applied a de facto residence rule for group quarters residents, although residents of some types of group quarters were allowed the option of indicating another usual place of residence. (An unduplication process was used to determine the correct enumeration for people listed at the group quarters and the other residence; such a process would not be possible for the ACS because it is not embedded in a census.) For many people, their ACS residence will be the same as their long-form-sample residence. However, some people may report a different residence: for example, people who live in a house or apartment in New York most of the year but reside in Florida in December through March should report Florida as their address if sampled for the ACS in Florida in the winter, whereas their Census Day address is in New York. 2-A.3 Content and Reference Periods The 2005 ACS includes about 55 questions for every person and 30 questions for every housing unit in the sample—approximately the same content as in the 2000 census long-form sample. There are some differences: The ACS mail questionnaire uses a matrix layout for questions on sex, age, race, ethnicity, and household relationship, compared with a person-by-person format in the long-form questionnaire. The ACS mail questionnaire provides room to respond for 5 household members compared with 6 on the long-form questionnaire (telephone follow-up is used to obtain information on additional household members).

OCR for page 27
Using the American Community Survey: Benefits and Challenges TABLE 2-1 Types of Residences in the American Community Survey (ACS) Residence Type 2000 Population (Percentage) Housing Unit Residencea 97.2     Single-family, detached   64.6   Single-family, attached   5.4   2-or-more-unit structure   20.4     In an apartment building (including condominium or co-op)         In an assisted living facility with separate apartments         In a group quarters (e.g., house master’s residence)         In a home (e.g., basement apartment, upstairs apartment)         In multi-unit military family housing on or off base       Mobile home that is occupied or, if vacant, that is permanently sited   6.7   Boat at a mooring, RV, or occupied van   0.1 Institutional Group Quarters Residence (beginning in 2006 ACS) 1.4     Nursing home or other long-term care facility   0.6   Correctional institution (for example, prison or jail)   0.7   Other institutions (for example, hospital or residential school for people with disabilities, long-term care home for juveniles)   0.1 Noninstitutional Group Quarters Residence (beginning in 2006 ACS) 1.3     College dormitory   0.7   Military quarters (in barracks on a base; on a ship assigned to home port)   0.1   Other noninstitutional group quarters   0.5b     Residence in the ACS and the 2000 Long-Form Sample           Convent, monastery           Group home           Halfway home           Hospice           Job Corps center           Migrant worker quarters           Shelter, emergency shelter           YMCA, YWCA, hostel         Residence NOT in the ACS but in the 2000 Long-Form Sample           Circus quarters           Crews on merchant ships           Domestic violence shelter           Recreational vehicle in a campground           Soup kitchen or mobile food van site           Street location for the homeless     aHousing units are separate living quarters with direct access from the outside or through a common hall (U.S. Census Bureau, 2006:D-17). bIncludes 170,706 people (0.06 percent of the population of 281.4 million in 2000) living in emergency shelters for the homeless, shelters for runaway children, transitional shelters, and hotels and motels used to provide shelter for people without conventional housing (U.S. Census Bureau, 2001). SOURCES: Types of residences adapted from U.S. Census Bureau (2006:Ch. 8, Attachment A); population percentages from http://factfinder.census.gov, Summary File 1, Table P37; Summary File 3, Table H33.

OCR for page 27
Using the American Community Survey: Benefits and Challenges Many ACS items refer to a time period different from that of the corresponding items on the long-form questionnaire: for example, usual hours worked per week, weeks worked per year, and income items on the ACS refer to the 12 months prior to the day when the household filled out the questionnaire, whereas these items on the long form always referred to the previous calendar year (1999 for the 2000 census). The ACS currently includes three items not on the 2000 long form: (1) whether the household received food stamps in the previous 12 months and their value; (2) the length of time and main reason for staying at the address (for example, permanent home, vacation home, to attend school or college); and, for women ages 15–50, whether they gave birth to any children in the past 12 months. Table 2-2 compares the items on the 2005 ACS questionnaire with the items on the 2000 census long form. The Census Bureau is proposing several changes to the ACS questionnaire beginning in 2008. These changes, if approved, will include the addition of three new questions on marital history, health insurance coverage, and veterans’ service-related disability, the deletion of the question on length of time and main reason for staying at the address, changes to the basic demographic items for consistency with the 2010 census questionnaire, and changes in wording and format to improve reporting of several other questions as determined by a 2006 test. A question on field of bachelor’s degree will be tested in 2007 and may be added to the ACS beginning in 2009. 2-A.4 Sample Design and Size The ACS sends out questionnaires to about 250,000 housing unit addresses every month that have been sampled from the Census Bureau’s Master Address File (MAF; see Chapter 4 for details of the sampling operation). Each month’s sample includes addresses in every one of the nation’s 3,141 counties. The monthly samples cumulate to about 3 million addresses over a year, or about 2.3 percent of the total number of about 129.5 million housing unit addresses in the United States in 2005. The sample is constructed so that no housing unit address will be included more than once every 5 years. The ACS sample is very large compared with the samples for major national household surveys. However, the long-form sample was even larger: in 2000, the long form was sent to about 18 million addresses, or one-sixth of total housing unit addresses in the United States at the time, and 16.4 million usable long-form questionnaires were included in the final edited data file. The ACS monthly and even yearly samples cannot be as large as

OCR for page 27
Using the American Community Survey: Benefits and Challenges TABLE 2-2 Items on the 2005 ACS Questionnaire and the 2000 Census Long Form 2005 ACS Item (in question order) Asked in 2000 Census? Person Items   Sex Yes (short-form item) Age (at interview) Yes (as of April 1, short form) Date of birth (month, day, year) Yes (short form) Relationship to household reference person (person 1) Yes (more detail, short form) Marital status Yes Hispanic origin Yes (short form) Race (option for multiple races) Yes (short form) Place of birth Yes Citizenship Yes Year of immigration Yes Attended school in last 3 months Yes (since February 1, 2000) Grade attending Yes Highest degree completed Yes Ancestry or ethnic origin Yes Language spoken at home Yes How well speaks English Yes Place of residence 1 year ago (city or town, county, state) Yes (5 years ago) Disability involving sight or hearing Yes Disability limiting physical activity Yes Difficulty learning, remembering due to disability of 6 or more months Yes Difficulty dressing, bathing, or getting around the home Yes For people ages 15 and older   Difficulty going outside the home alone to shop, etc. Yes (ages 16 and older) Difficulty working at a job or business Yes (ages 16 and older) Given birth in past 12 months (women ages 15–50) No Responsible for own grandchildren in the home Yes How long responsible for grandchildren Yes Veteran status (active duty) Yes Period of active military service Yes Number of automobiles, vans, trucks for use by household members Yes Years of active military service (less than 2 years, 2 years or more) Yes Working last week for pay or profit Yes Place of work (address) Yes Usual means of transportation to work last week Yes If by car, truck, or van, how many people used it Yes Time left home for work Yes Minutes to work Yes On layoff last week Yes Temporarily absent from work last week Yes Whether will be recalled to work Yes Looking for work last 4 weeks Yes Could have worked last week Yes

OCR for page 27
Using the American Community Survey: Benefits and Challenges 2005 ACS Item (in question order) Asked in 2000 Census? When last worked Yes Weeks worked, last 12 months Yes (1999) Hours usually worked per week, last 12 months Yes (1999) Class of worker of current or most recent employment Yes Industry of current or most recent employment Yes Occupation of current or most recent employment Yes Wage and salary income, last 12 months Yes (1999) Self-employment income (farm and nonfarm), last 12 months Yes (1999) Interest, dividend, net rent, royalty, and trust income, last 12 months Yes (1999) Social Security income, last 12 months Yes (1999) Supplemental Security Income, last 12 months Yes (1999) State or local public assistance income, last 12 months Yes (1999) Retirement, survivor, or disability pension income, last 12 months Yes (1999) Any other regular income, last 12 months Yes (1999) Total income, last 12 months Yes (1999) Housing Items   Type of building/number of units in structure Yes Year building built Yes When household reference person (person 1) moved in Yes Number of acres on property (single-family or mobile home) Yes Agricultural sales, last 12 months (single-family or mobile home on 1 or more acres) Yes (1999) Whether business on property (single-family or mobile home) Yes Rooms in unit Yes Bedrooms in unit Yes Complete plumbing facilities Yes Complete kitchen facilities Yes Telephone service available Yes Number of automobiles, vans, trucks for use by household members Yes Heating fuel most used Yes Electricity cost, last month Yes (annual cost) Gas cost, last month Yes (annual cost) Water and sewer cost, last 12 months Yes (annual cost) Oil, coal, kerosene cost, last 12 months Yes (annual cost) Receive food stamps, value last 12 months No Monthly condominium fee Yes Owner or renter (tenure) Yes (short-form item) Monthly rent (and whether includes various utilities) Yes Whether rent includes meals Yes Value of property if were for sale Yes

OCR for page 27
Using the American Community Survey: Benefits and Challenges 2005 ACS Item (in question order) Asked in 2000 Census? Annual real estate taxes Yes (last year) Annual hazard insurance Yes Monthly mortgage payment Yes Whether mortgage payment includes taxes Yes Whether mortgage payment includes insurance Yes Whether a second mortgage and/or home equity loan Yes Second mortgage/home equity loan monthly payment Yes Annual costs for mobile home and site (personal property taxes, site rent, fees and licenses) Yes (last year, also includes installment loans) Whether any household members live here year round No Number of months members live here No Main reason members stay at this address No NOTES: The 2005 ACS and 2000 census long-form sample provided room on the mailback questionnaire for characteristics of up to 5 and 6 household members, respectively. Questionnaires should be consulted for precise question wording. SOURCES: See http://www.census.gov/acs/www/SBasics/SQuest/SQuest1.htm for the 2005 ACS; Anderson (2000:388-399) for the 2000 long form. the long-form sample because the costs would be too great. Accumulated over 5 years, the ACS sample will total about 15 million housing unit addresses, but the ACS sample is then reduced by the subsampling for in-person follow-up of households not responding to the mail and telephone data collection procedures (see below). This subsampling may reduce the ACS 5-year sample to 10–11 million housing unit addresses. Because data on governmental jurisdictions will be an important output of the ACS and because many governmental units are very small in population size, the ACS oversamples housing unit addresses in small governmental units relative to other areas similar to the 2000 long-form-sample design. Oversampling provides more precise estimates for small counties, places, townships, school districts, and American Indian and Alaska Native areas than would otherwise be possible. In a similar manner, for the personal visit follow-up operation, the ACS oversamples mail and telephone nonrespondents in census tracts that are expected to have low mail and telephone response rates relative to other census tracts. In order to afford the costs for the additional follow-up, not only are smaller subsamples followed up in person in census tracts that are expected to have high mail and telephone response, but also the initial sample for these tracts is reduced by 8 percent. Tables 2-3a, 2-3b, and 2-3c provide initial annual and 5-year sampling rates for governmental units and census tracts of different popu-

OCR for page 27
Using the American Community Survey: Benefits and Challenges TABLE 2-3a Housing Unit Addresses, 2005 ACS and 2000 Census Long-Form Sample: Approximate Initial Block-Level Sampling Rates Type and Size of Smallest Area Containing a Block 2005 American Community Survey 2000 Long-Form- Sample Census Day Sampling Rate Annual Initial Sampling Rate Cumulative 5-Year Initial Sampling Rate Governmental unit (county, place, township in 12 states, school district, American Indian or Alaska Native area)       With < 200 occupied housing units (fewer than about 500 people) 10.0% (1 in 10) 50.0% (1 in 2) 50.0% (1 in 2) With 200–800 occupied housing units (about 500-2,000 people) 6.9% ( 1 in 14) 34.5% (1 in 3) 50.0% (1 in 2) With 800–1,200 occupied housing units (about 2,000-3,000 people) 3.5% (1 in 28) 17.5% (1 in 6) 25.0% (1 in 4) Census tract with > 2,000 occupied housing units (more than about 5,000 people)a 1.7% (1 in 59) or 1.6% (1 in 63) 8.5% (1 in 12) or 8.0% (1 in 13) 12.5% (1 in 8) Other areaa 2.3% (1 in 44) or 2.1% (1 in 48) 11.5% (1 in 9) or 10.5% (1 in 10) 16.7% (1 in 6) Overal l 2.3% (1 in 44) 11.5% (1 in 9) 16.7% (1 in 6) NOTES: Number of occupied housing units is estimated from the MAF. Because the initial ACS sample size will be kept at approximately 3 million residential addresses per year, the initial sampling rates shown will be slightly reduced as the number of occupied housing units grows. Townships and other minor civil divisions are recognized for sampling purposes in 12 states where they are functioning governments: Conne cticut, Maine, Massachusetts, Michigan, Minnesota, New Hampshire, New Jersey, New York, Pennsylvania, Rhode Island, Vermont, and Wisconsin. aThe smaller of the two ACS sampling rates shown applies for blocks in census tracts with predicted mail/CATI response rates gre ater than 60% (see Table 2-3b). SOURCE: Adapted from U.S. Census Bureau (2006:Tables 4.1, 4.2) for the ACS.

OCR for page 27
Using the American Community Survey: Benefits and Challenges by the original design, but also by nonresponse and, in the case of the ACS, by the extent of CAPI subsampling that is done for personal visit follow-up to contain costs. Tables 2-7a, 2-7b, and 2-7c provide rough, approximate estimates of sampling error for an estimated 15 percent poor school-age children from the ACS (1-year, 3-year, and 5-year period estimates) and the 2000 long-form sample for areas ranging in population from 500 to 2.5 million people. The calculations assume that school-age children are 20 percent of the total population and that areas with 3,000 or fewer people are oversampled. The calculations take account—for both the ACS and the long-form sample—of the added sampling error from household nonresponse but not the added error from item nonresponse. Specifically, Table 2-7a shows relative standard errors—that is, the standard error as a percentage of the estimate, also called the coefficient of variation (see Box 2-5). Table 2-7b shows approximate 90 percent margins of error (MOEs) plus or minus the estimate of 15 percent poor school-age children for each size area (90 percent MOEs are 1.65 times the corresponding standard error). Finally, Table 2-7c translates the MOEs into 90 percent confidence intervals surrounding the 15 percent school-age poverty estimates. The tables and text use 90 percent MOEs and confidence levels to follow the long-standing practice of Census Bureau publications; however, this practice is not standard in statistical work. It gives smaller MOEs and confidence intervals than is the case when the 95 percent standard is used: with the 95 percent standard, the MOEs and confidence intervals would be about 20 percent larger. The panel developed the sampling error estimates in the tables by starting with a generalized variance estimation function provided by the Census Bureau for the 2000 long-form sample; we then computed the sampling error estimates for the ACS as multiples of the long-form-sample estimates (see notes at the end of Table 2-7c). The multiplication factors are derived from Census Bureau research with the ACS test sites, the C2SS, and the 2001–2004 ACS test surveys. For the 2005 ACS, the Census Bureau directly estimated the sampling errors for specific estimates, including not only school-age poverty, but also other characteristics, using a repeated replication method (U.S. Census Bureau, 2006:Ch.12). The 2005 ACS data were only recently released, however, and the panel was not able to analyze their sampling errors; moreover, these estimates pertain only to areas with 65,000 or more people. Nevertheless, an unsystematic examination of the sampling errors for selected 2005 ACS poverty estimates suggests that they are similar to those shown in Tables 2-7a, 2-7b, and 2-7c.

OCR for page 27
Using the American Community Survey: Benefits and Challenges 2-C.2.c Assessment of Sampling Error from Illustrative Estimates Looking at Table 2-7a for the ACS 5-year period estimates, the relative standard errors, or coefficients of variation, for all but the smallest governmental units are half again as large (51 percent) as the corresponding relative standard errors for the 2000 long-form sample. For the ACS 3-year period estimates, the relative standard errors are, in turn, almost 30 percent larger than those for the ACS 5-year estimates and 95 percent larger than those for the long-form-sample estimates. For the ACS 1-year period estimates, the relative standard errors are more than 2 times larger than those for the ACS 5-year period estimates and more than 3 times larger than those for the long-form-sample estimates. To illustrate, consider first the best case shown in Tables 2-7a, 2-7b, and 2-7c, which is the long-form-sample estimate of 15 percent poor school-age children for an area with 2.5 million people. For this estimate, the relative standard error is only 1.1 percentage points, the 90 percent MOE is only ±0.3 percentage points, and the 90 percent confidence interval is quite narrow—14.7 to 15.3 percent. In other words, the estimate is very precise and provides useful information for a variety of applications, such as fund allocation and program planning. For the same estimate for the same size area from ACS data accumulated over 5 years, the relative standard error is only somewhat larger at 1.7 percentage points, and the ACS data have the advantage of being more up to date. At the other extreme, the worst case is for estimates of 15 percent poor school-age children for areas with 500 people. These areas are oversampled in both the ACS and the long-form sample, but the sample sizes are so small that the estimates are very imprecise. The 90 percent confidence interval for the estimate of 15 percent poor school-age children from the long-form sample ranges from 5.8 to 24.2 percent poor (90 percent MOE of ±9.2 percentage points), while that from the ACS 5-year period estimates ranges from 3.9 to 26.1 percent poor (90 percent MOE of ±11.1 percentage points). Intervals this wide are not helpful to users, and the range would be wider yet for areas with 500 people that are not oversampled—for example, a township in one of the 38 states for which the Census Bureau does not recognize townships as functioning governments for purposes of oversampling (refer back to Table 2-3), or a block group in a large area. What constitutes an acceptable level of precision for a survey estimate depends on the uses to be made of the estimate. A commonly used standard for many uses is that a sample estimate should have a relative standard error, or coefficient of variation, of 10 percent or less—sometimes increased to 12 percent or less for a characteristic like poverty, which is clustered within a household or family. This standard does not apply in some instances: specifically, for estimates of proportions that are less than 5 percent

OCR for page 27
Using the American Community Survey: Benefits and Challenges TABLE 2-7a Illustrative, Approximate Relative Standard Errors (Coefficients of Variation, or CVs) for an Estimate of 15 Percent Poor School-Age Children from the ACS and the 2000 Census Long-Form Sample, by Population Size of Area Population Size of Area (1) Children Ages 5–17 ACS 1-Year Period Estimate ACS 3-Year Period Estimate ACS 5-Year Period Estimate 2000 Long-Form Sample Estimate Total (20% of total pop.) (2) Poor (15% of ages 5–17) (3) CV(%) (4a) (Sample Cases) (4b) CV(%) (5a) (Sample Cases) (5b) CV(%) (6a) (Sample Cases) (6b) CV(%) (7a) (Sample Cases) (7b) 2,500,000 500,000 75,000 3.8 % (7,900) 2.2 % (23,700) 1.7 % (39,550) 1.1 % (77,500) 1,000,000 200,000 30,000 6.0 (3,150) 3.5 (9,500) 2.7 (15,800) 1.8 (31,000) 500,000 100,000 15,000 8.5 (1,600) 4.9 (4,750) 3.8 (7,900) 2.5 (15,550) 250,000 50,000 7,500 12.1 (800) 7.0 (2,350) 5.4 (3,950) 3.6 (7,750) 100,000 20,000 3,000 19.1 (300) 11.0 (950) 8.5 (1,600) 5.6 (3,100) 65,000 13,000 1,950 23.7 (200) 13.6 (600) 10.6 (1,050) 7.0 (2,000) 50,000 10,000 1,500 27.0 (150) 15.6 (450) 12.1 (800) 8.0 (1,550) 25,000 5,000 750 38.2 (80) 22.0 (250) 17.1 (400) 11.3 (800) 20,000 4,000 600 42.7 (60) 24.6 (200) 19.1 (300) 12.6 (600) 10,000 2,000 300 60.4 (30) 34.8 (90) 27.0 (150) 17.9 (300) 5,000 1,000 150 85.4 (20) 49.2 (50) 38.1 (100) 25.2 (150) 3,000 600 90 95.6 (10) 55.0 (40) 42.7 (80) 28.3 (150) 1,500 300 45 72.8 (10) 41.9 (40) 32.5 (70) 21.5 (150) 500 100 15 100.2 (10) 57.7 (20) 44.7 (30) 37.3 (50) NOTES: The coefficient of variation (CV) is the standard error (SE) of an estimate expressed as a percentage of the estimate (see Box 2-5). CVs that indicate often acceptable levels of precision are in bold italics. ACS estimates are not published below the solid line in column 4 (below 65,000 people) and column 6 (below 20,000 people). Column 1: Assumed population size of an area. Column 2: Assumed number of school-age children (ages 5–17); assumed to be 20 percent of column 1. Column 3: Assumed number of poor school-age children; assumed to be 15 percent of column 2. Column 4a: ACS 1-year period estimate CV; based on SE estimated as 2.24 times ACS 5-year period estimate SE (see column 6a).

OCR for page 27
Using the American Community Survey: Benefits and Challenges Column 4b: Approximate number of completed sample cases of school-age children (column 2) for 1 year of ACS; estimated as 20 percent of the ACS 5-year number of completed sample cases in column 6b; rounded to nearest 50 (nearest 10 when fewer than 100 cases). Column 5a: ACS 3-year period estimate CV; based on SE estimated as 1.29 times ACS 5-year period estimate SE (see column 6a). Column 5b: Approximate number of completed sample cases of school-age children (column 2) for 3 years of ACS; estimated as 60 percent of the ACS 5-year number of completed sample cases in column 6b; rounded to nearest 50 (nearest 10 when fewer than 100 cases). Column 6a: ACS 5-year period estimate CV; for areas with 1,500 or more people (column 1), based on SE estimated as 1.51 times 2000 long-form-sample SE (from Starsinic, 2005); for areas with 500 people (column 1), based on SE estimated as 1.2 times 2000 long-form sample SE (see column 7a). The factor of 1.51 accounts for the smaller initial 5-year ACS sampling rate compared with the 2000 long-form-sample rate, as well as CAPI subsampling and nonresponse in the ACS. The factor of 1.2 takes into account that the ACS initial sampling rate is the same as the long-form sampling rate for areas of this small size (see Table 2-3, Part A). Column 6b: Approximate number of completed sample cases of school-age children (column 2) for 5 years of ACS; estimated as 0.51 and 0.73 times the long-form number of completed sample cases (column 7b) for areas with 1,500 or more people and 500 people, respectively, times 0.97 to allow for nonresponse in the ACS; rounded to nearest 50 (nearest 10 when fewer than 100 cases). The 0.51 and 0.73 factors are based on the ratio of ACS 5-year period cumulative rates of completed sample cases to the 2000 long-form-sample rate (see Table 2-3, Part C). These factors assume a 60 percent mail and CATI response rate from the initial sample as in the 2005 ACS. Column 7a: 2000 long-form sample CV; based on SE estimated according to the formula in U.S. Census Bureau (2005:8-23), which is: SE(p) = F(√(5/b)p (100-p)), where b is the population base of the estimated percentage, p, and F is a design factor. The base, b, is the number of school-age children in column 2; p is 15 percent poor school-age children; and F varies by the characteristic estimated (poverty) and the assumed long-form-sample sizes for different size areas (from U.S. Census Bureau, 2005:Table C): • 1.5 design factor for areas of 5,000 or more people, with assumed sample sizes of about 15 percent, instead of 16.7 percent, of school-age children (allowing for unit nonresponse); • 1.3 design factor for oversampled areas of 3,000 people, with assumed sample sizes of about 20 percent instead of 25 percent of school-age children; and • 0.7 design factor for oversampled areas of 1,500 people or fewer, with assumed sample sizes of about 45 percent instead of 50 percent of school-age children. SEs for areas of 3,000 or fewer people that are not oversampled (including census tracts in larger governmental units and townships not in one of the 12 states in which they are recognized as functioning governments for purposes of oversampling—see Table 2-3, Part A) will be larger than those calculated. Column 7b: Approximate number of completed sample cases of school-age children (column 2) for 2000 long-form sample; estimated using 2000 long-form sampling rates from Table 2-3 times 0.93, which is the percentage of usable cases of the total sample in 2000. These estimates do not enter into the SE and CV calculations, which are based on design factors estimated for the actual 2000 long-form sample.

OCR for page 27
Using the American Community Survey: Benefits and Challenges TABLE 2-7b Illustrative, Approximate 90 Percent Margins of Error (MOEs), Plus or Minus an Estimate of 15 Percent Poor School-Age Children from the ACS and the 2000 Census Long-Form Sample, by Population Size of Area Population Size of Area Children Ages 5–17 ACS 1-Year Period Estimate 90% MOE ACS 3-Year Period Estimate 90% MOE ACS 5-Year Period Estimate 90% MOE 2000 Long-Form Sample Estimate 90% MOE Total (20%of total pop.) Poor (15% of ages 5–17) 2,500,000 500,000 75,000 ±0.9% ±0.5% ±0.4% ±0.3% 1,000,000 200,000 30,000 ±1.5 ±0.9 ±0.7 ±0.4 500,000 100,000 15,000 ±2.1 ±1.2 ±0.9 ±0.6 250,000 50,000 7,500 ±3.0 ±1.7 ±1.3 ±0.9 100,000 20,000 3,000 ±4.7 ±2.7 ±2.1 ±1.4 65,000 13,000 1,950 ±5.9 ±3.4 ±2.6 ±1.7 50,000 10,000 1,500 ±6.7 ±3.8 ±3.0 ±2.0 25,000 5,000 750 ±9.5 ±5.4 ±4.2 ±2.8 20,000 4,000 600 ±10.6 ±6.1 ±4.7 ±3.1 10,000 2,000 300 ±14.9 ±8.6 ±6.7 ±4.4 5,000 1,000 150 (±21.1) ±12.2 ±9.4 ±6.2 3,000 600 90 (±23.6) ±13.6 ±10.6 ±7.0 1,500 300 45 (±18.0) ±10.4 ±8.0 ±5.3 500 100 15 (±24.8) ±14.3 ±11.1 ±9.2 NOTES: The 90 percent margin of error (MOE) is plus or minus (±) the standard error of an estimate times 1.65 (see Table 2-7a notes). The MOEs these cases, the subtraction of the MOE from the 15 percent estimate yields a negative value, which is an impossible result. Although the standard procedure for deriving the MOE is applied throughout the table, the underlying assumption of that procedure—that the sampling distribution of the estimate is approximately the normal distribution—is not applicable in these cases.

OCR for page 27
Using the American Community Survey: Benefits and Challenges TABLE 2-7c Illustrative, Approximate 90 Percent Confidence Intervals (CIs) Around an Estimate of 15 Percent Poor School-Age Children from the ACS and the 2000 Census Long-Form Sample, by Population Size of Area Population Size of Area Children Ages 5–17 ACS 1-Year Period Estimate 90% CI ACS 3-Year Period Estimate 90% CI ACS 5-YearPeriod Estimate 90% CI 2000 Long-Form Sample Estimate 90% CI Total (20%of pop. total) Poor (15% of ages 5–17) 2,500,000 500,000 75,000 14.1–15.9% 14.5–15.5% 14.6–15.4% 14.7–15.3% 1,000,000 200,000 30,000 13.5–16.5 14.1–15.9 14.3–15.7 14.6–15.4 500,000 100,000 15,000 12.9–17.1 13.8–16.2 14.1–15.9 14.4–15.6 250,000 50,000 7,500 12.0–18.0 13.3–16.7 13.7–16.3 14.1–15.9 100,000 20,000 3,000 10.3–19.7 12.3–17.7 12.9–17.1 13.6–16.4 65,000 13,000 1,950 9.1–20.9 11.6–18.4 12.4–17.6 13.3–16.7 50,000 10,000 1,500 8.3–21.7 11.2–18.8 12.0–18.0 13.0–17.0 25,000 5,000 750 5.5–24.5 9.6–20.4 10.8–19.2 12.2–17.8 20,000 4,000 600 4.4–25.6 8.9–21.1 10.3–19.7 11.9–18.1 10,000 2,000 300 0.1–29.9 6.4–23.6 8.3–21.7 10.6–19.4 5,000 1,000 150 (0.0–36.1) 2.8–27.2 5.6–24.4 8.8–21.2 3,000 600 90 (0.0–38.6) 1.4–28.6 4.4–25.6 8.0–22.0 1,500 300 45 (0.0–33.0) 4.6–25.4 7.0–23.0 9.7–20.3 500 100 15 (0.0–39.8) 0.7–29.3 3.9–26.1 5.8–24.2 NOTES: The 90 percent confidence interval (CI) ranges from an estimate minus the 90 percent margin of error to the estimate plus the 90 percent margin of error (see Table 2-7b). The 90 percent confidence intervals in parentheses are inexact. The lower limit of the confidence interval calculated in the standard way is a negative number, which is not possible. For simplicity, the lower limit has been set to 0 in these cases. See also the notes for Table 2-7b.

OCR for page 27
Using the American Community Survey: Benefits and Challenges of a population group in an area. The formula for estimating the coefficient of variation is very unstable for estimates of small proportions, and the estimated coefficients can be misleadingly large. Table 2-7a shows that estimates from the 2000 long-form sample of 15 percent poor school-age children meet the 12 percent standard of precision for areas with a minimum population between 20,000 and 25,000 people (4,000–5,000 school-age children), but estimates from accumulated ACS 5-year data meet this standard only for areas with at least 50,000 people (10,000 school-age children). Estimates from the ACS 3-year and 1-year data meet this standard only for areas with at least 80,000 people (16,000 school-age children) and 250,000 people (50,000 school-age children), respectively. The relative standard errors in Table 2-7a are calculated for estimates of 15 percent poor children among all school-age children. The latter group, in turn, is assumed to be 20 percent of the total population, so that poor school-age children are only 3 percent of the total population. If, instead, the table were to provide relative standard errors for estimates of 15 percent poor people—including all children and adults—among the total population, then the levels of precision shown would be considerably improved (see Table 2-8). Thus, the long-form sample would provide estimates that meet the 12 percent or less precision standard for areas as small as 1,500 people, while estimates from accumulated ACS 5-year data would meet this standard for areas as small as 10,000 people. Estimates from accumulated ACS 3-year and 1-year data would meet this standard for areas as small as about 15,000 and 50,000 people, respectively (see Table 2-8). In other words, simple one-way tabulations from the ACS may meet common standards for precision for relatively small areas, although that is not likely to be the case once another variable is introduced, such as age or race. Users should not simply rely on commonly cited precision standards in deciding whether to use particular estimates. They also need to take into account the specific requirements of their application. For example, deciding which subset of school districts should receive additional funding directed to low-income students may require a narrower confidence interval than the standard. Thus, a 90 percent confidence interval of 12 to 18 percent poor school-age children, which corresponds to a 12 percent relative standard error for an estimate of 15 percent poor school-age children, may be too wide an interval for purposes of fund allocation. Still, for some applications, a ballpark estimate with an even wider confidence interval may suffice. In deciding which set of ACS estimates is best suited for a particular application, users will need to make trade-offs between timeliness and sampling error. For example, a user could decide that a 3-year period estimate is preferable to a 1-year period estimate for a large city or county in order to achieve a greater level of precision. Alternatively, a user could decide that

OCR for page 27
Using the American Community Survey: Benefits and Challenges TABLE 2-8 Illustrative, Approximate Relative Standard Errors (Coefficients of Variation, or CVs) for an Estimate of 15 Percent Poor People from the ACS and the 2000 Census Long-Form Sample, by Population Size of Area Population Size of Area (1) Poor People (15% of total pop.) (2) ACS 1-Year Period Estimate ACS 3-Year Period Estimate ACS 5-Year Period Estimate 2000 Long-Form-Sample Estimate CV(%) (3a) (Sample Cases) (3b) CV(%) (4a) (Sample Cases) (4b) CV(%) (5a) (Sample Cases) (5b) CV(%) (6a) (Sample Cases) (6b) 2,500,000 375,000 1.7 %(39,550) 1.0% (118,600) 0.8% (197,650) 0.5% (387,500) 1,000,000 150,000 2.7 (15,800) 1.6 (47,450) 1.2 (79,050) 0.8 (155,000) 500,000 75,000 3.8 (7,900) 2.2 (23,700) 1.7 (39,550) 1.1 (77,500) 250,000 37,500 5.4 (3,950) 3.1 (11,850) 2.4 (19,750) 1.6 (38,750) 100,000 15,000 8.5 (1,600) 4.9 (4,750) 3.8 (7,900) 2.5 (15,500) 65,000 13,000 10.6 (1,050) 6.1 (3,100) 4.7 (5,150) 3.1 (10,100) 50,000 7,500 12.1 (800) 7.0 (2,350) 5.4 (4,050) 3.6 (7,750) 25,000 3,750 17.1 (400) 9.8 (1,200) 7.6 (2,000) 5.1 (3,900) 20,000 3,000 19.1 (300) 11.0 (950) 8.5 (1,600) 5.6 (3,100) 10,000 1,500 27.0 (150) 15.6 (450) 12.1 (800) 8.0 (1,550) 5,000 750 38.2 (80) 22.0 (250) 17.1 (400) 11.3 (800) 3,000 450 42.7 (70) 24.6 (200) 19.1 (350) 12.6 (700) 1,500 225 32.5 (70) 18.7 (200) 14.5 (350) 9.6 (700) 500 75 44.8 (30) 25.8 (100) 20.0 (150) 16.7 (250) NOTES: See Notes for Table 2-7a—columns 3a–6b in Table 2-8 correspond to columns 4a–7b, respectively, in Table 2-7a. Population sizes for calculating standard errors are in column 1. To obtain an approximate 90 percent margin of error, multiply 15 percent by the estimated coefficient of variation (CV) above to obtain the estimated standard error and multiply the result by 1.65. For example, the 90 percent margin of error for an ACS 1-year period estimate of 15 percent poor people in an area of 65,000 total population is 15 times 0.106 equals 1.6, times 1.65 equals ±2.6, which, in turn, gives a 90 percent confidence interval of 12.4–17.6 percent poor people.

OCR for page 27
Using the American Community Survey: Benefits and Challenges several years of 1-year period ACS estimates will be informative regarding trends and the current situation for the city, even though the estimates are less precise (see discussion in Chapter 3). 2-C.2.d Documentation of Sampling Error The Census Bureau commendably is trying to impress upon users the extent of sampling error in the ACS estimates. Originally, for data products issued through mid-2005 from the C2SS and the ACS test surveys for 2001–2004, the Census Bureau published upper and lower 90 percent confidence interval bounds (for example, 13–17 percent for a 15 percent estimate of poor school-age children). In response to users, who are more accustomed to the MOE concept (as reported in the media for public opinion polls), the Census Bureau decided to replace the upper and lower bounds in tables with the 90 percent MOEs for specific estimates (such as ±0.2 percentage points). In addition, the Census Bureau will not publish 1-year or 3-year estimates when their imprecision is deemed to be too great. In these instances, the standard tabulation categories will be combined to the point at which the tabulations meet the Census Bureau’s threshold for a minimally acceptable level of precision. The 5-year period estimates will not be treated in this manner, even for very small areas for which they are highly imprecise, because the 5-year small-area estimates are the building blocks for a wide range of user applications similar to how the long-form-sample data were used (see Section 4-D.2). In contrast, the sampling error of the long-form-sample estimates was not highlighted, but instead was contained in footnotes and auxiliary documentation. Moreover, margins of error were not provided for specific estimates; instead, users were provided with general formulas for making their own computations of sampling error. As a result, many users have been unaware of the sampling error in the long form-sample estimates they have been using. 2-D SUMMARY ASSESSMENT The ACS promises to be of great benefit to many users for a wide range of applications for which they previously relied on information from the decennial census long-form sample. The three major benefits of the ACS are its timeliness, frequency, and the improved quality of the responses when compared with the long-form sample. Not only will the ACS information be released within 8–10 months of completion of data collection, compared with 2 years or more for the long-form sample, but it will also be updated every year instead of once a decade. Moreover, there is strong evidence that

OCR for page 27
Using the American Community Survey: Benefits and Challenges the ACS will provide data with reduced nonsampling error because of such factors as the use of trained interviewers to collect the information from nonrespondents. In tests of the ACS, improvements in quality are evident in more complete response to almost every item compared with the long-form sample. Furthermore, in personal interviews, some items will be more accurately reported because the computer-assisted interviewing can more readily correct respondent misperceptions about what is being asked and resolve inconsistent responses. A complication for users of switching from the census long-form sample to the ACS is the continuous fielding and processing of the ACS. This design produces estimates that pertain to periods of time—averages over 12, 36, or 60 months—instead of the traditional point-in-time estimates with which users are familiar from the long-form sample and other household surveys. Users will need to work together and with the Census Bureau to develop strategies for application of the ACS information that take account of the survey’s continuous design. In Chapter 3 we outline some of these strategies. Sampling error or imprecision of the estimates is a problematic aspect of the ACS, although users should remember that many long-form-sample estimates did not meet common standards of precision for small areas, either (see Tables 2-7a, 2-7b, 2-7c, and 2-8). When the data are averaged over 5 years, it appears that the ACS will provide reasonably precise estimates for small population groups, such as poor school-age children, for areas with 50,000 or more people but not for smaller areas. The ACS 1-year estimates for such a small population group will have low precision unless the area has at least 250,000 people. For larger population groups, such as total poor, the ACS 5-year estimates will likely provide reasonably precise estimates for areas of at least 10,000 people, while the ACS 1-year estimates will meet that standard for areas of at least 50,000 people. ACS estimates for census tracts, which average 4,000 people, and block groups, which average 1,500 people, will be very imprecise. Indeed, they were not precise from the long-form sample for other than large population groups. However, these areas can be combined in various ways by users who want to compare planning districts, wards, or other components of large cities, counties, and other areas. The bottom line for large geographic areas—such as states, congressional districts, and large metropolitan areas, cities, and counties—is that the ACS estimates will be a great asset to data users. The data will be timely, up to date, of good quality, and reasonably precise. The 5-year data for census tracts and block groups, while not precise in and of themselves, will provide building blocks that should enable detailed analyses of the populations of large geographic areas. Estimates from the ACS for small governmental units, even with over-

OCR for page 27
Using the American Community Survey: Benefits and Challenges sampling, are the most problematic from the perspective of sampling error. Consider a place of 1,500 people and 300 school-age children, of whom 45 children or 15 percent are estimated to be poor. Table 2-7c shows a 90 percent confidence interval of 7 to 23 percent poor school-age children from 5 years of ACS data. Based on the calculations used to derive Table 2-7c, the margin of error of the ACS estimate is 51 percent greater than that from the 2000 long-form sample, which already has a high margin of error, and this increase may be somewhat underestimated. Moreover, the option of combining small governmental units into larger analytical units in order to improve the precision of estimates is less applicable than in the case of combining census tracts or block groups within a larger jurisdiction. Chapter 3 discusses possible strategies for data users who are interested in very small governmental units to make effective use of the ACS estimates. It will also be imperative to maintain the planned sample sizes for the ACS over time and, furthermore, for the Census Bureau, in cooperation with users, to seek ways to improve the precision of the estimates for small areas (see Section 4-A.5).