Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 27
Using the American Community Survey: Benefits and Challenges
PART I
Using the American Community Survey
OCR for page 28
Using the American Community Survey: Benefits and Challenges
This page intentionally left blank.
OCR for page 29
Using the American Community Survey: Benefits and Challenges
2
Essentials for Users
This chapter addresses users of the decennial census long-form sample who want to know, in general terms, what benefits—and challenges—the new American Community Survey (ACS) presents to them. The chapter first summarizes the basics that every user should know about the ACS and key ways in which it is similar to and differs from the decennial census long-form sample. It then addresses two central issues: (1) why users should care about the ACS in terms of the benefits it offers and (2) some of the challenges those benefits present for users. Finally, it offers the panel’s assessment of the value of the ACS to users based on the available knowledge about its properties.
2-A
ACS DESIGN BASICS
To work with data from the ACS, users should be acquainted with the following features of its design and operations: the population or universe covered, rules for assigning people to a place of residence, questionnaire content and reference periods, sample size and design, data collection procedures, data products, and data-processing procedures to generate the products. The key factor to keep in mind is that, unlike the census long-form sample, the ACS is continuous: a fresh sample of addresses is surveyed every month, and data products represent cumulations of monthly data for 1-year, 3-year, and 5-year periods. The discussion below pertains to the ACS
OCR for page 30
Using the American Community Survey: Benefits and Challenges
BOX 2-1
The Puerto Rico Community Survey (PRCS)
The Puerto Rico Community Survey (PRCS) is identical in most respects to the American Community Survey in the 50 states and the District of Columbia. There were no PRCS test surveys or test sites for Puerto Rico in 2000–2004, so that 2005 PRCS data are the first post-2000 long-form-type data available for Puerto Rico.
Following the same basic design as the ACS, the initial sample size of the PRCS is 3,000 housing units each month or 36,000 housing units each year—about 2.4 percent of the total of about 1.5 million residential addresses in Puerto Rico. Initial sampling rates for blocks vary by the estimate of occupied housing units in the governmental jurisdiction or census tract (see Table 2-3, Part A), although the PRCS rates are slightly higher than the ACS rates (see U.S. Census Bureau, 2006:Table 4.1).
Data collection in the PRCS uses mailout, CATI, and CAPI, like the ACS. However, because of low mail response, all mailout/CATI nonrespondents are sampled at a 50 percent rate for the CAPI follow-up as of June 2005.
Areas for which PRCS products are published include:
78 municipios (county equivalents): 12 will have 1-year estimates; 65 will have 3-year estimates
455 barrios (subdivisions of municipios, similar to minor civil divisions): 5 will have 1-year estimates; 34 will have 3-year estimates (based on 2000 census counts)
225 zonas urbanas (census designated places that are governmental centers of municipios) and communidads (other census designated places): 9 will have 1-year estimates; 20 will have 3-year estimates (based on 2000 census counts)
871 census tracts and 2,477 block groups (5-year estimates only)
The PRCS is explained in greater detail by U.S. Census Bureau (2006).
in the United States; see Box 2-1 for a brief overview of the Puerto Rico Community Survey (PRCS).1
2-A.1
Population Coverage (Universe)
The ACS for 2005 covered the household population. The 2006 ACS covered not only the household population, but also people who live in college dormitories, armed forces barracks, prisons, nursing homes, and other group quarters. The 2006 ACS population coverage was the same as the census long-form-sample coverage, except that the ACS did not in-
1
All of Section 2-A draws heavily on U.S. Census Bureau (2006).
OCR for page 31
Using the American Community Survey: Benefits and Challenges
clude people found at soup kitchens or street locations frequented by the homeless and a few other transient situations. Table 2-1 lists the types of residences included in the 2006 ACS.
2-A.2
Residence Rules
The ACS instructs the respondent for a household to provide data on all people who, at the time of filling out the questionnaire, are living or staying at the household address for more than 2 months (including usual residents who are away for less than 2 months). In contrast, the long-form sample asked household respondents to report all people who usually lived at the address as of Census Day, April 1, meaning they lived or stayed there most of the time. People whom the ACS samples in group quarters beginning in 2006 are counted at the group quarters location, in effect applying a de facto residence rule regardless of how long an individual has lived or expects to live in the group quarters. The long-form sample also in effect generally applied a de facto residence rule for group quarters residents, although residents of some types of group quarters were allowed the option of indicating another usual place of residence. (An unduplication process was used to determine the correct enumeration for people listed at the group quarters and the other residence; such a process would not be possible for the ACS because it is not embedded in a census.)
For many people, their ACS residence will be the same as their long-form-sample residence. However, some people may report a different residence: for example, people who live in a house or apartment in New York most of the year but reside in Florida in December through March should report Florida as their address if sampled for the ACS in Florida in the winter, whereas their Census Day address is in New York.
2-A.3
Content and Reference Periods
The 2005 ACS includes about 55 questions for every person and 30 questions for every housing unit in the sample—approximately the same content as in the 2000 census long-form sample. There are some differences:
The ACS mail questionnaire uses a matrix layout for questions on sex, age, race, ethnicity, and household relationship, compared with a person-by-person format in the long-form questionnaire.
The ACS mail questionnaire provides room to respond for 5 household members compared with 6 on the long-form questionnaire (telephone follow-up is used to obtain information on additional household members).
OCR for page 32
Using the American Community Survey: Benefits and Challenges
TABLE 2-1 Types of Residences in the American Community Survey (ACS)
Residence Type
2000 Population (Percentage)
Housing Unit Residencea
97.2
Single-family, detached
64.6
Single-family, attached
5.4
2-or-more-unit structure
20.4
In an apartment building (including condominium or co-op)
In an assisted living facility with separate apartments
In a group quarters (e.g., house master’s residence)
In a home (e.g., basement apartment, upstairs apartment)
In multi-unit military family housing on or off base
Mobile home that is occupied or, if vacant, that is permanently sited
6.7
Boat at a mooring, RV, or occupied van
0.1
Institutional Group Quarters Residence (beginning in 2006 ACS)
1.4
Nursing home or other long-term care facility
0.6
Correctional institution (for example, prison or jail)
0.7
Other institutions (for example, hospital or residential school for people with disabilities, long-term care home for juveniles)
0.1
Noninstitutional Group Quarters Residence (beginning in 2006 ACS)
1.3
College dormitory
0.7
Military quarters (in barracks on a base; on a ship assigned to home port)
0.1
Other noninstitutional group quarters
0.5b
Residence in the ACS and the 2000 Long-Form Sample
Convent, monastery
Group home
Halfway home
Hospice
Job Corps center
Migrant worker quarters
Shelter, emergency shelter
YMCA, YWCA, hostel
Residence NOT in the ACS but in the 2000 Long-Form Sample
Circus quarters
Crews on merchant ships
Domestic violence shelter
Recreational vehicle in a campground
Soup kitchen or mobile food van site
Street location for the homeless
aHousing units are separate living quarters with direct access from the outside or through a common hall (U.S. Census Bureau, 2006:D-17).
bIncludes 170,706 people (0.06 percent of the population of 281.4 million in 2000) living in emergency shelters for the homeless, shelters for runaway children, transitional shelters, and hotels and motels used to provide shelter for people without conventional housing (U.S. Census Bureau, 2001).
SOURCES: Types of residences adapted from U.S. Census Bureau (2006:Ch. 8, Attachment A); population percentages from http://factfinder.census.gov, Summary File 1, Table P37; Summary File 3, Table H33.
OCR for page 33
Using the American Community Survey: Benefits and Challenges
Many ACS items refer to a time period different from that of the corresponding items on the long-form questionnaire: for example, usual hours worked per week, weeks worked per year, and income items on the ACS refer to the 12 months prior to the day when the household filled out the questionnaire, whereas these items on the long form always referred to the previous calendar year (1999 for the 2000 census).
The ACS currently includes three items not on the 2000 long form: (1) whether the household received food stamps in the previous 12 months and their value; (2) the length of time and main reason for staying at the address (for example, permanent home, vacation home, to attend school or college); and, for women ages 15–50, whether they gave birth to any children in the past 12 months.
Table 2-2 compares the items on the 2005 ACS questionnaire with the items on the 2000 census long form. The Census Bureau is proposing several changes to the ACS questionnaire beginning in 2008. These changes, if approved, will include the addition of three new questions on marital history, health insurance coverage, and veterans’ service-related disability, the deletion of the question on length of time and main reason for staying at the address, changes to the basic demographic items for consistency with the 2010 census questionnaire, and changes in wording and format to improve reporting of several other questions as determined by a 2006 test. A question on field of bachelor’s degree will be tested in 2007 and may be added to the ACS beginning in 2009.
2-A.4
Sample Design and Size
The ACS sends out questionnaires to about 250,000 housing unit addresses every month that have been sampled from the Census Bureau’s Master Address File (MAF; see Chapter 4 for details of the sampling operation). Each month’s sample includes addresses in every one of the nation’s 3,141 counties. The monthly samples cumulate to about 3 million addresses over a year, or about 2.3 percent of the total number of about 129.5 million housing unit addresses in the United States in 2005. The sample is constructed so that no housing unit address will be included more than once every 5 years.
The ACS sample is very large compared with the samples for major national household surveys. However, the long-form sample was even larger: in 2000, the long form was sent to about 18 million addresses, or one-sixth of total housing unit addresses in the United States at the time, and 16.4 million usable long-form questionnaires were included in the final edited data file. The ACS monthly and even yearly samples cannot be as large as
OCR for page 34
Using the American Community Survey: Benefits and Challenges
TABLE 2-2 Items on the 2005 ACS Questionnaire and the 2000 Census Long Form
2005 ACS Item (in question order)
Asked in 2000 Census?
Person Items
Sex
Yes (short-form item)
Age (at interview)
Yes (as of April 1, short form)
Date of birth (month, day, year)
Yes (short form)
Relationship to household reference person (person 1)
Yes (more detail, short form)
Marital status
Yes
Hispanic origin
Yes (short form)
Race (option for multiple races)
Yes (short form)
Place of birth
Yes
Citizenship
Yes
Year of immigration
Yes
Attended school in last 3 months
Yes (since February 1, 2000)
Grade attending
Yes
Highest degree completed
Yes
Ancestry or ethnic origin
Yes
Language spoken at home
Yes
How well speaks English
Yes
Place of residence 1 year ago (city or town, county, state)
Yes (5 years ago)
Disability involving sight or hearing
Yes
Disability limiting physical activity
Yes
Difficulty learning, remembering due to disability of 6 or more months
Yes
Difficulty dressing, bathing, or getting around the home
Yes
For people ages 15 and older
Difficulty going outside the home alone to shop, etc.
Yes (ages 16 and older)
Difficulty working at a job or business
Yes (ages 16 and older)
Given birth in past 12 months (women ages 15–50)
No
Responsible for own grandchildren in the home
Yes
How long responsible for grandchildren
Yes
Veteran status (active duty)
Yes
Period of active military service
Yes
Number of automobiles, vans, trucks for use by household members
Yes
Years of active military service (less than 2 years, 2 years or more)
Yes
Working last week for pay or profit
Yes
Place of work (address)
Yes
Usual means of transportation to work last week
Yes
If by car, truck, or van, how many people used it
Yes
Time left home for work
Yes
Minutes to work
Yes
On layoff last week
Yes
Temporarily absent from work last week
Yes
Whether will be recalled to work
Yes
Looking for work last 4 weeks
Yes
Could have worked last week
Yes
OCR for page 35
Using the American Community Survey: Benefits and Challenges
2005 ACS Item (in question order)
Asked in 2000 Census?
When last worked
Yes
Weeks worked, last 12 months
Yes (1999)
Hours usually worked per week, last 12 months
Yes (1999)
Class of worker of current or most recent employment
Yes
Industry of current or most recent employment
Yes
Occupation of current or most recent employment
Yes
Wage and salary income, last 12 months
Yes (1999)
Self-employment income (farm and nonfarm), last 12 months
Yes (1999)
Interest, dividend, net rent, royalty, and trust income, last 12 months
Yes (1999)
Social Security income, last 12 months
Yes (1999)
Supplemental Security Income, last 12 months
Yes (1999)
State or local public assistance income, last 12 months
Yes (1999)
Retirement, survivor, or disability pension income, last 12 months
Yes (1999)
Any other regular income, last 12 months
Yes (1999)
Total income, last 12 months
Yes (1999)
Housing Items
Type of building/number of units in structure
Yes
Year building built
Yes
When household reference person (person 1) moved in
Yes
Number of acres on property (single-family or mobile home)
Yes
Agricultural sales, last 12 months (single-family or mobile home on 1 or more acres)
Yes (1999)
Whether business on property (single-family or mobile home)
Yes
Rooms in unit
Yes
Bedrooms in unit
Yes
Complete plumbing facilities
Yes
Complete kitchen facilities
Yes
Telephone service available
Yes
Number of automobiles, vans, trucks for use by household members
Yes
Heating fuel most used
Yes
Electricity cost, last month
Yes (annual cost)
Gas cost, last month
Yes (annual cost)
Water and sewer cost, last 12 months
Yes (annual cost)
Oil, coal, kerosene cost, last 12 months
Yes (annual cost)
Receive food stamps, value last 12 months
No
Monthly condominium fee
Yes
Owner or renter (tenure)
Yes (short-form item)
Monthly rent (and whether includes various utilities)
Yes
Whether rent includes meals
Yes
Value of property if were for sale
Yes
OCR for page 36
Using the American Community Survey: Benefits and Challenges
2005 ACS Item (in question order)
Asked in 2000 Census?
Annual real estate taxes
Yes (last year)
Annual hazard insurance
Yes
Monthly mortgage payment
Yes
Whether mortgage payment includes taxes
Yes
Whether mortgage payment includes insurance
Yes
Whether a second mortgage and/or home equity loan
Yes
Second mortgage/home equity loan monthly payment
Yes
Annual costs for mobile home and site (personal property taxes, site rent, fees and licenses)
Yes (last year, also includes installment loans)
Whether any household members live here year round
No
Number of months members live here
No
Main reason members stay at this address
No
NOTES: The 2005 ACS and 2000 census long-form sample provided room on the mailback questionnaire for characteristics of up to 5 and 6 household members, respectively. Questionnaires should be consulted for precise question wording.
SOURCES: See http://www.census.gov/acs/www/SBasics/SQuest/SQuest1.htm for the 2005 ACS; Anderson (2000:388-399) for the 2000 long form.
the long-form sample because the costs would be too great. Accumulated over 5 years, the ACS sample will total about 15 million housing unit addresses, but the ACS sample is then reduced by the subsampling for in-person follow-up of households not responding to the mail and telephone data collection procedures (see below). This subsampling may reduce the ACS 5-year sample to 10–11 million housing unit addresses.
Because data on governmental jurisdictions will be an important output of the ACS and because many governmental units are very small in population size, the ACS oversamples housing unit addresses in small governmental units relative to other areas similar to the 2000 long-form-sample design. Oversampling provides more precise estimates for small counties, places, townships, school districts, and American Indian and Alaska Native areas than would otherwise be possible. In a similar manner, for the personal visit follow-up operation, the ACS oversamples mail and telephone nonrespondents in census tracts that are expected to have low mail and telephone response rates relative to other census tracts. In order to afford the costs for the additional follow-up, not only are smaller subsamples followed up in person in census tracts that are expected to have high mail and telephone response, but also the initial sample for these tracts is reduced by 8 percent. Tables 2-3a, 2-3b, and 2-3c provide initial annual and 5-year sampling rates for governmental units and census tracts of different popu-
OCR for page 37
Using the American Community Survey: Benefits and Challenges
TABLE 2-3a Housing Unit Addresses, 2005 ACS and 2000 Census Long-Form Sample: Approximate Initial Block-Level Sampling Rates
Type and Size of Smallest Area Containing a Block
2005 American Community Survey
2000 Long-Form- Sample Census Day Sampling Rate
Annual Initial Sampling Rate
Cumulative 5-Year Initial Sampling Rate
Governmental unit (county, place, township in 12 states, school district, American Indian or Alaska Native area)
With < 200 occupied housing units (fewer than about 500 people)
10.0% (1 in 10)
50.0% (1 in 2)
50.0% (1 in 2)
With 200–800 occupied housing units (about 500-2,000 people)
6.9% ( 1 in 14)
34.5% (1 in 3)
50.0% (1 in 2)
With 800–1,200 occupied housing units (about 2,000-3,000 people)
3.5% (1 in 28)
17.5% (1 in 6)
25.0% (1 in 4)
Census tract with > 2,000 occupied housing units (more than about 5,000 people)a
1.7% (1 in 59) or 1.6% (1 in 63)
8.5% (1 in 12) or 8.0% (1 in 13)
12.5% (1 in 8)
Other areaa
2.3% (1 in 44) or 2.1% (1 in 48)
11.5% (1 in 9) or 10.5% (1 in 10)
16.7% (1 in 6)
Overal l
2.3% (1 in 44)
11.5% (1 in 9)
16.7% (1 in 6)
NOTES: Number of occupied housing units is estimated from the MAF. Because the initial ACS sample size will be kept at approximately 3 million residential addresses per year, the initial sampling rates shown will be slightly reduced as the number of occupied housing units grows. Townships and other minor civil divisions are recognized for sampling purposes in 12 states where they are functioning governments: Conne cticut, Maine, Massachusetts, Michigan, Minnesota, New Hampshire, New Jersey, New York, Pennsylvania, Rhode Island, Vermont, and Wisconsin.
aThe smaller of the two ACS sampling rates shown applies for blocks in census tracts with predicted mail/CATI response rates gre ater than 60% (see Table 2-3b).
SOURCE: Adapted from U.S. Census Bureau (2006:Tables 4.1, 4.2) for the ACS.
OCR for page 66
Using the American Community Survey: Benefits and Challenges
by the original design, but also by nonresponse and, in the case of the ACS, by the extent of CAPI subsampling that is done for personal visit follow-up to contain costs.
Tables 2-7a, 2-7b, and 2-7c provide rough, approximate estimates of sampling error for an estimated 15 percent poor school-age children from the ACS (1-year, 3-year, and 5-year period estimates) and the 2000 long-form sample for areas ranging in population from 500 to 2.5 million people. The calculations assume that school-age children are 20 percent of the total population and that areas with 3,000 or fewer people are oversampled. The calculations take account—for both the ACS and the long-form sample—of the added sampling error from household nonresponse but not the added error from item nonresponse.
Specifically, Table 2-7a shows relative standard errors—that is, the standard error as a percentage of the estimate, also called the coefficient of variation (see Box 2-5). Table 2-7b shows approximate 90 percent margins of error (MOEs) plus or minus the estimate of 15 percent poor school-age children for each size area (90 percent MOEs are 1.65 times the corresponding standard error). Finally, Table 2-7c translates the MOEs into 90 percent confidence intervals surrounding the 15 percent school-age poverty estimates.
The tables and text use 90 percent MOEs and confidence levels to follow the long-standing practice of Census Bureau publications; however, this practice is not standard in statistical work. It gives smaller MOEs and confidence intervals than is the case when the 95 percent standard is used: with the 95 percent standard, the MOEs and confidence intervals would be about 20 percent larger.
The panel developed the sampling error estimates in the tables by starting with a generalized variance estimation function provided by the Census Bureau for the 2000 long-form sample; we then computed the sampling error estimates for the ACS as multiples of the long-form-sample estimates (see notes at the end of Table 2-7c). The multiplication factors are derived from Census Bureau research with the ACS test sites, the C2SS, and the 2001–2004 ACS test surveys.
For the 2005 ACS, the Census Bureau directly estimated the sampling errors for specific estimates, including not only school-age poverty, but also other characteristics, using a repeated replication method (U.S. Census Bureau, 2006:Ch.12). The 2005 ACS data were only recently released, however, and the panel was not able to analyze their sampling errors; moreover, these estimates pertain only to areas with 65,000 or more people. Nevertheless, an unsystematic examination of the sampling errors for selected 2005 ACS poverty estimates suggests that they are similar to those shown in Tables 2-7a, 2-7b, and 2-7c.
OCR for page 67
Using the American Community Survey: Benefits and Challenges
2-C.2.c
Assessment of Sampling Error from Illustrative Estimates
Looking at Table 2-7a for the ACS 5-year period estimates, the relative standard errors, or coefficients of variation, for all but the smallest governmental units are half again as large (51 percent) as the corresponding relative standard errors for the 2000 long-form sample. For the ACS 3-year period estimates, the relative standard errors are, in turn, almost 30 percent larger than those for the ACS 5-year estimates and 95 percent larger than those for the long-form-sample estimates. For the ACS 1-year period estimates, the relative standard errors are more than 2 times larger than those for the ACS 5-year period estimates and more than 3 times larger than those for the long-form-sample estimates.
To illustrate, consider first the best case shown in Tables 2-7a, 2-7b, and 2-7c, which is the long-form-sample estimate of 15 percent poor school-age children for an area with 2.5 million people. For this estimate, the relative standard error is only 1.1 percentage points, the 90 percent MOE is only ±0.3 percentage points, and the 90 percent confidence interval is quite narrow—14.7 to 15.3 percent. In other words, the estimate is very precise and provides useful information for a variety of applications, such as fund allocation and program planning. For the same estimate for the same size area from ACS data accumulated over 5 years, the relative standard error is only somewhat larger at 1.7 percentage points, and the ACS data have the advantage of being more up to date.
At the other extreme, the worst case is for estimates of 15 percent poor school-age children for areas with 500 people. These areas are oversampled in both the ACS and the long-form sample, but the sample sizes are so small that the estimates are very imprecise. The 90 percent confidence interval for the estimate of 15 percent poor school-age children from the long-form sample ranges from 5.8 to 24.2 percent poor (90 percent MOE of ±9.2 percentage points), while that from the ACS 5-year period estimates ranges from 3.9 to 26.1 percent poor (90 percent MOE of ±11.1 percentage points). Intervals this wide are not helpful to users, and the range would be wider yet for areas with 500 people that are not oversampled—for example, a township in one of the 38 states for which the Census Bureau does not recognize townships as functioning governments for purposes of oversampling (refer back to Table 2-3), or a block group in a large area.
What constitutes an acceptable level of precision for a survey estimate depends on the uses to be made of the estimate. A commonly used standard for many uses is that a sample estimate should have a relative standard error, or coefficient of variation, of 10 percent or less—sometimes increased to 12 percent or less for a characteristic like poverty, which is clustered within a household or family. This standard does not apply in some instances: specifically, for estimates of proportions that are less than 5 percent
OCR for page 68
Using the American Community Survey: Benefits and Challenges
TABLE 2-7a Illustrative, Approximate Relative Standard Errors (Coefficients of Variation, or CVs) for an Estimate of 15 Percent Poor School-Age Children from the ACS and the 2000 Census Long-Form Sample, by Population Size of Area
Population Size of Area (1)
Children Ages 5–17
ACS 1-Year Period Estimate
ACS 3-Year Period Estimate
ACS 5-Year Period Estimate
2000 Long-Form Sample Estimate
Total (20% of total pop.) (2)
Poor (15% of ages 5–17) (3)
CV(%) (4a)
(Sample Cases) (4b)
CV(%) (5a)
(Sample Cases) (5b)
CV(%) (6a)
(Sample Cases) (6b)
CV(%) (7a)
(Sample Cases) (7b)
2,500,000
500,000
75,000
3.8 %
(7,900)
2.2 %
(23,700)
1.7 %
(39,550)
1.1 %
(77,500)
1,000,000
200,000
30,000
6.0
(3,150)
3.5
(9,500)
2.7
(15,800)
1.8
(31,000)
500,000
100,000
15,000
8.5
(1,600)
4.9
(4,750)
3.8
(7,900)
2.5
(15,550)
250,000
50,000
7,500
12.1
(800)
7.0
(2,350)
5.4
(3,950)
3.6
(7,750)
100,000
20,000
3,000
19.1
(300)
11.0
(950)
8.5
(1,600)
5.6
(3,100)
65,000
13,000
1,950
23.7
(200)
13.6
(600)
10.6
(1,050)
7.0
(2,000)
50,000
10,000
1,500
27.0
(150)
15.6
(450)
12.1
(800)
8.0
(1,550)
25,000
5,000
750
38.2
(80)
22.0
(250)
17.1
(400)
11.3
(800)
20,000
4,000
600
42.7
(60)
24.6
(200)
19.1
(300)
12.6
(600)
10,000
2,000
300
60.4
(30)
34.8
(90)
27.0
(150)
17.9
(300)
5,000
1,000
150
85.4
(20)
49.2
(50)
38.1
(100)
25.2
(150)
3,000
600
90
95.6
(10)
55.0
(40)
42.7
(80)
28.3
(150)
1,500
300
45
72.8
(10)
41.9
(40)
32.5
(70)
21.5
(150)
500
100
15
100.2
(10)
57.7
(20)
44.7
(30)
37.3
(50)
NOTES: The coefficient of variation (CV) is the standard error (SE) of an estimate expressed as a percentage of the estimate (see Box 2-5). CVs that indicate often acceptable levels of precision are in bold italics. ACS estimates are not published below the solid line in column 4 (below 65,000 people) and column 6 (below 20,000 people).
Column 1: Assumed population size of an area.
Column 2: Assumed number of school-age children (ages 5–17); assumed to be 20 percent of column 1.
Column 3: Assumed number of poor school-age children; assumed to be 15 percent of column 2.
Column 4a: ACS 1-year period estimate CV; based on SE estimated as 2.24 times ACS 5-year period estimate SE (see column 6a).
OCR for page 69
Using the American Community Survey: Benefits and Challenges
Column 4b: Approximate number of completed sample cases of school-age children (column 2) for 1 year of ACS; estimated as 20 percent of the ACS 5-year number of completed sample cases in column 6b; rounded to nearest 50 (nearest 10 when fewer than 100 cases).
Column 5a: ACS 3-year period estimate CV; based on SE estimated as 1.29 times ACS 5-year period estimate SE (see column 6a).
Column 5b: Approximate number of completed sample cases of school-age children (column 2) for 3 years of ACS; estimated as 60 percent of the ACS 5-year number of completed sample cases in column 6b; rounded to nearest 50 (nearest 10 when fewer than 100 cases).
Column 6a: ACS 5-year period estimate CV; for areas with 1,500 or more people (column 1), based on SE estimated as 1.51 times 2000 long-form-sample SE (from Starsinic, 2005); for areas with 500 people (column 1), based on SE estimated as 1.2 times 2000 long-form sample SE (see column 7a). The factor of 1.51 accounts for the smaller initial 5-year ACS sampling rate compared with the 2000 long-form-sample rate, as well as CAPI subsampling and nonresponse in the ACS. The factor of 1.2 takes into account that the ACS initial sampling rate is the same as the long-form sampling rate for areas of this small size (see Table 2-3, Part A).
Column 6b: Approximate number of completed sample cases of school-age children (column 2) for 5 years of ACS; estimated as 0.51 and 0.73 times the long-form number of completed sample cases (column 7b) for areas with 1,500 or more people and 500 people, respectively, times 0.97 to allow for nonresponse in the ACS; rounded to nearest 50 (nearest 10 when fewer than 100 cases). The 0.51 and 0.73 factors are based on the ratio of ACS 5-year period cumulative rates of completed sample cases to the 2000 long-form-sample rate (see Table 2-3, Part C). These factors assume a 60 percent mail and CATI response rate from the initial sample as in the 2005 ACS.
Column 7a: 2000 long-form sample CV; based on SE estimated according to the formula in U.S. Census Bureau (2005:8-23), which is: SE(p) = F(√(5/b)p (100-p)), where b is the population base of the estimated percentage, p, and F is a design factor. The base, b, is the number of school-age children in column 2; p is 15 percent poor school-age children; and F varies by the characteristic estimated (poverty) and the assumed long-form-sample sizes for different size areas (from U.S. Census Bureau, 2005:Table C):
• 1.5 design factor for areas of 5,000 or more people, with assumed sample sizes of about 15 percent, instead of 16.7 percent, of school-age children (allowing for unit nonresponse);
• 1.3 design factor for oversampled areas of 3,000 people, with assumed sample sizes of about 20 percent instead of 25 percent of school-age children; and
• 0.7 design factor for oversampled areas of 1,500 people or fewer, with assumed sample sizes of about 45 percent instead of 50 percent of school-age children.
SEs for areas of 3,000 or fewer people that are not oversampled (including census tracts in larger governmental units and townships not in one of the 12 states in which they are recognized as functioning governments for purposes of oversampling—see Table 2-3, Part A) will be larger than those calculated.
Column 7b: Approximate number of completed sample cases of school-age children (column 2) for 2000 long-form sample; estimated using 2000 long-form sampling rates from Table 2-3 times 0.93, which is the percentage of usable cases of the total sample in 2000. These estimates do not enter into the SE and CV calculations, which are based on design factors estimated for the actual 2000 long-form sample.
OCR for page 70
Using the American Community Survey: Benefits and Challenges
TABLE 2-7b Illustrative, Approximate 90 Percent Margins of Error (MOEs), Plus or Minus an Estimate of 15 Percent Poor School-Age Children from the ACS and the 2000 Census Long-Form Sample, by Population Size of Area
Population Size of Area
Children Ages 5–17
ACS 1-Year Period Estimate 90% MOE
ACS 3-Year Period Estimate 90% MOE
ACS 5-Year Period Estimate 90% MOE
2000 Long-Form Sample Estimate 90% MOE
Total (20%of total pop.)
Poor (15% of ages 5–17)
2,500,000
500,000
75,000
±0.9%
±0.5%
±0.4%
±0.3%
1,000,000
200,000
30,000
±1.5
±0.9
±0.7
±0.4
500,000
100,000
15,000
±2.1
±1.2
±0.9
±0.6
250,000
50,000
7,500
±3.0
±1.7
±1.3
±0.9
100,000
20,000
3,000
±4.7
±2.7
±2.1
±1.4
65,000
13,000
1,950
±5.9
±3.4
±2.6
±1.7
50,000
10,000
1,500
±6.7
±3.8
±3.0
±2.0
25,000
5,000
750
±9.5
±5.4
±4.2
±2.8
20,000
4,000
600
±10.6
±6.1
±4.7
±3.1
10,000
2,000
300
±14.9
±8.6
±6.7
±4.4
5,000
1,000
150
(±21.1)
±12.2
±9.4
±6.2
3,000
600
90
(±23.6)
±13.6
±10.6
±7.0
1,500
300
45
(±18.0)
±10.4
±8.0
±5.3
500
100
15
(±24.8)
±14.3
±11.1
±9.2
NOTES: The 90 percent margin of error (MOE) is plus or minus (±) the standard error of an estimate times 1.65 (see Table 2-7a notes). The MOEs these cases, the subtraction of the MOE from the 15 percent estimate yields a negative value, which is an impossible result. Although the standard procedure for deriving the MOE is applied throughout the table, the underlying assumption of that procedure—that the sampling distribution of the estimate is approximately the normal distribution—is not applicable in these cases.
OCR for page 71
Using the American Community Survey: Benefits and Challenges
TABLE 2-7c Illustrative, Approximate 90 Percent Confidence Intervals (CIs) Around an Estimate of 15 Percent Poor School-Age Children from the ACS and the 2000 Census Long-Form Sample, by Population Size of Area
Population Size of Area
Children Ages 5–17
ACS 1-Year Period Estimate 90% CI
ACS 3-Year Period Estimate 90% CI
ACS 5-YearPeriod Estimate 90% CI
2000 Long-Form Sample Estimate 90% CI
Total (20%of pop. total)
Poor (15% of ages 5–17)
2,500,000
500,000
75,000
14.1–15.9%
14.5–15.5%
14.6–15.4%
14.7–15.3%
1,000,000
200,000
30,000
13.5–16.5
14.1–15.9
14.3–15.7
14.6–15.4
500,000
100,000
15,000
12.9–17.1
13.8–16.2
14.1–15.9
14.4–15.6
250,000
50,000
7,500
12.0–18.0
13.3–16.7
13.7–16.3
14.1–15.9
100,000
20,000
3,000
10.3–19.7
12.3–17.7
12.9–17.1
13.6–16.4
65,000
13,000
1,950
9.1–20.9
11.6–18.4
12.4–17.6
13.3–16.7
50,000
10,000
1,500
8.3–21.7
11.2–18.8
12.0–18.0
13.0–17.0
25,000
5,000
750
5.5–24.5
9.6–20.4
10.8–19.2
12.2–17.8
20,000
4,000
600
4.4–25.6
8.9–21.1
10.3–19.7
11.9–18.1
10,000
2,000
300
0.1–29.9
6.4–23.6
8.3–21.7
10.6–19.4
5,000
1,000
150
(0.0–36.1)
2.8–27.2
5.6–24.4
8.8–21.2
3,000
600
90
(0.0–38.6)
1.4–28.6
4.4–25.6
8.0–22.0
1,500
300
45
(0.0–33.0)
4.6–25.4
7.0–23.0
9.7–20.3
500
100
15
(0.0–39.8)
0.7–29.3
3.9–26.1
5.8–24.2
NOTES: The 90 percent confidence interval (CI) ranges from an estimate minus the 90 percent margin of error to the estimate plus the 90 percent margin of error (see Table 2-7b). The 90 percent confidence intervals in parentheses are inexact. The lower limit of the confidence interval calculated in the standard way is a negative number, which is not possible. For simplicity, the lower limit has been set to 0 in these cases. See also the notes for Table 2-7b.
OCR for page 72
Using the American Community Survey: Benefits and Challenges
of a population group in an area. The formula for estimating the coefficient of variation is very unstable for estimates of small proportions, and the estimated coefficients can be misleadingly large.
Table 2-7a shows that estimates from the 2000 long-form sample of 15 percent poor school-age children meet the 12 percent standard of precision for areas with a minimum population between 20,000 and 25,000 people (4,000–5,000 school-age children), but estimates from accumulated ACS 5-year data meet this standard only for areas with at least 50,000 people (10,000 school-age children). Estimates from the ACS 3-year and 1-year data meet this standard only for areas with at least 80,000 people (16,000 school-age children) and 250,000 people (50,000 school-age children), respectively.
The relative standard errors in Table 2-7a are calculated for estimates of 15 percent poor children among all school-age children. The latter group, in turn, is assumed to be 20 percent of the total population, so that poor school-age children are only 3 percent of the total population. If, instead, the table were to provide relative standard errors for estimates of 15 percent poor people—including all children and adults—among the total population, then the levels of precision shown would be considerably improved (see Table 2-8). Thus, the long-form sample would provide estimates that meet the 12 percent or less precision standard for areas as small as 1,500 people, while estimates from accumulated ACS 5-year data would meet this standard for areas as small as 10,000 people. Estimates from accumulated ACS 3-year and 1-year data would meet this standard for areas as small as about 15,000 and 50,000 people, respectively (see Table 2-8). In other words, simple one-way tabulations from the ACS may meet common standards for precision for relatively small areas, although that is not likely to be the case once another variable is introduced, such as age or race.
Users should not simply rely on commonly cited precision standards in deciding whether to use particular estimates. They also need to take into account the specific requirements of their application. For example, deciding which subset of school districts should receive additional funding directed to low-income students may require a narrower confidence interval than the standard. Thus, a 90 percent confidence interval of 12 to 18 percent poor school-age children, which corresponds to a 12 percent relative standard error for an estimate of 15 percent poor school-age children, may be too wide an interval for purposes of fund allocation. Still, for some applications, a ballpark estimate with an even wider confidence interval may suffice.
In deciding which set of ACS estimates is best suited for a particular application, users will need to make trade-offs between timeliness and sampling error. For example, a user could decide that a 3-year period estimate is preferable to a 1-year period estimate for a large city or county in order to achieve a greater level of precision. Alternatively, a user could decide that
OCR for page 73
Using the American Community Survey: Benefits and Challenges
TABLE 2-8 Illustrative, Approximate Relative Standard Errors (Coefficients of Variation, or CVs) for an Estimate of 15 Percent Poor People from the ACS and the 2000 Census Long-Form Sample, by Population Size of Area
Population Size of Area (1)
Poor People (15% of total pop.) (2)
ACS 1-Year Period Estimate
ACS 3-Year Period Estimate
ACS 5-Year Period Estimate
2000 Long-Form-Sample Estimate
CV(%) (3a)
(Sample Cases) (3b)
CV(%) (4a)
(Sample Cases) (4b)
CV(%) (5a)
(Sample Cases) (5b)
CV(%) (6a)
(Sample Cases) (6b)
2,500,000
375,000
1.7
%(39,550)
1.0%
(118,600)
0.8%
(197,650)
0.5%
(387,500)
1,000,000
150,000
2.7
(15,800)
1.6
(47,450)
1.2
(79,050)
0.8
(155,000)
500,000
75,000
3.8
(7,900)
2.2
(23,700)
1.7
(39,550)
1.1
(77,500)
250,000
37,500
5.4
(3,950)
3.1
(11,850)
2.4
(19,750)
1.6
(38,750)
100,000
15,000
8.5
(1,600)
4.9
(4,750)
3.8
(7,900)
2.5
(15,500)
65,000
13,000
10.6
(1,050)
6.1
(3,100)
4.7
(5,150)
3.1
(10,100)
50,000
7,500
12.1
(800)
7.0
(2,350)
5.4
(4,050)
3.6
(7,750)
25,000
3,750
17.1
(400)
9.8
(1,200)
7.6
(2,000)
5.1
(3,900)
20,000
3,000
19.1
(300)
11.0
(950)
8.5
(1,600)
5.6
(3,100)
10,000
1,500
27.0
(150)
15.6
(450)
12.1
(800)
8.0
(1,550)
5,000
750
38.2
(80)
22.0
(250)
17.1
(400)
11.3
(800)
3,000
450
42.7
(70)
24.6
(200)
19.1
(350)
12.6
(700)
1,500
225
32.5
(70)
18.7
(200)
14.5
(350)
9.6
(700)
500
75
44.8
(30)
25.8
(100)
20.0
(150)
16.7
(250)
NOTES: See Notes for Table 2-7a—columns 3a–6b in Table 2-8 correspond to columns 4a–7b, respectively, in Table 2-7a. Population sizes for calculating standard errors are in column 1. To obtain an approximate 90 percent margin of error, multiply 15 percent by the estimated coefficient of variation (CV) above to obtain the estimated standard error and multiply the result by 1.65. For example, the 90 percent margin of error for an ACS 1-year period estimate of 15 percent poor people in an area of 65,000 total population is 15 times 0.106 equals 1.6, times 1.65 equals ±2.6, which, in turn, gives a 90 percent confidence interval of 12.4–17.6 percent poor people.
OCR for page 74
Using the American Community Survey: Benefits and Challenges
several years of 1-year period ACS estimates will be informative regarding trends and the current situation for the city, even though the estimates are less precise (see discussion in Chapter 3).
2-C.2.d
Documentation of Sampling Error
The Census Bureau commendably is trying to impress upon users the extent of sampling error in the ACS estimates. Originally, for data products issued through mid-2005 from the C2SS and the ACS test surveys for 2001–2004, the Census Bureau published upper and lower 90 percent confidence interval bounds (for example, 13–17 percent for a 15 percent estimate of poor school-age children). In response to users, who are more accustomed to the MOE concept (as reported in the media for public opinion polls), the Census Bureau decided to replace the upper and lower bounds in tables with the 90 percent MOEs for specific estimates (such as ±0.2 percentage points). In addition, the Census Bureau will not publish 1-year or 3-year estimates when their imprecision is deemed to be too great. In these instances, the standard tabulation categories will be combined to the point at which the tabulations meet the Census Bureau’s threshold for a minimally acceptable level of precision. The 5-year period estimates will not be treated in this manner, even for very small areas for which they are highly imprecise, because the 5-year small-area estimates are the building blocks for a wide range of user applications similar to how the long-form-sample data were used (see Section 4-D.2).
In contrast, the sampling error of the long-form-sample estimates was not highlighted, but instead was contained in footnotes and auxiliary documentation. Moreover, margins of error were not provided for specific estimates; instead, users were provided with general formulas for making their own computations of sampling error. As a result, many users have been unaware of the sampling error in the long form-sample estimates they have been using.
2-D
SUMMARY ASSESSMENT
The ACS promises to be of great benefit to many users for a wide range of applications for which they previously relied on information from the decennial census long-form sample. The three major benefits of the ACS are its timeliness, frequency, and the improved quality of the responses when compared with the long-form sample. Not only will the ACS information be released within 8–10 months of completion of data collection, compared with 2 years or more for the long-form sample, but it will also be updated every year instead of once a decade. Moreover, there is strong evidence that
OCR for page 75
Using the American Community Survey: Benefits and Challenges
the ACS will provide data with reduced nonsampling error because of such factors as the use of trained interviewers to collect the information from nonrespondents. In tests of the ACS, improvements in quality are evident in more complete response to almost every item compared with the long-form sample. Furthermore, in personal interviews, some items will be more accurately reported because the computer-assisted interviewing can more readily correct respondent misperceptions about what is being asked and resolve inconsistent responses.
A complication for users of switching from the census long-form sample to the ACS is the continuous fielding and processing of the ACS. This design produces estimates that pertain to periods of time—averages over 12, 36, or 60 months—instead of the traditional point-in-time estimates with which users are familiar from the long-form sample and other household surveys. Users will need to work together and with the Census Bureau to develop strategies for application of the ACS information that take account of the survey’s continuous design. In Chapter 3 we outline some of these strategies.
Sampling error or imprecision of the estimates is a problematic aspect of the ACS, although users should remember that many long-form-sample estimates did not meet common standards of precision for small areas, either (see Tables 2-7a, 2-7b, 2-7c, and 2-8). When the data are averaged over 5 years, it appears that the ACS will provide reasonably precise estimates for small population groups, such as poor school-age children, for areas with 50,000 or more people but not for smaller areas. The ACS 1-year estimates for such a small population group will have low precision unless the area has at least 250,000 people. For larger population groups, such as total poor, the ACS 5-year estimates will likely provide reasonably precise estimates for areas of at least 10,000 people, while the ACS 1-year estimates will meet that standard for areas of at least 50,000 people.
ACS estimates for census tracts, which average 4,000 people, and block groups, which average 1,500 people, will be very imprecise. Indeed, they were not precise from the long-form sample for other than large population groups. However, these areas can be combined in various ways by users who want to compare planning districts, wards, or other components of large cities, counties, and other areas.
The bottom line for large geographic areas—such as states, congressional districts, and large metropolitan areas, cities, and counties—is that the ACS estimates will be a great asset to data users. The data will be timely, up to date, of good quality, and reasonably precise. The 5-year data for census tracts and block groups, while not precise in and of themselves, will provide building blocks that should enable detailed analyses of the populations of large geographic areas.
Estimates from the ACS for small governmental units, even with over-
OCR for page 76
Using the American Community Survey: Benefits and Challenges
sampling, are the most problematic from the perspective of sampling error. Consider a place of 1,500 people and 300 school-age children, of whom 45 children or 15 percent are estimated to be poor. Table 2-7c shows a 90 percent confidence interval of 7 to 23 percent poor school-age children from 5 years of ACS data. Based on the calculations used to derive Table 2-7c, the margin of error of the ACS estimate is 51 percent greater than that from the 2000 long-form sample, which already has a high margin of error, and this increase may be somewhat underestimated. Moreover, the option of combining small governmental units into larger analytical units in order to improve the precision of estimates is less applicable than in the case of combining census tracts or block groups within a larger jurisdiction.
Chapter 3 discusses possible strategies for data users who are interested in very small governmental units to make effective use of the ACS estimates. It will also be imperative to maintain the planned sample sizes for the ACS over time and, furthermore, for the Census Bureau, in cooperation with users, to seek ways to improve the precision of the estimates for small areas (see Section 4-A.5).
Representative terms from entire chapter:
census bureau
