National Academies Press: OpenBook

Using the American Community Survey: Benefits and Challenges (2007)

Chapter: PART II: Technical Issues, 4 Sample Design and Survey Operations

« Previous: 3 Working with the ACS: Guidance for Users
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

PART II
Technical Issues

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

This page intentionally left blank.

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

4
Sample Design and Survey Operations

The panel is impressed by the extent of research and development that the Census Bureau has devoted to the design and operation of the American Community Survey (ACS) throughout 10 years of testing and partial implementation. Given that the ACS has just been fully implemented and given its complex design, it will continue to require a high level of research, evaluation, and experimentation that can not only inform users and ACS managers, but also lead, as appropriate, to modifications that increase the quality and usefulness of the data and the efficiency of the survey operations. Such research needs to systematically evaluate various aspects of the survey in the context of full implementation and also to address unforeseen problems that may arise in data collection and processing.

The sections of this chapter address the following specific aspects of the ACS sample design and operations that, in the panel’s judgment, require continued research, evaluation, and experimentation by the Census Bureau:

  • Sampling operations for housing units, including initial sampling using the Master Address File (MAF) as the sampling frame and subsampling for nonresponding housing units;

  • Data collection for housing units, including mode of data collection and residence rules;

  • Sampling and data collection for group quarters; and

  • Data preparation, including confidentiality protection, collapsing of tables for large sampling errors, inflation adjustments, tabula-

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

tion specifications with respect to the universe and geographic areas for which various estimates are provided, and data quality review.

In each section, descriptive information precedes a discussion of issues and the panel’s assessment. Weighting procedures are discussed tangentially; for a detailed discussion, see Chapter 5, which reviews the construction and interpretation of the ACS estimates for 12 months (1-year period estimates), and Chapter 6, which reviews the construction and interpretation of the ACS estimates for 36 months (3-year period estimates) and 60 months (5-year period estimates). A report from the U.S. Government Accountability Office (2004) discusses some of the same ACS issues as this report, including residence rules, methods for deriving independent population and housing controls, inflation adjustments for dollar amounts, and understanding the ACS multiyear period estimates.

Chapters 4, 5, and 6 necessarily emphasize aspects of the ACS that appear to be or may be problematic and hence require continued research and evaluation. Readers should keep in mind the substantial benefits of the ACS in comparison with the 2000 long-form sample that are spelled out in Chapters 2 and 3. These benefits include timeliness, frequency of updating, improved data quality in terms of completeness of response, and consistency of measurement with the long-form sample for most items.

4-A
SAMPLING OPERATIONS FOR HOUSING UNITS

This section briefly describes the development of the initial ACS sample of housing units from the MAF (4-A.1), sampling rates for the initial sample (4-A.2), and subsampling rates for nonresponse follow-up (4-A.3). It then outlines the panel’s concerns and recommendations for the MAF (4-A.4) and the ACS sample size and design (4-A.5).

4-A.1
Developing the Initial Sample

The initial ACS sample of housing unit addresses in the 50 states and the District of Columbia for 2005 and subsequent years consists of approximately 250,000 housing units per month and approximately 3 million housing unit addresses for the year (about 2.3 percent of 129.5 million housing units on the MAF in 2005).1 The initial sample—that is, the sample before subsampling for nonresponse follow-up by computer-assisted per-

1

Refer back to Box 2-1 for a brief description of sampling and other procedures in the Puerto Rico Community Survey; for further information about the housing unit sampling procedures in the United States and Puerto Rico, see Asiala (2004, 2005); Hefter (2005a); U.S. Census Bureau (2006:Ch. 4).

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

sonal interviewing (CAPI)—is selected using systematic sampling from the MAF so that each monthly sample is spread throughout the United States in an unclustered way. The initial sampling occurs in two phases (see Box 4-1): subdivision of the MAF into yearly segments (first phase) and selection of addresses for the ACS sample for each data collection year from the applicable segment (second phase).

The first phase is designed to allocate housing unit addresses on the MAF to five equal segments, each of which is assigned to year t1 through year t5 of specified 5-year periods, which are 2005–2009, 2010–2014, 2015–2019, and so on. The addresses in each segment are eligible to be selected for the ACS sample only for the years to which they are assigned (for example, 2005, 2010, 2015, and so on for t1 addresses; 2006, 2011, 2016, and so on for t2 addresses). This segmentation procedure ensures that no address will be included in the ACS sample more than once every 5 years.

The first-phase segmentation of MAF addresses proceeds on a continuous basis in two waves each August and January. The process began in August 2004 when all of the housing unit addresses on the MAF at that time were assigned to equal segments for years 2005–2009. Then, each January and August, newly added addresses are assigned equally to one of the five segments for the period then in progress: for example, new addresses identified in January 2006 were assigned equally to years 2005–2009. In August 2009, all addresses assigned to segments for 2005–2009 that still exist as housing unit addresses on the MAF at that time will be reassigned to the same segments for 2010–2014, and the process of assigning newly added addresses to segments for these 5 years will proceed each January and August until August 2014, when the process will begin anew. The assignment to the five yearly segments is carried out using systematic sampling of addresses, which are arranged in each county by sampling rate stratum (see below) and geographical location.

The second-phase sampling is designed to select ACS sample addresses from a given year’s first-phase segment to meet specified sampling rates that are chosen to improve the precision of estimates for small governmental units. The second-phase sampling proceeds in two stages, corresponding to the stages of first-phase sampling. In August of year t – 1, a main sample is selected from the segment of MAF addresses assigned to year t; then, in January of year t, a supplemental sample is selected from newly added MAF addresses assigned to year t’s segment. The main sample addresses are assigned equally to the 12 months of year t for data collection, while the supplemental sample addresses are assigned equally to months April–December of year t for data collection.

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

BOX 4-1

Developing the Initial ACS Sample, Phases One and Two, Area X with 20,000 Housing Units (50,000 People)

Phase One: Allocate Master Address File (MAF) Housing Unit Addresses to Five Segments August 2004:

1. MAF addresses for Area X:

20,000 housing unit addresses

 

1a. Divide into 5 equal segments:

4,000 addresses each

 

Segment 1: 2005

Segment 2: 2006

Segment 3: 2007

Segment 4: 2008

Segment 5: 2009

 

January 2005:

 

2. Newly added MAF addresses for Area X:

500 housing unit addresses

 

2a. Divide into 5 equal segments as above:

100 addresses each

August 2005:

 

3. Newly added MAF addresses for Area X:

625 housing unit addresses

 

3a. Divide into 5 equal segments as above:

125 addresses each

January 2006:

 

4. Newly added MAF addresses for Area X:

250 housing unit addresses

 

4a. Divide into 5 equal segments as above:

50 addresses each

August 2006:

 

5. Newly added MAF addresses for Area X:

100 housing unit addresses

 

5a. Divide into 5 equal segments as above:

20 addresses each

January 2007:

 

6. Newly added MAF addresses for Area X:

100 housing unit addresses

 

6a. Divide into 5 equal segments as above:

20 addresses each

And so on … until:

August 2009:

MAF addresses for Area X, including all additions, demolitions, and ineligible units:

 

Divide into 5 equal segments

 

Segment 1: 2010 (addresses previously assigned to 2005)

Segment 2: 2011 (addresses previously assigned to 2006)

4-A.2
Initial Sampling Rates

Initial sampling of housing unit addresses from the applicable segment for a data collection year each August and January (prior to nonresponse follow-up subsampling) uses one of five different sampling rates for the addresses within each geographic block (an area of, on average, about 15–20 housing units). The five sampling rates pertain to five strata containing the following kinds of blocks:

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

 

Segment 3: 2012 (addresses previously assigned to 2007)

Segment 4: 2013 (addresses previously assigned to 2008)

Segment 5: 2014 (addresses previously assigned to 2009)

January 2010:

 

Newly added MAF addresses for Area X

Divide into 5 equal segments as above, and so on …

MAF Addresses in Segments 1-3 After Phase One, August 2004, 2005, 2006

Assumes no demolitions or ineligible units.

 

7. Segment 1 - August 2004 (line 1a):

4,000 addresses

 

8. Segment 2 - August 2005 (lines 1a + 2a + 3a):

4,225 addresses

 

9. Segment 3 - August 2006 (lines 1a + 2a + 3a + 4a + 5a):

4,295 addresses

Phase Two: Select Housing Unit Addresses for Each Year’s ACS Sample from Applicable Segment

Assume sampling rate is 11.5 percent (2.3 percent times 5—see Section A.2).

2005 ACS:

 

August 2004: Draw main sample from Segment 1 (line 7)

460 units

 

January 2005: Draw supplemental sample from Segment 1 (line 2a)

12 units

 

TOTAL ACS sample for 2005

472 units

2006 ACS:

 

August 2005: Draw main sample from Segment 2 (line 8)

486 units

 

January 2006: Draw supplemental sample from Segment 2 (line 4a)

6 units

 

TOTAL ACS sample for 2006

492 units

2007 ACS:

 

August 2006: Draw main sample from Segment 3 (line 9)

494 units

 

January 2007: Draw supplemental sample from Segment 3 (line 6a)

2 units

 

TOTAL ACS sample for 2007

496 units

And so on …

NOTE: Phase Two initial sampling rates will be reduced as the size of the MAF grows to maintain the overall ACS initial sample size of about 3 million housing unit addresses.

  1. Blocks in the smallest governmental units that are eligible for oversampling (refer back to Tables 2-3 and 2-4)—defined as eligible governments with an estimated fewer than 200 occupied housing units;

  2. Blocks in smaller governmental units—defined as eligible governments with an estimated 200 to fewer than 800 occupied housing units;

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
  1. Blocks in small governmental units—defined as eligible governments with an estimated 800 to fewer than 1,200 occupied housing units;

  2. Blocks in large census tracts in large governmental units—defined as census tracts with an estimated more than 2,000 occupied housing units; and

  3. All other blocks.

The designation of initial sampling rates is based on estimates of occupied rather than total housing units because blocks in governmental units or census tracts with large numbers of seasonally vacant housing units would be undersampled if total housing units were the criterion. The estimates of occupied housing units are obtained from the current MAF address count times an estimate from the 2000 census of the proportion of occupied housing units for blocks in the governmental unit or census tract. These estimated proportions will presumably be updated at each census.2

Initial sampling rates are calculated for each of the five strata that will produce approximately equal precision for estimates of a given characteristic for small governmental units and large census tracts outside these units and keep the overall initial ACS sample at about 3 million housing unit addresses each year. A budget constraint necessitates that the initial sampling rate be reduced for some census tracts in order to pay for a higher level of CAPI nonresponse follow-up in tracts with lower-than-average response by mail and computer-assisted telephone interviewing (CATI). For this purpose, the initial sampling rate is reduced by 8 percent for census tracts in strata 4 and 5 (see above) for which at least 75 percent of addresses are mailable and it is projected that at least 60 percent of responses will be obtained by mail or CATI.

For 2005, the initial (and reduced initial) overall sampling rates for the five strata were as follows:

1. blocks in the smallest governmental units eligible for oversampling: 10 percent;

2. blocks in smaller governmental units: 6.9 percent;

3. blocks in small governmental units: 3.5 percent;

4a. blocks in large census tracts in which sample reduction not made: 1.7 percent;

2

In Alaska Native and American Indian areas, blocks are assigned to a stratum by applying the estimated percentage of the population that is Alaska Native and American Indian to the estimate of occupied units for the block; the purpose of this procedure is to boost the sample in Alaska Native and American Indian areas.

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

4b. blocks in large census tracts in which sample reduction made as above: 1.6 percent;

5a. all other blocks in tracts in which sample reduction not made: 2.3 percent; and

5b. all other blocks in tracts in which sample reduction made as above: 2.1 percent.

The second-phase sampling of housing unit addresses for a data collection year uses the above sampling rates multiplied by 5 to allow for the fact that only one-fifth of the addresses in the MAF are included in the first-phase segment for that year. For example, 50 percent of the addresses in the first-phase segment for blocks in category 1 above will be sampled (10 percent multiplied by 5), as will 34.5 percent for blocks in category 2, 17.5 percent for blocks in category 3, and so on. In years after 2005, the 2005 initial sampling rates will be reduced as necessary to maintain an overall initial sample size of about 3 million housing unit addresses. The exception is that no reduction will be made in the sampling rate for stratum 1.

4-A.3
Subsampling for CAPI Follow-up

Even though response to the ACS, like the decennial census, is mandatory, the Census Bureau has never expected that as high a proportion of housing units sampled in the ACS would return their questionnaires by mail as occurs in the publicity-rich environment of the census. In order to reduce costs, the Census Bureau planned from the beginning to use CAPI to collect data for a subsample of nonresponding ACS sampled housing units instead of all of them. Before drawing the subsample, the Census Bureau planned to try to collect data by telephone using CATI for as many as possible of the sampled housing units not responding by mail.

The Census Bureau specified three CAPI subsampling rates to apply to housing unit addresses that were mailed a questionnaire but did not respond by mail or CATI (see Hefter, 2005a, for details):

  1. addresses in census tracts with predicted levels of mail and CATI responses between 0 and 35 percent: 50.0 percent (1 in 2);

  2. addresses in census tracts with predicted levels of mail and CATI responses between 36 and 51 percent: 40.0 percent (2 in 5); and

  3. addresses in other census tracts: 33.3 percent (1 in 3).

In addition, two-thirds (66.7 percent) of nonmailable addresses and addresses in remote Alaska are followed up in the CAPI operation.

The higher (lower) rates are used to subsample nonresponding housing units in census tracts with predicted lower (higher) levels of mail and

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

CATI response in order to roughly equalize the precision of the estimates for areas with differing levels of predicted mail and CATI response rates. The predicted levels for the 2005 subsampling operation were developed from mail response rate information from the Census 2000 Supplementary Survey and the 2001–2003 ACS test surveys when available and otherwise from a model that included data from the 2000 census; ACS mail and CATI response rate information will be used for all areas in the future.

4-A.4
MAF Concerns and Recommendations

The MAF plays a critical role as the sampling frame for the ACS. It is the Census Bureau’s inventory of known residential addresses (housing units and group quarters) and selected nonresidential units in the United States and Puerto Rico. It contains mailing and location address information and other attribute information about each address. It also contains geographic codes, such as county and place codes, obtained by linking to the Census Bureau’s Topologically Integrated Geographic Encoding and Referencing (TIGER) database.

For purposes of sampling housing unit addresses for the ACS, the following types of housing unit records are currently included in the ACS version of the MAF (see U.S. Census Bureau, 2006:Ch. 3):

  • housing units in existence in the 2000 census and those added from the postcensus program to resolve challenges by localities to their population counts (the count question resolution program);

  • new housing units added from semiannual updates of the U.S. Postal Service’s (USPS) Delivery Sequence File (DSF), along with housing units that were deleted in the 2000 census but continue to appear on the DSF;3

  • new housing units added from ongoing listings of addresses in areas of new construction that are conducted for the Census Bureau’s other household surveys; and

  • new housing units added from the Community Address Updating System (CAUS), which annually lists addresses in about 20,000 blocks, out of a total of 750,000 largely rural blocks, where use of the DSF does not provide adequate coverage.

Corrections to housing unit addresses are obtained from all of the above updating programs and from ACS interviewers.

Because the ACS is a continuous monthly survey nationwide, it is es-

3

To the extent that demolished housing units are not systematically deleted from the DSF, then the retention of housing units that remain on the DSF but were deleted from the 2000 census MAF may result in unnecessary follow-up costs in areas with heavy demolitions.

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

sential that its sampling frame—the MAF—be as complete and accurate as possible and that it be updated on a continuous basis in all areas of the country. The panel is concerned about the quality of the MAF updating, not only in areas with city-style addresses (house number and street name—see Section A.4.a below), but also in rural areas (see Section A.4.b; see also National Research Council, 2004a, which raises many of the same points).

4-A.4.a
The MAF in Urban Areas

The MAF updating for city-style-address areas between censuses depends almost entirely on the USPS DSF, for which the Census Bureau receives updated versions every 6 months. The DSF is a mail delivery file and is not meant to be a complete address list. Research conducted prior to the 2000 census indicated that the DSF is deficient as a source for the MAF in urban areas in at least three respects (see U.S. General Accounting Office, 1998:17-18):

  1. The DSF misses many addresses in new construction areas, where it takes time to establish separate mailboxes and mailing addresses.

  2. Portions of the DSF are not updated at the same rate all around the country.

  3. The DSF often does not clearly identify addresses in small multi-unit structures—in many of these units, mail may be delivered to a central hall or desk and not to the individual apartments.

These deficiencies in the DSF led to a decision by the Census Bureau for the 2000 census to conduct a complete canvass of all 8.2 million blocks in 1999 in order to bolster the completeness of the 2000 Decennial Master Address File. Previously, the Census Bureau had planned to conduct a complete canvass only in rural areas and to spot-check addresses in urban areas.

For the 2010 census, the Panel on Research on Future Census Methods (National Research Council, 2004a) recommended partnerships with state, local, and tribal governments to collect address list and geographic information throughout the decade in order to reduce the need for block canvassing in 2009, but such partnerships were not developed. Instead, the Census Bureau plans to repeat the very costly complete block canvass operation in 2009. It also plans to conduct a Local Update of Census Addresses (LUCA) program in 2008, in which local governments are given the opportunity to review and update the residential address listings for their jurisdiction, similar to a LUCA program conducted just prior to the 2000 census.4

4

The 2000 LUCA program experienced scheduling and communication problems, and participation was spotty across the country (National Research Council, 2004b:145).

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

While the block canvass in 2009, supplemented by a LUCA Program, should improve the MAF not only for the 2010 census, but also for the ACS, the improvements will be made at a point in time rather than continuously over the period leading up to 2010. The consequence is that the ACS sample for years prior to the block canvass will to some extent underrepresent growing areas of the country.

The ACS may also not accurately represent residences in small multi-unit structures (those with 2–9 apartments). Evidence from the 2000 census indicates that the problem of missed or erroneously identified addresses in these types of structures persisted in the 2000 MAF even after the block canvass and LUCA programs. At present there is no research in progress to investigate the problems of addresses in multiunit structures or duplicate addresses, even though research on the accuracy of the 2000 MAF shows that over 2 million duplicate housing unit addresses may not have been weeded out and, at the same time, over 2 million housing unit addresses may have been missed (National Research Council, 2004b:140-141).

Finally, research is needed on questionnaires that are returned by postal carriers because they are “undeliverable as addressed”—about 12 percent of mailed-out ACS questionnaires in 2005 (U.S. Census Bureau, 2006:7-7). These questionnaires are not missed in the ACS sample because they are included in the workload for CAPI follow-up, but they indicate problems in the MAF that need investigation.

Independent housing unit control totals for 1,951 estimation areas (counties and groups of small counties) are used to adjust the weights of ACS housing unit responses with the intent to reduce net coverage error (see Section 5-C for how these controls are developed using the previous census and local information on building permits). The application of the controls will increase (decrease) the housing unit weights in areas for which the unadjusted ACS estimates fall short of (exceed) the controls. While the effectiveness of these controls requires more research, their use may help identify and adjust for possible housing unit coverage errors in the ACS. However, their use will not adjust for coverage errors for specific kinds of housing—for example, the same weight adjustment is made to single-family homes, small multiunit apartments, and large building apartments in an estimation area. Moreover, because the census-based housing unit controls are subject to error, there will likely be inconsistencies between ACS estimates of housing units for 2010 and the 2010 census results.

4-A.4.b
The MAF in Rural Areas

In rural areas, the CAUS was developed because of the difficulty of using the DSF to identify addresses that should be added to the MAF. Many DSF addresses in rural areas are rural route or post office box number ad-

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

dresses that do not indicate street name and house number. Consequently, when the Census Bureau receives an updated DSF that has more addresses than are currently listed on the MAF for a rural area, it is not easy to determine which addresses are new. (Research is under way to determine if there are effective ways to use the DSF for address updates in rural areas.)

The identification of CAUS counties for listing is based on an algorithm that considers the address characteristics of existing MAF records for the county, changes in postcensal housing unit estimates for the county, and changes in the DSF tallies for the county. A second stage takes the counties identified for some potential CAUS listings and identifies blocks that would be expected to yield the most new units. Two ACS sources are used to identify CAUS-eligible blocks. One is blocks with addresses in which ACS fieldwork returned numerous outcomes such as “unable to locate” or “address nonexistent.” The other is blocks with a high percentage of addresses that were unmailable from the ACS mailout operation because they lacked a house number/street name/ZIP code address. A third source is from field representatives who identify blocks needing updating while they are in the field completing other block listing assignments. The number of selected blocks from those ranked highly by the algorithm is dictated by budget and operational constraints.

Dean and Peterson (2005) conducted the first evaluation of CAUS. They examined the CAUS listings completed between September 2003 and August 2004 to evaluate the targeting of blocks for CAUS work, review the quality of the CAUS listings, and find out if other Census Bureau operations would have captured the address updates or if CAUS was the only means to collect the information. The study found that CAUS was successful in adding addresses to the MAF that would not have been added by other means, but the study was limited in scope and did not address the issue of addresses that are missed because of constraints on the CAUS operation. No further evaluation has been conducted of CAUS.

4-A.4.c
Recommendations for MAF Research and Development

Recommendation 4-1: Given the centrality of the MAF to the ACS, the Census Bureau should ensure that adequate resources are provided to attain the highest possible completeness and accuracy of MAF address information on a continuous basis.


Recommendation 4-2: The Census Bureau should plan now for programs to follow the 2010 census to ensure that the MAF is updated on a continuous basis more completely than is being done prior to 2010. These programs should include not only the current updates from the DSF and the CAUS but also such initiatives as continuing local review,

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

the use of ACS field interviewers to investigate address problems, and the use of address information from the Census Bureau’s e-StARS database of linked administrative records.


Recommendation 4-3: The Census Bureau should support a continuing research program on the quality of the MAF and the cost-effectiveness of the various operations that are designed to update the MAF. This program should include periodic field checks on MAF addresses, comparisons with housing unit estimates for specific areas, comparisons with the e-StARS database, and comparisons with the results of the 2009 complete block canvass that will be used to prepare the 2010 census MAF. The program should also include studies of methods to improve the listing of small multiunit addresses in urban areas, characteristics of duplicate housing units, and characteristics of undeliverable mail addresses. In addition, the program should examine the effectiveness of the CAUS and explore ways to improve its performance.

The e-StARS database referenced in recommendations 4-2 and 4-3 is the Census Bureau’s electronic Statistical Administrative Records System (StARS). This database consists of addresses and other linked information for households and people from a number of federal and state administrative records, including Social Security, unemployment compensation, Medicare, and others. Addresses are geocoded to small geographic areas. The Census Bureau is using the e-StARS database for wide-ranging research on such topics as ways in which administrative records could improve census operations (Resnick and Obenski, 2006). The Bureau is also experimenting with the use of e-StARS to reduce the variance of ACS estimates for subcounty areas (Fay, 2006; see Section 6-B).

4-A.5
Sample Design Concerns

The ACS sample design is complex and differs from the point-in-time design of the decennial census long-form sample. The panel is concerned that users understand the differences, including the role of housing and population controls. The panel is also concerned about the consequences of smaller sample sizes (compared with the long-form sample) and variable sampling rates (including CAPI subsampling) for the sampling errors of ACS estimates.

4-A.5.a
Long-Form and ACS Sampling Frames

The 2000 census long-form sample was a systematic sample of housing units from the DMAF, using variable sampling rates to provide more precise

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

estimates for small governmental jurisdictions. A number of steps were taken to achieve high coverage for the DMAF and to remove duplicate and nonexistent addresses. Although the final DMAF had some remaining errors of omission and erroneous inclusion (as noted above), there is evidence that, on balance, it included virtually all of the housing stock at the time of the census in spring 2000 (National Research Council, 2004b:140-141). In turn, the long-form sample represented the characteristics of the housing stock as of spring 2000 and the people living in that housing as of springsummer 2000 (allowing that not all enumerations were completed until August and that there were undoubtedly errors in reporting where people lived as of Census Day, April 1). Recall that long-form-sample questions refer to the time of the census or the prior calendar year and that long-form-sample population and housing estimates are adjusted to agree with Census Day control totals from the complete count census of both short and long forms for small weighting areas.

The ACS representation of housing and people in any one year differs in at least three respects from the point-in-time long-form-sample representation for the census year. These differences involve: (1) the composition of the housing sample each year; (2) the composition of the sample of people in the sampled housing stock, and (3) the housing unit and population estimates that are used as ACS controls.

  1. The ACS sample of housing units in a given year t represents addresses recorded on the MAF as of January of year t; there is no provision to add newly constructed or newly identified units until the following year.5 Given that the ACS collects data continuously throughout the year, measures of the characteristics of the January housing stock from the ACS (for example, occupancy status, owner/renter, housing value) represent averages over the entire year, which may not be the same as how these characteristics would be measured in January. For housing units newly added to the MAF between August of year t – 1 and January of year t, they are included in the year t sample for only 9 months of the year (April-December), so their characteristics (for example, vacancy status) will represent averages over the last 9 months rather than all 12 months of the year.

  2. The ACS sample of household members in year t represents people who lived in the January MAF housing stock at some time in the

5

Each year’s ACS estimates actually pertain to responses obtained during that year, some of which may come from housing units that were included in the sample for November or December of the preceding year but did not provide data until January or February. This does not materially affect the discussion in the text.

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

year according to the residence rules for the ACS, which uses a 2-month rule and not the census “usual residence” rule (see Section B.2 below). The measures of household member characteristics in year t from the ACS represent averages over the entire year. Reflecting the continuous data collection in the ACS, the questionnaire items (population and housing) refer to the time during the year when respondents fill out the survey form, or, for some items, to the previous 12 months.

  1. The housing units and people for whom the ACS collects data in year t are weighted to agree with independently derived estimates of total housing units and population by age, race, sex, and ethnicity as of July 1 for each of 1,951 estimation areas (described above). The application of July 1 controls may make it appear that the ACS representation of housing and population is not so different from the long-form sample, which represents April, but this is not the case. As explained below, the July 1 controls, which derive from the previous census, are not consistent with the underlying ACS data.

4-A.5.b
ACS Housing and Population Controls

For 1-year period estimates from the ACS for housing, the application of a single control for total housing units—assuming it is accurate—will capture growth (or decline) in the housing stock in an estimation area that occurred between January and June. However, it will not capture changes in the composition of the housing stock—for example, in single-unit versus multiunit dwellings—due to growth (or decline) between January and June, nor will it capture changes in the housing stock between July and December. Moreover, the application of housing unit controls for estimation areas will not capture differences in housing growth (or decline) among smaller areas within an estimation area, such as cities for which independent housing unit estimates are available but are not currently used as ACS controls. Finally, the ACS estimates of characteristics of the January housing stock will be averages over the year and not point-in-time estimates for July 1 or any other time during the year. (See Section 5-C for further discussion of the housing unit controls.)

For 1-year period estimates from the ACS for people living in the January housing stock, the application of the population controls will adjust a few more dimensions than just total population to a July 1 reference date—namely, sex, age (13 categories), and race and ethnicity (6 categories), although in practice some collapsing of the cross-classification of these dimensions is common. Yet the population adjustments will have all of the problems of the housing unit adjustments enumerated above. In addition,

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

in areas with seasonal fluctuations in population, the use of July 1 controls derived from the previous census can distort the ACS average estimates of the numbers of various types of people in the area over the entire year (refer back to Table 3-6 for an example; see Section 5-D for further discussion of the population controls).

For many—and perhaps most—areas of the country, the somewhat different representation of population and housing in the ACS 1-year period estimates compared with the decennial long-form sample will not be a significant problem. For some areas, however, the differences may be more pronounced. In fast-growing areas, the restriction of the sample to the January MAF housing stock, even when weighted to represent the housing stock as of July, may cause the ACS estimates to lag the situation on the ground. This could happen, not only for total housing, but also for some housing characteristics if new construction differs markedly from older housing stock. In areas with large seasonal fluctuations in population, as was just noted, the application of census-based July 1 population controls to data that were collected throughout the year may result in estimates of household member characteristics that represent neither a point in time nor an average number.

4-A.5.c
Smaller Initial Sample Size and CAPI Subsampling

Budget constraints limit the size of the sample initially selected for the ACS to 3 million housing units per year, cumulating to 15 million housing units over 5 years. Even if data were collected for the full initial sample, the total 5-year ACS initial sample size is smaller than the 18 million housing units that received the 2000 census long-form questionnaire (16.4 million housing units with usable data were included on the final edited data file). The 5-year ACS initial sample size is smaller yet than the expected sample of about 21.7 million housing units that would result if the average 1-in-6 long-form sampling rate were applied to the 130 million MAF housing unit addresses in 2005. Moreover, the initial ACS sample is reduced by 8 percent in census tracts outside oversampled jurisdictions that are expected to have high mail and telephone response rates.

In addition, unlike the long-form sample design, the ACS design sub-samples housing units that do not respond by mail or telephone for follow-up with CAPI. The CAPI subsampling uses three different rates in order to approximately equalize the precision of estimates for areas with higher and lower mail/CATI response rates. The effect of the CAPI subsampling and the 8 percent reduction in the initial sample in census tracts expected to have high mail and telephone response is to reduce the size of the final sample to about 2.1 million housing units per year nationwide, or about 10.5 million housing units cumulated over 5 years. This reduced sample size

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

is only 70 percent of the initial full sample and less than half of the likely size of a long-form sample in 2010.

The use of CAPI subsampling increases the sampling error of estimates from the ACS in two ways compared with a design that follows up all nonrespondents. First, as noted above, the nominal sample size after subsampling (the number of mail, CATI, and CAPI responses) is reduced to about 70 percent of the initial size. Second, the effective sample size (the size that determines the precision of the estimates) is further reduced from the nominal size. The reason is the variation in sampling rates due to the subsampling, which equates to variation in the weights assigned to respondents. This variation in weights leads to a loss of precision of estimates compared with estimates from an equally weighted sample of the same size. (See Box 4-2 for a simple illustration.) The benefits of the CAPI subsampling are cost savings from reducing the number of expensive CAPI interviews and the size of the CAPI interviewing staff.

4-A.5.d
Variable Initial Sampling Rates

Similar to the long-form sample design, the ACS sample design specifies a limited set of variable initial sampling rates that are introduced in order to make the estimates for small governmental jurisdictions about as precise as the estimates for census tracts in larger jurisdictions. Yet many of the estimates for small areas will not meet commonly accepted statistical standards given the overall size limit of the ACS sample. Small areas must be aggregated into larger geographic areas to obtain reasonably precise estimates from the ACS for them, particularly for small population groups (refer back to Table 2-7a). Such aggregation makes sense for block groups and census tracts for some forms of analysis in larger counties and cities, but it is not likely to be suitable for analyses of small governmental units.

Moreover, the use of a small set of discrete initial sampling rates for different-sized governmental units, when combined with features of governmental organization in the United States, has at least three adverse consequences for the sampling errors of ACS estimates for some jurisdictions. (These problems also affected the sampling errors of long-form-sample estimates.) First, because of the variety of governmental units among and within states, there will likely be some anomalous situations. For example, some states have many small school districts, places, and functioning townships, while other states are principally organized into counties and larger cities. States of the first type will have larger samples, proportionate to their population, than states of the second type.

Second, the use of a small number of discrete initial sampling rates means that areas that differ little in population size may have markedly different sampling errors because they fall into different sampling rate categories. For example, the standard errors of estimates for a governmental

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

BOX 4-2

Illustration of the Effect of CAPI Subsampling on Precision of ACS Estimates

(1) MAF universe for area (housing units)

4,350

(2) ACS 1-year sample (2.3 percent annual sampling rate—see Table 2-3a)

500

(3) Mail or CATI respondents (52 percent)

260

(4) Remaining sample (line 2 – line 3)

240

(5) CAPI subsample (one-third of line 4)

80

(6) Realized number of sample cases (line 3 + line 5—assume there are no final nonrespondents, vacant units, or ineligible units)

340

(7) Effective sample size for estimation (line 6 reduced by the loss of precision due to unequal weights—the 80 CAPI housing units are assigned a weight 3 times as large as the 260 mail and CATI units)

255

(8) Standard errors for estimates based on an effective sample size of 255 housing units compared with an equally weighted sample of 340 units

15% larger

(9) Standard errors for estimates based on an effective sample size of 255 housing units compared with a long-form-sample size of 725 units

69% larger

NOTES: The effective sample size for the unclustered ACS design (line 7) is given by (Σwi)2wi2, where wi is the weight of the respondent—see, for example, Kish (1992).The calculation of differences in standard errors when compared to the long-form sample (line 9) does not take account of weighting factors, such as housing unit and population controls, that are intended to reduce sampling error in both the ACS and the long-form sample. Including them would further favor the long-form sample because the census-based controls used for the long-form sample are applied at a local area level. Moreover, inaccuracies in the estimated controls used for the ACS can lead to bias in ACS estimates, a feature that is not reflected in the sampling errors (see Chapter 5).

unit with 801 households that is initially sampled at an annual rate of 3.5 percent will be about 40 percent larger than that for a governmental unit with 799 households that is initially sampled at an annual rate of 6.9 percent (assuming equal CAPI subsampling rates). The use of discrete CAPI subsampling rates also has this effect.

Third, governmental units may be restructured in ways that have implications for their sampling rate categories and the sampling error of

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

their estimates. For example, a county with 1,000 occupied housing units and five equal-size school districts that are then consolidated into a single countywide district could move from an initial annual sampling rate of 10 percent (because of the constituent districts) to one of 3.5 percent, with a 69 percent increase in standard errors for county and subcounty estimates (assuming equal CAPI subsampling rates).

There are alternative approaches that could be considered. For example, if the initial sampling rate were a smoother function of the measure of governmental unit size, there would be no jurisdictions of very similar size with markedly different sampling rates (see Kalton et al., 1998:19). It is also possible that making school districts ineligible for oversampling could reduce the number of anomalous situations, such as states with disproportionately larger samples. School districts were first made eligible for oversampling in the 2000 census because of the need for more precise estimates for allocation of federal elementary and secondary education funds (see Section 3-A). This need persists, but the costs of oversampling school districts may outweigh the benefits. Oversampling school districts contributes to anomalous situations, as noted above. In addition, school districts frequently change boundaries, and in the ACS context, such changes could contribute to abrupt changes in sampling rates when districts combine or split up.

Regardless of the approach used to oversample small jurisdictions, one result is that many larger jurisdictions, such as counties and cities, contain blocks with very different sampling rates. For example, a county with a large city surrounded by small townships may have initial sampling rates that vary on an annual basis from as much as 10 percent (for the smallest government units) to as little as 1.6 percent (for large census tracts predicted to have at least 60 percent mail/CATI response). After subsampling for CAPI follow-up, the final sampling rates may vary on an annual basis from as much as 7.0 percent (for the smallest government units predicted to have, say, 50 percent mail/CATI response) to as little as 1.2 percent (for large census tracts predicted to have, say, 60 percent mail/CATI response). Accumulated over 5 years, the final sampling rates after CAPI subsampling may vary from 35 to 6 percent—a 6-to-1 ratio; in contrast, the 2000 long-form sampling rates varied from 50 to 12.5 percent—a 4-to-1 ratio. (This discussion ignores the effects of other weight adjustments, such as population and housing unit controls.)

The wider variation in final sampling rates will increase the sampling error of ACS estimates relative to long-form-sample estimates for geographic areas and population groups that incorporate varying sampling rates—either from the initial sampling, the CAPI subsampling, or both sources. This increase in sampling error will be in addition to the increase

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

from the smaller overall initial sample size of the ACS and the use of CAPI subsampling.

4-A.5.e
Recommendations to Review Sample Size and Design

The above discussion of sampling errors did not have the advantage of actual data from the full ACS. Now that the first year of data collection for the full ACS has been completed, the Census Bureau can begin to estimate the expected 5-year sampling errors for small governmental units and census tracts in larger jurisdictions, investigate disparities in sample allocation among states that differ in governmental organization, and determine the extent of other anomalous situations, such as jurisdictions with similar populations that fall into disparate sampling rate categories. Using that information, the Census Bureau should review the sample design decisions that led to the initial sample sizes and effective sample sizes after CAPI subsampling and consider alternatives that might reduce anomalies and make the allocation of the sample as equitable as possible. A review should be conducted of such alternatives as making the CAPI subsampling rates a smoother function of mail and CATI response rates and informing the choice of subsampling rates by the theoretical results on optimum subsampling rates for initial nonrespondents developed by Hansen and Hurwitz (1946).

Yet whatever the particulars of the sample design, given the available budget, the bottom line is that the sampling error of ACS estimates for small governmental jurisdictions will be larger, often substantially so, than the corresponding long-form-sample estimates. The same conclusion applies to ACS estimates for census tracts in larger jurisdictions, although these estimates can much more readily be combined into larger areas for analytical purposes.

The panel thinks that it is critically important to maintain and, if possible, increase the overall size of the ACS sample. A goal could be to increase the final ACS 5-year sample size (after subsampling for CAPI follow-up) to at least the number of housing units in the 2000 long-form sample, which was 16.4 million. This increase would provide a sample about 55 percent larger than the current ACS. To attain this larger final sample size would require an initial 5-year ACS sample size of about 23.5 million housing unit addresses instead of the current 15 million.6

Even with an increase in the ACS sample size of the magnitude just outlined, many small-area estimates, particularly for small population groups,

6

The originally planned initial ACS sample size over 5 years was 30 million housing units, which would have generated a final sample size of about 19 million housing units (see Section 1-B.3).

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

would still not meet commonly accepted standards of precision or even the levels of precision of the long-form sample. They would, however, be 25 percent more precise than comparable estimates from the current ACS.

Recognizing fiscal constraints, increases in the ACS sample size would likely have to be made on an incremental basis. Eliminating the institutional group quarters population from the ACS, as suggested in Section 4-C below, could permit a small increase in the household sample size within the current budget. It is also possible that making school districts ineligible for oversampling would permit some redistribution of the sample to other types of small governmental units.

The panel urges the Census Bureau to work closely with the user community to identify and assess the merits of alternative sample sizes and designs for the ACS. It is unlikely that any single design will be optimal for all users, so that trade-offs and compromises will be necessary, as is true of the current design.


Recommendation 4-4: The Census Bureau should identify potential ways to increase the precision of ACS estimates for small geographic areas, particularly small governmental jurisdictions, through reallocation of the sample and through increases in the overall sample size. Cost savings should be sought to support such increases, although increases that could significantly improve the precision of estimates will require additional funding from Congress. Sample reallocation should also be considered to minimize anomalies across areas (for example, jurisdictions with very similar populations that fall into different sampling rate categories).

4-B
DATA COLLECTION FOR HOUSING UNITS

4-B.1
Mode of Collection

The ACS, like many surveys, uses a mixed-mode data collection design in order to maximize response while containing costs. The ACS uses three modes of data collection:

  1. mailout-mailback, assisted by an advance letter, postcard reminder, and second questionnaire mailed to nonrespondents;

  2. CATI from three telephone call centers to try to reach mail non-respondents (the telephone is also used to follow up mail respondents for whom edit checks indicate a problem with the coverage of household members or failure to answer a minimum number of items); and

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
  1. CAPI of a subsample of mail/CATI nonrespondents. CAPI interviewers, who operate from the Census Bureau’s 12 regional offices, may first attempt to complete an interview by telephone, but approximately 80 percent of CAPI cases require a personal visit to the sample address.

The 2000 long-form sample, in contrast, used two modes of data collection—mailout-mailback (assisted by an advance letter and reminder postcard) and personal paper-and-pencil interviewing.

There is evidence from comparisons of the 2000 long-form sample with the Census 2000 Supplementary Survey (C2SS), which used ACS procedures, that the professional, fully trained ACS CATI and CAPI interviewers, assisted by the built-in computer edits and questionnaire routing of the CATI and CAPI instruments, obtained more complete data than the minimally trained, temporary census enumerators (see Section 2-B.2). The CATI and CAPI interviews were even more complete for most items than the ACS mailout-mailback responses (National Research Council, 2004b: Table 7.5).

Yet the panel has two related concerns with the three different data collection modes in the ACS. One concern is that mode effects may bias responses for the same item in different ways. A second concern is that mode effects may vary among population groups and geographic areas because of differences in their response patterns by mode.

4-B.1.a
Mode Effects on Questionnaire Items

Survey literature documents that responses for the same item obtained in different ways—writing on a paper questionnaire, typing on an Internet questionnaire, responding over the telephone, responding in person—often have different properties (see, for example, de Leeuw, 2005; Dillman, 2000: Ch. 6). Some of these differences may be due to respondent-interviewer effects that are not present for mail or Internet reports; other differences may be due to different presentations of the items in the various modes—for example, providing marital status categories on a mail questionnaire but asking an open-ended “What is your marital status?” question in a telephone interview.

Only limited research has been conducted to date of mode effects in the ACS. Some mode differences were found in the Census Bureau studies that compared the C2SS and the 2000 long-form-sample responses for various questionnaire items, such as disability and race and ethnicity (see Section 2-B.2; see also Stern and Brault, 2005, which reports on the response effects of changing the placement of disability questions on the 2003 ACS mail questionnaire).

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

The panel recommends that research on mode effects on item reporting in the ACS be conducted using appropriate experimental designs. Even though it is difficult to design an experiment that can estimate the pure mode effect on reporting because of the confounding mode effect on unit nonresponse (see Biemer and Lyberg, 2003), some work is possible and should be done, given the centrality of multiple reporting modes to the ACS. For example, a sample of mail respondents could be reinterviewed by CATI or CAPI to compare the two sets of responses, or a subsample of mail nonrespondents for which telephone numbers are available could be sent to CAPI instead of CATI interviewing and their responses compared with responses obtained by CATI.

4-B.1.b
Differences in Response Mode for Population Groups

Census Bureau research has shown that households responding by mail in the decennial census differ from households requiring follow-up. Households that respond by mail are more likely to own their own homes and be headed by an older person; they are less likely to be headed by a nonwhite or Hispanic person (National Research Council, 2004b:101-102). Analysis of mail response rates for the C2SS, based on housing units in census tracts with 75 percent or more people reporting a specific race or ethnicity, found marked differences in mode of response by the race and ethnic composition of the tract—see Table 4-1.

TABLE 4-1 Weighted Distribution of Respondents by Mode for Census Tracts with Concentrations of Race and Ethnicity Groups, Census 2000 Supplementary Survey

Population Group (housing units) (weighted)

Response Mode (percent)

Total Response

Mail

CATI

CAPI

Predominantly white census tracts

60.5

7.4

28.1

96.0

Predominantly Asian census tracts

58.6

4.1

32.5

95.2

Predominantly black census tracts

34.9

8.9

48.6

92.4

Predominantly Hispanic census tracts

34.2

8.3

53.3

95.8

Predominantly American Indian and Alaska Native census tracts

16.6

2.6

69.9

89.1

Total housing units

56.2

7.3

31.9

95.4

NOTES: The distributions represent the percentages of housing units that responded by mail, CATI, and CAPI (with CAPI responses weighted to account for subsampling) among the estimated number of housing units that were eligible to be interviewed (excluding nonresidential addresses). The distributions shown apply to housing unit responses in census tracts in which 75 percent or more of the population reported a specific race or ethnicity.

SOURCE: U.S. Census Bureau (2002b:Tables 2, 3, 4).

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

It is likely that differences in response modes characterize other groups as well. For example, non-English-speaking households may be less likely to respond by mail or CATI and more likely to respond by CAPI compared with English-speaking households. There may also be important geographic area differences in response modes.

Overall, the use of mailout-mailback, CATI, and CAPI interviewing results in high housing unit response rates to the ACS. Thus, in the C2SS (see Table 4-1), the overall weighted response rate was 95.4 percent, including 56.2 percent mail response, 7.3 percent CATI response, and 31.9 percent CAPI response (applying weights to CAPI respondents to account for the subsampling). The 2005 ACS overall weighted response rate was even higher (97 percent), although, based on data from January to March 2005, the distribution of responses by data collection mode has changed. Thus, only about 51 percent of the eligible sample in January-March 2005 responded by mail, while 9 percent were interviewed by telephone and 38 percent were CAPI interviews, with 2 percent nonresponse (U.S. Census Bureau, 2006:Figure 7-2).

Differences in response patterns (the mix of the three modes) among population groups and geographic areas—and changes in response patterns over time—may result in different levels and directions of response biases among groups and areas. Whether such effects are important and for which characteristics remains to be established by research.

4-B.1.c
Recommendation for Mode Effects Research

Recommendation 4-5: The Census Bureau should conduct experimental research on the effects of the different data collection modes used in the ACS—mailout-mailback, CATI, and CAPI—on ACS estimates and, when possible, on response errors for questionnaire items. In addition, the Census Bureau should assess how different patterns of responding by mail, CATI, and CAPI among population groups and geographic areas affect comparisons of ACS estimates and inform data users of consequential differences.

4-B.2
Residence Rules
4-B.2.a
Two-Month Rule

The decennial census employs a usual place of residence concept; in the 2000 census, this meant that a person was to be counted at the place where he or she lived or stayed most of the time. Most other household surveys also use a similar concept. In contrast, because of its continuous design in which data collection occurs throughout the year, the ACS

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

changed to a current residence concept, as is more common in polls and other person-based surveys. Specifically, the ACS residence concept is based on a “2-month rule:” people who live for more than 2 months at a sample address are assumed to be residents of that unit. The rule is intended to be prospective as well as retrospective—that is, people who have lived in a unit for more than 2 months at the time of the ACS interview and people who have just moved into the unit and expect to stay there for more than 2 months are considered residents of the unit.

The Census Bureau has identified three exceptions to this general concept (U.S. Census Bureau, 2006:6-2, 6-3): (1) children younger than college age who are away at boarding schools or summer camps are to be considered residents of their parents’ or caregivers’ homes; (2) children who live under joint custody agreements and move often between the residences of their parents are to be considered current residents of the sample unit at which they are staying when contact is made; and (3) commuter workers who stay in a residence close to their work and return regularly to a family residence are to be considered residents of the family residence and not the work residence. In addition, people staying at a unit at the time of the interview who have no other place to stay are to be considered residents of the unit.

While the 2-month rule generally seems reasonable, it is not clear why 2 months was chosen and not another value (for example, 1 month or 3 months). Some of the exceptions to the 2-month rule, particularly for commuter workers, also do not have a clear conceptual basis. In addition, while the 2-month rule acknowledges that not everyone stays in the same “usual residence” all the time (for example, people with summer and winter homes, commuter workers), it does not address other kinds of situations in which people have multiple residences. Examples include people with weekday and weekend residences, people who live and travel throughout the year in recreational vehicles, and people who move among the residences of several relatives or friends.

The 2-month residence rule is applied at the time the data collection takes place. For example, if no mail return comes back from a sampled address and there is no success with CATI, but the address is included in CAPI in the third month of data collection, respondents are asked about residence under the 2-month rule at the time of the interview. Thus, the reference period is a function of the time of interview rather than a fixed time interval related to the month of mailout.

The CATI and CAPI computerized instruments may include questions to enable the ACS residence rules to be applied as intended. However, the mail questionnaire does not clearly or fully explain these rules, as shown in Box 4-3. The accompanying guide for respondents does not provide further instruction (see U.S. Census Bureau, 2006:App. B, which reprints the mail

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

BOX 4-3

Residence Rule Guidance on the ACS Mail Questionnaire

Page 1:


Asks respondent to provide the number of people who “are living or staying at this address.”


Page 2, left-hand margin:


Asks respondent to “READ THESE INSTRUCTIONS FIRST.”

  • LIST everyone who is living or staying here for more than 2 months.

  • LIST anyone else staying here who does not have another usual place to stay.

  • DO NOT LIST anyone who is living somewhere else for more than 2 months, such as a college student living away.

If this place is a vacation home or a temporary residence where no one in this household stays for more than 2 months, do not list any names in the List of Residents.


IF YOU ARE NOT SURE WHOM TO LIST, CALL [number].

questionnaire and instruction booklet; see also National Research Council, 2006, Section 8-C, for a more detailed discussion). To date, no research has been carried out to estimate the extent to which mail respondents follow the intended ACS residence concept.

4-B.2.b
Recommendation on Residence Rules Research

A separate Committee on National Statistics panel was charged to conduct a comprehensive review of the residence rules for the decennial census. In its report (National Research Council, 2006), the panel comments on the ACS residence rules, noting the lack of a clearly articulated basis for the rules (including the exceptions to the 2-month rule noted above) and the lack of clear instructions on the mail questionnaire on how to apply the rules. The report cites literature on relevant respondent behavior, such as the tendency to ignore instructional material, which can lead respondents to misapply residence rules even if they are clearly specified. The report recommends research leading to the addition of questions on the census about other places where people live to assist the Census Bureau to determine usual place of residence. The report also recommends research leading to the inclusion of a question on usual place of residence in the ACS

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

in addition to residence according to the 2-month rule, in order to make it possible to relate census and ACS results (National Research Council, 2006:Recs. 8-3, 8-4).

We support research with the ACS’s experimental methods panel (see Section 7-C.2) to assess the extent to which respondents give different answers to the decennial census usual residence rule and the ACS 2-month residence rule and the extent to which they follow the specific ACS rules, such as the rule to count boarding school students at home. The inclusion of questions on other residences at which respondents spend time would facilitate the determination of respondents’ usual residence and 2-month residence to use in analyzing the experimental results.7 Such research might in the future lead to improvements in the way in which the 2-month rule is explained to respondents, as well as possibly to a decision to modify the 2-month rule in some respect.

In addition, we support research on the effects of the residence rules, assuming they are applied as intended, on estimates for different geographic areas and population groups. For example, the application of the 2-month residence rule should provide a basis for identifying seasonal fluctuations in population in ways that would not be possible with a usual residence rule. A possibly confounding effect could occur from the 3-month data collection window for each month’s sample that is part of the ACS design. What is the effect, for example, on estimates of occupied versus vacant housing units when a seasonal resident does not respond by mail or CATI and has left the area by the time of the CAPI interview? Questions such as these should be addressed through appropriate research, including experimentation.


Recommendation 4-6: The Census Bureau should conduct experiments to determine the extent to which ACS respondents give different answers to the decennial census usual residence rule and the ACS 2-month residence rule and the extent to which they apply the specific ACS residence rules (for example, reporting commuter workers at the family residence, applying the 2-month rule prospectively). To help clarify residence according to the census and ACS concepts, the experimental questionnaire should ask about other residences at which respondents spend time. The Census Bureau should assess the implications of the experimental results for ACS population estimates for different geographic areas and population groups. Depending on the results, the Census Bureau should consider appropriate changes

7

The ACS questionnaire currently asks three relevant questions (see Table 2-2): whether any household members live at the address year round, number of months members live here, and main reason members stay at the address, but these questions are slated to be eliminated in 2008. Moreover, the questionnaire does not ask for information on other residences.

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

in the ACS questionnaire instructions on residence or in the residence rules themselves.

4-C
GROUP QUARTERS SAMPLING AND DATA COLLECTION

About 3 percent of the U.S. population resides in group quarters and not in households. A group quarters (GQ) is defined as a place where people stay that is normally owned or managed by an entity or organization providing housing (and often other services) for the residents. People living in GQs are normally not related to one another. Group quarters include not only institutions, such as prisons and nursing homes, but also noninstitutional group quarters, such as college dormitories, military quarters, and group homes of various kinds (see listing in Table 2-1). Housing unit addresses at which large numbers of (mostly) unrelated people live used to be classified as GQs, but in 2000, these units were classified as households. Similarly, they are included in the ACS household population. Boarding schools and summer camps for children below college age are not included in the ACS GQ universe because of the Census Bureau’s rule that children at these facilities are to be reported at their parental or caregivers’ residences (see Section B.2.a above). Data collection procedures for GQs in the ACS were tested in 1999 and 2001 in the 36 test counties and revised as appropriate. GQs were not included in the C2SS or the 2001–2004 ACS test surveys, nor were they included in the 2005 ACS because of budget constraints. They were included in the 2006 ACS and are included in the 2007 ACS. Some GQ types are out of scope for the ACS for privacy reasons or because monthly data collection would be too difficult and costly: domestic violence shelters, soup kitchens, mobile food vans, targeted non-sheltered outdoor locations, natural disaster shelters, and quarters for crews of maritime vessels.

This section describes the development of the MAF for GQs (4-C.1), sampling of GQs and residents within them (4-C.2), data collection for GQs (4-C.3), and the panel’s concerns and recommendations about GQs (4-C.4, 4-C.5). For details about MAF development, sampling, and data collection procedures for GQs in the ACS, see U.S. Census Bureau (2006:3-7 to 3-8; 4-8 to 4-10; Ch. 8).

4-C.1
Group Quarters and the MAF

For the 2000 census, the Census Bureau originally constructed separate MAFs for GQs and housing units using somewhat different procedures. In the 1990s, the Census Bureau developed an inventory of GQs from various sources. It did not check the GQ inventory against the housing unit MAF until late 1999; these checks identified problems of duplicate GQ and hous-

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

ing unit enumerations, as well as erroneous geographic coding of some GQs (for example, college dormitories that were assigned the geographic location of the university administrative headquarters). Additional geographic coding problems were identified by localities after release of the census counts. After August 2000, when the GQ enumeration was completed, addresses for GQs were added to the MAF, with a flag to indicate that an address was a GQ.

The GQ MAF for the 2006 ACS was constructed by merging an updated GQ inventory file, extracts from the final 2000 MAF, and a file of GQs that were closed on April 1, 2000, but may be open at other times of the year. The Census Bureau also obtained a file of federal prisons and detention centers from the U.S. Bureau of Prisons and a file of military bases and vessels from the Department of Defense. In addition, the Census Bureau conducted Internet research to identify new state prisons and state prisons that had closed. On an ongoing basis, information on new GQs and updated address information for existing GQs is collected by CAUS and the current demographic surveys listing operations.

There has been no formal evaluation of the GQ MAF for the quality of the GQ addresses or for the completeness of the list of GQs. In 2009, there will be a validation of GQ addresses in preparation for the 2010 census. It is likely that the ACS GQ population based on the current GQ MAF will differ in some respects from the 2010 census GQ population.

4-C.2
Sample Design for Group Quarters

The sampling for GQs is different from the sampling for housing units (see Hefter, 2005b). All GQ samples are selected in the main sampling phase in August preceding the data collection year. Two strata are created to sample GQs: the first stratum includes small GQs estimated to have 15 or fewer people as well as GQs listed as closed on Census Day, 2000; the second stratum includes larger GQs estimated to have more than 15 people.

For the small GQ stratum, a two-phase sample of GQs is selected, similar to how the housing unit sample is obtained. The first-phase sampling began in August 2005, when all small GQs were assigned to one of five 20 percent segments or subuniverses. One of these subuniverses is the 2006 first-stage sample, and the rest are assigned to 2007–2010. The 2006 subuniverse will not be eligible for sampling again until 2011. In August 2006, all small GQs that were new since the previous year were assigned equally to the five existing subuniverses, as will be done each August through 2009. In August 2010, the plan is to reassign the GQs in the 2006–2010 subuniverses to subuniverses for 2011–2015 and to assign new GQs likewise.

The second-phase sample of small GQs is designed to yield a 2.5 percent systematic sample of such GQs within each state, sorted by GQ type

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

and geography. To achieve this sampling rate for each year, 12.5 percent (1 in 8) of GQs in the appropriate one-fifth subuniverse are selected. Every person in the selected GQs is eligible to be interviewed, although, if there turn out to be more than 15 people residing in the GQ, a field subsampling operation is implemented to reduce the sample to 10 people.

For the larger GQ stratum, there is no assignment to subuniverses. Instead, all of these GQs in a state are sorted by type and geography, and a measure of size is calculated, which equals the estimated number of residents divided by 10. These groups of 10 constitute the first-stage unit of sample selection: a 2.5 percent (1 in 40) systematic sample of groups is selected each year. GQs with a measure of size of 40 or more will have one or more selections or hits; those with a smaller measure of size may have one hit or no hits. If there is more than one hit in a larger GQ, the hits are allocated to different months for data collection (if there are more than 12 hits, then more than one hit is assigned to one or more months). All GQs in this stratum may be selected in any year regardless of whether or not they were previously selected.

The second-stage and ultimate sampling unit for larger GQs is the person. Field representatives implement the selection of people to be interviewed when they visit the GQs assigned to them each month with at least one first-stage hit. They determine the total number of residents at the GQ and use an automated listing instrument to select 10 residents to be interviewed for that month. The field representatives will return in a subsequent month (as assigned) to large GQs with more than one hit to select another group of 10 to be interviewed.

The assignment to each month of the year of sampled small GQs and one or more sampled groups of 10 people in larger GQs is similar to the procedure for housing units, in that the sampled small GQs and sampled groups of 10 people in large GQs for a state are combined, sorted, and systematically assigned to months January–December. The exceptions to the assignment procedure, due to budgetary and operational constraints, occur for correctional facilities and military barracks. While sampled state and local correctional facilities and military barracks are assigned evenly to all months in the year, all groups of 10 people in a state or local correctional facility or barracks with more than one sampled group are assigned to the same month, instead of being spread across months as is the case for other GQ types. In the case of all sampled federal prisons, all sampled groups of 10 people are assigned to September, with a period of up to 4.5 months allowed for data collection. The U.S. Bureau of Prisons generates the person samples for the federal prisons that are selected by the Census Bureau for the year; the Bureau of Prisons must also conduct security clearances for all field representatives who will conduct interviews in the sampled federal prisons.

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
4-C.3
Data Collection for Group Quarters

For the 2006 ACS, about 18,000 GQ facilities with one or more hits are in the sample (about 20,000 hits). Of these facilities, 850 are military facilities and 148 are federal prisons. The GQ data collection for 2006 was accomplished primarily by field representative personal visits, using an automated Group Quarters Facility questionnaire and a bilingual paper ACS questionnaire for each sampled resident. The facility questionnaire is used at the only or first visit to a GQ to collect address, contact information, and type of GQ for the sampled GQ, record up to two other GQ types for a GQ, ascertain the maximum and current population at the facility, and then generate the person-level sample. The individual GQ resident questionnaire contains the same person items as the household questionnaire but none of the housing unit questions, except for the question on receipt of food stamps.

It is clear that field representatives cannot do all of the interviewing of GQ sample persons face to face, although that is the preferred procedure. Other methods are permitted: the field representative may fill in the questionnaire by telephoning the sample person; conduct an in-person interview with a proxy, such as a relative or guardian; leave the questionnaire with the sample person to complete after ascertaining that the person is physically and mentally able to do so; or leave questionnaires with the contact person for the GQ to distribute them to sample persons and collect them after they are filled in. Any GQ contact person who is enlisted to distribute and collect questionnaires must first be sworn in as a special sworn agent of the Census Bureau, bound to protect the confidentiality of individual responses and subject to the same penalties for breach of confidentiality as regular Census Bureau employees.

4-C.4
Concerns About Group Quarters

Almost every aspect of survey operations for group quarters residents presents challenges for the Census Bureau, and successful data collection for this population requires substantial effort and resources. Feedback from ACS managers is that, after some start-up problems, the 2006 data collection for group quarters residents proceeded relatively smoothly but at considerable expense to complete a sample case. To ensure data of good quality from the GQ component of the ACS, sufficient resources must be devoted to intensive, continuing research and development to fine-tune all GQ procedures, from construction of the MAF and sampling of facilities to the collection of data from individual group quarters members, and then to rigorous control of the quality of operational procedures.

In the 2000 census, the group quarters operation was a stepchild of

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

the household data collection operation, and poor quality of the GQ data was the result. In particular, missing data rates for most long-form-sample items on GQ questionnaires were very high (20 percent or more for four-fifths of the items and 40 percent or more for one-half of the items). The rates were much higher than missing data rates for household members and considerably higher than missing data rates for GQ residents in the 1990 census (National Research Council, 2004b:Tables 7-9, H-8). Missing data rates were particularly high for people in prisons, nursing homes, and other institutions, perhaps because of heavy reliance on administrative records for collecting the data. These and other problems in 2000 led a Committee on National Statistics panel to recommend that the Census Bureau “redesign the processes related to group quarters populations for the 2010 census, adapting the design as needed to different types of group quarters” (National Research Council, 2004b:156).

The Census Bureau has devoted considerable effort to refining its procedures for collecting data from GQ residents in the ACS, and presumably missing data rates for GQ residents, including inmates of institutions, are much reduced in the 2006 ACS compared with the 2000 long-form sample. Yet the panel is concerned about the costs of collecting high-quality GQ information relative to the benefits of the data.

The argument for collecting information on GQ residents in the ACS is so that the survey will cover the entire population similar to the long-form sample. Most national household surveys, in contrast, cover just the civilian noninstitutional population, including residents of housing units and noninstitutional GQs. The Current Population Survey Annual Social and Economic Supplement (CPS ASEC), which produces official income and poverty statistics, covers the civilian noninstitutional population plus members of the armed forces living with their families in housing units or military barracks. The CPS ASEC does not conduct interviews in college dormitories but asks parents to report college students who reside in dormitories as household members.8

The census will continue to obtain basic demographic information about all types of GQ residents once every 10 years for all size geographic areas. The Census Bureau’s population estimates program could publish annual estimates of GQ residents—total and broken down by institutional and noninstitutional—by age, sex, race, and ethnicity for counties, cities, and townships, although the quality of these estimates is not known. National surveys have targeted some GQ populations, although they do not provide small-area estimates (for example, the periodic National Nursing Home Surveys, sponsored by the National Center for Health Statistics, and

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

the periodic Surveys of Inmates in Federal and State Correctional Facilities and Local Jails, sponsored by the Bureau of Justice Statistics).

The question is whether users require continuous collection of detailed long-form-type information for GQ residents for counties, cities, and smaller areas and whether their requirements are sufficiently pressing to justify the high cost of obtaining high-quality responses in the ACS. Indeed, for the institutional population, one can question the relevance of much long-form-type information. For example, what does it mean to ask a prisoner about his or her income, and how useful are the responses? Most residents of nursing homes and long-term-care hospitals likely have income from such sources as Social Security or retirement or disability benefits, but it is not clear how they or their proxies may report other income sources, such as support from family members. In fact, in 2000, fully 78 and 77 percent of prisoners and nursing home residents, respectively, had all of their income imputed because they did not answer any of the income questions. In comparison, 25 percent of household residents had all of their income imputed (National Research Council, 2004b:Tables 7-5, H-8).

The panel thinks that the Census Bureau should give serious consideration to whether long-form-sample-type data from the continuous ACS for the institutional population—and perhaps other types of GQs—is needed to an extent that justifies the costs. Dialogue with the user community could identify items that are important to collect every year on a comparable basis and items that are not needed or for which data are not likely to be of sufficient quality to be useful. Discussion with users could also determine whether it is necessary to collect any data at all for residents of some or all types of GQ. A decision to alter the universe for the ACS by excluding some or all GQ residents would require the use of an appropriate set of population estimates to use as controls for the ACS estimates. For example, household population estimates are used in the 2005 ACS estimates, and noninstitutionalized population estimates are used in other household surveys. The quality of these estimates for estimation areas (counties and groups of small counties) would need to be carefully evaluated (see Section 5-D). A decision to alter the universe for the ACS would also have implications for ACS tabulations and other data products (see Section 4-D.4 below).

4-C.5
Recommendation for Group Quarters

Recommendation 4-7: The Census Bureau should discuss with data users their requirements for detailed information from the ACS for residents of institutions and other types of GQs, particularly at the local level. The discussions should assess benefits against costs, and the results should be used to determine any changes to the GQ com-

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

ponent of the ACS—for example, the possible deletion of institutions from the ACS universe—that would be cost-beneficial for users and stakeholders.

4-D
DATA PREPARATION

This section briefly describes key procedures to prepare the ACS data products, including confidentiality protection measures (4-D.1), the collapsing of tables because of large sampling errors (4-D.2), inflation adjustments of income and housing value and costs (4-D.3), tabulation specifications with respect to the population universe and geographic areas for which various estimates are provided (4-D.4), and data quality review (4-D.5). Recommendations for research and development on these topics are contained within the applicable subsection.

4-D.1
Confidentiality Protection
4-D.1.a
Confidentiality Protection Procedures

The Census Bureau uses three primary methods of disclosure avoidance to minimize the risk that someone could identify an individual respondent in the ACS data products: data swapping, categorizing variables, and top-coding. The first two methods are used for tabulations; all three methods are used for the ACS public use microdata sample (PUMS) files. The PUMS files also protect confidentiality by deleting names and addresses from the individual records, limiting geographic identification to large areas containing about 100,000 people called public use microdata areas (PUMAs), and perturbing the ages of people in households with 10 or more members. In addition, the subsampling for generating the PUMS files affords protection even if one knows a person who was in the full ACS sample because one does not know whether the person is in the PUMS subsample.

Data swapping occurs when a household has rare characteristics (such as being the only minority household in a block group). In such instances, the entire household may be swapped with a demographically similar household in a different geographic region. Only a small percentage of households are ever swapped, and they are never identified. The purpose of swapping is to ensure that users will not be able to identify a household with certainty. All data products are created from the ACS records after swapping.

Categorizing variables refers to collapsing categories of a variable within a table, or on the PUMS records, to avoid small cell sizes. For example, a table may combine some race categories, such as races other than white and black, into a single category, or a table may combine income

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

amounts into intervals of $10,000 or more, with a wide top category, such as $100,000 or more.

Top-coding refers to assigning a value to an individual record that is the same as that assigned to other individuals, all of whom have actual values above a specified limit. For example, all individuals with wages and salaries of $100,000 or more might be assigned the value of $100,000. Top codes for the ACS PUMS files are developed separately according to the distribution of responses by state.9

4-D.1.b
Confidentiality Protection Concerns

The panel strongly supports the protection of respondents’ individual information, because a breach of confidentiality would not only undercut the Census Bureau’s ability to collect information, but also break trust with respondents. At the same time, the panel is concerned that confidentiality protection not be ratcheted up without a careful consideration of the need not only to minimize disclosure risk, but also need to provide useful information for public- and private-sector decision making, research, and analysis. It is not possible to reduce the risk of disclosure to zero; the goal instead must be to minimize risks while not unduly suppressing valuable information.


Microdata Products A recent report of a panel of the Committee on National Statistics, Expanding Access to Research Data (National Research Council, 2005), addresses issues in balancing confidentiality and privacy protection with obtaining an adequate return on taxpayers’ investment through providing users with access to rich microdata sets from government surveys. The report recommends research on techniques for providing useful, innovative public-use microdata sets that increase informational utility without increasing disclosure risk.

In the context of ACS microdata, the panel encourages the Census Bureau to revisit its decision not to include month of data collection on the PUMS as a confidentiality protection measure. Given that individual PUMS records are not identified geographically for areas with fewer than 100,000 people, it could be argued that omitting month of data collection is not necessary to protect confidentiality. Including this variable on the PUMS files would be immensely valuable for analytical purposes in light of the moving ACS reference period. For instance, knowing the month of data collection would permit data users to make their own adjustments for inflation for income amounts (see Section 4-D.3 below). It would also fa-

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

cilitate research on seasonal variations in population. If, upon investigation, it appears too risky to include the exact month of interview, then perhaps the value could be perturbed within a range of plus or minus a month (for example, a month of interview labeled as “March” might actually have occurred in February or April).


Multiyear Estimates The panel thinks that the continuous design of the ACS affords a measure of protection for respondents that the Census Bureau should take into account when considering appropriate confidentiality protections for multiyear estimates for small areas. The U.S. population is highly mobile with respect to geographic location, employment, family composition, commuting patterns, and other characteristics within and across years. Thus, the fact that 60 months of data are averaged to provide 5-year period estimates for block groups, census tracts, and small governmental jurisdictions should go a long way toward protecting individual respondents, even without additional steps to protect confidentiality. The Census Bureau, of course, will not, and should not, rely solely on averaging as a protection, but it should carefully consider the costs and benefits of each additional protection procedure and conduct research to identify the most useful protection techniques.

In this regard, the Census Bureau should consider developing selected tables with reasonably precise estimates for seasonal populations (for example, winter and summer residents) for geographic areas that experience seasonal population changes. Thought would need to be given to whether appropriate population controls can be developed for such tables or whether to use controls at all.

In addition, the Census Bureau should conduct research to determine an appropriate number of cases that need to be in the sample for a table or table cell to be released. To date, the Census Bureau appears to be using rule-based procedures for determining which tables must be deleted from publication in order to protect confidentiality. For example, the Census Bureau has developed rules for publication of worker and journey to work tabulations for traffic analysis zones and other geographic areas (Zayatz, 2005). Some of these rules appear to be reasonable, but others appear to lack a rationale.

One of these rules is that an area must have at least 10 unweighted or 60 weighted cases of workers in the sample over the year for 1-year workplace tables to be published. For 3-year and 5-year workplace tables, the corresponding minimums are, respectively, 30 unweighted or 180 weighted cases of workers in sample over the last 3 years and 50 unweighted or 300 weighted cases of workers in sample over the last 5 years. In other words, the average minimums, year by year, are the same—namely, 10 unweighted or 60 weighted cases. Assuming the 1-year period estimate minimums are

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

reasonable, then having the same average yearly minimums for the 3-year and 5-year period estimates makes sense: even though the 3-year and 5-year period estimates are published for smaller geographic areas than the 1-year period estimates, they represent averages over longer periods of time.

A second rule is that 5-year period estimates of mode of transportation to work cross-tabulated by another variable will not be published for an area for a particular mode unless it has at least 3 unweighted workers in the sample. If it does not, then the mode must be collapsed with other modes to reach the minimum sample size requirement. Such a restriction is not imposed on the 1-year or 3-year period estimates. Given the skewed distribution of mode of transportation in the United States, whereby three-fourths of the population drives alone to work, another 10 percent carpools, and very small percentages take public transit, bicycle, walk, or work at home, this restriction may curtail the publication of needed information on transportation to work in many areas. In turn, such curtailment will handicap users who want to aggregate data for traffic analysis zones into larger areas of their own definition.

The reason for the restriction for 5-year period estimates is not clear. Mode of transportation to work is highly variable: the same individual may decide to walk to work in the summer and drive in the winter or may walk to work for 4 years and then decide to switch to a new bus line or vice versa. Collectively, the workers in a traffic analysis zone are unlikely to include the same individuals over the 5-year period because of changes in residence and employment.

The Census Bureau has time before 5-year period estimates become available in which to develop appropriate confidentiality protection strategies and techniques for transportation tables and other data products. Such strategies should seek to minimize disclosure risk in ways that recognize the protection afforded by averaging over 60 months of data. When developing confidentiality protection procedures for cross-tabulations, the Census Bureau should also, whenever possible, prefer procedures that make it possible to aggregate the data for smaller units into larger units. Thus, instead of suppressing cells of a cross-tabulation, it might be possible to use techniques that perturb the data for individual cells while preserving the marginal totals for each variable.

4-D.1.c
Confidentiality Protection Recommendations

Recommendation 4-8: Because of the potential value of month of data collection for analysis of the ACS PUMS, the Census Bureau should revisit its decision to omit this variable as a confidentiality protection measure. If further research determines that including exact month of data collection would significantly increase disclosure risk, the Census

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

Bureau might consider perturbing the month of data collection or taking other steps to protect confidentiality. Similarly, the Census Bureau should consider developing selected summary tables that identify the season of collection (such as winter and summer) for geographic areas for which such information would be useful.


Recommendation 4-9: The Census Bureau should undertake research to develop confidentiality protection rules and procedures for tabulations from the ACS that recognize the protection afforded to respondents by pooling the data over many months. Whenever possible, the Census Bureau should prefer confidentiality protection procedures that preserve the ability to aggregate smaller geographic areas into larger, user-defined areas.

4-D.2
Collapsing Tables for Large Sampling Errors

In addition to procedures to protect confidentiality, the Census Bureau applies collapsing (or suppression) rules to the ACS 1-year and 3-year period standard tabulations that are designed to reduce the dimensions of tables, or to eliminate whole tables, that do not meet minimum standards for precision of the estimates. These collapsing rules are not applied to the 5-year period tabulations, even though the estimates will be very imprecise for small areas, because the small areas are intended to be building blocks for larger, user-defined areas.

The rules for determining which tables, or categories of tables, need to be suppressed involve examining the standard errors of every cell of a tabulation for individual tabulation areas (U.S. Census Bureau, 2006:13-10 to 13-11). For a specified table and area, the coefficient of variation (CV, the standard error of an estimate as a percentage of the estimate—see Box 2-5) is calculated for each cell of the table. If the cell entry is zero, the CV is set to 100 percent. The CV values are arrayed from high to low, and if the median CV value—the value that divides the distribution into equal halves—is greater than 61 percent, then the full table cannot be released. The categories of the table are then combined into fewer categories, and the median CV for the new table is calculated anew and the test is reapplied. If the median CV is still greater than 61 percent, then even the simpler table cannot be released (see Box 4-4 for an example).

It is difficult to evaluate this rule, but it could to lead to anomalous situations that make the data harder to use. For example, a table could be completely or partially suppressed one year and not the next year for the same geographic area, or a table could be suppressed for some, but not all, of the component areas of a large city or county. The suppression will affect small areas and minority population groups disproportionately.

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

BOX 4-4

Illustrative Calculation for Suppressing Table Cells with Large Sampling Error, 1-Year ACS Period Estimates

Assume a city of population 100,000, with 2,000 school-age children in a particular population group (e.g., Hispanic).

First Pass of Table

Ratio of Family Income to Poverty Threshold

Percent Chldren in Category

Coefficient of Variation (CV)

Below poverty threshold

15.0

60.4

100–149 percent of poverty

10.0

76.1

150–199 percent of poverty

10.0

76.1

200–249 percent of poverty

10.0

76.1

250–299 percent of poverty

10.0

76.1

300–349 percent of poverty

20.0

50.7

350 percent or more of poverty

25.0

43.9

What is the result?

  • Median CV is 76.1.

  • The table may not be released because the median CV is greater than 61.0.

Second Pass of Table after Combining Categories

Ratio of Family Income to Poverty Threshold

Percent Chldren in Category

Coefficient of Variation (CV)

Below poverty threshold

15.0

60.4

100–199 percent of poverty

20.0

50.7

200–299 percent of poverty

20.0

50.7

300–349 percent of poverty

20.0

50.7

350 percent or more of poverty

25.0

43.9

What is the result?

  • Median CV is 50.7.

  • The table may be released because the median CV is less than 61.0.

Recommendation 4-10: The Census Bureau should monitor the extent of collapsing of cells that is performed in different tables to meet minimum precision standards of 1-year and 3-year period tabulations from the ACS and assess the implications for comparisons among geographic areas and over time. After sufficient information has been gleaned about the extent of data collapsing, the Census Bureau, in consultation with data users, should assess whether its collapsing rules are sound or should be modified for one or more subject areas.

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
4-D.3
Inflation Adjustments

Chapter 3 discussed the procedures used by the Census Bureau to adjust income amounts for the 1-year, 3-year, and 5-year period estimates and housing value and cost amounts for the 3-year and 5-year period estimates to reflect changes in the national all-item consumer price index (CPI) over the period (see Section 3-A.2.c and Table 3-1). The discussion underlined the importance of users understanding the resulting estimates—for example, a 5-year period estimate of income or housing value is the average of all of the reported amounts over the 5 years expressed in dollars for the latest year using a national CPI adjustment. Moreover, as with any period estimate, the same inflation-adjusted average dollar amount for two areas may reflect different underlying patterns—for example, average income for 2005–2009 expressed in 2009 dollars of, say, $40,000 could result from income growth, stability, or decline over the 5-year period.

For many applications, users may prefer the Census Bureau’s adjustment to latest-year dollars by using the national CPI to some other inflation adjustment or to no inflation adjustment at all. One advantage is that 1-year, 3-year, and 5-year period estimates for a large city or county will all be expressed in dollars for the same (latest) year—for example, 2009 dollars in the case of estimates for 2009, 2007–2009, and 2005–2009.

For some applications, however, users might prefer an inflation adjustment that is specific by geographic area. The problem is that area price data are limited. Currently, the Bureau of Labor Statistics (BLS) collects price data for over 100 specific areas, but it publishes price indexes for only the four regions (Northeast, Midwest, South, and West), population size classes of metropolitan statistical areas (MSAs), and the 20 largest MSAs. No price data are collected for rural areas.10 Moreover, variation in price changes may be as great within areas for which price indexes are available as among them—for example, prices for housing and other goods may increase at a very different rate in the central city and suburbs, let alone individual neighborhoods, of an MSA. Finally, area-specific price indexes are less precise than the national all-item CPI.

For still other applications, users may require latest-year estimates for income, housing costs, or housing value. Averages of reported amounts over 3 or 5 years adjusted for inflation to the latest year are not likely to be the same as latest-year amounts. For income, this is true even for the 1-year period estimates: inflation-adjusted averages of reported income over the 23 months covered in 1-year period estimates are not likely to be the same as latest-year income estimates.

For estimating latest-year housing amounts from multiyear averages,

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

the problem is a lack of subnational price indexes for specific items, such as housing value, rent, and different utilities or other housing costs. For income amounts, the problem is that incomes are not prices: income (in total and by component, such as wages or pension income) may increase faster (or slower) than inflation. A possibility to investigate in this context is to use estimation methods that are appropriate by type of income. For Social Security and Supplemental Security Income, it would be appropriate to use the applicable national CPI to which these payments are indexed by law. For property and self-employment income, it might be more appropriate to use an average interest rate, whereas, arguably, some types of income—specifically, public assistance and other retirement income—should not be inflated at all unless it is known that a jurisdiction has increased such payments. For wages, it could be possible to use changes at the county level in average quarterly wages for employees covered under state and federal unemployment insurance programs. These data, which are part of the BLS Quarterly Census of Employment and Wages, are released each quarter about 6–7 months after data collection.

It might be possible to develop simpler models to estimate latest-year amounts by using the published multiyear estimates. For example, by examining how well the trends in BLS county wage data estimate 1-year period income from the 5-year period estimates for large counties, a user might be able to develop a procedure for estimating latest-year incomes from the 5-year period estimates for small counties.

To determine how to produce the most helpful data on income, housing costs, and housing value, the Census Bureau should initiate a two-part discussion with users. The Census Bureau should first clearly illustrate to users the nature of the current inflation adjustment procedures. Then it should ascertain users’ needs for income and housing amount information, the resultant implications for what adjustment procedures can best serve most users, and what steps to take to assist users whose needs are not satisfied by the standard procedures. Finally, the Census Bureau should consider providing tables that reflect unadjusted dollar amounts whenever it provides adjusted amounts. So doing will make clearer to users the effects of inflation and enable them to determine if another kind of adjustment would better suit their purposes.


Recommendation 4-11: The Census Bureau should provide users with a full explanation of its inflation adjustment procedures and their effects on multiyear ACS estimates of income, housing costs, and housing value. It should consult with users about other kinds of income and housing amount adjustments they may need and conduct research on appropriate estimation methods (for example, methods to produce latest-year amounts from multiyear averages). It should consider pub-

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

lishing selected multiyear averages in nominal dollars as well as inflation-adjusted dollars.

4-D.4
Tabulation Specifications

The long-standing release plan for tabulations from the ACS includes two major elements: (1) the universe or population covered and (2) the geographic areas for which tabulations are produced. The full universe for ACS data products, beginning in 2006, will include the housing unit and GQ populations, although some tables may be published for subuniverses, such as households or the noninstitutional population. (Prior to 2006, tabulations included just the housing unit population.) For geographic areas, the available products (1-year, 3-year, and 5-year period estimates) will depend on the population size of the geographic area (refer back to Tables 2-4 and 2-5).

The Census Bureau will need to follow its plan for a number of years, not only to allow time for collection of sufficient data to begin release of 3-year period estimates in 2008 and 5-year period estimates in 2010, but also to allow both the Census Bureau and the data user community sufficient opportunity to gain experience with the various sets of tabulations. Yet the Census Bureau should not neglect to consult with users to determine if the population universe and the geographic area specifications are optimal or might be modified to produce more useful information.

With regard to population coverage, the key question is the role of GQ residents, particularly those in institutions. The Census Bureau will need to consult with users regarding appropriate universe definitions for ACS tabulations—for example, employment and income tabulations may be most useful if they are restricted to the noninstitutional population. In 2000, confidentiality concerns sometimes precluded the publication of the same tabulations separately for households and GQ residents in very small areas. Because the ACS estimates for small areas are averages over multiyear periods, confidentiality concerns could be less of a problem in this regard. Ideally, consultation with users on the most useful tabulation universes would precede and feed into the production of tables for 2006 (for release in summer 2007), which will be the first year to include GQ residents.

For the geographic area release schedule, one issue is the population size cutoff for publication of 1-year period estimates, for which the Census Bureau might consider the usefulness of lowering the current threshold of 65,000 residents to one of, say, 50,000 residents. The discussions in Chapters 2 and 3 emphasize the large sampling errors of 1-year period estimates for a small population group (such as school-age children in poverty) for geographic areas with fewer than 250,000 people, so lowering the threshold might appear to be deleterious. However, estimates for major population

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×

groups will often meet common standards of precision for areas of 50,000 population (see Table 2-8). Moreover, 50,000 is a common threshold for allocation of various types of federal assistance. Yet another advantage of lowering the threshold to provide 1-year period estimates for additional areas is that users would have more flexibility in combining the data and, consequently, would less often have to request special tabulations from the Census Bureau. For example, users could average two 1-year period estimates for a small city or county to obtain a 2-year period estimate that was more precise than the individual 1-year period estimates.

A second issue for release of geographic area tabulations concerns the feasibility of producing 3-year period estimates for user-defined statistical subareas of large cities and counties. Such subareas could be a set of aggregations of census tracts or block groups in cities and of places and towns in counties, where the city or county has at least 40,000 people (so that, at a minimum, there are two subareas, each with at least 20,000 people). If the city or county is large enough to have more than one PUMA, then the subareas could usefully nest within a PUMA to maximize the ability to relate the data for the PUMA and its subareas. (PUMA boundaries may need to be redrawn in some areas to achieve the most useful designation of subareas within PUMAs.) Finally, it may be possible to produce 1-year period estimates for large statistical subareas of PUMAs, particularly if the threshold for 1-year period estimates is lowered to 50,000 people. The Census Bureau will need to explore with users the desirability of providing additional estimates for statistical subareas of large cities and counties and weigh user needs against the feasibility of increasing the production workload for the ACS.


Recommendation 4-12: If some or all GQ residents continue to be included in the ACS, the Census Bureau should consult with users regarding the most useful population universe for tabulations, which, depending on the table, could be the entire population, the household and GQ populations separately, or the noninstitutional and institutional populations separately.


Recommendation 4-13: The Census Bureau should consider expanding the geographic areas for ACS tabulations in order to afford users greater flexibility for aggregating small areas into larger user-defined areas. Two possibilities to investigate are to lower the population threshold for 1-year period estimates to, say, 50,000, and to produce 3-year (and possibly 1-year) period estimates for user-defined statistical subareas of large cities (aggregations of census tracts or block groups) and counties (aggregations of places and towns).

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
4-D.5
Data Quality Review

The final step in the production and release of tabulations and other ACS data products is review by subject matter analysts to be sure there are no obvious errors or anomalies in the data. Each year the ACS processing staff and subject matter analysts must complete the entire process of preparing and reviewing data products within the span of a few months. In contrast, the preparation and review of data products from the long-form sample typically required well over a year to complete.

The volume of estimates to be reviewed each year led the Census Bureau to develop automated tools to facilitate the work of the staff. One tool, ART II, was developed in 2005 as an improved version of a similar tool (ART) used in 2003–2004. This tool automates the process of identifying statistically significant differences in estimates from one year to the next and facilitates other aspects of the review process. Other tools enable analysts and managers to track the process of review for tabulations and PUMS (U.S. Census Bureau, 2006:13-11).

We support continued efforts by the Census Bureau to automate and standardize the review process for ACS products, including not only the final review, but also review at earlier stages, such as when imputations for missing data and weighting adjustments are applied to the data records. As the time approaches when 1-year, 3-year, and 5-year period estimates must be provided for thousands of geographic areas every year (including 5-year estimates for over 200,000 individual block groups), the immensity of the review task threatens to overwhelm the analyst staff. They will run the risk of inadvertently releasing poor-quality data unless they receive a high level of technical assistance.

The Census Bureau recently identified prerelease review of demographic data, including from the ACS and other household surveys, as an important problem that merits research (Bell, 2006:10). The panel urges the Census Bureau to not only continue, but also to step up its investment of resources for automated tools, standardized protocols, and other means to facilitate an appropriate level of review of ACS data products that will ensure a high standard of quality before they are released each year. Consulting with computer software development firms and with computer scientists in academia may generate useful ideas and identify existing automated tools that are relevant to the Census Bureau’s needs (see National Research Council, 2003b).


Recommendation 4-14: The Census Bureau should increase its research and development on automated tools and standardized procedures to facilitate timely review and quality control of the large volume of ACS data products.

Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 139
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 140
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 141
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 142
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 143
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 144
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 145
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 146
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 147
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 148
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 149
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 150
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 151
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 152
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 153
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 154
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 155
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 156
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 157
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 158
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 159
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 160
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 161
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 162
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 163
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 164
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 165
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 166
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 167
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 168
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 169
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 170
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 171
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 172
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 173
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 174
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 175
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 176
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 177
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 178
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 179
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 180
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 181
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 182
Suggested Citation:"PART II: Technical Issues, 4 Sample Design and Survey Operations." National Research Council. 2007. Using the American Community Survey: Benefits and Challenges. Washington, DC: The National Academies Press. doi: 10.17226/11901.
×
Page 183
Next: 5 The Weighting of ACS 1-Year Period Estimates »
Using the American Community Survey: Benefits and Challenges Get This Book
×
Buy Paperback | $80.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The American Community Survey (ACS) is a major new initiative from the U.S. Census Bureau designed to provide continuously updated information on the numbers and characteristics of the nation’s people and housing. It replaces the “long form” of the decennial census. Using the American Community Survey covers the basics of how the ACS design and operations differ from the long-form sample; using the ACS for such applications as formula allocation of federal and state funds, transportation planning, and public information; and challenges in working with ACS estimates that cover periods of 12, 36, or 60 months depending on the population size of an area.

This book also recommends priority areas for continued research and development by the U.S. Census Bureau to guide the evolution of the ACS, and provides detailed, comprehensive analysis and guidance for users in federal, state, and local government agencies, academia, and media.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!