Cover Image

PAPERBACK
$44.00



View/Hide Left Panel

1

Introduction

In the early 1990s, the Census Bureau proposed a program of continuous measurement as a possible alternative to the gathering of detailed social, economic, and housing data from a sample of the U.S. population as part of the decennial census. The idea was based on earlier visions of nonoverlapping monthly samples that could be cumulated over different time periods to provide data for geographic areas of different sizes or for different subsets of the population (Kish, 1981). When the Census Bureau began to actively pursue the possibility of implementing a continuous measurement approach, the goal was to make the same data that were collected on the census “long form” available on a more timely basis than what was possible through a decennial data collection, at the same time reducing the burden imposed on the census enumeration by the fielding of the long form.

Pilot testing of the new survey began a few years later, and the full-fledged American Community Survey (ACS) became a reality in 2005, with nationwide implementation at the household level. Group quarters (GQ) facilities have been included in the sample since 2006. The replacement of the long-form sample with the new, ongoing survey—and the consequent casting of the decennial census as “short-form only”—became a key part of the Census Bureau’s strategy for the 2010 census.

The design of the ACS relies on monthly samples that are cumulated to produce sufficient data to enable the release of estimates for increasingly smaller geographic areas over multiyear rolling intervals. As such, the ACS data products are period estimates, as opposed to point-in-time estimates. In other words, they are based on aggregating and averaging data collected over a



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 11
1 Introduction In the early 1990s, the Census Bureau proposed a program of continu- ous measurement as a possible alternative to the gathering of detailed social, economic, and housing data from a sample of the U.S. population as part of the decennial census. The idea was based on earlier visions of nonoverlapping monthly samples that could be cumulated over different time periods to pro - vide data for geographic areas of different sizes or for different subsets of the population (Kish, 1981). When the Census Bureau began to actively pursue the possibility of implementing a continuous measurement approach, the goal was to make the same data that were collected on the census “long form” available on a more timely basis than what was possible through a decennial data collec - tion, at the same time reducing the burden imposed on the census enumeration by the fielding of the long form. Pilot testing of the new survey began a few years later, and the full-fledged American Community Survey (ACS) became a reality in 2005, with nationwide implementation at the household level. Group quarters (GQ) facilities have been included in the sample since 2006. The replacement of the long-form sample with the new, ongoing survey—and the consequent casting of the decen- nial census as “short-form only”—became a key part of the Census Bureau’s strategy for the 2010 census. The design of the ACS relies on monthly samples that are cumulated to produce sufficient data to enable the release of estimates for increasingly smaller geographic areas over multiyear rolling intervals. As such, the ACS data products are period estimates, as opposed to point-in-time estimates. In other words, they are based on aggregating and averaging data collected over a 11 11

OCR for page 11
12 SMALL POPULATIONS, LARGE EFFECTS period of time, instead of providing a snapshot as of a particular point in time, as the census long form did for the April 1 reference date, once every 10 years. Table 1-1 shows the initial ACS sample sizes and number of completed interviews between 2006 and 2010 (U.S. Census Bureau, 2011a). These sample sizes cumulate to approximately 15.4 million cases (10.4 million completed interviews) over the 5-year period, including the GQ residents. In an ideal version of the ACS design, data cumulated over 5 years would have comparable estimation reliability to that achieved by recent census long- form samples, even in small areas. Although long-form sample sizes have varied in recent censuses, the last time the long form was administered (as part of the 2000 census), the sample included approximately 18 million housing units, and the data collection resulted in 16.4 million completed questionnaires (National Research Council, 2007). During the first few years of the ACS, the survey was not funded at a level necessary for a comparable sample size, but beginning in June 2011 the target was increased to 3.54 million sampled addresses annually, which, if continued at that level, will bring the ACS housing unit sample size closer to the census 2000 level over 5-year intervals in the future (U.S. Census Bureau, 2011b). Table 1-2 shows the ACS data release schedule from the survey’s inception through 2013, along with the population thresholds required for each release. Beginning in 2006, the Census Bureau published annual 1-year estimates of characteristics of the U.S. population and housing units for all geographic entities with populations of at least 65,000. Since 2008, 3-year estimates for geographic entities with populations of at least 20,000 have also been reported. The end of 2010 marked a crucial milestone for the ACS, when the first set of estimates based on 5 years of continuous data collection were published for all statistical, legal, and administrative entities, including areas as small as census block groups. TABLE 1-1 Initial ACS Sample Sizes and Completed Interviews, 2006-2010 Housing Units GQ Residents Initial Initial Addresses Final Sample Final Year Selected Interviews Selected Interviews 2010 2,899,676 1,917,799 197,045 144,948 2009 2,897,256 1,917,748 198,808 146,716 2008 2,894,711 1,931,955 186,862 145,974 2007 2,886,453 1,937,659 187,012 142,468 2006 2,885,384 1,968,362 189,641 145,311 NOTE: Only a subsample of the housing units that do not respond by either mail or telephone are included in the in-person follow-up, which is the final stage of the ACS nonresponse follow-up effort. More information about response rates is available on the Census Bureau website (http:// www.census.gov/acs/www/methodology/response_rates_data/). SOURCE: U.S. Census Bureau (2011a).

OCR for page 11
13 INTRODUCTION TABLE 1-2 ACS Data Release Schedule, 2006-2013 Year of Data Release Data Population Product Threshold 2006 2007 2008 2009 2010 2011 2012 2013 Year(s) of data collection 1-year 65,000+ 2005 2006 2007 2008 2009 2010 2011 2012 estimates 3-year 20,000+ 2005- 2006- 2007- 2008- 2009- 2010- estimates 2007 2008 2009 2010 2011 2012 5-year All areas 2005- 2006- 2007- 2008- estimates 2009 2010 2011 2012 NOTE: Group quarters have been included in the ACS sample since 2006. SOURCE: U.S. Census Bureau (2011c). The GQ data collection has always been more challenging than the housing unit data collection, even in the decennial census, and some of these challenges are the natural consequence of the more complex living arrangements associ - ated with residence in GQ facilities. The replacement of the census long-form sample with the ACS promises data users major benefits, but it also presents new challenges. In terms of the benefits, the critical advantages of the ongoing, continuous ACS are the timeliness of the estimates and the increased frequency of data releases. The continuous ACS data collection also has some advantages in terms of data quality. Whereas the decennial census relies heavily on a vast temporary workforce that must be hired, trained, and deployed quickly, the continuous nature of the ACS can accommodate a staff of well-trained, per- manent field representatives. This, in turn, may contribute to reducing various kinds of nonsampling errors, including item nonresponse rates and proxy responses compared with data from the census long-form approach. However, the ACS has some offsetting disadvantages. Chief among these are the larger sampling errors associated with the estimates. Higher levels of estimate uncertainty are a consequence of the smaller overall sample size (com - pared, for example, with the 2000 census long-form sample), even cumulating over 5 years, and the fact that only a sample of nonresponding housing units is included in the follow-up stages of data collection. In addition, large numbers of GQ facilities included in the GQ sample are found to be ineligible or eligible but unoccupied during the data collection. Furthermore, the ACS uses control totals based on postcensal population estimates from the Population Estimates Program (PEP)—instead of the census itself—to reduce variation in the ACS estimates, which means that controls are not available for geographic areas as small as from the census, and that any errors associated with the population estimates will also affect the ACS estimates. Although these comments refer to the ACS as a national survey and there - fore apply primarily to the sample of housing units and estimates of the popu -

OCR for page 11
14 SMALL POPULATIONS, LARGE EFFECTS lation living in households, they apply to the GQ part of the ACS operation as well. However, several aspects of sampling GQ facilities and estimating the numbers and characteristics of individuals living in them are uniquely prob - lematic to this segment of the population. This more narrow set of issues is the focus of this report. ISSUES FOR THE PANEL When the ACS entered full-scale production in 2005, it did so only for the household population. One year later, in 2006, the Census Bureau was also able to include what it refers to as group quarters—such places as correctional facilities for adults, student housing, nursing facilities, inpatient hospice facili - ties, and military barracks—primarily to more closely replicate the design and data products of the census long-form sample. The decision to include group quarters in the ACS enables the Census Bureau to provide a comprehensive benchmark on the total U.S. population (not just those living in households), but it also brings about additional challenges and data quality implications. Box 1-1 provides the definition of group quarters used by the Census Bureau for purposes of the ACS and lists the major types of group quarters included in the survey (additional details are provided in Box 2-1). The GQ population was 2.6 percent of the total U.S. population at the time of the 2010 census. Although this represents only a small fraction of the total population, its unique characteristics present major challenges for the ACS. In addition to the operational hurdles associated with collecting data from non- household populations, there are statistical challenges as well, partly because group quarters are unevenly distributed across the country and their residents are often systematically different from the household population in the communities in which they are located. Some jurisdictions have no group quarters at all, and others may have a large prison facility, military barracks, student housing, or a mix of different GQ types. For national and state-level population estimates, this is not a particularly large concern. However, the goal of the ACS is to provide data for geographic areas as small as census tracts and block groups as well as for sparsely popu - lated villages and towns across rural America. In such small areas, the accuracy and precision of population estimates will be affected by data limitations. These limitations necessarily affect estimates pertaining to the GQ popula - tions. However, as a secondary consequence, errors in the GQ estimates can often profoundly affect the estimates and population characteristics of the total population as well. For a variety of reasons, which are described in detail in subsequent sections, the design of the ACS and the data collection, weighting, and estimation procedures pertaining to GQ residents are not optimized for small-area estimates. As a result, a thorough evaluation of the implications of these design issues for small-area estimates is essential.

OCR for page 11
15 INTRODUCTION BOX 1-1 ACS Definition and Major Types of Group Quarters Definition A group quarters is a place where people live or stay, in a group living arrangement, that is owned or managed by an entity or organization providing housing and/or services for the residents. This is not a typical household-type living arrangement. These services may include custodial or medical care as well as other types of as- sistance, and residency is commonly restricted to those receiving these services. People living in group quarters are usually not related to each other. Group quar- ters include such places as college residence halls, residential treatment centers, skilled nursing facilities, group homes, military barracks, correctional facilities, and workers’ dormitories. Major Types of Group Quarters 1. Correctional facilities for adults 2. Juvenile facilities 3. Nursing facilities and skilled nursing facilities 4. Other institutional facilities 5. College or university student housing 6. Military group quarters 7. Other noninstitutional facilities SOURCE: U.S. Census Bureau (2008a). As illustrated in Table 1-1, in any given year, the number of completed GQ interviews is less than 150,000 nationally. The annual sampling rate for the GQ population varies by state, but in most states it is approximately 2.5 percent of the expected number of GQ residents. Currently, a stratified sample of group quarters is selected for each state, without controlling for the allocation of GQ populations at substate levels of geography, such as counties, municipalities, tracts, and block groups (unlike the 2000 census long-form sample, which was generally controlled to census counts at subcounty levels of geography). Consequently, while the household sample is suitable for producing estimates of characteristics of people resid - ing in households for substate geographies, the measurement and estimation approaches developed for the GQ population are designed to be optimal only for estimates at the state level and higher levels of geography. A serious challenge posed by the inclusion of group quarters in the ACS is “sampling zeroes”—small geographic areas that have no group quarters represented in the sample, even after a 5-year period of data collection, despite

OCR for page 11
16 SMALL POPULATIONS, LARGE EFFECTS the presence of GQ facilities in the sampling frame and nonzero GQ popula - tion counts revealed in the most recent decennial census for those areas. In substate areas, this can adversely affect the estimates of characteristics for the total population, and it can elevate estimated standard errors for characteristics of the total population. To the extent that group quarters are skipped over in the sample selection for some small jurisdictions or census tracts or block groups, the sampling weights of group quarters in other jurisdictions may be disproportionately increased. Other challenges include the complexities and costs associated with main - taining an accurate and up-to-date inventory of GQ facilities, independent of the inventory of household addresses. This is especially difficult in the case of smaller group quarters, which open and close at rates faster than larger facili - ties, and group quarters in structures that may have been recently converted from housing units. This affects not only the efficiency of the GQ sampling frame but also the GQ estimates produced by the Census Bureau’s PEP, which are used as controls in the ACS. Based on the factors described above, the panel concludes that the main data quality concern is not necessarily the estimates of GQ characteristics, but rather the effects that the GQ sample has on the estimates of total population characteristics, especially in smaller areas. Although largely unanticipated by the panel early on, this conclusion solidified as it became evident over the course of the study that, for the GQ population, very limited data would be made available below the state level, even based on the 5-year data release. As part of the panel’s research it also became clear that the presence of group quarters can play an important role in many smaller places, where the quality of the GQ estimates often means the difference between an accurate statistical portrait of the area and one that is substantially distorted. PANEL CHARGE The Census Bureau asked the Committee on National Statistics of the National Academies to convene a panel to evaluate the ACS methodology for measuring the GQ population, taking into consideration data user needs (for the exact wording of the panel’s charge, see Box 1-2). The panel was asked to recommend alternatives to the current study design, with the primary goal of making the ACS data more useful for small-area data users. The panel was not asked to conduct a cost-benefit analysis of the inclusion of the GQ population in the ACS, although the panel conducted its deliberations being mindful of the costs associated with the GQ data collection, as well as of the costs of pos - sible alternatives that would involve a major reconceptualization of the survey’s design. In response to this charge, the panel appointed by the National Research Council undertook a range of activities over the course of approximately 2

OCR for page 11
17 INTRODUCTION BOX 1-2 Statement of Task An ad hoc panel will conduct an in-depth review of the statistical methodology for measuring the group quarters (GQ) population in the continuous American Com- munity Survey (ACS). The panel will consider user needs for ACS data on the various components of the GQ population, including inmates of federal, state, and local correctional facilities, residents of nursing homes and other long-term health care facilities, college students living in campus housing, military personnel in bar- racks or on a ship in home port, and residents of noninstitutional group quarters, such as hospices, convents, monasteries, group homes, and migrant workers quarters. In light of user needs and considerations of operational feasibility and compatibility with the treatment of the household population in the ACS, the panel will recommend alternatives to the current sample design, weighting procedures, and other methodological features that can make the ACS GQ data more useful for small-area data users, particularly users of ACS 5-year period estimates for small governmental jurisdictions, census tracts, and block groups. The panel will issue an interim report at the end of the first year of the study with recommenda- tions for near-term improvements in the sample design and weighting of group quarters in the ACS and a final report at the conclusion of a 24-month study with findings and recommendations for longer term improvements to the measurement of the GQ population. years. The panel met with staff from the Census Bureau’s American Commu- nity Survey Office on several occasions to learn about the design of the ACS, the GQ data collection methodology, the challenges experienced, and plans for the future. The panel also consulted with staff from other Census Bureau offices whose work has implications for the ACS. This included such units as the Decennial Census Division, the Population Estimates Program, and the Geography Division. To evaluate data user needs, the panel held a workshop with users of the ACS data on December 13, 2010, in Washington, DC (for a list of participants, see Appendix A). The goal of the meeting was to gain a thorough understand- ing of how the GQ data are used and what the data user needs are and to discuss enhancement and alternatives to the current ACS design. In an effort to reach as many potential stakeholders as possible, panel members also discussed the study at several meetings and conferences attended by data users interested in census and ACS data. These included meetings of the Association of Public Data Users, the Council of Professional Associations on Federal Statistics, and the Population Association of America. The panel’s efforts to better understand data user needs were also assisted by consultants engaged by the panel to review the role of GQ data in the distribution of federal funds as well as the use of GQ

OCR for page 11
18 SMALL POPULATIONS, LARGE EFFECTS data in programs primarily focused at the state and local levels. The panel also researched the availability of GQ data from sources other than the decennial census and the ACS. Panel members discussed data collection strategies with other researchers, including staff from the Bureau of Justice Statistics and the National Center for Health Statistics. OVERVIEW OF THE REPORT At the request of the Census Bureau, the panel prepared an interim report, which focused on recommendations for near-term improvements in the sample design, weighting, and estimation of the GQ population (National Research Council, 2010). This final report incorporates the findings and recommenda - tions from the interim report and discusses them in the broader context of long-term goals for the ACS, with special emphasis on data user needs. Following this introduction, Chapter 2 describes the measurement of the GQ population in the ACS. Chapters 3 through 6 contain the panel’s recom- mendations related to data user needs and to different aspects of the ACS GQ methodology. Chapter 3 discusses data user needs. Chapter 4 focuses on the challenges related to developing and maintaining a sampling frame of GQ facilities and makes recommendations for increasing the efficiency of the updating process and sample design. Chapter 5 describes the sample allocation and selection process and offers suggestions for refining these aspects of the survey design. Chapter 6 discusses possible improvements and alternatives to the weighting and estimation procedures.