3
Working with the ACS: Guidance for Users

The American Community Survey (ACS) can benefit decision makers, planners, and analysts in virtually every type of setting, including federal executive and legislative agencies, state and local government agencies, metropolitan planning organizations, nonprofit organizations, professional associations, universities, think tanks, and private businesses in many sectors. The ACS will also be invaluable to educators, students, the media, and the public.

This chapter addresses how users can work with the various ACS products that are planned to become available and the factors to consider when deciding which products to use for particular purposes. Because not every potential application can be included (or indeed foreseen), the chapter highlights key applications for federal, state, and local government agencies, transportation planners, researchers, the media, and the public who currently use long-form-sample data. The specific users and applications that are discussed include:

  • Federal agency users (Section 3-A). Highlighted applications include the use of ACS 1-year, 3-year, and 5-year period estimates for fund allocation to states and localities (3-A.1) and to update the U.S. Department of Housing and Urban Development’s income limits for housing assistance programs (3-A.2).

  • State agency users (Section 3-B). A highlighted application is the use of ACS 5-year period estimates for fund allocation and grants to localities.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 77
Using the American Community Survey: Benefits and Challenges 3 Working with the ACS: Guidance for Users The American Community Survey (ACS) can benefit decision makers, planners, and analysts in virtually every type of setting, including federal executive and legislative agencies, state and local government agencies, metropolitan planning organizations, nonprofit organizations, professional associations, universities, think tanks, and private businesses in many sectors. The ACS will also be invaluable to educators, students, the media, and the public. This chapter addresses how users can work with the various ACS products that are planned to become available and the factors to consider when deciding which products to use for particular purposes. Because not every potential application can be included (or indeed foreseen), the chapter highlights key applications for federal, state, and local government agencies, transportation planners, researchers, the media, and the public who currently use long-form-sample data. The specific users and applications that are discussed include: Federal agency users (Section 3-A). Highlighted applications include the use of ACS 1-year, 3-year, and 5-year period estimates for fund allocation to states and localities (3-A.1) and to update the U.S. Department of Housing and Urban Development’s income limits for housing assistance programs (3-A.2). State agency users (Section 3-B). A highlighted application is the use of ACS 5-year period estimates for fund allocation and grants to localities.

OCR for page 77
Using the American Community Survey: Benefits and Challenges Local government users (Section 3-C). Uses of different ACS estimates are discussed separately for big cities (3-C.1) and small, oversampled jurisdictions (3-C.2). Also discussed is the special case of jurisdictions with large seasonal populations (3-C.3). Transportation planners (Section 3-D). Their applications will rely heavily on the ACS 5-year period estimates and also the public use microdata sample (PUMS) files. Academic and other researchers (Section 3-E). Researchers will make heavy use of the PUMS files and of an ACS summary file, similar to Summary File 3 from the 2000 long-form sample that is currently under development. Media outlets and the public (Section 3-F). These groups will likely make the most use of the ACS 1-year period summary estimates provided in profiles, ranking tables, and change tables. Whatever their category (federal agency, local government, other), users should review other sections in addition to the one addressed to them. Many of the specific applications discussed—each of which illustrates some but not all issues regarding use of ACS data products to replace long-formsample data products—pertain to more than one category of user. Section 3-G discusses an issue that affects all users—namely, the fact that new population and housing numbers from the decennial census every 10 years will likely interrupt the time series of ACS estimates. The reason is that the ACS estimates for calendar years ending in 0 through 9 each decade will be calibrated at the level of an individual county (or a group of small counties) to annual population estimates updated from the previous census by records of births and deaths and estimates of net migration. A similar calibration will be made to housing unit totals updated from the previous census. When a new census is taken, the census counts will not necessarily coincide with the updated estimates, thereby producing discontinuities in the ACS time series. The chapter concludes (Section 3-H) by summarizing the panel’s general guidelines for effective use of the ACS and suggesting ways in which users who expect to work extensively with the ACS small-area data can prepare during the ramp-up period from 2006 to 2010, as the first sets of 1-year, 3-year, and 5-year period estimates become available. Many users are rightly concerned, first and foremost, with how well the ACS can serve as a replacement source of useful and usable estimates for planning, research, public education, and a host of other applications that currently rely on the long-form sample. The examples in this chapter serve principally to address this underlying concern about the functionality of ACS data to meet current needs. The decoupling of long-form-type data from the once-a-decade census,

OCR for page 77
Using the American Community Survey: Benefits and Challenges however, promises to allow the ACS to develop in ways that, while not clear today, will allow this new survey to become much more powerful than the long-form sample could ever be. We urge users to take a long view of the ACS and be open to new uses that were not possible with the long-form sample but that the continuously updated ACS data can support. The Census Bureau, for its part, needs to provide as much guidance and training as possible to users to help them maximize the upside and minimize the downside of working with this complex data set. As discussed in Chapter 7, the Census Bureau should proactively identify ways to assist the occasional user who will not be in a position to master the ins and outs of the ACS data—for example, by highlighting estimates that meet reasonable standards for precision. The Census Bureau should also support an ongoing education and outreach program for users who plan to work extensively with ACS data, including the staffs of state data centers and other groups whose mission is to assist the broad user community. As discussed in Chapter 4, the Census Bureau should consider the development of new data products that would help many users, such as 3-year period estimates for statistical areas that are larger than census tracts and smaller than public use microdata areas (PUMAs). 3-A FEDERAL AGENCY USES Federal government agencies have historically used data from the long-form sample for a wide range of purposes. For at least the past two censuses, the Census Bureau and the U.S. Office of Management and Budget (OMB) have required that each item on the census short and long forms be justified as serving a federal agency need. For the long-form sample, each item had to be needed for federal government use for small areas, often as small as census tracts. Uses were classified into three categories: (1) mandated—that is, the use of census data was specified in legislation; (2) required—that is, data were required by legislation and, although the census was not named as the source, it was the only or the historical source of data; and (3) programmatic—that is, the census data were used for agency program planning, implementation, or evaluation or to provide legal evidence. The same general criteria are being applied with the ACS, although congressional oversight committees have indicated that it is not mandatory to pass legislation in order to add a question to the ACS.1 It should be noted that where laws or regulations specify the use of census long-form-sample estimates, changes in legislation may be required to permit the use of ACS estimates instead. 1 Personal communication, Lynda T. Carlson, Director, Division of Science Resources Statistics, National Science Foundation.

OCR for page 77
Using the American Community Survey: Benefits and Challenges BOX 3-1 Selected Federal Agency Uses of Census Long-Form-Sample Data The U.S. Department of Justice uses the long-form-sample data on race, Hispanic origin, educational attainment, language spoken at home, how well English is spoken, and citizenship for census tracts and American Indian areas to implement sections of the Voting Rights Act that deal with bilingual voting assistance. The U.S. Equal Employment Opportunity Commission uses the data on occupation, industry, and demographic characteristics for ZIP codes and other geographic areas to analyze statistical evidence in class action charges of employment discrimination. The OMB Statistical and Science Policy Office uses the data on place of work in relation to place of residence, together with population size and density, for counties and places to define metropolitan and micropolitan statistical areas. These areas have many public- and private-sector applications, including use in determining eligibility for some types of federal funding. The U.S. Department of Health and Human Services uses the data on older people, such as marital status, educational attainment, ancestry, disability status, income, year last worked, and housing characteristics, for counties, cities, and census tracts to measure social isolation and housing needs under the Older Americans Act. The U.S. Department of Transportation uses the data on disability, means of transportation to work, and automobile ownership for traffic analysis zones (small areas made up of one or more block groups) to monitor compliance with the Federal Transit Act and the Americans with Disabilities Act. Types of federal agency uses of the 2000 long-form-sample data and, prospectively, of the ACS data vary widely (Citro, 2000a; National Research Council, 2004b:Ch. 2; National Research Council, 1995:Apps. C, G, H, M). Ten selected long-form-sample uses are summarized in Box 3-1; they give a flavor of the importance of these data to the operation of the federal government. The ACS should be able to serve all of these federal agency uses and more, providing more up-to-date information of higher quality than the long-form sample. Some of the issues that must be considered in using ACS estimates for federal applications are illustrated below in the discussion of two specific uses: formula fund allocation (3-A.1) and determination of income limits for housing assistance programs (3-A.2). 3-A.1 Allocation of Federal Funds In sheer dollar terms, perhaps the most important use by federal agencies of long-form-sample data is to allocate billions of dollars of federal funds annually to states and localities (National Research Council, 2000b, 2003a; U.S. General Services Administration, 2006). Long-form-sample

OCR for page 77
Using the American Community Survey: Benefits and Challenges The U.S. Department of Housing and Urban Development uses the data on rent and utilities, number of bedrooms, plumbing facilities, kitchen facilities, type of heating fuel, and date when the occupant moved into the unit to determine fair market rents for a base year for some metropolitan areas and nonmetropolitan counties. American Housing Survey data and telephone surveys are used to estimate base-year fair market rents for the remaining areas. Fair market rents, updated yearly from the shelter component of local consumer price indexes and telephone surveys, are used to administer rental housing subsidies and to analyze housing costs relative to household income. The Bureau of Labor Statistics uses the data on sex, age, race, Hispanic origin, labor force status, occupation, industry, and class of worker to develop state-level labor force projections, which are used by program planners, policy makers, job training administrators, and career counselors. The Federal Reserve Board uses the data on race, Hispanic origin, and the year a structure was built for census tracts to report on the record of financial institutions in meeting the credit needs of low- to moderate-income neighborhoods under the Home Mortgage Disclosure Act and Community Reinvestment Act. The U.S. Department of Veterans Affairs uses the data on veteran status and other characteristics of veterans for counties and ZIP code areas to assess changes in the veteran population and to allocate resources, such as outreach specialists and employment and training directors. The U.S. Department of Agriculture uses the data on farm acreage and sales to distribute agricultural research and extension funds to states. data are used in two ways in allocation formulas: directly, in that long-form-sample estimates provide one or more factors in a formula, or indirectly, in that the formula relies on estimates for which long-form-sample data are one input to an estimation process that also uses other data sources.2 Whether formulas use long-form-sample estimates directly or indirectly has implications for how proactive the responsible program agency needs to be in deciding how best to use ACS estimates in place of long-form-sample estimates. 3-A.1.a. Use of Long-Form-Sample Estimates in Fund Allocation Formulas Most federal allocation formulas that incorporate long-form-sample data use the long-form-sample estimates directly; see Box 3-2 for seven ex- 2 Allocation formulas that use long-form-sample estimates (or estimates that incorporate long-form-sample data) may also include other factors that are often based on administrative records, such as per pupil expenditures or taxable resources.

OCR for page 77
Using the American Community Survey: Benefits and Challenges BOX 3-2 Selected Uses of Long-Form-Sample Estimates in Federal Fund Allocation Formulas Special Education Grants to States ($10.6 billion obligated in fiscal 2005): Allocates funds to states for the education of handicapped children in part by a formula that includes long-form-sample estimates of the number of children in the age ranges mandated by the state’s program and the number of children in poverty in those age ranges. Head Start ($6.7 billion obligated in fiscal 2005): Allocates funds to states according to long-form-sample estimates of the number of children ages 0–4 living in poor families. Organizations that operate Head Start programs use long-form-sample data as part of their applications to the U.S. Department of Health and Human Services for funding (within the limit of the funds allocated to their state). Community Development Block Grants, Entitlement Grants and State’s Program ($4.1 billion authorized in fiscal 2005): Allocates 70 percent of funds to large jurisdictions (metropolitan counties with 200,000 or more people and cities with 50,000 or more people) and 30 percent of funds to the remaining areas of states on the basis of the larger amount computed under two formulas. One formula uses long-form-sample estimates of total population, poverty population, and overcrowded housing units; the other formula uses long-form-sample estimates of total population, poverty population, and housing units built before 1940. Home Investment Partnerships Program ($1.9 billion authorized in fiscal 2005): Allocates funds to states, cities, urban counties, and consortia of local governments by a formula that uses various long-form-sample estimates, such as the estimated number of rental units built before 1950 occupied by poor families. Workforce Investment Act Adult and Youth Activities Programs ($1.9 billion obligated in fiscal 2005): Allocate funds to states, which reallocate most funds to local areas, by formulas that include long-form-sample estimates of unemployment and economic disadvantage for youths and adults. Title V Maternal and Child Health Services Block Grant to the States ($586 million obligated in fiscal 2005): Allocates funds to states as the sum of the state share of funds received for eight antecedent programs as of 1981 plus a share of any funds appropriated above the fiscal year 1983 level according to the state’s number of poor children under age 18 estimated from the long-form sample. The New Freedom Program, enacted August 2005 in the Safe, Accountable, Flexible, Transportation Equity Act: A Legacy for Users (SAFETEA-LU, P.L. 109-59): Allocated $78 million in fiscal 2006 for improved transportation services for people with disabilities. Funds are allocated to urbanized areas with 200,000 or more people (60 percent of the funds), and to states for smaller urbanized areas (20 percent) and for nonurbanized areas (20 percent). Within each group, funds are allocated to urbanized areas and states on the basis of the number of people with disabilities.

OCR for page 77
Using the American Community Survey: Benefits and Challenges amples. Long-form-sample estimates enter indirectly into the allocation of funds under Title I of the No Child Left Behind Act (estimated $12.7 billion obligated in fiscal 2005). This program allocates funds to school districts to meet the needs of educationally disadvantaged children by formulas that include estimates of poor school-age children. In the past these estimates were obtained from the most recent census long-form sample; currently, more up-to-date estimates are obtained from statistical models developed by the Census Bureau in its Small Area Income and Poverty Estimates (SAIPE) program.3 The SAIPE state- and county-level models include long-form-sample poverty estimates as one input together with more up-to-date information from administrative records to predict school-age poverty from a 3-year average of data from the Current Population Survey Annual Social and Economic Supplement (CPS ASEC). The school district-level model uses the previous census long-form-sample estimates of within-county school district shares of poor school-age children to apply to the updated county model estimates of the number of poor school-age children. The SAIPE program produces annual estimates with a 2-year lag between release and the estimates’ income reference year; the lag is due to delays in acquiring administrative records that are required for the modeling. 3-A.1.b Using ACS Estimates in Formulas Because the 2010 census will not include a long-form sample, policy makers and program managers must develop strategies for introducing ACS estimates into funding program allocation formulas that previously used long-form-sample estimates and decide whether such a change will require legislation or can be handled by regulation. The primary benefits of using ACS estimates are that they will be more timely and up-to-date and probably of higher quality than estimates from the long-form sample, so that the resulting fund allocations will more accurately reflect the distribution of needs among eligible areas.4 Still, the ACS estimates will have higher sampling error than long-form-sample estimates. Role of Policy Makers The role that policy makers and program managers play in decisions about the use of ACS estimates in allocation formulas 3 See National Research Council, 2000a; http://www.census.gov/hhes/www/saipe/saipe.html. 4 This discussion does not address whether the variables in a formula (in the absence of data quality concerns) produce the most equitable fund distributions in light of a program’s original goals (see National Research Council, 2003a). The need to replace long-form-sample estimates with ACS estimates could trigger reconsideration of the variables and other features in a formula, but that is outside the panel’s charge.

OCR for page 77
Using the American Community Survey: Benefits and Challenges depends at least in part on whether the estimates will enter into a formula indirectly or directly. Indirect uses will require less in-depth consideration by program and policy people because the statistical agency that produces the relevant estimates will presumably tackle the matter. Thus, the Census Bureau SAIPE staff will presumably determine effective ways of including ACS data in their model-based estimates of poor school-age children that are used in the allocation of education funds to school districts under the No Child Left Behind Act. The Bureau of Economic Analysis (BEA) is already incorporating ACS data into its county-level per capita income estimates, which could be considered for possible use in fund allocation. At present, only the BEA state-level per capita income estimates, which do not require 2000 long-form-sample (or ACS) data, are used in federal fund allocation programs, including the largest program—Medicaid ($193 billion of federal funds obligated in fiscal 2005)—and other programs that use the Medicaid formula. BEA develops county (and state) per capita income estimates from federal and state administrative records, censuses and surveys, and census-based population estimates (as denominators). Currently, BEA is moving to use the ACS, in place of the 2000 long-form sample, as a source of data on intercounty commuting. This information is needed to convert estimates of per capita income by county of workplace to those by county of residence. The BEA estimates are produced annually for counties about 15 months after the end of a year.5 When ACS estimates are to replace long-form-sample estimates directly in a fund allocation formula, then program and policy people must be more involved. Factors in choosing which ACS period estimates to use (1-year, 3-year, or 5-year) include not only the extent of sampling error, but also the desired frequency with which funds are to be reallocated among areas and the types and population sizes of eligible geographic areas. Of course, during the ramp-up period between 2005 and 2010, agencies’ choices are constrained by whether the estimates that best serve their needs are available. For example, if 5-year period estimates must be used to obtain an acceptable level of precision, then agencies will need to rely on the long-form-sample estimates until 2010 when ACS 5-year period estimates become available for the period 2005–2009. Currency, Precision, and Stability Considerations In determining which ACS estimates to use in an allocation formula (assuming they are available for all eligible areas), decision makers should identify key characteristics that the estimates must satisfy. If currency of the information is paramount, so that areas with the greatest present need receive the most funding, then 5 See http://www.bea.gov/bea/regional/articles.cfm?section=methods.

OCR for page 77
Using the American Community Survey: Benefits and Challenges 1-year period ACS estimates will be preferable to 3-year or 5-year period estimates, and 3-year period estimates will be preferable to 5-year period estimates. However, 1-year (or 3-year) period estimates may not be sufficiently precise—that is, may not have low enough sampling error—for fund allocation purposes. If estimates are not precise, then nontrivial changes in funding allocations from year to year may be an artifact of sampling error.6 A related consideration is the weight to give to currency for the most equitable allocations versus the practical arguments for moderating the magnitude of year-to-year changes to facilitate program planning and implementation. Many programs moderate fluctuations in program allocations through features of the formula. For example, under a hold-harmless provision, every locality is entitled to receive at least as many dollars as a specified percentage—which could be 100 percent—of its prior-year dollars. Such legislative provisions have drawbacks, in that their use can delay the responsiveness of the funding formula to changes in need and also create inequitable allocations that are an artifact of sampling error in the estimates. For example, if legislation sets a threshold for eligibility, such as a minimum number of poor school-age children, and an area exceeds that threshold in a particular year because the estimate is greater than the threshold level due to sampling error, it will erroneously receive funding at that time. Moreover, the application of a hold-harmless provision will enable the area to retain funding in subsequent years, even though it was not eligible in the first place. An alternative approach to achieve more stable funding streams, while still responding to changes in need, is to eliminate thresholds and hold-harmless provisions and instead smooth the estimates themselves—for example, by using 3-year period estimates rather than 1-year period estimates for allocations to states (see Zaslavsky and Schirm, 2002). Implementation of this approach could require changes in legislation. Geographic Area Considerations Yet another consideration in the selection of ACS estimates for fund allocation is the types and population sizes of geographic areas that are eligible for funding. Some formulas apply to a single type of geographic area, such as states, while others include several types of areas, such as states, cities, and counties, and still others have population size thresholds that may vary by type of area. Consider first a formula allocation program, such as Special Education Grants to States, which uses state-level estimates of all children and poor 6 See Box 2-5 for definitions of sampling error and related terms, such as coefficient of variation and margin of error.

OCR for page 77
Using the American Community Survey: Benefits and Challenges children in specific age ranges, leaving it to state agencies to make further allocations to localities. In this instance, the most straightforward method for taking advantage of the ACS would simply be to substitute up-to-date ACS 1-year period estimates for the long-outdated 2000 census long-form-sample estimates in the formula. The ACS 1-year period estimates should have low sampling error for all 50 states and the District of Columbia. For example, estimates of poor school-age children may have a coefficient of variation of less than 8 percent for the smallest states, with about 600,000 people (see Table 2-7a), while the coefficient of variation of these estimates may be only 1 percent for the largest states, with 20 million people. Moreover, the Special Education Grants Program has minimum funding provisions that would moderate year-to-year fluctuations in allocation amounts from the use of annually updated ACS 1-year period estimates in place of the once-a-decade long-form-sample estimates. Should it be deemed desirable to further smooth funding amounts, the Special Education Grants Program could average 2 years of 1-year period estimates or use 3-year period estimates, which should have very low sampling error for all 50 states and the District of Columbia. Programs like Community Development Block Grants and Home Investment Partnerships, however, provide funds to different types of governmental units, some of which are smaller in population size than the cutoff of 65,000 people or more for ACS 1-year period estimates. For these programs, it will not be possible to take the simple approach outlined above because ACS 1-year period estimates will not be available for all eligible areas. Moreover, while ACS 3-year period estimates may be available for all eligible areas, they may not be sufficiently precise for some of them. For example, should the needed estimates represent a group as small as poor school-age children, then the 3-year period estimates will not have a reasonably small coefficient of variation until the eligible area has a population of at least 80,000 people (see Table 2-7a).7 For such programs as Community Development Block Grants, for which governmental units as small as 50,000 people are eligible for funding, agencies must carefully balance the need for more up-to-date information from using 3-year period estimates against precision requirements that will be better satisfied with 5-year period estimates. For programs for which governmental units must have at least 100,000 people to be eligible for 7 Table 2-7a should be used only as a very rough guide to expected levels of sampling error for estimates for different size areas from ACS 1-year, 3-year, and 5-year period estimates. The sampling error will differ from that shown in the table for a characteristic that is a different percentage of the population from poor school-age children (as seen in Table 2-8). The sampling error will also depend on the sample size that the ACS achieves in the field for the particular governmental unit.

OCR for page 77
Using the American Community Survey: Benefits and Challenges funding, agencies must trade off the timeliness of 1-year period estimates and the greater precision of 3-year period estimates. When agencies decide that there is no choice but to use 5-year period estimates from the ACS in a funding formula in order to gain sufficient precision, they should be aware that inequities may result. For example, two areas may have the same 5-year period poverty rate and therefore receive the same allocation, even though one area may have a sharply increasing poverty rate and the other area a sharply decreasing poverty rate over the period. Even in this case, however, the use of ACS 5-year period estimates would represent an improvement over the continued use of the increasingly out-of-date 2000 long-form-sample estimates. At present, the only federal funding program that makes allocations to areas with fewer than 50,000 people is the No Child Left Behind Act, which allocates funds to school districts, varying in size from a few hundred to several million people (see Table 2-4). The SAIPE estimates that are used for the allocations are more up to date than the direct long-form-sample estimates later in the decade, but they rely on statistical models. The incorporation of ACS data into the SAIPE county and school district models should make it possible to improve their timeliness and precision. Consistency of Period Estimates In trading off such considerations as currency and precision, in no instance should agencies use in their allocation formulas a mix of different periods of ACS estimates—for example, 1-year (or 2-year) period estimates for larger areas and 3-year or 5-year period estimates for smaller areas—in an attempt to equalize the sampling error across areas. The reason has to do with equity: formulas generally allocate shares of a fixed pie, so that the data used in the allocation should reference the same time period. Otherwise, inequitable outcomes may occur. For example, consider a large county and a medium-sized city, both of which experience rapidly increasing poverty over 5 years. If in a poverty-based formula, 1-year period estimates are used for the large county and 3-year period estimates are used for the medium-sized city, then the county will likely receive more than its fair share of funds over the 5 years compared with the city because the 1-year period estimates will likely exhibit more growth in poverty than the 3-year period estimates. 3-A.2 Determination of Median Incomes for Counties The U.S. Department of Housing and Urban Development (HUD) obligates $27 billion annually for assisted housing programs in which families that have incomes below specified limits are eligible to live in public housing or receive rent subsidies. The income limits are determined separately for every metropolitan area and nonmetropolitan county as a function of

OCR for page 77
Using the American Community Survey: Benefits and Challenges that housing permit data do not necessarily correspond with actual housing units constructed and occupied and given the problems of estimating demolitions and conversions to nonresidential use. The errors in the housing unit controls may well vary across geographical areas and may also cumulate over time. While one cannot be sure what the magnitude or direction of the errors will be in the 2000 census-based estimates of the population and housing for 2010, one can be reasonably sure that there will be discrepancies between the estimates and the 2010 census counts and, furthermore, that the discrepancies will be greater for many counties and combinations of counties that are the basis for the ACS weighting controls. For the ACS, this means that there will be a discontinuity in many areas in totals for important demographic groups between 1-year period estimates for years preceding a census and for years including and following a census. This discontinuity will also exist for 3-year and 5-year period estimates between those that completely antedate a census year and those that include and follow a census year (for example, when comparing 5-year period estimates for 2005–2009 and 2010–2014). For 3-year and 5-year period estimates that span a census year (for example, a 5-year period estimate for 2008–2012), the Census Bureau plans to use an average of controls in which the population estimates for precensus years are adjusted to be consistent with the census counts. One might consider that ACS estimates of percentages, as opposed to levels, would not be affected by the problem of differences in precensus and postcensus population controls. This will be the case, however, only if the discrepancies between the two sets of controls are relatively uniform by demographic category. If the discrepancies differ by category, which is likely, then the percentages will be affected as well (see Table 3-7 for an example). There is no universal solution for the problem that will result from discrepancies between precensus and postcensus population controls. Users must address the situation for their applications and areas of interest, given that the problem will be more significant for some areas and population groups than others. The Census Bureau can help users in this regard by producing concurrent series of estimates that are based on precensus and postcensus controls. For example, the Census Bureau could produce two series of 1-year period estimates for, say, 2008–2010, in which the first series would use the 2000 census-based controls (the official series for those years), while the second series would backcast the 2010 census-based controls.

OCR for page 77
Using the American Community Survey: Benefits and Challenges TABLE 3-7 Hypothetical Effect of Decennial Census on ACS 1-Year Period Estimates, BIG CITY, 2008–2012 Year Population Control Estimated Number of Poor People (Equal to the Percent Poor Times the Control) Non-Hispanic Hispanic Non-Hispanic (10% Poor) Hispanic (20% Poor) Total ACS, Control Based on 2000 Census           2008 200,000 50,000 20,000 10,000 30,000 2009 200,000 52,000 20,000 10,400 30,400 2010 200,000 54,000 20,000 10,800 30,800 2010 Census 200,000 64,000 20,000 12,800 32,800 ACS, Control Based on 2010 Census           2011 200,000 67,000 20,000 13,400 33,400 2012 200,000 70,000 20,000 14,000 34,000 NOTES: • The population controls for Hispanics and non-Hispanics (and for age, sex, and race groups) are implemented for the ACS by estimation area (county or group of small counties; this example assumes that BIG CITY is its own county). The controls are developed from the previous census updated with administrative records on births, deaths, and net migration. • For ease of presentation, the example unrealistically assumes constant poverty rates from the ACS for the Hispanic population (20%) and the non-Hispanic population (10%) and that the city experienced no growth in the non-Hispanic population. • For ease of presentation, the example unrealistically assumes that all of the error in the population controls prior to 2010 pertained to the Hispanic population. What happened? • The 2010 census gave the same non-Hispanic count (200,000) as the 2000-based population controls updated to 2010. But the 2010 census gave a different Hispanic count from the updated controls—64,000 instead of 54,000. Consequently, the controls were revised going forward from 2010. What does the example tell the user? • The number of Hispanics and non-Hispanics is determined by the population controls, while the percentage of poor people in each group is determined by the ACS. Hence, the number of poor people in each group is the product of the control and the estimated ACS percentage. • The 2010 census results indicate that the Hispanic population and, consequently, the poverty population grew faster prior to 2010 than previously estimated, so that the ACS estimates of the number of poor Hispanics and total poor were too low for the years 2008–2010. • Users will not know until the 2020 census is taken the extent of error that may occur in the 2010 census-based population estimates that are used as controls for the ACS in the period 2011–2020.

OCR for page 77
Using the American Community Survey: Benefits and Challenges 3-H PREPARING TO USE THE ACS Users can ease the transition from using long-form-sample estimates to using ACS estimates for their applications by becoming knowledgeable about general guidelines for effective use of the ACS (3-H.1) and by taking concrete steps in advance to prepare for the time when 1-year, 3-year, and 5-year period estimates will be released each year (3-H.2). 3-H.1 General Guidelines for ACS Use Abstracting from the specific applications discussed above, this section provides the panel’s basic general guidelines for appropriate use of ACS estimates. a. Always examine margins of error before drawing conclusions from a set of estimates. Users should follow this practice for the long-form sample, the ACS, and any other survey on which they rely. More specifically: When using ACS data to estimate a number or percentage for a single area or population group, such as a city or county, the ACS period estimate chosen—1 year, 3 years, or 5 years—should satisfy the precision requirements appropriate for the purpose for which it is being used (refer back to Table 2-7a). Five-year period estimates will not be precise for estimates of small population groups (for example, poor school-age children, poor elderly people, minorities, high school dropouts) for areas with fewer than about 50,000 people, which includes most counties, cities, towns, townships, and school districts, as well as every census tract and block group (refer back to Table 2-7a). Consequently, users should work with 5-year period estimates for small areas only with great care. When it is unduly burdensome to examine numerous individual margins of error—as, for example, when working with a large number of 5-year period estimates for multiple geographic areas—users should at least examine some of the individual error margins to check that the estimates are of adequate precision for their purpose. b. Review the available information about nonsampling errors for estimates of interest and use this information in interpreting findings from the ACS.

OCR for page 77
Using the American Community Survey: Benefits and Challenges Research on nonsampling errors that may systematically bias survey estimates upwards or downwards is difficult to conduct, and the available information is rarely complete or definitive about the magnitude of the biases. Hence, users are rarely in a position to adjust estimates of interest to correct for nonsampling errors. Nonetheless, users should acknowledge known nonsampling errors in their uses of the ACS data. As examples of possible biases in the ACS, a comparison of the C2SS with the 2000 long-form sample found significantly lower estimates of median income in the C2SS than in the long-form sample, while comparisons of the C2SS and the 2001–2003 ACS test surveys with the CPS consistently found significantly lower estimates of unemployment in the ACS surveys than in the CPS (see Section 2-B.2.e). Further research is needed to determine which survey is more accurate. c. Carefully consider the pros and cons of alternative strategies for extracting value from ACS 5-year period estimates for very small areas, such as aggregating small-area estimates into estimates for larger, user-defined areas. Large cities and counties should use ACS 5-year period estimates for census tracts and block groups as building blocks to define larger areas that are meaningful for analysis and for which 5-year period estimates are sufficiently precise. For example, a city might aggregate census tracts into several planning areas, or it might use combinations of block groups that do not necessarily respect census tract boundaries. Statistical mapping techniques may help identify which tracts and block groups would be most useful to combine into subareas for analyzing such phenomena as commuting patterns. For user-defined subareas, a city might ask the Census Bureau to develop 3-year period estimates for large population groups to obtain more information on trends. Small governmental units may not be able to satisfy their data needs by aggregating 5-year period estimates into larger areas. However, with due care they may be able to work with 5-year period estimates for large population groups in their jurisdiction and 5-year period estimates for smaller groups for a larger area, such as their county, to assess changes in the composition of their own area. Small governmental units might also ask the Census Bureau to develop ACS estimates for their area for periods longer than 5 years.

OCR for page 77
Using the American Community Survey: Benefits and Challenges In addition to the basic strategies outlined above, in the future it may be possible to extract more value from the ACS 5-year period estimates by linking them with administrative records and other sources of local-area information (see Section 7-D.1). d. When using ACS data to estimate shares of some total, compare estimates among areas or population groups, or assess trends over time, use ACS estimates that pertain to the same time period (1-year, 3-year, or 5-year) for all geographic areas or population groups that are being compared. Do not use a mixture of different period estimates. For example, when determining the share of federal or state program funds that is to be allocated to each county in a state, the ACS estimates that are used will most likely need to be 5-year period estimates. The reason is that 1-year and 3-year period estimates are available for only about one-fourth of counties (refer back to Table 2-5), and it is not equitable to use a mixture of 1-year, 3-year, and 5-year period estimates to determine each county’s share of funds. An exception to the need to use 5-year period estimates for fund allocations to counties is when a state has only a small number of counties that lack 1-year (or 3-year) period estimates. In this case, it may be appropriate to update the 5-year period estimates for the smaller counties by using information for larger areas, so that the equivalent of 1-year (or 3-year) period estimates can be used for all counties in the state. A simple procedure for accomplishing the updating is described in Section B.2 above; the use of this or another procedure depends on the reasonableness of the underlying assumptions. As a matter of good practice, differences that are observed in comparing areas or population groups or in assessing trends over time should be evaluated not only for statistical significance, but also for substantive importance—that is, whether the differences are large enough to matter for policy, planning, or research purposes. e. When analyzing trends over time for an area or population group, use ACS 1-year period estimates whenever they are available and sufficiently precise for the purpose of interest and be cognizant of changes in geographic area boundaries that may affect comparability. Keep in mind that the sampling error for the estimate of the difference between pairs of 1-year period estimates will be larger than the sampling error of either estimate.

OCR for page 77
Using the American Community Survey: Benefits and Challenges f. If only 3-year or 5-year period estimates are available or sufficiently precise, use them with care for analyzing trends over time for an area or population group. In general, avoid analyses of changes over time that are based on overlapping period estimates (for example, 5-year period estimates for 2010–2014 and 2011–2015). It is not straightforward to interpret the meaning of differences that are observed between pairs of 3-year or 5-year period estimates: an observed difference may reflect a gradual change over the period, or it may reflect another pattern of change, such as stability in a characteristic followed by a sudden increase or decrease. Examining 1-year (or 3-year) period estimates for a larger area may help determine the appropriate interpretation of differences that are observed between pairs of 5-year period estimates for smaller areas within the larger area. The less that pairs of 5-year (or 3-year) period estimates overlap in time, the more precise will be an estimate of differences between them—for example, a difference observed between estimates for 2010–2014 and 2015–2019 will be more precise than a difference observed between estimates for 2010–2014 and 2011–2015. Indeed, to obtain an acceptable level of precision for analysis of population groups, it will generally be necessary to wait for a second, nonoverlapping estimate to become available to compare to an earlier estimate. g. Take advantage of the availability of 1-year and 3-year period estimates for PUMAs, which include about 100,000 people, to assist with analyses for smaller areas. As one example, 5-year period estimates for small areas (census tracts in a city, towns in a county, small counties in a state) could be updated by adjusting their 5-year period estimates to the latest 1-year (or 3-year) period estimates for the applicable PUMA, as in Section B.2 above. Such adjustments need to be performed with care. As another example, 1-year period estimates for large cities or counties could be compared with the PUMA estimates for the rest of the state (or the rest of the county in the case of a large city within a very large county). h. Take care to label ACS estimates, including those for 1 year, 3 years, and 5 years, as period estimates.

OCR for page 77
Using the American Community Survey: Benefits and Challenges ACS 3-year and 5-year period estimates do not refer to a particular year, such as the end year or the middle year. They are period estimates—averages of characteristics over a 36-month or 60-month period—and should be labeled as such. Otherwise, readers may draw an incorrect inference—for example, assuming that a 5-year period estimate of 15 percent poverty is the rate for the end year, when the end-year rate could be considerably higher or lower. ACS 1-year period estimates are also an average over 12 months (except for the special estimates released in June 2006 for January–August 2005 and September–December 2005 for areas affected by Hurricanes Katrina and Rita). i. Use ACS 3- and 5-year period estimates for income, housing value, and housing costs with care. To compensate for the differing time periods for which dollar amounts are collected, those amounts are adjusted to a common calendar year by the change in the national CPI. This inflation adjustment expresses all of the reported dollar amounts in a comparable manner with regard to purchasing power as of the most recent calendar year in the period. However, the resulting estimates should not be interpreted as current-year estimates. j. Use care in comparing ACS estimates with estimates from other data sources, including the 2000 long-form sample and other surveys, and be cognizant of the differences that could affect the comparisons. Such differences may include population coverage, sample size and design, reference periods, residence rules, and interview modes. 3-H.2 Suggestions for Users During the Ramp-up Period In the next few years, users who plan to make extensive use of the ACS will have an opportunity to prepare for the full range of 1-year, 3-year, and 5-year data products that will be available beginning in 2010. It is important for users—including federal, state, and local agencies, and private organizations—not to stint attention or resources in order to make the most of this opportunity, so that they are well prepared to work with the full richness of the ACS data by 2010. We outline below some of the steps that the technical staff of an agency can take to ensure that their agency is well prepared to work with the ACS data (see also the recommendations in ORC Macro, 2002:Ch. 10). a. Steps to prepare agency management:

OCR for page 77
Using the American Community Survey: Benefits and Challenges Schedule briefings with agency program managers to acquaint them with the ACS and how it will replace and improve on the once-a-decade long-form sample. Make the case for sufficient resources from management to support planning for effective use of the various ACS products, which should save resources in the long run by minimizing inappropriate or ineffective use of the data. As methods are developed to work with the ACS data for key applications, keep agency management informed of solutions. Apprise management of the need to take such actions as seeking legislative authority or modification of regulations to permit ACS data to be used in place of long-form-sample data for particular applications. b. Steps to determine which data and methods to use for particular applications: Make use of information from the Census Bureau about the likely sampling error for different size areas to determine the most useful ACS estimates for the agency’s application(s). For example, if a city will have 1-year period estimates provided but their sampling error will be high, then the city may want to rely on the 3-year period estimates for planning and program applications. Use the detailed 1-year period estimates that first became available in summer and fall 2006 to help develop the most useful profiles and other products to generate from the detailed 3-year and 5-year period estimates when they become available. Use the training data sets released by the Census Bureau in spring 2007 of 1-year, 3-year, and 5-year period estimates fo 34 ACS test site counties for the years 1999–2005. Use of these data can provide valuable experience in working with multiple sets of estimates prior to the availability of the full set of 3-year and 5-year period estimates. Determine the most appropriate geographic aggregations of 5-year period estimates for subareas of, say, a city or county. For example, the technical staff might divide a city of 500,000 into 10 service areas of approximately 50,000 population and aggregate 5-year period estimates for census tracts and block groups accordingly. Similarly, a large county might aggregate 5-year period estimates for townships into subcounty regions. If resources permit, commission a detailed, comprehensive analysis of the alternatives for using various ACS data products for key

OCR for page 77
Using the American Community Survey: Benefits and Challenges applications, similar to the study commissioned by HUD (ORC Macro, 2002). Consider how often to analyze updated ACS estimates in light of the agency’s needs and resources. For example, it is unlikely that updates of small-area data analyses can be conducted more than twice a decade, nor may it be effective or efficient to do so. Determine if any special tabulations will be needed from the Census Bureau, develop detailed specifications for them, and discuss feasibility and costs with the Census Bureau well in advance. For example, areas with large seasonal populations may want to request special tabulations. Determine whether and how additional data sources may be helpful in some applications of ACS data. For example, a state might want to use administrative records information in conjunction with ACS estimates in fund allocation formulas. Determine if model-based or composite estimates that are developed from the ACS and other data sources by statistical agencies could support particular applications, thereby saving on the program agency’s technical resources. Request that the Census Bureau inform users of helpful guides that are developed by State Data Centers and other organizations and individuals to assist users—for example, the recent publication, American Community Survey Data for Community Planning (Taeuber, 2006). c. Steps to work with public officials, the media, and other constituents: Develop templates for appropriate interpretative language to use in press releases and talking points about each summer’s issuance of the latest ACS estimates from the Census Bureau. Given that the media and public officials will inevitably want to compare trends across time and levels across areas using the most recent estimates regardless of their precision, the agency technical staff should develop suitably cautionary language to include in statements by public officials and in speaking with the media. Develop standard formats for tables to provide to constituent groups (for example, neighborhood advisory commissions or council members in a city or county). Be sure to include appropriate explanatory material about sampling error and other aspects of the data.

OCR for page 77
Using the American Community Survey: Benefits and Challenges d. Steps to look toward the future: Keep up to date with Census Bureau information on the ACS web site, such as users’ guides and design and methodology reports, and take advantage of training opportunities afforded by the Census Bureau, state data centers, and other organizations. Feed back questions, concerns, and data needs to the state data centers and to the Census Bureau. On one hand, be cognizant that the Census Bureau has a heavy workload in collecting, processing, and disseminating the continuous ACS, but, on the other hand, remind the Census Bureau that the ACS must be an evolving data system that responds to user needs. Liaise with other users with similar interests to develop and evaluate strategies for effective use of ACS data products, and put forward coordinated requests for new and improved data products, training materials, and other support from the Census Bureau. If there is a need for new or modified questions, work with the Census Bureau and stakeholders to determine what is the minimum set of changes that would serve the purpose. The Census Bureau has a protocol for the testing that must be undertaken before a new question can be added to the ACS (see Section 7-C.2). Similarly, work with the Census Bureau and stakeholders to adjust geographic boundaries for census tracts and block groups in ways that reflect population change but minimize discontinuities in local geographic boundaries over time. If, for example, most changes to these small areas involve splitting them to reflect population growth (or, alternatively, combining them to reflect population decline), then it will be easier to use successive 5-year period estimates. Participate in forums in which users share their experiences with analysis and presentation techniques that make effective use of the ACS data for a range of applications. In conclusion, the ACS will offer not only significant challenges to data users, but also significant benefits. Having more timely and up-to-date information that is likely of higher quality will benefit all applications that previously used the long-form-sample estimates. In the future, there will be opportunities for new uses of the ACS that would never be possible with the long-form sample. Users should take steps during the ramp-up period to prepare for the ACS, anticipate problems, and work together and with the Census Bureau on solutions.

OCR for page 77
Using the American Community Survey: Benefits and Challenges This page intentionally left blank.