The American Community Survey (ACS), to be run by the Census Bureau, will be a large (250,000 housing units a month), predominantly mailout/mailback survey that will collect information similar to that on the decennial census long form. The development of this new survey raises interesting questions about methods used for combining information from surveys and from administrative records, weighting to treat nonresponse and undercoverage, estimation for small areas, sample design, and calibration of the output from this survey with that from the long form. To assist the Census Bureau in developing a research agenda to address these and other methodological issues, the Committee on National Statistics held a workshop on September 13, 1998. This report summarizes that workshop.
When fully operational (currently planned for 2003), the ACS will provide continuous, small-area information on demographic characteristics, social welfare, education and health status, commuting patterns, crime patterns, and other important attributes of the population of the United States, including the interrelationships of these characteristics. Unlike any other national survey, the ACS will provide information on the American population at substate levels, i.e., counties and cities. Over a 5-year period, the survey's sample size will approximate that of the census long form, supporting the production of estimates, possibly through use of statistical modeling, for small and nonstandard geographical areas, such as school districts and traffic analysis zones. In addition, given the sample size, information will be available for specific demographic groups, including racial and ethnic groups, children, the elderly, people in specific occupations, people with specific health condi-
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 1
Page 1 1 Introduction The American Community Survey (ACS), to be run by the Census Bureau, will be a large (250,000 housing units a month), predominantly mailout/mailback survey that will collect information similar to that on the decennial census long form. The development of this new survey raises interesting questions about methods used for combining information from surveys and from administrative records, weighting to treat nonresponse and undercoverage, estimation for small areas, sample design, and calibration of the output from this survey with that from the long form. To assist the Census Bureau in developing a research agenda to address these and other methodological issues, the Committee on National Statistics held a workshop on September 13, 1998. This report summarizes that workshop. When fully operational (currently planned for 2003), the ACS will provide continuous, small-area information on demographic characteristics, social welfare, education and health status, commuting patterns, crime patterns, and other important attributes of the population of the United States, including the interrelationships of these characteristics. Unlike any other national survey, the ACS will provide information on the American population at substate levels, i.e., counties and cities. Over a 5-year period, the survey's sample size will approximate that of the census long form, supporting the production of estimates, possibly through use of statistical modeling, for small and nonstandard geographical areas, such as school districts and traffic analysis zones. In addition, given the sample size, information will be available for specific demographic groups, including racial and ethnic groups, children, the elderly, people in specific occupations, people with specific health condi-
OCR for page 1
Page 2 tions, and people with various levels of educational attainment. The ACS questionnaire may also be able to address current issues of national and regional importance by including supplementary questions (though this raises various complications). Obviously, since the ACS data will be collected continuously throughout the decade, the information will generally be more timely than that from the census long form. The three primary types of information sources for individuals and households in the federal statistical system are the decennial census, various household surveys, and administrative records systems, e.g., tax and food stamp records. The planned ACS has various advantages in comparison to these other sources. With respect to household surveys, the ACS will have advantages of size and scope: it will sample more households and will provide a wider range of information than is typically found on individual household surveys, which are by necessity targeted to a specific set of issues. With respect to administrative records systems, the ACS will have the advantages of representativeness (e.g., tax records have information only on filers, and food stamp records have information only on participants) and scope—since administrative records do not contain information other than that required for the associated program (e.g., they typically do not include demographic information). Also, the associated programs can change from year to year, making interyear comparisons difficult, and the administration of a program may differ by geographic region, especially by state, which complicates interregional comparisons. However, these other information sources also have advantages in comparison to the ACS. It is expected that the information from household surveys will generally be of higher quality than that from the ACS for the outputs directly related to the purpose of the household survey. This is due to at least three factors: (1) many household surveys use personal interviews as a primary source of data collection, which tends to result in higher-quality responses than for mailout-mailback surveys; (2) many household surveys visit the same households over time, which can help to improve response; and (3) household surveys targeted on a specific characteristic typically ask a large number of questions concerning that characteristic, which can aid in recall and in the accuracy of response. Therefore, ACS responses are likely to have greater measurement error than household surveys requesting the same information. Also, administrative records systems, which are collected as a by-product of the associated programs, can provide more detailed information and often provide information for larger numbers of individuals or families than will be possible for ACS. From the time of the initial planning of the ACS (initially referred to as continuous measurement), the Census Bureau has been aware that its introduction would raise a number of complicated methodological issues. However, given the substantial work entailed in the fielding of such a new, large
OCR for page 1
Page 3 survey, the focus of the ACS group to date has been on refining data collection, leaving the final answers to the difficult analysis questions for later. Thus, procedures for nonresponse and undercoverage adjustment were modeled, to the extent possible, after current procedures used for the census long form. Now that data collection has matured as the ACS demonstration phase is well under way, the Census Bureau is developing a research plan and initiating research to address all issues related to ACS methodology. Fall 1998 therefore seemed an opportune moment for a workshop to assist the Census Bureau in developing a research agenda to deal with many of these challenging issues. The hope was that the workshop would facilitate a dialogue between Census Bureau staff, interested academics, and other researchers. DATA COLLECTION AND SAMPLE DESIGN The idea for the ACS follows a suggestion made nearly two decades ago by Leslie Kish (1981), who used the term “rolling samples.” The survey instrument for the ACS is a questionnaire that will be mailed out to households, with 100 percent nonresponse follow-up using computer-assisted telephone interviewing (CATI) in the month following questionnaire mailout, followed by field follow-up (computer-assisted personal interviewing, or CAPI) of a random one-third of the remaining nonrespondents in the month following CATI follow-up. The size of this survey will make direct small-area estimates possible, though the estimates for smaller areas typically will be produced by aggregating information over 2 to 5 years, depending on the size of the area. (At this time moving averages are planned.) The current plan also is that governmental jurisdictions of less than 2,500 population will be oversampled. Each month's sample is intended to be a self-weighting sample of the population of each area of the United States (except for possibly a few complications such as oversampling small areas). Thus, cluster sampling will not be used to facilitate field follow-up. Also, both to reduce respondent burden and to lower variances of direct estimates aggregated over months and years, a household cannot be in the sample more than once every 5 years. When the ACS is fully fielded, it will use as a sampling frame the Census Bureau's Master Address File, an update of the address list used to conduct the 2000 census. The annual sample will be divided into monthly mailout panels, where each month's panel is a systematic sample across the complete address list. As mentioned above, the CATI follow-up of mail nonresponse lags 1 month from the questionnaire mailout, and the field follow-up lags 1 month from the CATI operation. To accommodate this schedule and provide timely estimates, a given month's estimates will make use of the totality of information collected in that month, which will not be the information corresponding
OCR for page 1
Page 4 to the systematic sample selected for that month. The information collected in a month will include mail responses received in that month, the CATI interviews relative to the sample for the previous month, and the information from the field follow-up relative to the sample for the month before that. This treatment of nonresponse, along with undercoverage and other complications, raises difficult weighting challenges. PILOT TESTING There was a 3-year demonstration pilot test for the ACS in 1996-1998. The survey was administered in Brevard County, FL; Multnomah County, OR; Rockland County, NY; Fulton County, PA; Franklin County, OH; Douglas County, NE; Ft. Bend and Harris Counties, TX; and Otero County, NM. The sampling rate for the first year was 15 percent in most areas with 30 percent in small governmental areas. The rate was lowered to 3 percent during the latter part of the demonstration period. During 1998, the decennial census dress rehearsal site in South Carolina also implemented the demonstration ACS. The goals of the demonstration period were to: (1) illustrate the usefulness of ACS data every year and over time; (2) to improve operations and reduce and understand costs; and (3) to provide a comparison with the dress rehearsal data. The demonstration period is now to be followed by two comparison studies—in 1999-2001 and 2000-2002—comparing ACS and census long-form information. Full implementation of ACS will begin in 2003, with an ongoing sample size of 3 million housing units a year (a sampling rate of approximately 3%—15% over five years—compared with 17%, on average, for the census long form). The first comparison study was based on an implementation of ACS in 31 comparison sites for 3 years, 1999-2001. An experimental design was used to select the 31 areas with characteristics for which differences between the ACS and long-form responses were anticipated. The sampling rate in the sites was planned to be roughly 5 percent annually so that the sample sizes of the ACS (over the 3-year period) and the long form would be comparable, though budget limitations have reduced this sampling rate in some areas. For the second comparison study, in 2000-2002, the ACS will have a national sample of 700,000 addresses per year (0.7% sampling rate). This study is designed to make comparisons between the long form and ACS for all states, large metropolitan areas, large substate areas, and population groups. The objective of the 1999-2001 comparison is to understand the factors associated with the differences between the 1999-2001 ACS and the 2000 long form in the 31 areas, using the second comparison study to develop a calibration model to adjust the 2000 long-form estimates to roughly represent what the full ACS would have yielded in 2000. The adjustment based on this
OCR for page 1
Page 5 calibration model will reflect differences in question wording, residence rules, reference periods, interviewer training and other field operations, and differences in coverage and nonresponse. Once adjusted, the “calibrated” long-form data for 2000 can be compared with ACS data that are collected following full field implementation in 2003, in order to understand the dynamics over time of such characteristics as poverty and employment. THE WORKSHOP: PURPOSE AND STRUCTURE To maximize the interchange of ideas during the limited time available in a 1-day workshop, selected experts were asked to prepare thought pieces to address one of six methodological issues: combination of information cross-sectionally, combination of information across time, the impact of variance on ACS outputs as inputs into fund allocation formulas, weighting to accommodate nonresponse, undercoverage, etc., issues related to sample and questionnaire design, and calibration of the ACS with the census long form. For some of the issues, discussants were also selected. As background for both the writers and discussants, committee staff prepared a document elaborating on the specific problems posed by each of the methodological issues, including research directions that might prove beneficial. Following this, Charles Alexander of the Census Bureau provided a “response” to the background document, and both were provided in advance to the writers and discussants. The staff document, the response from Charles Alexander, the thought pieces, and the discussant papers when completed were made available to all of the thought piece writers, discussants, and special invited guests in advance of the workshop, so that the floor discussion could be more informed as to each presenter's ideas. (These documents are available in the workshop agenda book, “The American Community Survey Workshop: Technical Papers.”) The next six chapters describe, in turn, the methodological areas of focus of the workshop. Although the subjects chosen for focus at the workshop are some of the more important methodological issues facing the Census Bureau, it is important to note that these issues do not encompass all the issues of concern and that there are many others worthy of study. Even in these six methodological areas of focus, the presenters were free to choose to address various subtopics. Some of the remaining issues not examined in the workshop are listed in Chapter 8. To set the stage, and especially to illustrate concerns for the development of small-area estimates and issues related to the combination of information from different data sources, Graham Kalton started off the workshop by describing the work of the Panel on Estimates of Poverty for Small Geographic Areas (which he chairs). (For more information, see the Appendix and National Research Council, 2000.) The remainder of the report has the following structure: Chapter 2, Chapter 3, Chapter 4, Chapter 5, Chapter 6
OCR for page 1
Page 6 through Chapter 7 treat the individual methodological issues in turn, namely, combination of information across areas, combination of information across time, concerns involved in the use of the ACS as input to funding formulas, weighting issues in the ACS to treat nonresponse and undercoverage, sample and questionnaire design issues, and calibration of the ACS to the census long form. In each chapter, the topics are first introduced, followed by summaries of the presentations and the floor discussion; Chapter 8 provides some final comments.