National Academies Press: OpenBook

Designing the 2010 Census: First Interim Report (2000)

Chapter: Research Experiments

« Previous: The Master Trace Sample
Suggested Citation:"Research Experiments." National Research Council. 2000. Designing the 2010 Census: First Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9802.
×

Research Experiments

THE NON-AREX 2000 EXPERIMENTS

The Census Bureau is planning on carrying out five experiments during the 2000 census. The Administrative Records Experiment (AREX 2000) is separately described and discussed below. The four other experiments, for which we have no suggested changes, are briefly described here.

  • Incentives for Alternative Response Modes For a sample of households, the Census Bureau will examine whether offering free time on a telephone calling card increases the frequency of respondents' use of the telephone or the Internet to provide their census responses. There will be a control group and two treatment groups, both of which are offered the calling card. Each group is made up of three panels, one for each response mode: (1) reverse computer- assisted telephone interviewing, (2) automated spoken questionnaire, and (3) the Internet. While households may respond with the paper questionnaire, the calling card is activated after one of the above alternative response modes (selected for each panel) is used.

  • Use of Noncognitive Tests of Enumerators For a sample of enumerators, the Census Bureau will examine whether the use of a test of noncognitive characteristics can help to predict enumerator effectiveness. A sample of local census offices will administer a noncognitive test to a sample of their new hires. (This test will not be used in the selection

Suggested Citation:"Research Experiments." National Research Council. 2000. Designing the 2010 Census: First Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9802.
×
  • process.) The test to be used is the Wonderlic Employee Inventory, which measures long-term job commitment, trustworthiness, courteous job performance, conscientiousness, emotional maturity, safe job performance, and freedom from disruptive substance abuse. The goal of the experiment is to see if there is an association between the results from this test and employee turnover and work performance. The test would have to be judged to be job-relevant, have no between-group differences, and be criterion-related to be used in the selection process.

  • Alternative Questionnaire Study For a sample of households, the Census Bureau will examine whether alternative questionnaires with changes to one of the following— the overall appearance of the census questionnaire, the order and wording of questions on ethnicity and race, and the wording of the question concerning the definition of household residence—result in improved levels or accuracy of response.

  • Study of Attitudes on Confidentiality and the Use of Social Security Numbers For a sample of households, the Census Bureau will examine attitudes toward requests for social security numbers on census forms and notification of their use in linking to administrative records to support statistical analyses. There are two components. The SSN-notification component examines the effects of a request for social security numbers or a public notification on mail or item response rates in the decennial census. There will be different panels, depending on whether the household receives the long form or the short form, whether there is a request for social security numbers, whether the request is for the entire household or only for the person completing the form, and whether there is a notification that mentions the Census Bureau 's possible use of statistical data from other federal agencies. The SPA 2000 component is a household survey that assesses the public 's attitudes toward the Census Bureau's possible use of administrative records. The objective is to measure the public's opinion (of the federal government and the Census Bureau) of expanding uses of administrative records and of the Census Bureau 's possible interest in collecting social security numbers in the future. SPA 2000 will collect data twice, once before advertising and once immediately after April 1, 2000.

Suggested Citation:"Research Experiments." National Research Council. 2000. Designing the 2010 Census: First Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9802.
×

THE AREX 2000 STUDY

Description

The Administrative Records Experiment in the 2000 Census (AREX 2000) is an experiment that will test, in a limited set of sites (Baltimore City, Baltimore County, and three counties in Colorado), the use of administrative records as a data collection method. The goal of this test is to better understand the potential for merged lists of administrative records to deliver content and geographic detail, especially sufficient coverage, that meet reapportionment and redistricting requirements. Two methods, which differ by whether individuals are linked to households, will be used in the experiment to generate block-level population counts. The “top-down” approach merges a number of administrative records files, unduplicates them, and then assigns individuals block-level codes using the MAF/TIGER (Master Address File/Topologically Integrated Geographic Encoding and Referencing) system. The “bottom-up” approach matches residences found on the merged administrative records list with those on the census master address file, resolving any inconsistencies. This approach therefore forms households. The resulting population counts, disaggregated by race, Hispanic origin, and gender, will be compared with census counts.

Four evaluations are planned: (1) comparison of population counts (disaggregated demographically) to census counts; (2) analysis of the components of the implementation of the experiment, including effects on coverage and costs; (3) comparison of household level information for the “bottom-up” method; and (4) assessment of the feasibility of the use of administrative records for the purposes of delivering geographically disaggregated count and content information that meets reapportionment and redistricting requirements. Several eventual uses of merged lists of administrative records are under consideration: nonresponse follow-up, reducing differential undercoverage, imputation for item nonresponse, address list improvement, and special procedures to target individual groups. For nonresponse follow-up, the question to be answered is whether there are advantages to the use of (pre-follow-up) census data and AREX in comparison with the current use of (pre-follow-up) census data combined with nonresponse follow-up data.

Suggested Citation:"Research Experiments." National Research Council. 2000. Designing the 2010 Census: First Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9802.
×
Analysis and Recommendation

The panel questions whether the goals for AREX 2000 are consistent with the current plans. Specifically, how will this experiment inform the Census Bureau sufficiently to support more refined testing of administrative records (with respect to any of the above purposes, e.g., for nonresponse follow-up or for reducing differential undercoverage) for 2010? Clearly, assessing coverage of the merged list, possibly in conjunction with the census, is the first crucial step toward any of the possible uses of administrative records. A two-way match of the list of census enumerations and the merged administrative records list is a very important addition to this experiment.

Recommendation: The Census Bureau should modify the plans for AREX 2000 to include a match of the merged administrative recordslist with the 2000 census households. Costs of this new activity could be controlled through the use of field follow-up of unresolved matches on a sample basis.

Without a match, one could imagine a situation in which administrative records were considered deficient due to a lack of coverage, but matching (with additional later work) could have shown that missing people were generally included on an administrative records list that had been omitted in the test. Inclusion of that administrative records list would have eliminated the problem. More generally, a match might indicate why the merged list was inferior. Conversely, one could imagine a situation in which administrative records had excellent coverage in comparison with the census, but the process used to merge the lists was deficient, or the lists were of poor quality, so that a positive assessment of coverage was erroneous and the merged list was not nearly as useful as it seemed.

Furthermore, without a match, how will AREX 2000 adequately demonstrate the value of a merged list of administrative records for nonresponse follow-up? Distributions of information on characteristics will not indicate whether the answers are of high enough accuracy to be used for this purpose. Matching the census and AREX 2000 files will provide important additional information on the ways in which administrative records data are inferior or superior to census data, which could be easily missed by comparison of aggregates. If costs are a primary concern, sampling could be used to limit the costs of field reconciliation of differences. In ACE

Suggested Citation:"Research Experiments." National Research Council. 2000. Designing the 2010 Census: First Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9802.
×

block clusters, one could also use matching to determine whether individuals that are enumerated by the ACE survey and missed by the census appear on administrative records lists, which would help to assess whether administrative records could be used to reduce census undercoverage. This matching is a first step toward measurement of correlation bias and is also a preliminary step toward implementation of triple system estimation.

In addition to its recommendation for matching, the panel has six other proposals for AREX 2000. First, the Census Bureau should investigate whether AREX 2000 could be used to assess the possibility of using administrative records, in nonresponse follow-up, to supply census data only for the individuals who have a good match, i.e., those cases with a high likelihood of matching as derived from the Fellegi-Sunter algorithm, reserving field enumeration for the remaining, more difficult cases. The panel has not examined the overall feasibility of this approach; if it can be done with only a modest increase in costs, the Census Bureau should consider it.

Second, given the possibly limited utility of the current combination of administrative records used in AREX 2000 to cover the non-elderly poor, the Census Bureau should explore the process of obtaining approval from each state (and some counties) to acquire their food stamp records, as well as the process of obtaining Medicaid files. While obtaining 50 separate approvals from states for food stamp records may be time consuming, the great majority of them might be straightforward. Moreover, the immediate concern is to acquire files for the test areas, which lie in only two states. It would be extremely useful if the Census Bureau could acquire food stamp and Medicaid files for the test areas to assess the value of these lists for these purposes. The Census Bureau should give high priority to getting approval for the acquisition of these files for the test areas (which has been generally feasible in the past). (Besides providing information on the non-elderly poor, both food stamp and Medicaid files are the two obvious candidate record systems for augmenting Internal Revenue Service [IRS] data files for improving coverage.)

Third, given that facilitating acquisition of administrative records through legislative or legal change can sometimes take a considerable amount of time, the Census Bureau should assign a relatively high priority to beginning these activities as soon as possible. This is especially important if food stamp records are to be used.

Fourth, the second evaluation that is planned—analyzing components of the implementation of the experiment, including effect on coverage and

Suggested Citation:"Research Experiments." National Research Council. 2000. Designing the 2010 Census: First Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9802.
×

costs—does not appear to be fully thought out. It has not yet been fully described to the panel, and questions remain as to exactly how the data files are going to be merged, the effects resulting from the current choice of data files, how costs will be measured, etc. There is a concern that AREX 2000 is too specific a test on which to base what might be a stop-go decision with respect to the use of administrative records in the 2010 census. Some of the specificity of this test may stem from what appear to be limited resources allocated to it. The panel is concerned that AREX 2000 might not have adequate staff support. Given the importance and complexity of this experiment and given the additional suggestions made here, the panel believes that additional resources are needed. Therefore, the panel suggests that the Census Bureau consider allocating additional resources to this experiment and that it plan additional tests of administrative records for these purposes to learn as much as possible about the choices that are related to better or worse outcomes.

Fifth, the panel is concerned that the above uses of census data often require information on race and ethnicity. The panel strongly recommends that the Census Bureau, very likely in conjunction with other federal agencies and in conjunction with academia and research organizations through directed research, examine how to improve the race/ethnicity data available from administrative records and what the social and political implications would be as a result of the addition of this information to administrative records.

Sixth, these applications obviously require the matching and unduplication of records both from within a single administrative records system and across systems. Therefore, the Census Bureau should include in AREX 2000 an assessment of the accuracy of the matching and unduplicating operations used and an assessment of which data from duplicate records are the most accurate.

Suggested Citation:"Research Experiments." National Research Council. 2000. Designing the 2010 Census: First Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9802.
×
Page 19
Suggested Citation:"Research Experiments." National Research Council. 2000. Designing the 2010 Census: First Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9802.
×
Page 20
Suggested Citation:"Research Experiments." National Research Council. 2000. Designing the 2010 Census: First Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9802.
×
Page 21
Suggested Citation:"Research Experiments." National Research Council. 2000. Designing the 2010 Census: First Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9802.
×
Page 22
Suggested Citation:"Research Experiments." National Research Council. 2000. Designing the 2010 Census: First Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9802.
×
Page 23
Suggested Citation:"Research Experiments." National Research Council. 2000. Designing the 2010 Census: First Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9802.
×
Page 24
Next: Auxiliary Data Systems »
Designing the 2010 Census: First Interim Report Get This Book
×
Buy Paperback | $29.00 Buy Ebook | $23.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The Panel on Research on Future Census Methods was formed to examine alternative designs for the 2010 census and to assist the Census Bureau in planning tests and analyses to help assess and compare the advantages and disadvantages of them. Designing the 2010 Census: First Interim Report, examines whether the auxiliary information that is planned to be collected (and retained) during the 2000 census could be augmented to help guide the Census Bureau in its assessment of alternative designs for the 2010 census.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!