This final report of the Panel on Alternative Census Methodologies provides an assessment of the Census Bureau's plans for the 2000 census as of the time of the 1998 census dress rehearsal. It examines changes in census plans following, and to a modest extent in reaction to, the panel's second interim report, regarding the use of sampling for nonresponse follow-up, construction of the master address file, use of multiple response modes and respondent-friendly questionnaires, and the use of administrative records. It also describes evaluation plans for the census dress rehearsal and plans for data collection and experimentation during the 2000 census. Most of the results from the dress rehearsal were not yet available to the panel, so this report does not offer any suggested changes to 2000 census plans in response to the dress rehearsal.
The Census Bureau plans to introduce several new features for the 2000 census: enhanced procedures for developing the master address list to which census forms are mailed; modern mail survey techniques to enhance response, including revised forms and multiple mail contacts; making census forms available in a variety of public places; use of random sampling during nonresponse follow-up; an expanded coverage measurement survey of 750,000 housing units to obtain information about census coverage rates; and incorporation of the results of the nonresponse follow-up and the coverage measurement survey into the census counts using statistical estimation procedures.1
New Census Plans
Address List and Mail Procedures
Of all the new census procedures, producing a nearly complete address list and obtaining a high mail response rate remain the cornerstone of a high-quality census. The other procedures are designed to complement these two main steps by ensuring that high quality is maintained even if address list development and the mail return process fall short of perfection.
During the 1990s the Census Bureau conducted an address list improvement program that included making use of U.S. Postal Service files and input from local officials. Because these efforts were not sufficient, the Bureau has instituted plans for a nationwide field check of addresses prior to the 2000 census. The panel strongly endorses these newly instituted procedures.
Through research early in the 1990s, the Census Bureau also determined that provision of a replacement form targeted to mail nonrespondents would likely yield substantial improvements in mail response rates. The panel endorsed both the process used to evaluate that research and the subsequent changes to the form and to the techniques used to encourage mail response. However, all research regarding the mailing of a replacement form was specific only for households that failed to respond to the initial mailing—and the Census Bureau subsequently determined that it could not implement such a targeted replacement operation in the time available. Therefore, the 1998 census dress rehearsal tested the process of sending a replacement form to all households. The panel believes that it is critical to measure the effects of the use of an untargeted replacement form, especially its effects on respondent cooperation and the effectiveness of the unduplication process used for households that return both forms. An analysis of dress rehearsal results should be performed to help decide whether to use an untargeted replacement form in the 2000 census.
Making forms available at public places was successfully tested in 1995. If the dress rehearsal results confirm that this program is beneficial (i.e., if the number of duplicate submissions of census questionnaires is considered small or if the unduplication process is considered to be of high enough quality), the panel believes this procedure will afford some gains in response at relatively little cost. In addition, the concept may have public relations benefits, as suggested in work with focus groups.
Sampling for Nonresponse Follow-Up
The panel has concluded that a properly designed and well-executed sampling plan for field follow-up of census mail nonrespondents will save over $100 million (assuming an overall sampling rate of 75 percent). Furthermore, sampling for nonresponse follow-up will reduce the Census Bureau's total workload, which will permit improvements in the control and management of field operations, and will allow more complete follow-up of difficult cases that could lead to an increase in the quality of the census data collected by enumerators. In addition, nonresponse follow-up interviews could be completed in a more timely fashion, which would lead to improvements in quality when the planned integrated coverage measurement operation is used.
Of course, sampling for nonresponse follow-up will add sampling variability to census counts. However, imprecision in the census counts at low levels of geographic aggregation due to added variance through use of sampling for nonresponse follow-up will not cause any systematic biases, because under sampling for nonresponse follow-up only characteristics of people found in a tract contribute to the estimates for that tract. Furthermore, the relative amount of variance due to sampling decreases as the population of an area increases, and the amount of sampling variance can be measured statistically. As a result of this, and also due to the possibly higher quality of collected data resulting from the use of sampling, the panel believes that sampling for nonresponse follow-up will provide data of equal or better quality when used for congressional apportionment and that it will approximately replicate, at lower levels of aggregation, what would be obtained with 100 percent follow-up.
The panel further concludes that the prespecified nature of the sampling design for nonresponse follow-up and the fact that enumerators will not know whether households they are not visiting are mail respondents or nonrespondents that are not sampled ensures that sampling does not present a new opportunity for manipulation of census counts by enumerators.
Adjusting for Differential Undercoverage
Because the master address list is incomplete, because households are sometimes missed in listed housing units, and because individuals who live in otherwise enumerated households are at times missed, there is (gross) undercoverage in the decennial census. At the same time, people can be enumerated in multiple ways, possibly at several residences, so there is also (gross) overcoverage in the decennial census. The difference between undercoverage and overcoverage is referred to as net under-
coverage, and this is what is relevant to the allocation of political representation and public funds. This net undercoverage affects some groups more than others—that is, the census has differential (net) undercoverage—and there are demographic groups for which this differential undercoverage has persisted over several censuses.
The 1950 through 1990 decennial censuses all made use of various evaluation programs to assess the extent of gross and net census undercoverage and its causes. The only cost-effective methodology available for measuring the degree of differential undercoverage for subnational areas is a large-scale post-enumeration survey coupled with dual-system estimation.
Dual-system estimation, the methodology used in 1990 to join the information from the post-enumeration survey and the census to measure net census undercoverage, depends on several assumptions. After considering the criticisms related to the validity of, and the impact of departures from these assumptions, which have been used to argue against the use of integrated coverage measurement to produce official population counts, the panel concludes that the criticisms of this approach are not compelling reasons to halt plans to use integrated coverage measurement in 2000. If the Supreme Court prohibits use of integrated coverage measurement for apportionment, the panel still strongly supports a post-enumeration survey of the currently budgeted size of 750,000, for purposes other than apportionment.
The decennial census, as planned for 2000, will require estimation methods that were not needed or used in 1990. They include supplying imputed records as a result of sampling for nonresponse follow-up and for carrying the results of the dual-system estimation down to small areas. The Census Bureau has made an effort to keep these estimation methods as simple as possible. While the panel supports this decision for the 2000 census, it hopes that several more promising techniques can be adequately tested over the next decade and used in 2010 if shown to have advantages over the techniques used in 2000.
Experimentation in the 2000 Census
As the U.S. population continues to change in various ways, the best methods for enumerating the population also change. Therefore, a cycle of experimentation and data collection during a census followed by evaluation, further development, and experimentation and testing between censuses, is necessary for an effective census methodology. The decennial
census provides a unique opportunity for testing new methodologies because of its size and its general level of public acceptance and awareness. Data collection is essential to support later simulation studies and generally to understand what happened. Plans for research experimentation and data collection during the 2000 census are now being finalized. These plans begin the process of developing methodologies for 2010.
One concern with respect to testing as part of the decennial census is whether it is possible to predict 11 or 12 years in advance what methodologies might be effective. After all, technologies change at a rapid pace and the U.S. population itself is dynamic. Yet in previous censuses the Census Bureau staff carried out tests that were useful for advancing census methodology for the subsequent census. An important example is the introduction of a mailout/mailback census, which was tested in 1960 and introduced on a broad scale in 1970.
When unanticipated problems arise during the decennial census that require additional funds, field staff, or other resources, there is a natural tendency to draw off resources from research experimentation and data collection. Unfortunately, this may ''mortgage the future" of census taking for smaller, immediate benefits. Many issues involving the methodology to be used for the 2000 census would have been clarified if additional data collection had been incorporated into the 1990 census.
The panel strongly supports a renewal and modest expansion of the suggestion by a previous National Research Council panel for a master trace sample—that is, a sample of tracts in which essential information on all respondents with respect to enumeration is saved. Given the variety of innovations in the 2000 census, it would be extremely useful if the planned data management system could collect and save for research purposes a trace sample in, say, 100 tracts spread around the country. The trace sample would provide information about phases of data collection, which would be extremely valuable in guiding future methodological advances.
Planning for a decennial census begins at least 10 years before the first questionnaire is mailed. Some decisions must be made relatively early in the decade because of the need to procure equipment or because of limited testing opportunities. While the panel supports the fundamental decisions that the Census Bureau has made in planning for the 2000 census regarding sampling for nonresponse follow-up and integrated coverage measurement, various decisions that the Census Bureau was required to make early in the 1990s that cannot be changed until the 2010 census planning cycle—some supported by this panel, some not—need to be revisited for 2010. Two examples are whether the Census Bureau should in sampling for nonresponse follow-up—in combination with mail response—be obligated to directly enumerate at least 90 percent of the
households in each tract, and whether it would be more effective for the Census Bureau to make use of estimation methods that borrow information across states.
Time for Planning
All of the innovations planned for first use in the 2000 census, along with the methods used in 1990, have received their final test in the 1998 census dress rehearsal in Sacramento, California; Columbia, South Carolina, and its 11 surrounding counties; and Menominee County, Wisconsin. The evaluation studies based on the dress rehearsal will provide the final, important input to the decisions the Census Bureau must make as to the final plans for the 2000 census. The 37 evaluation studies are well designed, covering all aspects of census taking. The panel considers it important that they be completed in time to inform the decisions for 2000.
Finally, there is clearly a need for the Census Bureau to have sufficient time to plan whether the 2000 census may or may not use statistical sampling in either or both nonresponse follow-up and integrated coverage measurement. The fact that the Bureau is now less than 15 months from the start of the 2000 census without a firm decision on that issue presents an enormous problem to the Bureau in planning and implementing the complex process that is the U.S. decennial census.