This chapter summarizes the panel's most recent findings concerning the six new processes proposed for introduction in the 2000 census and described in Chapter 1: (1) master address file development, (2) efforts to enhance the mail return rate, (3) availability of "Be Counted" forms in public places, (4) sampling for nonresponse follow-up, (5) integrated coverage measurement, and (6) statistical estimation.
Master Address File Development
In its second interim report (National Research Council, 1997b), the panel reiterated the importance of a high-quality address list for the 2000 census. One way to improve the quality of the 2000 census relative to the 1990 census while reducing costs is to make the master address file (MAF) more complete than it was in 1990. The collection of high-quality data for small geographic areas is greatly facilitated through the use of an address list of uniformly high quality for the entire nation. A poor address list can contribute greatly to increased rates and poor estimates of the rates of census omissions and erroneous enumerations, including duplicates. Also, to increase the level of confidence in decennial census procedures, local stakeholders—officials,business leaders, interest group representatives—must be convinced that the address lists for the 2000 census are better than the 1990 lists for their areas.
To improve the MAF, the Census Bureau has made use of updates from the U.S. Postal Service and has solicited input from local officials. The Census Bureau recognizes, however, that these efforts have not been
as effective as initially hoped. Information received from the Postal Service has not been timely, and a greater proportion of the country remains without city-style postal addresses than was anticipated. At the same time, it has become clear that a substantial number of local authorities lack the resources to provide timely updates of address information of adequate quality in a usable format (i.e., referenced to the correct location on census block boundary maps). Therefore, the Census Bureau has decided that it will implement a nationwide check of addresses immediately prior to the 2000 census, although this additional check will be costly. The panel strongly endorses this change in plans.
Efforts to Enhance the Mail Return Rate
The Census Bureau conducted systematic research early in the 1990s to identify procedures that would increase the proportion of households that return their census form by mail (see National Research Council, 1994). The research indicated that mail response rates would likely increase, relative to 1990, as a result of (1) improvements in the design of census envelopes and forms, (2) the use of prenotification letters, (3) clear information about the mandatory nature of the census, and (4) sending nonrespondents a reminder notice and then a replacement questionnaire. Tests and other research indicate that the resulting reduction in the need for nonresponse follow-up will more than offset the increase in census costs from these changes.
In the development work and testing carried out before the dress rehearsal, replacement forms had been sent only to households that did not return the original form by a specified date. In developing operations for the 2000 census, the Census Bureau has learned that the scale of the decennial census and timing constraints will not permit the mailing of replacement forms only to nonrespondent households (a targeted replacement questionnaire). Instead, replacement questionnaires must be mailed to all households on the MAF (a blanket replacement questionnaire). This nontargeted mailing of replacement questionnaires to all households was tested for the first time in the 1998 census dress rehearsal. While early indications were that the second mailout significantly increased response rates in the test,1 there was also a considerable amount of duplication. Therefore, the panel remains concerned that mailing replacement forms to all households could generate millions of duplicate submissions in 2000, which the Bureau must identify and exclude, as well as reduce
cooperation by creating an impression of wastefulness and by increasing respondent burden. The generation of millions of duplicate forms will likely result in delays for later census operations and additional errors. It is therefore critical that thorough evaluation of this procedure follow the 1998 census dress rehearsal on a schedule that allows the findings to influence plans for the 2000 census.2
Sampling for Nonresponse Follow-up
In its first and second interim reports (National Research Council, 1996, 1997b), the panel endorsed using sampling, combined with statistical estimation, to efficiently and effectively collect information on households that do not respond by mail or to other opportunities for enumeration. The panel expects that a properly designed and well-executed sampling plan for nonresponse follow-up can save more than $100 million (assuming a sampling rate of 75 percent, see Brown et al., 1998) and possibly increase the quality of the census data collected by enumerators.
The likely improvement in quality has both direct and indirect aspects. Directly, by reducing the total workload, sampling for nonresponse follow-up will allow for improvements in the control and management of field operations that in turn would lead to an increase in the quality of the census data collected by enumerators. Indirectly, the nonresponse follow-up interviews of a sample of nonrespondents can be completed in a more timely fashion than follow-up of all nonrespondents, which will lead to improvements in quality in the integrated coverage measurement operation.
It is important to point out that given the overall sampling rate of roughly 70 percent (depending on the 2000 mail-return rate), the benefits gained through greater control and management of field operations and the completion of the field work more expeditiously are substantially limited in comparison with what would be gained with a more typical (lower) rate of sampling. However, as argued in Chapter 3, the panel agrees with the Census Bureau's more conservative approach to this planned, initial use of sampling for nonresponse follow-up in the decennial census. Given the sampling rate, it is difficult to argue for large, simultaneous benefits both in time saved and in field control and management.
In its second interim report (National Research Council, 1997b), the panel strongly endorsed the Census Bureau's switch to an approach re
ferred to as direct sampling, in which the mailout/mailback phase is followed directly by nonresponse follow-up on a sample basis, with no intermediate period of 100 percent nonresponse follow-up. The panel expressed concern that the sampling rates proposed at that time would result in coefficients of variation that were too high in areas with high primary response rates, in comparison with other areas. The Census Bureau has since modified the sampling plan for nonresponse follow-up so that census tracts with primary response rates of more than 85 percent will be sampled at a rate of 1 in 3. That rate, combined with the planned sampling rates for areas with lower initial response rates, means that most areas will have similar levels of sampling error. Should any tracts achieve an initial response rate of more than 95 percent, they will have somewhat lower levels of sampling error than the rest of the country.
The panel strongly endorses this most recent change in sampling rates for nonresponse follow-up. The overall nonresponse follow-up sampling plan now is more efficient, and the field work will be easier to control. Given no unanticipated operational problems, all census tracts will be enumerated with high reliability with respect to nonresponse follow-up.
One concern that has been expressed about the plan to use sampling for nonresponse follow-up is that it could lead to results with relatively high levels of sampling error for areas with small populations. While the panel addressed this issue in its first and second interim reports, we now provide additional detail. It is also useful to point out that this concern has been greatly alleviated because the sampling rate will be a minimum of 1 in 3 for all census tracts and will likely be considerably higher in most of the country. Thus, assuming a mail response rate of 65 percent, the Census Bureau will be following up approximately 25 million nonresponding households in 6 weeks.
The Census Bureau recognizes that the use of sampling for nonresponse follow-up will introduce sampling errors in essentially each census block. The estimation procedures to be used in conjunction with sampling for nonresponse follow-up use information from sampled households to project results for nonsampled households from the same block, to the extent feasible. Inevitably, those who are included in the sample will differ from those who are not, so that the results obtained from the sample data will include some error.3 However, the scientific sampling procedure that the Census Bureau proposes, known as stratified probability sampling, yields counts with three important properties
with regard to these errors. First, the error in any one block is not systematic or predictable in its direction. In particular, the direction of the sampling error cannot be manipulated to reach a predetermined outcome in a given area. Second, the range of possible estimates derived from a sample can be determined reliably for any level of geography. Third, the relative size of these sampling errors decreases as the size of the population in the area increases.
Furthermore, the results from using sampling in this way could lead to more accurate counts (depending on the size of the area) because errors from the amount of proxy enumeration in the census will probably be reduced and therefore the data that are collected could be of higher quality. In this context, it is important to realize that every recent decennial census has had a considerable amount of proxy response. The less intense and uneven quality of past efforts to collect data from initial nonrespondents with 100 percent nonresponse follow-up resulted in the collection of poorer quality data, particularly because of relatively high rates for proxy enumeration4 than would be achieved with a sample-based nonresponse follow-up. With sampling for nonresponse follow-up, the extent of proxy response could be considerably reduced, at the cost of adding sampling variability, which diminishes quickly with increasing population size.
Consider an area 25 times the size of a census tract, i.e., roughly 40,000 housing units,5 a level of census geography well below that of a congressional district. (In this calculation we ignore the likely modest benefits from stratification that the Census Bureau is planning on using, which would make the argument stronger.) To take the worst case with respect to the nonresponse sampling rate, assume this area has a mailback rate of 85 percent (and therefore a nonresponse sampling rate of 1 in 3), and that there are a mean number of 2.5 people per housing unit (100,000 people in all) and a standard deviation of 1.5 people per housing unit. The standard error of the average number of people per housing unit due to sampling for nonresponse follow-up would be approximately .027, and the standard error on the total population would be 162. Now assume that nonresponse follow-up would miss (or overcount) 500 of the 15,000 people to be followed up, or 3.33 percent. Then sampling for nonresponse
follow-up would have lower mean square error (and be preferred) if it missed less than 3.15 percent of the 5,000 people followed up—if it missed less than 158 out of 5,000 (rather than 167 without sampling). Since the workload is reduced from 6,000 to 2,000 households, this seems quite feasible.
For another example, assume that the mail return rate is 65 percent, the standard error for the total population is 112, and nonresponse follow-up again has an error of 500, this time in counting the 35,000 nonresponding people for a miss rate of 1.43 percent. Sampling for nonresponse follow-up, in counting 25,000 people, would have lower mean square error if it missed (less than or equal to) 348 of these 25,000 for a miss rate of 1.39 percent (rather than 357 without sampling). This again seems like a feasible gain given the reduction in workload from 14,000 to 10,000 households. Finally, we point out that it is certainly arguable that rushing nonresponse follow-up could increase the error rate since the average rate for closeout and last-resort enumeration for the 1990 census was 3.5 percent, and the erroneous enumeration rate for these cases was around 40 percent (Ericksen et al., 1991).
We now add to the above argument the possible reduction in the number of movers and errors caused by movers through the more rapid completion of nonresponse follow-up through use of sampling. On the basis of this evidence and reasoning, the panel believes that the results from sampling for nonresponse follow-up will be of equal or better quality than would result from the continuation of the procedures used in 1990 when used for important purposes, such as delineating congressional districts, and that for other uses of census data, sampling for nonresponse follow-up at the very least approximately replicates what would be obtained, in terms of data quality, with 100 percent follow-up. It is useful to point out that many problems will be common to follow-up regardless of whether or not sampling is used. People's attitudes towards being enumerated, their work schedule, and ease of access to residences are the same, whether sampling is used or not.
Finally, there are some valid concerns about the implementation of various administrative operations with this first application of such a large, time-constrained sample survey. One worry is what difficulties may result from the constraint to produce a "one-number" census, and whether this may result in time-abbreviated nonresponse follow-up or integrated coverage measurement. The dress rehearsal is a key for understanding what implementation issues need to be addressed before 2000.
Concern has also been expressed that sampling for nonresponse follow-up presents an opportunity for political manipulation. Such manipulation is simply impossible. In addition to the enormous complexity of any manipulation, the constraint that the census methodology be
prespecified, a requirement that the Census Bureau is strictly observing, would make identification of any manipulation easy. At this time, many of the details concerning sample design and estimation are currently fixed for the 2000 census; for a few important issues yet to be resolved, their resolution depends primarily on results from the 1998 census dress rehearsal. Also, the estimation for sampling for nonresponse follow-up is simple and has been prespecified (except for the treatment of late responses, for which a procedure has been suggested and will be decided on well before the census).
To manipulate sampling for nonresponse follow-up at the design stage, someone at the Census Bureau would have to know which of the more than 30 million mail nonrespondent households had more or fewer residents and then manipulate the computer-generated random-number-based selection of households for follow-up so that the households in a particular area that were included in the sample were on average larger or smaller. This is, of course, unimaginable.
Furthermore, sampling does not provide additional opportunities for manipulation in the field. Without sampling an enumerator would visit the nonrespondents on a certain block. With sampling the same enumerator would simply visit a subset of the same nonrespondents. An enumerator would have no idea whether sampling was occurring on a particular block (sampling would not be used on integrated coverage measurement blocks) since the enumerator would not know whether an address was skipped because it was not in the sample or because a response had already been received by mail. Finally, it is important to state in response to this concern that the Census Bureau, except for a handful of top management positions, is staffed by career civil servants who have a long-standing reputation for integrity and professionalism.
Integrated Coverage Measurement
Both because the master address list, despite the Census Bureau's best efforts, is incomplete and because individuals who live in otherwise enumerated households are at times missed, all decennial censuses fail to count everyone. In addition, due to people moving, having more than one residence, and confusion as to the census definition of residence, many individuals are counted more than once or are counted erroneously. The net effect of undercoverage and overcoverage is referred to as net undercoverage. This net undercoverage affects some groups and geographic areas more than others—that is, the census has differential (net) undercoverage. For example, for 1990 the net undercount of black males aged 25 to 54 was measured by demographic analysis to have been around 12 percent (see Robinson et al., 1993), compared with a net under-
count of non-black males 25 to 54 that was estimated to be around 3 percent. In 1990, again as estimated by demographic analysis for blacks, the largest undercounted group for females was children 0 to 9 years old (Robinson et al., 1993), for whom the undercount was around 8.0 percent. Since the differential net undercoverage has persisted for some demographic groups, especially blacks, over several censuses, these groups have been consistently underrepresented in census figures, resulting in possible misallocations of political representation and government funds.6
The decennial censuses from 1950 through 1990 all made use of various evaluation programs to assess the extent of gross and net census undercoverage and its causes. (For a description of these programs and their findings, see National Research Council, 1985; Hogan, 1992.) The only methodology that has been shown to be feasible to measure the amount of differential undercoverage at relatively low levels of geographic aggregation is a large-scale post-enumeration survey with dual-system estimation. (This is the approach planned for use in the 2000 census and referred to in that context as integrated coverage measurement.)
The basic statistical model represented by the term dual-system estimation (ignoring some complications) is as follows. A first enumeration, the census, is carried out, followed by a second enumeration, the post-enumeration survey. Those enumerated by both processes are identified through matching the two lists of those enumerated each time. A key assumption used in this model is that the probability of enumeration in the second process given enumeration in the first process is identical to the probability of enumeration in the second process given a miss in the first process. This is equivalent to the assumption that the events of enumeration in the first and second processes are statistically independent. This assumed identity provides a basis for estimating the number that were not enumerated with either process, and therefore the total population.7 This method was originally proposed by Sekar and Deming
(1949), who coined the term dual-system estimation (DSE). In addition to the fact that DSE depends on the important assumption that the events of inclusion in the census and in the post-enumeration survey are independent, it is also important that there be an appropriate random sampling scheme that chooses blocks for inclusion in the post-enumeration survey so that inferences from the sampled blocks extend to the unsampled ones. (The Census Bureau goes to great lengths to ensure this.) Finally, since the probabilities of inclusion in the census and in the post-enumeration survey are known to depend on various characteristics of members of the population, post-stratification is typically used to produce subgroups for which these probabilities are more homogeneous (see below).
DSE was the methodology used in 1980 and 1990 to join information from the post-enumeration survey and the census to measure census undercoverage. In the 1990 census, a post-enumeration survey of roughly 160,000 housing units collected information to measure the amount of under- and overcoverage in the census, along with other characteristics of those persons who were missed or erroneously enumerated.
In 1980 and 1990 the problem of differential undercoverage was addressed exclusively in the official counts through the use of coverage improvement programs. These programs (e.g., the nonhousehold sources check8) were used to try to increase the coverage of historically undercounted groups. Not only were many of these programs generally unsuccessful, they tended to be expensive, costing as much as $76 (in 1980 dollars) per added person in the 1980 census (see National Research Council, 1985). Furthermore, Ericksen et al. (1991) and Griffin and Moriarity (1992) showed that in the 1990 census these programs often added a substantial number of erroneous enumerations. (There was a direct relationship between the amount of erroneous additions and the distance from census day.) Therefore, the use of coverage improvement programs alone is unlikely to be effective in greatly reducing differential undercoverage.
A successful application of integrated coverage measurement has the advantage of greatly reducing the need for expensive and ineffective coverage improvement programs. Furthermore, as mentioned above, the elimination of unsuccessful coverage improvement programs reduces the rate of erroneous enumerations, which can reduce the error in integrated coverage measurement, and it may provide more time for integrated coverage measurement.
Finally, as we have noted, sampling for nonresponse follow-up has the promise of being able to conclude nonresponse follow-up more expeditiously. This, in turn, would permit the integrated coverage measurement survey interviewers to begin work earlier, which will reduce the number of individuals who have moved since the census, which will reduce the number of erroneous enumerations due to people being enumerated a second time at an address other than the census day address. Thus, overall, sampling for nonresponse follow-up will increase the quality of the information collected in integrated coverage measurement, which will facilitate matching, one of the major concerns arising with the use of integrated coverage measurement.
Response to Arguments Against Integrated Coverage Measurement
The panel is well aware of the controversy involving the proposed use of sampling in the 2000 census for nonresponse follow-up and for integrated coverage measurement. This final report presents an opportunity for the panel to comment on the controversy. The public debate surrounding sampling in the census has often confused the use of sampling for nonresponse follow-up and sampling as part of integrated coverage measurement. As argued above, the two activities interact, but they are different applications of sampling and target different problems in the census. This section first notes some arguments that have been given against sampling in the census in general and against sampling for nonresponse follow-up in particular. This is followed by an introduction to the leading technical criticisms of integrated coverage measurement ("adjustment" in 1990 terminology), which are treated in more depth in Chapter 4.
One argument against sampling for nonresponse follow-up concerns the possible manipulation of the results. We address the concern above; it is completely unfounded. Since sampling for nonreponse follow-up is a routine application of standard sampling techniques and practices, it is generally not subjected to technical criticism aside from the specific design and possible operational complications, except as a criticism of the addition of sampling variability to counts for areas, as discussed above (and
possibly as a criticism of the estimation used in conjunction with late returns). However, a nontechnical criticism has been expressed that sampling in the census, especially for nonresponse follow-up, ignores the constitutional requirement for a complete enumeration of the nation's population. The panel recognizes that a decennial census is constitutionally linked to the apportionment of political representation among the states, legally linked to the distribution of many types of federal funds, and used as a basis for forming congressional districts within states to conform to constitutional requirements. These links are the root of legal and constitutional debates about the census. They are rightly settled by the courts. The panel makes no attempt to anticipate judicial rulings on what restrictions on methods, if any, are implied by the Constitution and relevant legislation. The panel takes as its premise that the most accurate counts and shares are sought at the various levels for which they are needed in a cost-effective manner and that systematic and persistent errors are particularly problematic.
The panel notes, though, that census questionnaires or enumerators are sent to all the households with addresses on the MAF, which have an opportunity to respond. In addition, there are a variety of respondent-friendly opportunities for enumeration by telephone and mail. Sampling comes into play only for housing units from which no response is obtained from the initial mailout (or visit in nonmailout/mailback areas). We reiterate our conclusion that sampling for nonresponse follow-up is an excellent technical way to control census costs and potentially improve quality.
Chapter 4 focuses on technical arguments concerning the use of sampling in the census as part of integrated coverage measurement. The following issues are addressed there: (1) matching error and the bias from imputation of match status for unresolved cases, (2) unmodeled heterogeneity in census undercoverage for lower levels of geographic aggregation (violation of the so-called synthetic assumption), and (3) correlation bias, focusing on the heterogeneity of probabilities of enumeration of individuals in the census and in the integrated coverage measurement survey. Chapter 4 examines the studies of matching error, explores the synthetic assumption, and discusses the problem of correlation bias.
Finally, with respect to court decisions on sampling in the census, the panel is aware that there have been two recent decisions against the use of sampling to produce the counts that will be used to reapportion the U.S. House of Representatives after the 2000 census. If these decisions are upheld by the U.S. Supreme Court,9 it will not be possible to use sam
pling for nonresponse follow-up as part of the 2000 census. However, "adjusted" counts can be produced in addition to "traditional" census counts. Therefore, while the restriction not to use sampling for apportionment would prohibit the use of adjusted counts for apportionment, it would still be possible to benefit from integrated coverage measurement for many other important uses of census data, such as allocation of federal and state funds to states and localities. Also, for these purposes, the adjusted counts are not needed by December 31, 2000. Of course, the situation would not be the same as originally planned for integrated coverage measurement, in which there is one set of official estimates. In this case there may be two sets of estimates—one that makes no use of sampling and another that uses the integrated coverage measurement estimates of all areas.
The importance of the many uses of small-area census data argues strongly for retention of the current plans for the integrated coverage measurement survey as a sample of 750,000 housing units. This sample size may enhance public acceptance of census results since it permits making estimates for states directly without the need to use information from other states. In turn, estimates for substate areas, which are used for many program administration, planning, and research purposes, can be based on information specific to each state. Therefore, the panel strongly supports the large-scale integrated coverage measurement survey as planned for the 2000 census.
The panel examined one statistical estimation issue:10 how to assign the persons added through the integrated coverage measurement survey to households when all that is known about them are their demographic characteristics and some geographic information at a relatively high level.11 If information on census undercoverage had been used for offi
made public while this report was in the last stages of editing and final production. No changes were made to the report as a result of this decision other than the addition of portions of the preface, this footnote, and similar footnotes in Chapter 1 and the executive summary.
The estimation issue of what to do about the delivery of late mail returns was presented to the panel too late to be addressed in this report. However, this is an important issue that should be examined for potential bias.
This problem is not faced in sampling for nonresponse follow-up since there the imputation is done on a household basis, whereas in integrated coverage measurement the characteristics are determined on an individual basis. However, in integrated coverage measurement, the people counted in the post-enumeration survey but missed in the census do have a household affiliation based on the post-enumeration survey interview, but this information is not used in providing household characteristics of the undercounted population.
cial purposes in 1990, there was no statistically satisfactory way of assigning household characteristics to the added persons. Files for people included through adjustment would have been added to the census data files by imputing individuals from the appropriate poststrata. These individuals' housing types would have been listed as special quarters, that is, they would not have been placed in distinct households.
For the 2000 census the Census Bureau initiated a research project early in the 1990s to determine if something more effective could be done. Specifically, the Bureau examined the creation of a so-called transparent file in which all of the additions obtained through use of integrated coverage measurement would be incorporated into the distribution of households and thus be transparent to data users.12 The Bureau's initial plan would have altered the relative weighting of larger versus smaller households in order to obtain the desired counts of persons by demographic characteristics. Thus for example, three-person households with certain characteristics might be increased while two-person households with otherwise similar characteristics were reduced.
The initial procedure presented by the Census Bureau raised a concern for the panel. The panel's views were informed by other research in this general area (see Zaslavsky, 1988; Zanutto and Zaslavsky, 1996). However, no other approach had been demonstrated to be feasible in a large-scale production setting. It is our understanding that recent advances to the initial procedure may have addressed the panel's concern. But because the Census Bureau's decision on whether to integrate this into the production operation had to be made on the basis of the performance of the initial procedure, the panel understands the decision to essentially repeat the method of the 1990 census, i.e., persons added through use of integrated coverage measurement will be assigned for their household, to the special quarters category. Especially given the limited time for evaluation of statistical models, this seems to be a reasonable decision for 2000. However, the panel strongly supports the Bureau's intention to produce the transparent file at a later date.