The Bicentennial Census: New Directions for Methodology in 1990: 30th Anniversary Edition (2015)

1

Introduction

Periodic censuses of population are a long-established tradition in the United States of America, with roots going back to the earliest years of the colonial period. The royal colony of Virginia conducted the first census in North America in the early seventeenth century, and censuses of individual colonies were frequently attempted during the colonial era (Bureau of the Census, 1970b:3).

Political necessity led to the requirement for a periodic complete enumeration of the population in the new nation formed after the American Revolution. In the compromise between large and small states made at the 1787 Constitutional Convention, the delegates voted to provide equal representation for each state in the Senate and representation proportional to population in the House of Representatives; the population of each state was to be determined through a decennial census. Article 1, section 2, of the Constitution stipulates:

Although fundamental issues of the structure of government provided the motivation for the U.S. decennial census, the country’s leaders recognized from the beginning that the census could be a valuable source of in-

formation for many other purposes. James Madison noted in 1789 (Bureau of the Census, 1970b:4) that Congress:

The first census in 1790 asked the age, sex, and race of each resident. During the next 100 years, the census became firmly established as an important information resource. The centennial census in 1890 asked questions on more subjects than any census before or since, including 30 items on the basic population questionnaire, several housing inquiries, and special inquiries about decedents, inmates of almshouses and prisons, Indians on and off reservations, Civil War veterans and widows of veterans, and several categories of mentally and physically disabled people (Bureau of the Census, 1973b:74-91).

Work is now under way to plan for the nation’s bicentennial census of population and housing, scheduled to take place on April 1, 1990. Reflecting a long-standing tradition of improvement and modification to meet changing information needs and to take advantage of technological advances, census-taking in the twentieth century has come to differ in many important respects from census-taking in the nineteenth century. The 1990 census will undoubtedly incorporate the following features that have been introduced into modern U.S. censuses:

• As has been true since 1910, the 1990 census will be directed by a permanent organization, the U.S. Bureau of the Census, with an experienced, professional staff in charge of planning and supervising the operation;
• As in every census since 1940, statistical sampling methods will be used to obtain responses to many census items, so that a large volume of useful information can be gathered without placing the burden on every household of responding to all questions (the 1980 census asked 7 population and 12 housing items of all households, while about 20 percent of households were asked an additional 26 population and 20 housing questions);
• As in the 1970 and 1980 censuses, the U.S. Postal Service will deliver most of the census questionnaires, and households will be asked to mail their completed questionnaires to census offices. Enumerators will telephone or visit only those households that do not completely respond (95 percent of households were sent ques-

• tionnaires by mail in 1980 and 83 percent of them returned their questionnaires by mail);

• As has been true since 1960, large computers will be used to process the census returns in a relatively short span of time; in contrast, the 1890 census required almost a full decade to process, even with the introduction of punchcard machines to help the clerical work force; and
• As in every census since 1950, intensive effort will be devoted to evaluating the completeness of coverage of the total population and of important subgroups and geographic areas.

The 1990 census will undoubtedly differ as well from the most recent censuses in the United States. Most of the differences are likely to represent incremental improvements and modifications to tried and tested procedures: for example, mailout-mailback techniques may be extended to the remaining 5 percent of the population residing in sparsely settled rural areas that enumerators personally canvassed in 1980. But pressures are growing in this country, as in other Western nations, to address the problems of rising costs of traditional census practices on one hand and to satisfy expressed needs for greater accuracy in the numbers on the other. Consequently, exploration of changes in methods and techniques that mark a greater break with tradition is under way: for example, one proposal that has received much attention is the use of statistical techniques to adjust the field counts for deficiencies in the enumeration.

In the past, major changes in census methodology, such as the use of sampling for content and mailout-mailback enumeration, were often made on a small scale in one census and then more fully implemented in the next. The 1990 census will be part of a continuing evolution that may lead to a methodology in the twenty-first century that differs as significantly from current methodology as current methodology differs from that of the nineteenth century. This report is an attempt to assess the merits of proposed changes in the next decennial census that represent important departures from past practice and, specifically, to recommend concepts and procedures that should be assigned high priority in the Census Bureau’s research and testing program for the nation’s bicentennial census.

THE PLANNING CYCLE FOR 1990

Planning for the 1990 census officially began in fall 1983 with an appropriation for fiscal 1984. Well before that date, however, substantial work of direct relevance for 1990 was conducted. The 1980 decennial program included several experiments and post-enumeration studies designed to help plan improvements in methodology for subsequent censuses. Pre-

tests carried out in the late 1970s of concepts and procedures considered for 1980 also had results that are useful for 1990 census planning.

To the general public and many casual users of census data, it may appear that the Census Bureau has ample time to plan wisely for the 1990 census, given the start of the planning process more than 6 years prior to Census Day, April 1, 1990, and the foundation of research already completed in connection with prior censuses. In fact, as a review of the Census Bureau’s field test schedule for 1990 indicates, there are relatively few opportunities to test thoroughly changes or modifications to census procedures, particularly if the changes represent major departures from the past. Moreover, evaluation of the likely impact of important changes is hampered by the fact that pretests cannot adequately assess the effects of alternative procedures on public cooperation with the census—only tests conducted under census conditions, that is, experiments incorporated into an actual census as distinct from pretests, can fully address this important question.

The Census Bureau’s 1990 census testing program began in spring 1984 with tests of mailing list compilation methods in several localities around the country (Bureau of the Census, 1984b). Two large-scale pretests were fielded in spring 1985. Pretests will also be conducted in 1986 and 1987. Finally, the research and testing program will culminate in 1988 in “dress rehearsals” of the procedures planned for 1990. (See Chapter 3 for an overview of the Census Bureau’s 1990 census research program and for a description of the methodology used in 1980.)

This testing schedule means that the Census Bureau’s only opportunities for full field testing of new procedures and concepts for 1990 are the pretests in 1985, 1986, and 1987. The dress rehearsals, as the name implies, are not used to test new ideas but to run through the procedures the Census Bureau expects to follow in the decennial census itself. The only changes the Census Bureau anticipates from the dress rehearsals are corrections of problems encountered in the field, not innovations in census procedures at that late date. This schedule not only compresses into a few years the opportunities to test new methodology but also compresses the time available to evaluate the test results. The fact that tests are scheduled no more than a year apart makes it very difficult to complete the analysis of one set of tests in time to affect the design of the next set.

In addition to the compressed time schedule for testing and research, two other critical factors affect the ability of the Census Bureau to modify census methodology: staff and budget resources. The Census Bureau has long been known for the high quality and dedication of its technical staff. The current budget for research on decennial census methodology, particularly for research on the undercount, is large by the standards of earlier censuses. Nevertheless, no agency of government, particularly in the con-

strained world of the 1980s, can expect to have sufficient staff or resources to try out more than a few promising ideas and concepts. Pressures in the next few years to reduce the federal government’s large deficit may make it more than usually difficult to obtain adequate staff and funding to carry out a thorough research and testing program for 1990. Hence, it is critical to designing the best census for 1990 and to being in the best position to plan further design changes for 2000 that the Census Bureau make the most of the testing opportunities afforded over the next few years and establish priorities for testing and research wisely.

THE IMPORTANCE OF CHOICE OF METHODOLOGY FOR 1990

Controversy surrounding population censuses has as long a history in the United States as census-taking itself. According to one review (Bureau of the Census, 1982a:App. IIIb:73), censuses conducted during the colonial period, generally at the direction of the Privy Council or the British Board of Trade, “were seldom regarded as complete or successful, as people perceived them being for the purposes of taxation or conscription and were evasive and uncooperative.” The decennial censuses conducted in the new nation had a constitutional mandate according them legitimacy and support. Moreover, one historian (Conk, 1984:7) has noted that: “After the first few censuses, Americans became increasingly interested in the census results . . . [which] showed that the population was growing steadily and extremely rapidly.” It quickly became evident in the early nineteenth century, however, that not all areas were sharing equally in population growth and that reapportionment based on census results meant substantial shifts in political power. Hence, as the same historian comments (Conk, 1984:8):

The first extensive criticism of the census by statisticians occurred in 1843 when the American Statistical Association (ASA) issued a lengthy report that documented glaring errors in the data on education, occupation, and especially the classification by race of persons identified as insane, idiotic, and deaf and dumb. The ASA recommended that these results should be corrected or, at the least, disavowed. Problems with both undercount and fraudulent additions to the count were documented in many early censuses (Bureau of the Census, 1982a:App. IIIb:81-83).

Congress did not as a general rule respond directly to these criticisms, although occasionally it acted to alter the apportionment of the House when there was strong evidence of gross deficiencies in the population count. Congress gave a third representative to Alabama in 1823 when the claim was made that the 1820 census omitted two counties and in 1860 awarded an additional seat to California because of problems with the census in that state (Bureau of the Census, 1982a:App. IIIb:82). These actions were politically much more palatable than similar actions would be today, because reapportionment legislation up through 1910 added representatives to accommodate population growth rather than allocating a fixed number of seats among the states.

Despite the questions raised about the population enumeration in the past, a review of decennial census history suggests that social and political forces have converged in recent years to make the census in this country—and in other countries as well—a matter of demonstrably greater controversy than before. Several factors are involved.

On one hand lies increased concern with the need to protect the privacy of individual citizens and a sense that the public is oversurveyed and less willing to respond to government inquiries. Indeed, in the last few years, the level of public suspicion and hostility to plans for the census caused the governments of several Western European countries to delay their census programs or cancel them entirely (see Butz, 1984; Redfern, 1983).

On the other hand, legislators have increasingly turned to statistics in handling tough policy decisions. In fiscal 1984, federal grant-in-aid programs allocated at least $80 billion to states and local areas via formulas that depended in important ways on census figures (or statistics based on census figures, such as current population estimates) to determine who received how many dollars (Office of Management and Budget, 1985). As noted above, census data are used by constitutional mandate to determine the number of seats in the U.S. House of Representatives that are allotted to each state. They are used as well in drawing up congressional and state and local legislative districts to meet rigid criteria for equitable representation of the population. Data requirements for redistricting purposes in 1980 included census tabulations of the population by race (white; black; American Indian, Eskimo, and Aleut; Asian and Pacific Islander; other races) and Hispanic origin for each of several million city blocks in urban parts of the country and enumeration districts in unblocked areas (Bureau of the Census, 1982b:79).

In addition to these critical governmental needs, census data support many other major uses. Data from the latest census serve to document the social and economic condition of the country as a whole and are the single most important source of information for small areas and groups in the population. Comparative information from successive censuses illuminates

trends over time. Researchers, planners, and decision makers in business, government, and academic institutions make use of census data for a wide range of important planning and analysis purposes. Just a few of the many uses to which census data are put (see Chapter 2 for a detailed review) include:

• Site selection for public service facilities and commercial establishments based on evaluating the socioeconomic characteristics of alternative locations;
• Transportation planning using detailed data on commuting flows; and
• Research into changing rates of population growth for metropolitan versus nonmetropolitan areas and different regions of the country.

Many analyses based on census data have implications for the distribution of political power and wealth among various population groups in the country. For example, census data on the racial, ethnic, age, and sex makeup of occupational groups in labor market areas are used to assess the extent to which work forces reflect the characteristics of the local labor force. These data frequently form the basis of antidiscrimination lawsuits brought against employers. Census data on the makeup of the local population are used to assess—and challenge—the representativeness of grand and petit juries. Census data on earnings cross-tabulated by various characteristics are used to analyze wage disparities within and among occupations and important population subgroups. Findings from such studies can affect outcomes of public policy deliberations, such as the current debate over the issue of comparable pay for jobs of comparable worth. All of these uses have underscored more than ever before the importance of obtaining a complete and accurate count of the population as well as accurate data about characteristics.

Yet to obtain highly accurate data costs money. The 1980 census cost close to $1.1 billion dollars—about $4.75 for each inhabitant of the United States (Bureau of the Census, 1983b:88). The per capita amount is small compared with the per case cost of most government and private-sector sample surveys. Moreover, this total cost includes planning, collection, and processing activities that spanned most of a decade and provided data that are of value for the decade and beyond. Nonetheless, census costs exceeding $1 billion excite comment and invite close scrutiny to determine how they might be reduced. Recently in Canada, the quinquennial census scheduled for 1986 was cancelled because of budget constraints facing the government; it was subsequently reinstated, however, in response to widespread public expressions of concern and its demonstrated cost-effectiveness com-

pared with alternatives. The U.S. decennial census is constitutionally mandated; nevertheless, pressures are likely to be severe in the coming years to attempt drastic cost reductions both in census planning activities and in the enumeration, despite the fact that, compared with other ways of obtaining comparable information, the census is still cost-effective.

The Census Bureau’s own research has shown that there were inaccuracies in the 1980 census, both of underenumeration (i.e., persons who were missed) and overenumeration (i.e., persons who were inadvertently counted twice or otherwise included when they should not have been). Evaluation studies generally point to the conclusion that, overall, the 1980 census produced a small net undercount of the population—that is, the census count, including erroneous enumerations, fell somewhat short compared with an independent estimate. Most significantly, important race, sex, and age subgroups of the population experienced differential rates of net undercount. There is strong evidence that the black population experienced a net undercount of about 5 percent nationwide. Black men ages 25-54 appear to have had the highest net undercount rates—close to 15 percent on average (Passel and Robinson, 1984:Table 3). Coverage estimates for whites and other races are difficult to derive because of the lack of reliable estimates of net legal and illegal immigration. Making a range of reasonable assumptions about the size of the undocumented alien population, it appears very likely that whites and other races experienced net undercount in the 1980 census, but that the rate of undercount was smaller, perhaps significantly smaller than the 1.5 percent rate experienced in 1970 (see Passel et al., 1982:6-8; see also the review in Chapter 5 of coverage estimates for censuses from 1950 through 1980).

Differential net undercount means possible inequities in redistricting, fund allocation, and provision of social services based on census data as well as possibly erroneous conclusions drawn from studies used as the basis for antidiscrimination policies and lawsuits and other socially important purposes. The belief that errors in the census affected representation and fund allocation gave rise to an unprecedented number of lawsuits following the 1980 census. By October 1981, over 50 suits had been filed challenging the census results (Bureau of the Census, 1983b:85). Currently, the judge assigned to a major case in which the State and the City of New York are suing to have the Census Bureau adjust the 1980 census counts is reviewing testimony and preparing to hand down a decision; 23 other cases are awaiting settlement of the New York suit. Analyses by Kadane (1984) and Gilford (1983) indicate that the apportionment of congressional seats may have been affected by the differential undercount. For example, Kadane found that if one of the sets of estimates produced from the 1980 Post-Enumeration Program evaluation was used to adjust the census results, California would have received an additional seat at the expense of Pennsylvania.

PROPOSED CHANGES IN METHODOLOGY

Not surprisingly, many ideas have been proposed by the Census Bureau and others to improve the decennial census. Some are directed principally at improving coverage and reducing differential coverage errors. One idea in this class is to match administrative records, such as driver’s license lists and other sources, against the census on a scale even larger than that used in 1980 to identify people who should be followed up to determine if they were improperly omitted from the census count. (See Chapter 3 for a description of the efforts along these lines in the 1980 census.) Other ideas are directed principally at reducing costs. One such approach is to make use of sampling, not only to obtain information on characteristics, as is currently standard decennial census practice, but also as part of the procedure to obtain the count. For example, one could attempt contact with a sample of households that do not mail back their questionnaires, rather than all nonrespondents, in the follow-up stage of census operations. Special coverage improvement procedures could also be carried out on a sample basis.

Two important themes stand out in current discussions of methodology for the decennial census. One relates to the degree of emphasis that should be given to counting versus estimation. A census, no matter how diligently administered, can never be complete or without error. Moreover, it is true of current census methodology that not every record corresponds to a person actually named on a questionnaire; for example, a small percentage of records (about 1 percent in recent censuses—see Chapter 3) represents imputations in situations in which there is good evidence that a housing unit is occupied but repeated efforts have failed to find the residents. Hence, a census, strictly speaking, provides an estimate of the population.

From this recognition has come a view of the decennial process that emphasizes the role of estimation and argues that some of the resources for conducting the census should be shifted from efforts directed toward traditional coverage improvement procedures to efforts directed toward developing the best possible estimates of the total population and subgroups. Input to the decennial year population estimates, in one version of this view (Ericksen and Kadane, 1985), would include not only a well-conducted census, but also information obtained from various programs conducted on a sample basis that would provide a basis for adjusting the census field counts. (Such programs might include matching of samples of administrative lists to census records and follow-up of a sample of households that did not respond to an initial follow-up effort.) Whatever the merits of particular suggestions put forward to incorporate estimation into the census process, the known errors and the incompleteness of the census count mean that the issue of adjusting census figures needs to be addressed.

The other theme relates to the critical importance of evaluation programs in the methodology of the decennial census. Politicians, policy analysts, statisticians, economists, demographers, other social scientists, and users of census data in all sectors have expressed divergent views regarding the most appropriate methodology for conducting the census. But whether they view the census in traditional terms as strictly a counting operation or believe that the census should be the starting point for an estimating process, there is substantial agreement on the importance of evaluating the completeness and accuracy of census statistics.

The Census Bureau has conducted formal evaluation programs for every census since 1950 (Bureau of the Census, no date-a). All of the techniques used to date in this country and abroad, including demographic analysis, reverse record checks, administrative record matches, and post-enumeration surveys (whether recanvassing selected areas or matching independent surveys to census records), have important flaws (see the review of coverage evaluation methods in Chapter 4). In the United States today, the absence of adequate data for estimating net immigration, whether of legal or illegal residents (Marks, 1980), poses particularly severe problems for evaluating the census count even at the national level using the demographic method. Furthermore, if evaluation results were to be used for census adjustment purposes, then reasonably accurate information on the errors of evaluation estimates would also be needed. Nevertheless, with concern over possible inequities in political representation and the distribution of large amounts of federal dollars as well as concern over the adequacy of the data for analysis of the socioeconomic status of important population groups, there has never been a greater need for thorough evaluation of the decennial census. This evaluation is necessary whether the object is to inform users of known errors in the census or actually to modify census results.

While there is widespread agreement that evaluation is important and that the issue of adjustment must be faced, many decisions on methodology for 1990 remain to be made. It is clear that there is no lack of ideas and suggestions that appear useful to investigate. It is also clear that the process of determining a reasonable methodology for 1990 will involve difficult choices.

Ideally, one would like the 1990 and future censuses to improve the accuracy of the data over that in 1980, maintain the amount of useful information collected, and release the results on a more timely basis, while at the same time reducing the burden on the public and lowering costs. The Census Bureau has stated (Bailar, 1984:259) that its minimum goals for 1990 are to:

(c) Maintain a high rate of overall coverage and improve the accuracy of small area data while reducing the overall differential for population groups and geographic areas.

It may be possible to design a methodology that makes gains in the desired direction on each of these dimensions. The more likely situation is that it will be possible to make progress on one or two dimensions but at the price of giving up improvements on the others. Explicit trade-offs reflecting costs and benefits will need to be made in the choice of methodology for 1990 and beyond (see Keyfitz, 1979). Because of the high costs of censuses and the compressed time frame within which they are carried out, making mid-course corrections impossible, it is essential that the methodology to be used be thoroughly tested.

INDEPENDENT REVIEWS OF DECENNIAL CENSUS PLANS

The Census Bureau is actively working on methodology for the 1990 census and has assembled staffs to plan the census and specifically to work on issues of undercount and the possible adjustment of census counts. For many decades, the Census Bureau has also actively sought outside independent review of its plans and proposed procedures. In addition to ongoing advisory committees involving various professional disciplines and advisory committees representing the interests of population groups for whom census results are particularly important, the Census Bureau has asked the National Research Council (NRC) and the American Statistical Association (ASA) to conduct special studies of the decennial census. This report represents the fourth outside review conducted in recent years of key aspects of modern census methodology. A brief discussion of the scope and thrust of the predecessor NRC and ASA studies can help place the current study in context.

The 1969-1972 NRC Advisory Committee on Problems of Census Enumeration

The Bureau of the Census sponsored a study in 1969 by a committee of the National Research Council to provide advice on ways to improve completeness of coverage in the decennial census and intercensal household surveys. (The Office of Economic Opportunity and the Manpower Administration of the U.S. Department of Labor also contributed support for the study.) The Advisory Committee on Problems of Census Enumeration issued its final report, America’s Uncounted People, in 1972. The report focused on the need to understand the social and psychological context in which undercount occurs. For example, the committee noted that people

may be missed in central city areas because, although members of extended families, they are not attached to a family or household residence, which is the basic unit of enumeration in the census and household surveys. The committee strongly recommended that the Census Bureau broaden its research strategy and knowledge base to include methods and concepts not typically embraced in survey research. The report included specific recommendations to conduct experimental studies of questionnaire wordings and formats and their effects on respondents; explore the utility of communication research for better understanding the reasons for census and survey undercoverage; and carry out localized participant-observer studies to learn more about the impediments to census data collection in different kinds of areas.

The 1978 NRC Panel on Decennial Census Plans

The Census Bureau asked the National Research Council again in 1978 to review decennial census methodology, specifically the plans for the upcoming 1980 census. The NRC’s Committee on National Statistics set up the Panel on Decennial Census Plans, which worked within a very short time span, to: (1) examine field procedures, questionnaire design, and special procedures designed to improve the 1980 census coverage, (2) review proposed procedures for handling contested counts, (3) investigate the feasibility of adjusting census counts, and (4) consider evaluation plans for the 1980 census. The panel’s report, Counting the People in 1980: An Appraisal of Census Plans, made recommendations in many areas. This panel repeated the call of the earlier committee for imaginative work on the cultural and social problems associated with census-taking. In the area of adjustment, the 1978 panel concluded (National Research Council, 1978:132-133) that: “methods of adjustment with tolerable accuracy are feasible” and “on balance an improvement in equity would be achieved.” The panel supported implementation of procedures to adjust population counts for underenumeration for purposes of fund distribution and expressed confidence in the Census Bureau to determine the best technical procedures for adjustment. The panel recommended that adjustment “not be applied to the counts used for legislative apportionment nor to the body of census data on the characteristics of the population.”

The 1981-1982 ASA Technical Panel on the Census Undercount

The Census Bureau asked the American Statistical Association in 1981 to convene an expert group to review the methods and results of the programs used to evaluate completeness of coverage in the 1980 census and to make recommendations regarding research in the areas of coverage

evaluation and adjustment of census counts. This panel made a number of specific research suggestions and also recommended (American Statistical Association, 1984:256): “that the Bureau of the Census sponsor an outside technical advisory group on undercount estimation and related problems.”

The 1984 Panel on Decennial Census Methodology

In response to the recommendation of the 1981 ASA panel, the Census Bureau asked the Committee on National Statistics at the National Research Council to establish the Panel on Decennial Census Methodology. The charge to the panel was for an investigation of three major issues from a technical viewpoint, setting aside legal considerations:

1. Adjustment of census counts and characteristics, including exploration of formal criteria to evaluate measures of undercount and alternative adjustment procedures;
2. Uses of sampling in the decennial census, including investigation of whether, for a given cost, the sampling of lists and areas to improve coverage and sampling of nonrespondents for follow-up can improve accuracy for the total population and for important subgroups; and
3. Uses of administrative records, including investigation of various types of records to determine their possible utility in improving the accuracy of census counts and the efficiency of census operations.

We interpreted this charge to include investigation of closely related topics, notably methods of coverage evaluation and improvement. Coverage evaluation programs provide the necessary input data for any adjustment and serve the important function of apprising users of the quality of the census counts. Procedures for coverage improvement are viewed by the panel as important and desirable even if an adjustment procedure is incorporated into census methodology. The panel also investigated uses of census data and their dependence on the accuracy of the census figures. Proper evaluation of the consequences of changes in collection methodology requires an understanding of important uses of the data being collected.

The charge to the panel related to analysis of decennial census methodology and not to other population programs of the Census Bureau. However, in the course of the panel’s work, it became clear that the census could not be considered completely in isolation. Demographic and related social and economic statistics are used continually over the decade following each census, and current information is needed for these uses. The Census Bureau has a number of formal programs for updating some of the census information. Hence, the census is the central part of a broader statistical

system designed to produce data needed to implement legislation, assist in decision making both by industry and government, and help understand changes taking place in our society. Although the panel did not undertake a study of population statistics programs other than the census, we did consider the quality of census data compared with the quality of postcensal population estimates. The panel makes a recommendation to assess the need for a mid-decade census in 1995 in light of the impacts of errors in postcensal population estimates on major data uses, such as fund allocation (see the discussion in Chapter 2).

The work of the panel differs in several important ways from the efforts of its predecessors. This is the first panel to be asked explicitly to consider important changes in decennial census methodology from the perspective of cost as well as effectiveness. A theme running through the charge to the panel is to design a methodology that improves accuracy compared with previous censuses but costs no more, and ideally less, in constant dollars.

Other important differences have to do with the timing of the panel’s work in relation to the cycle of decennial census planning. The panel was convened at a point in the cycle when it could benefit from the availability of extensive material regarding the experience in the most recent census. At the same time, the panel carried out its work in an early stage of the planning cycle for the next decennial census before decisions on methodology were fixed. Hence, the panel has been in an unusually good position to provide suggestions and guidance regarding the research and testing program for 1990. In fact, the panel’s role has been one of assessing and reacting to Census Bureau research and pretest plans for 1990 rather than attempting to recommend, at this stage in the process, the adoption of specified procedures and concepts as the methodology for the 1990 census. The panel does not presume to have the answers regarding the “best” methodology for the decennial census. We have endeavored to state our position and to recommend directions for needed research on critical issues of what constitutes cost-effective methodology, particularly the relative emphasis to be given to counting versus estimation.

MAJOR THEMES OF THE REPORT

Several themes run through this report. The first major theme can be expressed as the need for balance between traditional and new procedures in the choice of census methodology for 1990. Indeed, balance has characterized the historical evolution of decennial census methodology. The panel does not propose that the Census Bureau make radical innovations in decennial census methodology in the near term. The census is a massive and complex operation and major changes should be made only with care and after thorough evaluation—including tests carried out under actual census

conditions. Nonetheless, the panel believes that it is important to implement changes on some dimensions for 1990 and to undertake planning that may lead to further changes in the future.

Most important, the panel argues for balance between efforts to achieve a complete enumeration and efforts to improve the accuracy of census figures through adjustment procedures. The panel believes that adjustment cannot be viewed as an alternative to obtaining as complete a count as possible through cost-effective means. The United States has a long tradition of a census as a complete enumeration in which it is a civic responsibility to participate in the census process. The panel believes it is important to continue this tradition and important that census methodology should strive for a complete enumeration via counting procedures, including the use of cost-effective special coverage improvement programs. However, the panel also believes that the ultimate goal of the census should be the accuracy of the census figures. The evidence is overwhelming that no counting process, however diligent, will in fact enumerate everyone. Hence, the panel recommends that the Census Bureau carry out a vigorous program of research on coverage evaluation and adjustment methods that, if successful, would permit adjustment of census figures as part of the methodology for the 1990 census (see the discussion in Chapters 7 and 8).

A second and related theme concerns cost-effectiveness. The panel has not attempted to apply formal cost-benefit analysis to decennial census methodology, but has endeavored to identify those proposed changes that show the most promise of improving accuracy without increasing costs or of reducing costs without importantly impairing accuracy. In this regard, the panel’s recommendation for research designed to develop appropriate and feasible methods of adjustment of the census counts, together with the Census Bureau’s stated goal to contain costs for the 1990 census, implies that some budget resources must be shifted from coverage improvement to coverage evaluation and adjustment. Specifically, the panel argues that coverage improvement programs used in previous censuses should be carefully reviewed to determine their efficacy. Costly programs that neither correctly added significant numbers of people to the count nor importantly reduced differential undercount should be dropped from the Census Bureau’s plans for 1990 (see the discussion in Chapter 5). Effective programs, however, should be further refined through testing and research, and the budget should make room for testing some new ideas in this area.

While not favoring extensive use of sampling to obtain the count, the panel supports research on using sampling in the later follow-up stages of census operations and in some coverage improvement programs, such as the program to recheck the vacancy status of housing units. Limited use of sampling may effect measurable cost savings with minimal sacrifice of accuracy (see the discussion in Chapter 6). Careful use of sampling for

certain coverage improvement programs may, in fact, improve accuracy by reducing duplications and other erroneous enumerations, in addition to identifying missed households and people.

In considering cost and accuracy, the panel believes it is important to look at the characteristics data collected in the census as well as the population count. There is strong evidence that important subject items have severe reporting problems. The panel recommends a strategy of looking closely at each item proposed for inclusion on the questionnaire to determine: (1) the need for that item; (2) the level of geographic detail required by users, and hence whether the item must be asked of all households on the short form, or whether it can be asked of a sample on either the long form or on a much smaller follow-on survey; and (3) whether some other source could provide higher-quality data. The panel suggests exploring the use of administrative records together with sampling to obtain data on some housing structure characteristics (see Chapter 6). Such data could be more accurate than individual responses on the census form. Costs initially may be high, but should decline over time. This particular use of administrative records has the advantage that it should present no actual or perceived threat to individual privacy.

A third major theme of the report concerns the strategy for designing the 1990 census, whatever the particulars of the methodology may turn out to be. The research plans drafted by the Census Bureau staff are extremely comprehensive and ambitious. The staff has clearly tried to include all reasonable ideas for consideration in the research and testing program. The panel commends the Census Bureau’s efforts to design and carry out a thorough research and testing program that will support sound decisions regarding methodology for the 1990 and later censuses.

The panel believes, however, that in most areas the Census Bureau must choose among all the ideas and procedures proposed for testing, given constraints on available staff and budget resources and the limited time available to analyze test results and use them to guide decisions on methodology. The exception concerns research related to adjustment, including research on coverage evaluation methods. In this area, the panel believes that research must proceed on a broad front if effective methodologies are to be developed for 1990. In other areas, the panel has endeavored to recommend strategies for choosing priority projects for inclusion in the 1990 census research and testing program and has also recommended the use of less costly research methods where appropriate, including more detailed analysis of 1980 census results, in place of full-scale field tests. Finally, the panel recommends specific concepts and procedures for research and testing that we believe show special promise for improving the methodology of the decennial census in 1990 and beyond.

OVERVIEW OF THE REPORT AND RECOMMENDATIONS

This section provides an overview of the report and a summary of the panel’s recommendations. The report is organized as follows: Chapters 2, 3, and 4 provide background on the decennial census that is helpful for understanding the subsequent exposition of the panel’s recommendations. Chapters 5 through 8 provide general and specific recommendations in several areas. Each chapter includes one or more appendixes that provide additional details for the interested reader on topics discussed in the text. Below we briefly summarize the contents of each chapter and list the panel’s recommendations.

Chapter 2—Purposes and Uses of the Decennial Census

In choosing an appropriate and cost-effective methodology for a data collection program, it is important to understand the kinds of uses that are made of the statistics. The chapter gives a brief overview of the uses of census data historically and then describes the range of uses of the modern census; most importantly, the census is virtually the only source of comparable data on basic counts and detailed characteristics for small areas and small groups in the population. Two appendixes provide additional information on uses and users of census data: Appendix 2.1 reviews state and local government uses, and Appendix 2.2 provides a case study of government, business, and academic uses in the State of New Jersey.

The chapter includes a review of the limited body of research that has attempted to measure the effects of census data errors on key uses, such as reapportionment, redistricting, and allocation of federal funds to states and localities. Finally, the chapter reviews data on errors in postcensal population estimates for small areas, which appear to greatly exceed errors in the census itself. The panel recommends serious consideration of the need for a mid-decade census program in 1995 to improve the quality of postcensal estimates:

Chapter 3—Census Methodology: Prior Practice and Current Test Plans

This chapter briefly reviews the procedures used to conduct the 1980 census and compares and contrasts the 1980 census methodology with

procedures used in previous modern censuses in the United States and in other Western countries. The discussion covers the following stages in the census process: development and checking of address lists, enumeration, follow-up, coverage improvement, data processing, and post-census evaluation. The chapter also summarizes the Census Bureau’s current research and testing plans for the upcoming 1990 census, with particular emphasis on the plans for pretests in spring 1986, and presents the panel’s overall assessment of these plans.

The panel has several major concerns with the research and testing program proposed for 1986. The program outlined appears too ambitious for the time remaining before the census and for the staff and budget resources likely to be available, particularly if key data are to be analyzed in time to support major decisions. In the panel’s view the program also places too much emphasis on field testing over other kinds of research, including further analysis of existing data. The panel suggests in this chapter some ways to scale back the 1986 testing program and in subsequent chapters provides detailed recommendations for research priorities in specific areas of census methodology.

1. Can be pursued with other research methods and omitted from the 1986 field test program;
2. Can be safely deferred for research or testing until 1987 or until the dress rehearsals;
3. Are unlikely to be viable for 1990 but should be incorporated on an experimental basis into the 1990 census as a test for future censuses; and
4. Should be omitted entirely from consideration for the 1990 census, based on previous census experience or other survey research results.

Appendixes to Chapter 3 provide more detailed expositions of two aspects of current census methodology—sequential hot-deck imputation, used

to assign values to census records for missing responses (Appendix 3.1), and iterative proportional fitting, used to calibrate responses obtained from samples of households to responses obtained from the entire population (Appendix 3.2). These techniques have potential use, as discussed in Chapter 7, in carrying out an adjustment of the census counts.

Chapter 4—Evaluating the Decennial Census: Past Experience

This chapter reviews the history of coverage evaluation of population censuses in the United States from 1950 through 1980. Broadly speaking, there are two major classes of coverage evaluation techniques—micro-level methods based on case-by-case analysis of samples of units such as persons or households and macro-level methods involving analysis of aggregate census data, including comparison of census totals with external data and analysis of internal consistency. The chapter identifies strengths and weaknesses of each of the major micro-level methodologies used in the United States and Canada, including post-enumeration surveys, reverse record checks, and administrative records matches, and of the major macro-level method, demographic analysis. Although the chapter does not contain recommendations, it does provide important background material for the panel’s conclusions presented in Chapter 8 regarding research on coverage evaluation estimation methods for the 1990 census. Chapter 4 includes two appendixes providing technical detail on coverage evaluation based on multiple lists (Appendix 4.1) and on operational aspects and modeling of computer matching (Appendix 4.2). Matching of records from the census to records obtained from one or more surveys or administrative lists is a critically important component of most methods of evaluating the completeness of coverage of the population obtained in the census.

Chapter 5—Taking the Census I: Improving the Count

This chapter focuses on the problem that not all population groups are counted equally well in the census and discusses procedures for improving the count, including procedures used in past censuses together with some new ideas. Most programs directed toward coverage improvement are expensive. They may also introduce error by duplicating or otherwise erroneously adding persons. In general, however, the panel believes that the costs of well-designed and well-executed coverage improvement programs represent money well spent for improving the census figures. The chapter first reviews what is known about hard-to-count groups in the population and about groups that have exhibited problems of erroneous enumeration. (The text provides a summary of the literature, and Appendix 5.1 provides a more in-depth review.) The chapter then reviews the performance of special pro-

grams directed toward coverage improvement in the 1970 and 1980 censuses, including estimates of cost and numbers of persons and housing units added to the count. The panel makes recommendations for coverage improvement related to items on the questionnaire and to enumeration procedures.

The panel first notes the importance of gaining understanding of the problems of undercount and overcount in the census:

The panel next discusses priorities for research and testing directed toward improvement of items on the questionnaire that relate to coverage, including the questions on race and Hispanic origin. It is important to understand what responses to the race and ethnicity questions mean if appropriate estimates of coverage rates for race and Hispanic groups are to be developed. This section reviews the history of race and ethnicity questions in the census; considers techniques for developing race and ethnicity questions; and discusses issues of data comparability, including comparability of race and ethnicity data from census to census and comparability with information collected in vital statistics records.

The next section of the chapter reviews experience in the 1970 and 1980 censuses with questions on the short form designed to aid in achieving a complete and accurate count, such as questions probing for a complete roster of household members. The discussion notes problems posed for an accurate count by the mobility of the population and recent trends in living arrangements that have resulted in growing populations with two or more usual residences (e.g., retired people with summer and winter homes). The panel suggests a question for testing directed toward improving coverage of young adults and children in hard-to-count areas.

The last section of the chapter provides the panel’s overall assessment of special enumeration procedures designed to improve the count. While believing that programs such as the recheck of vacant units can make important contributions to improving coverage, the panel does not subscribe to the view that every coverage improvement idea that is suggested or has been used in the past should be included in the plans for the next census. The panel recommends paring down the list of programs to be considered for 1990 and the list requiring early field testing.

Chapter 6—Taking the Census II: Uses of Sampling and Administrative Records

This chapter addresses two major methods that have been proposed to improve the cost-effectiveness of the decennial census—the uses of sampling in obtaining the count and the use of administrative records. The chapter considers the merits of replacing the census with a large sample survey, using sampling in the follow-up stage of census operations, and using sampling for various coverage improvement operations. It also discusses the traditional use of sampling to obtain characteristics detail. Finally, the chapter considers the use of administrative records along with sampling for improving the quality of certain census items.

With regard to sampling for the count, the discussion notes problems of replacing the census with a large sample survey. The panel believes that a survey would result in less complete coverage compared with a census and that there would be only minor cost savings in sampling on the scale necessary for satisfaction of present demands for small-area data from the census. The use of sampling for follow-up of households that do not return their census questionnaires has some of the same drawbacks, but sampling could prove cost-effective in the final stages of follow-up in which it is very expensive to count an additional person. Although the Census Bureau has dropped plans to study the use of sampling for follow-up and for coverage improvement programs such as the recheck of vacant units in 1986, the panel supports research in these areas. The panel also supports further testing of telephone follow-up of nonresponding households, which was tried experimentally in 1980. Finally, the panel discusses the need to maintain machine-readable records of the follow-up history of individual households that will permit detailed analysis and simulation of different sample designs.

The chapter then reviews the use of sampling for content items in the census. Historically, in every census since 1940, some items have been asked of only a sample of the population in order to reduce response burden and processing costs while obtaining the benefits of additional data. Sample designs and sampling fractions have differed in recent censuses. The Census Bureau is currently considering a design for 1990 that would include a short form containing items asked on a 100 percent basis, a long form containing additional items asked of a large sample, and a follow-on survey of a small percentage of short-form households administered within a few months of Census Day that would obtain yet other information. The panel did not offer specific recommendations in this area, but noted that the criteria for including items in the follow-on survey have not been explicitly articulated but should be to permit thorough assessment of the need for the survey and for the inclusion of particular items.

The last section of the chapter discusses the use of administrative records and sampling for improving the accuracy of content items. The concern over completeness of population coverage in the census can obscure equally valid concerns over the accuracy of the content. There are well-documented problems with the reporting of content items such as income, utility costs, and age of structure. The panel recommends research and testing directed toward improving the data quality of key items. The research program should include design of operations to verify, and possibly adjust, responses as part of the census operation and investigate the possibility of obtaining some items from administrative records sources. The panel makes a specific recommendation with regard to housing structure items.

Two appendixes to Chapter 6 provide additional information related to use of sampling to obtain the census count. Appendix 6.1 gives very rough cost estimates for conducting the census on the basis of different-sized samples. Appendix 6.2 develops illustrative costs for conducting follow-up operations of a sample of nonrespondents. Appendix 6.3 develops in further detail the panel’s suggestion for obtaining improved data on housing structure items by means of local administrative records.

Chapter 7—Adjustment of Population Counts

In this chapter, the Panel presents its basic position on the issue of adjustment of the census counts. The chapter considers criteria or yardsticks for measuring the increase in accuracy of census information that adjust-

ment might produce and addresses problems of consistency and timing posed by adjustment. The chapter discusses what is known and recommends further research on procedures that could be useful for adjustment. Three appendixes provide additional technical discussion. Appendix 7.1 supplies the mathematical expressions for various yardsticks discussed in the text; Appendix 7.2 discusses in greater detail hierarchical Bayesian techniques that have been proposed for use in adjustment; and Appendix 7.3 discusses a problem raised by the aggregation of synthetic estimates.

The chapter begins with consideration of the need for adjustment to improve the accuracy of the census numbers, particularly to reduce differential coverage errors across geographic locations and demographic groups. The panel is led to recommend development of adjustment procedures, but as a complement to—not a substitute for—continued efforts to improve census coverage. If public perception of the importance of being counted should deteriorate, this would have serious consequences for the accuracy of the figures, adjusted or not.

The chapter next considers criteria for evaluating the numbers produced by the census (based on either unadjusted or adjusted counts), considering both the errors in the numbers themselves and the resulting loss to society due to erroneous treatment of political jurisdictions in terms of representation, fund allocation, and other uses of the data. The panel discusses yardsticks or loss functions, that is, numeric measures of the impact of census errors, from the viewpoint of the data user and as they relate to adjustment. The discussion notes that no adjustment procedure can be expected to simultaneously reduce the error of every piece of census information for every geographic area; rather, there is an important net social gain if differential coverage error is generally reduced. The panel believes it is substantially more important to reduce the overall error per person than the overall error per place and recommends that loss functions for measuring total error take into account the population size of each jurisdiction.

In discussing technical considerations concerning choice of loss function, the panel concludes that good adjustment procedures should be expected to perform well for a range of loss functions. Where the choice of adjustment procedure depends importantly on the choice of loss function, this suggests that the particular adjustment procedure has weaknesses that need to be addressed.

The next section of the chapter discusses the problem of estimating the likely range of error introduced by the particular procedure adopted for an adjustment. Although error can be measured only imperfectly, information about the distribution of error is important in the same way that sampling variances for sample surveys provide useful information.

Adjustment of census data could create problems of internal consistency of microdata from the census with aggregate statistics. The panel believes that internal consistency is an important quality for general purpose statistics, such as those produced by the decennial census, which have a wide range of output and many uses. The section discusses reasons to prefer carrying down any adjustment of population estimates for larger geographic areas to the level of the individual micro-records and reviews methods, such as weighting and imputation, for accomplishing this.

Adjustment also presents problems of timing. Current law requires submission of state population counts for purposes of reapportionment within 9 months after Census Day and of small-area counts within 12 months after Census Day for purposes of redistricting. The panel discussed the pros and cons of various scenarios with regard to release of adjusted data if it does not prove possible to implement a full-scale adjustment in time to satisfy the above constraints. Congress clearly will need to stipulate which scenario is preferable for apportionment purposes.

The remaining sections of Chapter 7 review possible technical approaches to the use of data from coverage evaluation programs for adjusting the raw census figures (detailed discussion of these programs is in Chapter 8). The review covers procedures for starting out, that is, for developing estimates for a limited number of large geographic areas, and procedures for carrying down, that is, for using the large-area estimates to develop adjustments for small areas and ultimately for the microdata records. The discussion considers the Census Bureau’s plans for research and testing of adjustment procedures in upcoming pretests and makes recommendations for priority research areas.

Chapter 8—Measuring the Completeness of the 1990 Census

This chapter presents the panel’s recommendations for research and testing to design effective coverage evaluation programs for the 1990 census. For adjustment to be feasible, evaluation programs must be good enough to provide estimates of net undercoverage that are reliable for at least large geographic areas and have error properties that are broadly understood. Coverage evaluation programs also provide valuable information for users of the data and for the Census Bureau in planning subsequent censuses. Although in general the panel recommends that the Census Bureau narrow its 1990 census research and testing objectives, in the area of coverage evaluation the panel believes it is too soon to focus on one method to the exclusion of others. The panel makes several recommendations related to the Census Bureau’s currently preferred method of pre- or post-enumeration surveys and also recommends vigorous research related to alternative methods, including reverse record checks and systematic observation. Appendix 8.1 discusses estimation methods and estimates for the illegal alien population, and Appendix 8.2 provides estimates of variance and cost for a large systematic observer study.

The chapter begins with a review of the problems associated with each of the major methods of coverage evaluation and considers the Census Bureau’s current plans for research and testing directed toward coverage evaluation of the 1990 census. The panel argues against the Census Bureau’s decision to concentrate on post-enumeration (or possibly pre-enumeration) survey methodology as the principal means of coverage evaluation in 1990, notes that the Census Bureau should not put itself in the position of lacking a means of adjustment if there are problems with the operation for matching survey with census records, and urges completion of 1980-based studies related to coverage evaluation.

The chapter next considers possible improvements and recommends priority research areas for each major coverage evaluation method in turn. The demographic analysis method, which uses data from independent sources including birth and death records to estimate the number of persons at the time of the census in a given age-race-sex category, currently suffers from the absence of data on undocumented aliens. The panel recommends research into using demographic analysis for estimates of the native-born population. The reverse record check method, which traces the current location of a representative sample of newborns, immigrants, and persons counted in the previous census or coverage evaluation program, has a greater problem in tracing in the United States because of the 10-year interval between censuses (as opposed to 5 years in Canada, where the method has been used extensively). The panel recommends completion of a current experiment to test alternative methods of tracing. The chapter discusses at length the method of post-enumeration (or pre-enumeration) surveys, in which a sample of households is interviewed and matched with records in the census, and identifies several problem areas for particular attention in the Census Bureau’s research.

Finally, the chapter discusses the idea of using systematic observers to provide independent estimates of the population in a sample of areas, including but not limited to areas that have proved particularly hard to count in previous censuses. This method is suggested because it may have the potential to surmount the problem observed repeatedly in the history of coverage evaluation, namely that persons who are missed by the census are also likely to be missed by an independent survey or other data source.

1. Reduction of post-enumeration survey nonresponse;
2. Reduction of unresolved matches between records for individuals listed in the post-enumeration survey and the decennial census;
3. Validation of the assumptions and/or development of alternative methodologies with respect to netting-out of overcounts and undercounts with reference to the place of enumeration; and
4. Investigation of alternatives to the assumption that the inclusion of individuals in the post-enumeration survey is unrelated to their inclusion in the decennial census and the estimation of the strength of this relation.

In the area of adjustment-related research, including coverage evaluation methods, the panel acknowledges that many technical and operational issues need to be resolved if adjustment procedures are to be developed in time for their use in the nation’s bicentennial census in 1990. Overall, while much effort will be required, the panel is optimistic that substantial progress can be made.