Summary

THE DECENNIAL CENSUS IS A CORNERSTONE of the U.S. federal government and its statistical system. The census’s constitutionally mandated purpose is to provide population counts to reapportion the U.S. House of Representatives every 10 years; a closely related purpose is to provide block-level population counts by age, race, and ethnicity to draw the boundaries of congressional and legislative districts. In addition, census information is used to allocate hundreds of billions of dollars of federal funds to states and localities; as the basis for postcensal population estimates for states, counties, and cities; to support federal, state, and local government program planning; to provide sampling frames and population weights for household surveys; and to provide denominators for such vital statistics as birth and death rates by age, gender, race, and ethnicity.

To address these information needs, the Census Bureau is preparing to conduct the nation’s 23rd decennial census. In 2010, the Census Bureau will collect basic information for each U.S. resident, including name, gender, age, date of birth, race, ethnicity, relationship to principal respondent, number of people in the housing unit, and whether the housing unit is owned or rented. The additional socioeconomic information that was previously collected on the census long-form questionnaire for a sample of households is now collected by the continuous American Community Survey (ACS), resulting in a short-form-only 2010 census.

Census planning is itself a major effort, which since the 1950 census has involved extensive research and development over the better part of each decade. The planning culminates in a dress rehearsal for the immediately upcoming census, followed by experiments and evaluations conducted as part of that census to begin the planning for the next census.

The Census Bureau recognized that the experiments and evaluations conducted for the 2000 census did not provide an adequate basis for 2010 cen-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 3
Summary T of the U.S. federal gov- HE DECENNIAL CENSUS IS A CORNERSTONE ernment and its statistical system. The census’s constitutionally man- dated purpose is to provide population counts to reapportion the U.S. House of Representatives every 10 years; a closely related purpose is to provide block-level population counts by age, race, and ethnicity to draw the boundaries of congressional and legislative districts. In addition, census in- formation is used to allocate hundreds of billions of dollars of federal funds to states and localities; as the basis for postcensal population estimates for states, counties, and cities; to support federal, state, and local government program planning; to provide sampling frames and population weights for household surveys; and to provide denominators for such vital statistics as birth and death rates by age, gender, race, and ethnicity. To address these information needs, the Census Bureau is preparing to conduct the nation’s 23rd decennial census. In 2010, the Census Bureau will collect basic information for each U.S. resident, including name, gender, age, date of birth, race, ethnicity, relationship to principal respondent, number of people in the housing unit, and whether the housing unit is owned or rented. The additional socioeconomic information that was previously collected on the census long-form questionnaire for a sample of households is now col- lected by the continuous American Community Survey (ACS), resulting in a short-form-only 2010 census. Census planning is itself a major effort, which since the 1950 census has involved extensive research and development over the better part of each decade. The planning culminates in a dress rehearsal for the immediately upcoming census, followed by experiments and evaluations conducted as part of that census to begin the planning for the next census. The Census Bureau recognized that the experiments and evaluations con- ducted for the 2000 census did not provide an adequate basis for 2010 cen- 3

OCR for page 3
4 ENVISIONING THE 2020 CENSUS sus planning. The Bureau therefore requested a study from a panel con- vened by the National Academies Committee on National Statistics to re- view the 2010 Census Program of Experiments and Evaluations (CPEX) so that it would provide a strong basis for 2020 census planning. The panel was charged to: consider priorities for evaluation and experimentation in the 2010 cen- sus. [The panel] will also consider the design and documentation of the Master Address File and operational databases to facilitate research and evaluation, the design of experiments to embed in the 2010 cen- sus, the design of evaluations of the 2010 census processes, and what can be learned from the pre-2010 testing that was conducted in 2003– 2006 to enhance the testing to be conducted in 2012–2016 to support census planning for 2020. Topic areas for research, evaluation, and testing that would come within the panel’s scope include questionnaire design, address updating, nonresponse follow-up, coverage follow-up, unduplication of housing units and residents, editing and imputation procedures, and other census operations. Together, the panel’s interim and letter reports (included in this volume as Parts II and III) fulfill the core mandate of this charge to assess the 2010 CPEX program. In those reports, the panel generally concluded that the specific experiments chosen by the Census Bureau for the 2010 CPEX are not likely to inform changes that will substantially affect either census cost or quality in 2020, and so largely squander a valuable testing opportunity. Moreover, the panel concluded that the ability to evaluate the 2010 census (apart from or in addition to the formal CPEX evaluations) will depend on the Census Bureau’s ability to retain input and output data from each 2010 census operation, for later linkage and analysis. RESEARCH AND DEVELOPMENT FOR A COST-EFFECTIVE 2020 CENSUS This final report builds on the panel’s earlier work by discussing census experimentation and evaluation in a broader context. In particular, this re- port describes a strategy for research and development (R&D) leading to a 2020 census design that controls costs, maintains quality, and adapts to major social and technological changes. An effective research and develop- ment strategy for the 2020 census should keep a goal of containing costs while maintaining quality; it should also recognize that new technical ad- vances may not only reduce costs, but also drive them up, so that research and effective cost modeling are necessary to determine how best to use such advances for the most cost-effective census.

OCR for page 3
SUMMARY 5 Census Costs Since 1970, when the current approach of a mailout-mailback enumer- ation, based on a master address list, with in-person follow-up of addresses for which a questionnaire is not returned, was first used, the real dollar per housing unit cost (2009 dollars) of the U.S. census has increased by more than 600 percent—it was $17 in 1970 (about the same as in 1960), $33 in 1980 (91 percent increase over 1970), $43 in 1990 (30 percent increase over 1980), and $70 in 2000 (63 percent increase over 1990). It is antici- pated to be as much as $115 in 2010 (64 percent increase over 2000). In contrast, the real dollar per housing unit cost of recent Canadian censuses (in 2008 Canadian dollars) has remained constant at $39–$40 for the 1996, 2001, 2006, and 2011 censuses. The unprecedentedly high per housing unit cost expected for 2010 has resulted in part from planned long-term investments—improvements to the Census Bureau’s TIGER geographic database, which is essential for accurate geographic coding and mapping of addresses, and the testing and implemen- tation of the continuous ACS. High costs have also resulted from a failed initiative to automate the nonresponse follow-up operation, which will use traditional paper-and-pencil questionnaires instead of handheld computers as originally planned. Moreover, contributing to cost increases over the entire 40-year span has been the accretion of operations to address special situations, which have not been fully evaluated to determine their benefits and costs. Underlying this accretion and the failure to achieve major steps forward on a timely basis, such as the nonresponse follow-up automation initiative, is a largely incremental approach to census design and a research and development program that is often unfocused and ineffective. Census Quality The overall quality of the census enumeration using the current mailout/mailback methodology has improved between 1970 and 2000, to the point at which major continued improvement is unlikely. The net un- dercount of the population (measured by the demographic analysis method) decreased from 2.7 percent in 1970 to 0.1 percent in 2000. Moreover, the difference in net undercount rates between African Americans and all others decreased from 4.3 percentage points in 1970 to 3.1 percentage points in 2000. These figures mask large numbers of offsetting errors of omission of people who should have been counted and duplicate or other types of erroneous enumeration of people who should not have been counted. Some programs put in place to address coverage errors, such as a recheck of housing units initially classified as vacant, have contributed to reducing net undercount (they have also contributed to erroneous enumerations).

OCR for page 3
6 ENVISIONING THE 2020 CENSUS Other programs, sometimes put in place with no advance testing, such as a parolee and probationer check in 1990, have contributed little to cover- age improvement. Given trends that make census-taking more difficult, it is unlikely that the increased costs anticipated for the 2010 census will achieve much, if any, further increase in quality. In order to lead to successful quality improvements for 2020 any additional expenditures would, at a minimum, need to be properly focused and carefully researched and planned. Social and Technological Change Changes in the U.S. population since 1970 that have made census-taking more difficult are well known. They include increased immigration, includ- ing illegal immigration, and therefore the existence of communities that are wary of cooperating with the census and in which English is not the primary language. Changing norms in residence and living arrangements have also complicated the concept of a single, usual place of residence for many peo- ple, including children in joint custody, people with seasonal homes, com- muter workers or couples who maintain a separate workweek residence, and others. More broadly, the public’s willingness to respond to surveys has declined significantly over the past three decades. Given these factors, an incremental approach to census design and an unfocused R&D program are likely to result in a continuation of the pattern of escalating census costs in 2020 with little or no improvement in quality. Nevertheless, important dynamics that are dramatically changing the en- vironment in which the census is taken offer both challenges and opportuni- ties to make the 2020 census markedly more cost-effective. Administrative record files from government assistance and other programs are improving in quality and completeness, the Internet is now a ubiquitous means for communication, and the proportion of household business that is carried out through the ordinary mail is shrinking. It is difficult to paint a reliable picture now of the United States in 2020, but it is easy to conceive of the Internet as being the primary method for communication and conducting household business for the great majority of residents, and that a large frac- tion of U.S. households may opt not to receive ordinary mail or may ignore much or all of it. It is also likely that administrative records will provide timely, high-quality, and inexpensive information that would be a useful in- put to a variety of census operations. Clearly, the design of the 2020 census should be dramatically different to accommodate such changes. TOWARD A NEW VISION FOR THE 2020 CENSUS To escape the pattern of incremental and often unfocused research and development for the next census leading to escalating costs with diminishing

OCR for page 3
SUMMARY 7 returns on quality, the Census Bureau will need to completely overhaul its approach for planning the 2020 census. Planning for 2020 must begin with a set of clear goals for census costs and quality, a limited set of strategic visions that are likely to meet those goals, and an R&D program that is focused on determining which vision has the most promise and how best to implement it. The goal for census costs should recognize that an increase in real dollar per housing unit cost for 2020 over 2010 would be unjustified in comparison with the experience of other developed nations and unacceptable in a time of fiscal imbalance. Recommendation 2.1: The Census Bureau should approach the design of the 2020 census with the clear and publicly announced goal of reducing the inflation-adjusted per housing unit cost to that of the 2000 census (subtracting the cost of the 2000 census long-form sample), while holding coverage errors (appropriately defined) to approximately 2000 levels. To be clear, we do not make this recommendation to suggest cutting census costs simply for the sake of cost-cutting. The objective is a more effi- cient census that, for example, uses lower cost options like the Internet and administrative records to supplement or replace expensive follow-up opera- tions without compromising census quality; effective research and improved cost modeling (see below) are critical to determining to what extent these efficiencies can be realized. The reason for setting a cost goal that is stark, ambitious, and public is to convey commitment to a focused 2020 census planning process. This 2020 census cost goal should exclude the costs of the ACS, which will increasingly take on a life of its own as a source of intercensal data on a wide range of population characteristics. Recommendation 3.1: The Census Bureau should immediately develop a limited number of strategic visions for the 2020 cen- sus that are likely to meet its announced goals for costs and quality. By strategic visions, we mean start-to-finish strategies for conducting all major census operations in order to confront looming threats and implement new technologies. In addition, evaluation of each vision should include thorough review of the costs and benefits of individual census operations relative to the announced cost and quality goals, to determine whether opera- tions that are not demonstrably cost-effective may be eliminated or scaled back or whether new technologies or practices might usefully be introduced. Recommendation 3.2: The Census Bureau should develop its 2020 census research and development program by identifying a handful of research questions whose resolution will determine

OCR for page 3
8 ENVISIONING THE 2020 CENSUS which of the visions of the next census are feasible and cost- effective and which are not. Priorities for evaluations of the 2010 census should be arrived at consistent with this research program, as should priorities for experiments and tests in the 2011–2018 period. Recommendation 4.7: The Census Bureau’s planning for the 2020 census, particularly for research in the period 2010–2015, should be designed to permit proper evaluation of significant innovations and alternatives to the current decennial census de- sign that will accomplish substantial cost savings in 2020 with- out impairing census quality. Otherwise, the census design in 2020 will either be an incremental change from that in 2010 with increased costs, or the Census Bureau may be compelled to implement a poorly evaluated and tested alternative design under severe time and cost constraints with a risk of substan- tially reduced quality. All involved, including Congress and the administration, should recognize that substantial cost savings in 2020 can be achieved only through effective planning over the course of the 2010–2020 decade and should fund and pursue research efforts commensurately. PROVIDING TOOLS FOR CENSUS R&D To make possible a truly focused R&D program for 2020 census plan- ning, the Census Bureau needs to take immediate steps to develop the neces- sary tools for effective planning and evaluation. These tools include an im- proved, transparent cost model of census operations and well-documented data from all 2010 census operations in formats suitable for research and evaluation. Recommendation 3.3: In order to provide early indications of the costs of competing visions for the 2020 census and to sup- port effective planning throughout the decade, the Census Bu- reau should develop and validate a detailed cost model that not only represents the 2010 census, but also accommodates novel approaches to census-taking, including the use of data capture via the Internet and automated telephone systems, the use of handheld devices in nonresponse follow-up, the use of adminis- trative record information for some types of nonresponse follow- up cases, and innovative mechanisms for reducing the costs of updating the Master Address File between 2010 and 2020. This cost model should be able to assess the implications of intro- ducing specific changes to an existing design, singly and in com-

OCR for page 3
SUMMARY 9 bination, and to distinguish between the direct and indirect cost effects of specific changes. This cost model should be thoroughly documented and transparent so that the Census Bureau can ob- tain the benefit of expert advice on cost-effective improvements to census operations. Recommendation 3.4: The Census Bureau should retain suffi- cient input and output data, properly linked and documented, from each 2010 census operation to permit adequate evaluation of the contribution of the operation to census costs and data quality to feed into 2020 census planning. For this purpose, the Census Bureau should either establish an internal group or hire a contractor with database management expertise. This group would have the responsibility of retaining and documenting suf- ficient data from the 2010 census to be able to comprehensively represent the functioning of all census operations. Such a group would also have the responsibility of assisting Bureau research staff, using current database management tools, to produce re- search files to support the assessment of analytic questions con- cerning aspects of the 2010 census. REVITALIZING THE RESEARCH INFRASTRUCTURE To support a focused, goal-oriented R&D program for the 2020 census and, more broadly, to improve its research capabilities, the Census Bureau needs to take immediate steps to revitalize its research infrastructure. These steps include thorough assessment of the Census Bureau’s posture toward R&D compared with other data collection organizations and the develop- ment of means for research to break out of existing organizational “silos.” Recommendation 4.1: The Census Bureau should comprehen- sively review the research and development practices and orga- nization in other national statistics offices and in survey orga- nizations in academia and the private sector, with the goal of modernizing and strengthening the Bureau’s own research and development program. Such a review should include assess- ments of and recommendations about: • How to organize and direct basic and applied methods re- search to best serve the decennial census and other Census Bureau programs; • How to organize information technology and database management to best serve research and operations, includ- ing how to manage the development of new technologies

OCR for page 3
10 ENVISIONING THE 2020 CENSUS and ensure access to adequate expertise in these technical areas; • How to operate collaborative project teams to facilitate timely innovation; • How to ensure adequate training in survey methods and related fields; • How to achieve extensive and intensive interaction with external research organizations and academic departments so that Census Bureau researchers and methodologists can benefit from related research work and ideas elsewhere; and • How to fund and establish priorities for research and ap- plied methodology work. Recommendation 4.4: The Census Bureau should put in place incentives and structures so that research is fully integrated and collaborative not only across programs, but also with opera- tional planning. Research should be responsive to operational needs, and, in turn, research findings should play a primary role in informing operational decision making. Necessary steps also include reestablishing the position of Associate Di- rector of Methodology and Standards, reestablishing a strong Center for Survey Methods under that associate director, integrating research across programs and with operational planning, integrating census and ACS re- search, and renewing and refreshing mechanisms for obtaining outside ex- pert advice. Recommendation 4.2: To carry out the findings from the re- view recommended above, the Census Bureau should consider reestablishing and filling an associate director–level executive staff position to head the statistical and survey research activ- ities at the Census Bureau, with authority to organize the Bu- reau’s research and applied methods activities. Recommendation 4.3: The Census Bureau should give greater emphasis to survey methodology. One possibility for doing so would be to establish a core survey methods research center, staffed by full-time survey researchers and headed by a nation- ally recognized expert in census and survey data collection in- struments. Such a high-profile center could give priority to re- search on making effective initial contacts with census and sur- vey respondents, including those made with new technologies. Recommendation 4.5: The Census Bureau should integrate de- cennial census and American Community Survey research—for

OCR for page 3
SUMMARY 11 example, by using the ACS methods panel as a test bed for the Internet and other data collection methods to consider in the census and by matching census and ACS records to evaluate cov- erage in both programs. To support comparative census-ACS re- search and to inform users, the Census Bureau should carry out analyses that explore, at both the aggregate level and the level of individual households, the degree of differences and the source of differences in demographic characteristics and residence be- tween the ACS and the decennial census. Recommendation 4.6: The Census Bureau should renew and augment mechanisms for obtaining external expertise from lead- ing researchers and practitioners in survey and census method- ology and in relevant computer science fields. These mecha- nisms might include (1) a more active census professional ad- visory committee program in which the members have an op- portunity to work more closely with Census Bureau staff in de- veloping and evaluating ideas for improved census and survey methods; (2) increased opportunities for sabbaticals at the Bu- reau for university faculty and other short-term appointments for both senior- and junior-level (graduate student) academics at the Census Bureau; (3) increased opportunities for sabbaticals for Census Bureau staff at academic institutions and private- sector survey organizations; (4) the awarding of design contracts early in the decade to support research and development of in- novative technologies for census and survey data collection and processing; and (5) more effective use of contracting processes to obtain expert services. 2020 R&D FOR STAGES OF CENSUS-TAKING The strategic visions developed for the 2020 census will include differ- ent combinations of approaches for more cost-effective address list develop- ment, initial contact with and response mechanisms for household and group quarters residents, and follow-up operations. The panel suggests targets of opportunity for focused R&D for each of these stages of census-taking. Address List Development A priority for determining a more cost-effective strategy for developing the Master Address File (MAF) between 2010 and 2020 is to conduct a comprehensive evaluation of the accuracy and costs of the 2010 Decennial Master Address File (DMAF) at each stage of its development. This eval- uation should determine, by comparison with results from the 2010 cen-

OCR for page 3
12 ENVISIONING THE 2020 CENSUS sus, the 2010 ACS, and the 2010 Census Coverage Measurement, which operations—Postal Service updates, Local Update of Census Addresses, block canvass, Vacant/Delete Check, and others—resulted in accurate ad- dresses (occupied and vacant), which operations added nonresidential and nonexistent units, and which operations erroneously deleted valid addresses. The evaluation should examine the comparative performance of various ad- dress operations for geographic areas, types of neighborhoods, and types of structures, such as single-family residences, small multiunit structures, and large multiunit structures. It should simulate the effects on census costs and accuracy of deleting one or more address operations nationwide and for types of areas and structures. Research and development should investigate the use of administrative records from government programs to replace more expensive address up- dating operations in some areas and the use of information on population and housing trends from the ACS and the Census Bureau’s population es- timates program to target areas in which address canvassing may not be needed. R&D should also investigate the costs and benefits of continuous updating of the MAF with the cooperation of some technologically savvy state and local governments and pilot-test a continuous updating program in selected areas. A continuously updated MAF would have benefits through- out the decade for the ACS and other household surveys. Initial Contact and Response There are a number of ways of contacting households to let them know how and when to respond to the census, including not only the traditional mailing of a questionnaire but also mailing a postcard, sending an e-mail, or leaving a voice message. Similarly, Internet and interactive voice response could become much more prevalent modes of household response. There are issues of confidentiality protection and ensuring the ability to link a re- sponse to a particular address, yet other countries have solved these prob- lems and are making extensive use of Internet response to their censuses. Only about 3 percent of people reside in institutions and other kinds of group quarters, but in some local areas they constitute a large proportion of the population. Group quarters enumeration poses special challenges— first, of identifying all group quarters and then of gaining access and accu- rately enumerating group quarters residents. An early priority for R&D on more effective group quarters enumeration should be to investigate the use of administrative records to enumerate the residents of institutional group quarters, such as nursing homes, prisons, and college dormitories.

OCR for page 3
SUMMARY 13 Follow-Up Nonresponse follow-up and other field operations, such as the Vacant/ Delete Check, are time-consuming and add substantially to the costs of the census. Focused R&D for a cost-effective 2020 census should address ways to minimize field operations by making more extensive use of administrative records, as well as ways to reduce the costs of fieldwork, assuming that some level of in-person follow-up will continue to be required. R&D on minimizing field operations should analyze 2000 and 2010 in- formation to determine the likely effects on costs and data quality of using administrative records to replace some or all of nonresponse follow-up in some or all geographic areas. Options to consider include the use of ad- ministrative records to replace current last-resort procedures in late stages of follow-up; the use of administrative records after one or two attempts at in-person nonresponse follow-up; and the use of administrative records at the outset so that nonresponse follow-up is conducted only when there is no match for an address to a record. Options for reducing the cost of in-person follow-up include reducing the number of contact attempts be- fore using last-resort procedures (or administrative records) and the use of handheld technology in place of paper questionnaires. Cost-Benefit Review of Individual Census Procedures A close examination of the modern U.S. census reveals the accretion of many individual procedures to address special situations for address list de- velopment, initial contact, nonresponse follow-up, and telephone and in- person coverage improvement follow-up. To satisfy a goal of a substantial reduction in real dollar per housing unit cost for the 2020 census, the Cen- sus Bureau will need not only to test and develop major changes in census operations, but also to examine thoroughly each individual procedure to determine its contribution to census costs and data quality. This exercise will require the detailed, transparent cost modeling recom- mended above, so that stakeholders can be informed regarding trade-offs between cost savings and possible adverse effects on completeness of cen- sus coverage. While significant deterioration of data quality should not be contemplated, elimination of procedures that only marginally benefit quality and have significant costs should not be ruled out a priori. The U.S. census needs to be of high quality; it also needs not to consume disproportionate resources. Focused R&D that builds from strategically chosen evaluations of 2010 census results and is supported by detailed cost modeling is the nec- essary ingredient to achieving a new vision for a cost-effective 2020 census.

OCR for page 3