THE NATIONAL ACADEMIES
National Academy of Sciences
National Academy of Engineering
Institute of Medicine
National Research Council
Panel on Research on Future Census Methods
Committee on National Statistics Division on Behavoral and Social Sciences and Education
February 15, 2001
Dr. William Barron
U.S. Bureau of the Census Room 2049, Building 3 Washington, DC 20233
Dear Dr. Barron,
At the third meeting of the Panel on Research on Future Census Methods of the Committee on National Statistics (CNSTAT), convened on December 7, 2000, Census Bureau staff presented major elements of the Bureau’s current strategy for the 2010 census. The panel offers this letter report as an assessment of that strategy.
The panel commends the Census Bureau staff members for their presentations at the December 7 meeting. We understand the importance of early planning for census operations and, accordingly, we see the formation of a group with 2010 census planning responsibilities and the subsequent summary of that group’s facilitated discussions as a positive step. The panel also applauds the scope of evaluations and experiments currently under way to evaluate the 2000 census and inform future census efforts. In particular, the panel looks forward to the development of the Master Trace Sample—a compilation of major census databases for a systematic sample of addresses—because it will not only permit highly detailed evaluation of 2000 census work, but also fill a long-unmet need for a resource that can use past census experience to answer unanticipated design questions for future censuses.
As the panel understands it, the Census Bureau’s current strategy for the 2010 census is that only traditional short-form data will be collected. The Bureau expects that simplification to a short-form-only census will streamline data capture and facilitate questionnaire collection through the Internet. Moreover, the Bureau anticipates updating the Bureau’s Master Address File (MAF) and Geographic Data Base (TIGER) with Geographic Positioning System (GPS) data and local partners’ address contributions. The Bureau believes that this will improve the accuracy of census mailings, as well as allow the use of hand-held computers to pinpoint address locations and collect follow-up information.
The success of the strategy presented at the December 7 panel meeting is based on three critical assumptions:
that implementation of the American Community Survey (ACS) will eliminate the need for a long form in the 2010 census and that the development of a Long Form Transitional Database will allow a smooth transition from the traditional decennial snapshots of long-form characteristics to annual ACS measurements;
that the MAF and TIGER databases can be fully updated and modernized over the next decade; and
that early integrated planning across the various divisions of the Bureau will occur in a timely fashion.
If these assumptions are correct, the Bureau expects substantial cost savings in carrying out the 2010 census.
2101 Constitution Avenue, NW, Washington, DC 20418 • Telephone (202) 334 3096 • Fax (202) 334 3751 • national-academies.org
The panel generally agrees with the Census Bureau on the goals of the two major initiatives of the current 2010 census strategy—the ACS and the MAF/TIGER upgrade—and strongly agrees on the importance of early planning. However, the panel has two serious concerns about the current state of development of the 2010 census strategy. First, the strategy, as presented to the panel, appears to lack an overall framework. Second, the strategy relies on assumptions of uncertain validity and on assessments that currently have little analytic support. Thus, the panel cannot provide a considered evaluation and endorsement until the Bureau moves from the general outline of a strategy to a more detailed and better-integrated plan that addresses feasibility and cost-effectiveness. The panel also awaits the Bureau’s completion of evaluation studies and experiments since these empirical analyses will be crucial in building an overall 2010 census framework.
With regard to the lack of an overall framework, we mean that the Census Bureau has not developed a clear statement of the objectives that the 2010 census and the ACS are intended to accomplish. When possible, such objectives should be stated in quantifiable and measurable terms. An example is gross overcount and undercount and how they may differ among subpopulations. More generally, the panel would like to see a clearer case for components of the 2010 census strategy, itemizing the goals, costs, and benefits of each initiative and indicating how they integrate and contribute to a high quality census.
The description of the current strategy suggests planning that is overly compartmentalized. For example, the panel is unclear as to how the broad ACS and MAF/TIGER initiatives will be operationalized within the structure of the Census Bureau: how and when the two initiatives will interact (e.g., refreshing the ACS address file based on an updated MAF/TIGER), where the decision-making locus lies, and how these broad initiatives interact with other ongoing operations such as the Current Population Survey (CPS). An approximate timeline for completion of MAF/TIGER and ACS tasks, embedded in the framework of overall census processes, would help us in assessing the utility of the proposed strategy and in guiding our discussion of other crucial planning topics, such as the acquisition of new technology and outreach to special populations.
With regard to the Bureau’s current assumptions, the strategy critically assumes that no long form will be necessary in the 2010 census. The validity of this assumption appears to depend on the further assumption that the ACS and MAF/TIGER initiatives will be fully funded over the entire decade. The panel believes that it would be prudent to approach 2010 planning with a contingency plan. That is, the Bureau should consider its possible responses under various funding scenarios (for instance, that the ACS is funded only in the early part of the decade rather than throughout the decade) and have contingency plans for those possibilities.
The current strategy asserts that the census will achieve cost savings through adoption of new technologies and, particularly, through implementation of the ACS. But no cost or budget estimates have been provided to validate these claims, and the panel has not seen evidence that the proposed combination of a short-form census with the ACS and associated use of technology will reduce costs relative to a typical short-form/long-form census. The panel urges the Bureau to broaden its justification for the ACS, detailing the need for and use of long-form data and how those data needs will be addressed through the ACS, perhaps in conjunction with the CPS and other demographic surveys. Accordingly, the Bureau should expedite ongoing evaluations that assess the quality of ACS data relative to the quality associated with the traditional census long form.
The panel is also concerned that the current strategy does not appear to address techniques for coverage measurement and improvement or the assessment of undercount or overcount. One particular concern with respect to coverage improvement is the lack of an approach for dealing with missed persons in otherwise enumerated housing units. Consistent with its earlier suggestion that the Bureau develop
contingency plans—and mindful of the mid-course changes that had to be made to the 2000 census plan due to legal and political developments—the panel believes that it would be a mistake to defer these important issues to the latter part of the decade. Rather, the panel urges the Bureau to include coverage measurement and improvement in its early planning efforts and strive for a census plan that is flexible enough to accommodate potential changes late in the process.
The Bureau has emphasized early integrated planning as critical to a successful 2010 census, but the materials presented to the panel suggest compartmentalized thinking, without due attention to how efforts across divisions within the Census Bureau will be coordinated and synthesized. More broadly, the strategy at present lacks a “business plan”—that is, a clear statement of objectives and how they will be accomplished, when various steps in the census process must be completed, how much those steps will cost (in terms of both monetary and nonmonetary resources, including those that must be diverted from other areas to complete tasks in a timely fashion), the degree to which the steps will interact, and what benefits will accrue through each step. A critical feature of such a “business plan” for the 2010 census is a full enumeration of the costs and benefits—either monetary or nonmonetary—associated with each component of the census strategy. An example of a nonmonetary benefit is the improvement in quality of short-form data and timeliness of long-form data if the long form is replaced by the ACS.
The Bureau is currently conducting a wide array of evaluation studies and experiments designed to assess the quality of the 2000 census and inform approaches to the 2010 census. As noted above, the panel applauds the scope of these evaluation studies. However, the panel is concerned that the Bureau has not sufficiently focused its evaluation program and has instead labeled most of its evaluation categories as high priority. As recommended in its first interim report, Designing the 2010 Census, the panel urges the Bureau to “identify the benefits and resources required for each evaluation study, set priorities among them, and allocate sufficient resources for the completion of all or, at least, the highest priority evaluations.” Furthermore, efficient census planning requires the early completion of the evaluation studies that will shed light on the potential costs and operational efficiencies of the new approaches that are proposed. Therefore, the panel urges that—to the extent possible—high priority be given to completing evaluations that are most directly relevant to early 2010 census planning. These evaluations will also enable the panel to have a sound empirical foundation for its assessment of the Bureau’s plans.
In conclusion, the panel commends the Bureau’s early planning efforts for the 2010 census and looks forward to providing a fuller assessment of the Bureau’s 2010 strategy. In furthering these efforts, the panel recommends the following:
The Census Bureau should produce a “business plan” for the 2010 census that provides an overall framework for development. Such a plan should include (1) a clear statement of objectives, (2) an approximate timeline for completion of tasks, (3) a cost-benefit analysis of the various components of the plan, and (4) a fuller explanation of how intra-Bureau efforts will be coordinated. In assessing the costs and benefits (both monetary and nonmonetary) of a re-engineered 2010 census, attention should be given to potential effects of new processes on census coverage and differential undercount and their measurement.
The Census Bureau should give high priority to evaluation studies and data analyses that are important to building an overall 2010 census framework. Attention should be focused on evaluation studies that directly address the following priority issues:
the relative impact of various processes (such as the Local Update of Census Addresses [LUCA] and block canvassing) that were used to assemble the 2000 census MAF;
the effect of local partnerships on the data collection process;
comparison of estimates from the ACS and 2000 census long-form data, in sites where both are available;
coverage of the population, disaggregated by demographic and geographic subgroups;
the effectiveness of major automated systems for data collection, capture, and processing;
the quality and completeness of long-form data collection; and
the effectiveness of operations used to designate special places and enumerate the group quarters and homeless populations.
We hope that this critique will be seen as the constructive effort that we have intended and that it will clarify and assist the Bureau’s planning as it further develops its 2010 census strategy. Again, we thank the Census Bureau staff for its cooperation in the panel’s activities and look forward to continued interaction.
Benjamin F. King
encl: Panel Roster