CHAPTER 2
The General Plan for the 2010 Census

IN ITS EARLY PLANNING FOR THE 2010 CENSUS, the Census Bureau has, to its credit, suggested a general approach that promises much more than a mere incremental improvement of the census performed 10 years earlier. If carried through as planned and if all of its elements are fully realized, the 2010 census will result in the most dynamic change in census-taking since the 1970 switch to an enumeration conducted principally by mail rather than by ringing doorbells or knocking on doors. The success of the Bureau’s vision—an extensively reengineered 2010 census—is, however, highly contingent on the completion of much difficult work in the next few years.

As context for discussion of the developing plans for the 2010 census, we review the basic steps involved in performing a decennial census in Section 2-A. We then briefly review the major milestones and problems of the 2000 census and its planning process (Section 2-B) before outlining the major initiatives of the Census Bureau’s general plan for the 2010 census (Section 2-C). Finally, in Section 2-D, we express and elaborate on our belief that, based on the information that has been provided to us, many of the Bureau’s plans for 2010 are at serious risk of failure if certain deficiencies are not corrected. This is our great-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 23
Reengineering the 2010 Census: Risks and Challenges CHAPTER 2 The General Plan for the 2010 Census IN ITS EARLY PLANNING FOR THE 2010 CENSUS, the Census Bureau has, to its credit, suggested a general approach that promises much more than a mere incremental improvement of the census performed 10 years earlier. If carried through as planned and if all of its elements are fully realized, the 2010 census will result in the most dynamic change in census-taking since the 1970 switch to an enumeration conducted principally by mail rather than by ringing doorbells or knocking on doors. The success of the Bureau’s vision—an extensively reengineered 2010 census—is, however, highly contingent on the completion of much difficult work in the next few years. As context for discussion of the developing plans for the 2010 census, we review the basic steps involved in performing a decennial census in Section 2-A. We then briefly review the major milestones and problems of the 2000 census and its planning process (Section 2-B) before outlining the major initiatives of the Census Bureau’s general plan for the 2010 census (Section 2-C). Finally, in Section 2-D, we express and elaborate on our belief that, based on the information that has been provided to us, many of the Bureau’s plans for 2010 are at serious risk of failure if certain deficiencies are not corrected. This is our great-

OCR for page 23
Reengineering the 2010 Census: Risks and Challenges est concern in this report and it will be discussed in detail in the chapters that follow. 2–A BASIC STEPS IN THE DECENNIAL CENSUS PROCESS Conducting a decennial census of the United States presents massive logistical challenges on many levels. It has been said that the fielding of the 2000 census—with more than 860,000 short-term employees serving as enumerators—constituted the “largest peacetime civilian mobilization” in American history (U.S. Department of Commerce, Office of Inspector General, 2000:3). To motivate discussion of the proposed changes for the 2010 census process, it is helpful first to consider the basic steps that characterize the modern decennial census. We have listed these basic steps in Box 2.1 and describe them in more detail (with particular reference to their implementation in the 2000 census, as appropriate) in the remainder of this section. 2–A.1 Preparation The ultimate quality of a census depends critically on successful completion of a number of preparatory steps. First, the Census Bureau must establish an organizational structure (1) as it mobilizes for the count, from the definition of staff roles at Census Bureau headquarters in Suitland, Maryland, to the establishment and operation of hundreds of temporary local census offices (LCOs). The basic high-level organizational structure of the Census Bureau is described in Box 2.2. In addition to the human organizational structure, census planners must also piece together the broader technical infrastructure of the census—the amalgam of people, computer hardware and software systems, and telecommunication networks that will be used to support all aspects of the census process. Of the initial preparations that must be made for a census, the development of an address list (2) is arguably the most crucial. Recent decennial censuses have relied heavily on the delivery and return of questionnaires by mail for most of the population, and the quality of mail census operations relies on the strength

OCR for page 23
Reengineering the 2010 Census: Risks and Challenges Box 2.1 Basic Steps in Conducting the Decennial Census Preparation Establish organizational and technical infrastructure Develop an address list Determine questionnaire content Design questionnaires and mailing materials Develop enumeration procedures and assign to areas of the country Design plan for coverage evaluation Establish advertising and outreach programs Taking the Count Deliver questionnaires Follow up with nonresponding households Conduct coverage evaluation operations Data Processing Capture data from completed questionnaires Apply editing, imputation, and unduplication procedures Tabulate and release data Research (before, during, and after census) Test proposed operations and procedures Evaluate operations and procedures Design and conduct experiments and trials of the address list. The address list, combined with other geographic resources, is also essential to the accurate tabulation of census results. In 2000, the Census Bureau’s address list—the Master Address File (MAF)—was constructed by augmenting the preserved 1990 address list with inputs from additional sources. The development of the 2000 MAF is described in detail in Section 3-A.1. In 2000, as in previous censuses, a short-form questionnaire asking for basic data items was administered to most households, while a census long form asking the basic data items along with many other socioeconomic and demographic questions was administered to an approximate 1-in-6 sample of households. Hence, determination of questionnaire content (3) is an impor-

OCR for page 23
Reengineering the 2010 Census: Risks and Challenges Box 2.2 Organization of the Census Bureau The Census Bureau is headed by a director, appointed by the president with the advice and consent of the Senate. The directorship is a political, not a fixed-term, appointment. In addition to a deputy director, the Bureau’s current organizational chart also includes two principal associate director positions, one of which serves as chief financial officer, and the other of which is designated as principal associate director for programs (the principal associate director positions are currently not filled). Under these top levels of management, eight associate directors oversee particular aspects of the Census Bureau’s operations. Of these directorates, two are particularly key to decennial census operations: Decennial Census: The associate director for decennial census oversees all decisions for census planning, budget, and operations, and all funding for decennial-census-related activities is coordinated under this directorate. Under this directorate, the Decennial Management Division is responsible for planning and coordination as well as liaison with oversight groups such as the U.S. General Accounting Office, the U.S. Department of Commerce Inspector General, and congressional committees. The Decennial Statistical Studies Division develops the statistical methodology for use in census operations, while the Decennial Systems and Contracts Management Office is the primary contracting arm for decennial census operations (including external contracts for such operations as questionnaire printing and data capture). Finally, the Geography Division maintains the Master Address File and TIGER geographic database and administers the Bureau’s other cartographic activities. Field Operations: The associate director for field operations oversees the Field Division, which is charged with conducting the actual field enumeration for the census and other survey programs. This directorate also oversees the Bureau’s permanent National Processing Center in Jeffersonville, Indiana, and 12 regional offices; during conduct of the decennial census, hundreds of temporary local census offices also come under this directorate’s purview. Other directorates include divisions that play key roles in the decennial census: Demographic Programs: Until recently, the American Community Survey was administered by the Demographic Surveys Division under the Demographic Programs directorate; this authority was transferred to the Decennial Management Division. A key part of the Demographic Programs directorate is the Population Division, which is responsible for the production of postcensal population estimates between census years. Methodology and Standards: The Methodology and Standards directorate is home to the Planning, Research, and Evaluation Division, which administered the suite of evaluations of census operations and data quality in 2000 (see Section 8-A). Additional expertise is provided by the Statistical Research Division and the Computer Assisted Survey Research Office, which assists with the development and testing of electronic questionnaires.

OCR for page 23
Reengineering the 2010 Census: Risks and Challenges Basic administrative support for the Census Bureau and decennial census activities is provided by three directorates: Finance and Administration, Information Technology, and Communications. The eighth directorate in the Census Bureau hierarchy is Economic Programs; while it is not involved in decennial census operations, it administers the Census Bureau’s extensive portfolio of business and economic surveys, including the stand-alone Economic Census. SOURCES: Thompson (2000); Census Bureau organizational chart available at http://www.census.gov/main/www/m-img/orgchart.jpg [2/23/04]. tant part of census preparation. The exact set of questions has varied in the past, depending in part on legal and regulatory requirements on the collection of data, as has the mix of questions between the short and long forms. In the 2000 census, for instance, marital status was shifted to the census long form along with such questions regarding housing as number of rooms and amount spent on rent (see National Research Council, 2004:App. B). Once questionnaire content has been finalized, the actual questionnaires and mailing materials must be developed and tested (4). Even small changes on questionnaires can be consequential, and so the exact phrasing and layout for questions such as race and Hispanic origin are extensively debated. For 2000, the question on Hispanic origin was moved to immediately precede the race question and—for the first time—census respondents could choose to identify themselves as belonging to more than one racial category. Critical choices then involve determining exactly which enumeration procedures (5) will be applied to which areas of the country. Most of the population—some 82 percent in 2000—lives in areas with predominantly city-style (street number and name) addresses, and so is assigned for enumeration via mail (questionnaires are sent by mail and are expected to be returned by mail). But mailout/mailback was just one of nine types of enumeration areas (TEAs) used in the 2000 census, with a variety of other methods being applied to count the remaining 18 percent of the population. Among these other options are update/leave, used in predominantly rural areas, in which enumerators drop off questionnaires at specific housing units and update address informa-

OCR for page 23
Reengineering the 2010 Census: Risks and Challenges tion as possible; it is hoped that residents will then return the form by mail. In 2000, the Census Bureau applied update/leave methods in select portions of some urban areas in which it was thought that there might be problems with mail delivery. Other approaches used in different TEAs include list/enumerate (updating address records and collecting questionnaire responses in a single enumerator visit) and special operations for the military and remote areas in Alaska. In addition to the nine designated TEAs, separate operations are deployed to count the population residing in group quarters (those living in places such as hospitals, dormitories, and prisons) and the population served by facilities like soup kitchens and shelters. As we will discuss further in Section 2-B, the basic census step that garnered the most attention and focus in preparing for the 2000 census was designing a plan for coverage evaluation (6). Coverage evaluation programs are intended to use independent measures to estimate the accuracy of the census count, both in the aggregate and for specific demographic groups. Coverage evaluation plans in 1990 and 2000 centered around an independent postenumeration survey, though demographic analysis estimates (essentially, updating population counts by adding births and immigrants and subtracting deaths and emigrants) were also used as a corroborative measure. A final piece in the preparation for the census count is establishment of advertising and outreach programs (7) to facilitate cooperation in the census. The 2000 census departed from past programs that relied on donated commercial time; the Bureau was instead more aggressive in marketing and raising public awareness of the census. 2–A.2 Taking the Count The actual conduct of the census is done in two operational phases: delivery of questionnaires (8), either by mail or in person depending on the type of enumeration area, and visits by enumerators to follow up with nonresponding households (9). Of course, this basic breakdown masks a great deal of complexity, as each of these broad steps involves significant planning and constant management.

OCR for page 23
Reengineering the 2010 Census: Risks and Challenges Particularly complicated is the process of enumerator followup, which itself had two phases in the 2000 census. The first stage, nonresponse follow-up (NRFU), sent enumerators to each household that did not return a questionnaire to either collect questionnaire information or ascertain the status of the address (e.g., whether it was a vacant unit or nonresidential). Enumerators were instructed to collect questionnaire information whenever they could obtain an interview, even if the respondents indicated that they had already mailed back a form. In 2000, NRFU lasted 8 weeks, between April and June, and processed a total workload of 41.7 million addresses (National Research Council, 2004). The second stage of follow-up in 2000, coverage improvement follow-up (CIFU) made further checks on 8.9 million housing units, the majority of which had been labeled vacant or nonexistent in the NRFU operation. For mail returns, a coverage edit follow-up (CEFU) operation was conducted by telephone in order to resolve discrepancies between the number of residents claimed on the form and the number for which characteristics were actually reported (the 2000 questionnaire limited households to providing detail on 6 members). The CEFU phase did not involve the use of field enumerators in the event that telephone contact failed. After census field operations have concluded, coverage evaluation operations (10) are initiated. For coverage evaluation using a postenumeration survey (as in 2000), the basic goal is to conduct a survey that is completely independent of the census operations and that obtains information about people who lived in particular households on Census Day, April 1. Thus, the postenumeration survey must be conducted as quickly as possible after Census Day (to minimize reporting errors by respondents) but not so close that census and coverage evaluation enumerators are simultaneously working on getting responses from households (which could violate the assumption of independence). 2–A.3 Data Processing Particularly crucial segments of the census technical infrastructure come into play once questionnaire delivery begins and questionnaires are returned. To begin, the data must be extracted

OCR for page 23
Reengineering the 2010 Census: Risks and Challenges from the paper questionnaire, in a process known as data capture (11). For the 2000 census, the Census Bureau took the step of working with outside contractors and incorporating optical character recognition (OCR)—electronically parsing handwritten responses on paper questionnaires scanned into a computer. [The 1990 and earlier censuses had relied on optical mark recognition (OMR), in which the computer is intended to recognize marks only in particular locations on the questionnaire (e.g., a filled-in circle to indicate a particular response category).] Data were input by hand from scanned images of the questionnaire, as necessary, if the OMR and OCR methods failed for some reason. Once captured from the questionnaires, the census data are then processed. In particular, editing, imputation, and unduplication procedures (12) are applied to the data. Edits range from the very simple (e.g., calculating age if date of birth is given but age is not reported) to very complex, such as assessing the consistency of reporting of various types of employment information on the census long form. Imputation routines can be used to fill in values for missing data items on otherwise complete household data records or to fill in values for entire persons if their information is not included on the census return. For the past several censuses, the Bureau has relied on “hot-deck” methods that fill in responses using a pool of possible responses, drawn roughly from questionnaires in nearby households. Finally, the data are unduplicated to the extent possible, in terms of both persons and housing units. The 2000 census relied on a primary selection algorithm (PSA) to determine a single response if multiple data records (questionnaire results) were reported for the same housing units. Unduplication experience in 2000 is described further in Section 2-B. Finally, when the data have been processed, they are ready for tabulation and release (13). By law, census counts by state must be reported to the president by 9 months after Census Day for the purpose of reapportioning the U.S. House of Representatives. Under the now-traditional April 1 Census Day schedule, this means generation of apportionment population counts by December 31 of the census year. Census law also mandates that data for legislative redistricting must be issued within 1 year of Census Day.

OCR for page 23
Reengineering the 2010 Census: Risks and Challenges 2–A.4 Research A final broad set of basic census steps relate to research. These steps differ from the ones previously listed because they deviate from the rough chronological flow of the census process. In principle, census research should occur before, during, and after the census. In terms of testing proposed operations and procedures (14), it has become the norm for the last major test before the census to be a dress rehearsal 2 years prior. Each census is also accompanied by a suite of evaluations of census operations and procedures (15). For the 2000 census, the Bureau developed a program of 91 formal evaluation studies related to census operations (reduced from an initial total of 149). These operational evaluations were complemented in the 2000 census cycle by three separate waves of research studies connected to the Accuracy and Coverage Evaluation program, each of which informed a decision on the use of adjusted census figures (see Section 2-B). Finally, each recent census has been accompanied by several formal experiments (16), generally conducted at the same time as the census but focusing on major processes or operations that are not yet ready for full implementation in the census. In 2000, major experiments included a study of respondents’ privacy attitudes and willingness to divulge Social Security numbers on the census form, a response mode and incentive experiment that gauged differences in response when a reward (specifically, a prepaid telephone calling card) was offered for submitting census information via the Internet or telephone, and an alternative questionnaire experiment testing some question wording and formatting options. Significantly for 2010 planning, a major experiment of the 2000 census was fielding of the Census 2000 Supplementary Survey, conducted as an experiment to judge whether it was feasible to deploy this survey and run decennial census operations simultaneously. The Census 2000 Supplementary Survey was the first nationwide prototype of the American Community Survey, a centerpiece of the Bureau’s emerging plans for 2010.

OCR for page 23
Reengineering the 2010 Census: Risks and Challenges 2–B PLANNING AND CONDUCTING THE 2000 CENSUS The final report of the Panel to Review the 2000 Census includes a rich and detailed history of the evolution of the 2000 census design and a review of its operations (see especially National Research Council, 2004:Ch. 3, 5). Given the material in that volume, we do not intend to repeat a comprehensive history of the 2000 census in these pages but rather to review major milestones and problems in the 2000 census planning process. By many accounts, the planning process for the 2000 census was fraught with risk and ultimately chaotic. In 1997, the U.S. General Accounting Office (GAO) added the 2000 decennial census to its short list of high-risk programs, citing failure to finalize and justify to Congress a basic census design. In 1990, “the most expensive census in history produced results that were less accurate than those of the preceding census” (U.S. General Accounting Office, 1997:142), and GAO worried that the cycle could repeat—that billions of dollars invested in the 2000 census might produce results that were not of demonstrably better quality than those of the 1990 census. To a great extent, 2000 census planning was dominated by ongoing debate over the role of sample-based methods in the census. The Census Bureau’s initial plans in the mid-1990s relied heavily on sampling in two respects: first, the Bureau would focus nonresponse follow-up efforts on a sample of households that did not return their census forms and, second, the results of a major postenumeration survey would be used to adjust census totals to reflect estimated undercount. The former application became known as sampling for nonresponse follow-up (SNRFU) while the planned postenumeration survey was known as Integrated Coverage Measurement (ICM). The Census Bureau’s vision at the time was a “one-number census,” producing a single set of population estimates using ICM results rather than a dual-track approach of publishing both adjusted and unadjusted figures. Both applications of sampling—and, with them, the overall plan for the 2000 census—would ultimately be shaped by intervention first by Congress in 1997 and then by the U.S. Supreme Court in 1999. The 1997 legislation (Public Law 105-119, §209)

OCR for page 23
Reengineering the 2010 Census: Risks and Challenges required the Census Bureau “to plan, test, and become prepared to implement a 2000 decennial census, without using statistical methods, which shall result in the percentage of the total population actually enumerated being as close to 100 percent as possible.” While it did not expressly prohibit SNRFU or ICM, the legislation did signal Congress’s insistence that the Census Bureau be prepared to enact a more traditional census model. In addition, the legislation dispensed with the “one-number census” vision by mandating that apportionment counts, redistricting files, and other 2000 census products contain “the number of persons enumerated without using statistical methods” as well as those added or subtracted using sample-based techniques. The 1997 legislation also required expedited judicial review of cases filed by parties who believed themselves aggrieved by the use of methodologies based on sampling. Two major challenges arose as a result and were ultimately consolidated for hearing by the U.S. Supreme Court, setting the stage for the final decision on the design of the 2000 census. In January 1999, the Court issued a 5–4 ruling that the Census Act (Title 13 of the U.S. Code) prohibited the use of sample-based methods in tabulating population counts for reapportioning the U.S. House of Representatives (Department of Commerce v. United States House of Representatives, 525 U.S. 316). The Court’s ruling stopped short of declaring sampling unconstitutional in the census process, and it left open the possibility of adjustment of census data for other purposes (including legislative redistricting) based on a postenumeration survey. But, despite these remaining ambiguities, the Court’s ruling forced the Census Bureau to abandon ICM and revamp the 2000 census plan little more than a year from April 1, 2000, the census target date. ICM was ultimately replaced by a smaller postenumeration survey, the Accuracy and Coverage Evaluation (ACE) Program. The delay in finalizing a census design meant that the 1998 census dress rehearsal was a major experiment rather than a true dress rehearsal. Instead of permitting a full test of finalized plans and systems, the dress rehearsal conducted in spring 1998 was primarily a field comparison of competing basic designs. The original sampling-based (ICM and SNRFU) framework was tested in Sacramento, California; a traditional census

OCR for page 23
Reengineering the 2010 Census: Risks and Challenges bills, that acceded to the House funding levels for the 2010 census, save for a cut from $8.6 million to $3.6 million in the allotment for “operational design strategy” (H. Rept. 108-401). The bill allots $107,090,000 for 2010 census planning, $83,310,000 for the MAF/TIGER Enhancements Program, and $64,800,000 for the American Community Survey in fiscal 2004. The bill was approved by Congress in mid-January 2004. The Census Bureau requested a $180 million increase in fiscal 2005 funds for 2010 census activities.5 This increase from fiscal 2004 totals was included in the Bush administration’s budget message for fiscal 2005, which notes that the budget increase includes a first full year of funding for the American Community Survey (Budget of the United States Government, Fiscal Year 2005, p. 77). 2–D REENGINEERING THE 2010 CENSUS: A PROCESS AT RISK The Census Bureau has advanced an ambitious vision for the 2010 decennial census, and—as our previous reports and the balance of this report suggest—the panel strongly supports the major aims of the plan. The implementation of the ACS, for example, and with it the elimination of the long form from the decennial census process is a very good idea; the Bureau’s geographic databases are in dire need of comprehensive update; and the implementation of new technologies in census-taking is crucial to maintaining an accurate count. There is thus much to like about the emerging plans for the 2010 census, and we strongly support these efforts toward a modernized and improved census in 2010. To this end, the Census Bureau’s focus on planning early in the decennial cycle is highly commendable. Based on the information made available to us, however, the panel finds that some of the planned-for innovations in the reengineering of the 2010 census are at considerable risk of failure or partial failure. Many of the problem areas that we see 5   See “EXECUTIVE SUMMARY—FY ’05 Budget” issued by the Census Bureau, available at http://www.census.gov/Press-Release/www/releases/archives/miscellaneous/001675.html [2/23/04], for additional detail on the Census Bureau request.

OCR for page 23
Reengineering the 2010 Census: Risks and Challenges in the emerging 2010 census plan stem from what we believe to be a serious disconnect between research and operations in the census process. Put another way, the Census Bureau’s planning and research entities operate too often at either a very high level of focus (e.g., articulation of the “three-legged stool” concept for the 2010 census) or at a microlevel that tends toward detailed accounting without much analysis (e.g., the program of planned evaluation reports of the 2000 census; U.S. Census Bureau (2002a)). What is lacking is research, evaluation, and planning that bridges these two levels, synthesizing the detailed results in order to determine their implications for planning while structuring high-level operations in order to facilitate meaningful detailed analysis. Justifying and sustaining the 2010 census plan requires both research that is forward-looking and strongly tied to planning objectives, and rigorous evaluation that plays a central role in operations rather than being relegated to a peripheral, post hoc role. The Census Bureau still needs to do a great deal of work to develop a strong research and evidentiary base for the general 2010 census plan, carefully assessing operational data from the 2000 census to guide planned practice for 2010 and fully exploring the potential of new tools for evaluation (such as the Master Trace Sample containing results of all census operations for a limited national subset). Looking ahead, much work also remains in integrating and mapping the logical and technical infrastructures of the entire census process, and in developing a rigorous and timely testing program for new census systems and techniques. The consequences of failing to develop a strong research base for the 2010 census are serious. They entail: repeating past census processes that may be inefficient or suboptimal, conducting a census with methods that are out of step with the dynamics of the population it is intended to count, making limited technological innovations that may not match real needs, and marking a flawed beginning for 2020.

OCR for page 23
Reengineering the 2010 Census: Risks and Challenges The Census Bureau, acutely aware of the risks and problems that resulted from extended delay in finalizing plans for the 2000 census, has identified the reduction of risk as one of the key goals for the 2010 census plan and has drafted a risk management plan that identifies 14 perceived risks related to the 2010 census (Decennial Management Division, 2003); these are listed in Table 2-2. The draft risk management plan is a useful start but it needs extensive and continuous revision. In approach, it is perhaps overly formulaic in judging the severity of risks and may make it too easy to consign moderate-risk items to an amorphous “tracking list,” rather than performing the research necessary to more fully evaluate the risk and determining potential remedies. Of the 14 risks identified by the Bureau, 9 are designated as actionable by placement on the tracking list. In our assessment, this severely underestimates the risks related to the technical infrastructure of the census (e.g., estimating the risks of “systems not performing or obsolete” and “stovepipe systems that are not interoperable”6 to be low). As we discuss in greater detail in Chapter 6, although the Bureau is working on steps to effectively model its infrastructure, its success in using that modeling capability to the full extent (in order to abate risk) is not a fore-gone conclusion. Furthermore, the Census Bureau’s risk management plan does not identify risks associated with the major components of the general census design, such as the ACS and the MAF/TIGER modernization, and the plan’s soft-pedaling of the risk associated with the lack of integration of these components is puzzling. As the Bureau’s risk management plan notes, 2010 census reengineering faces two paramount risks: first, that the final design of the census (particularly the role of the ACS or a census long form) will be determined late in the process and, second, that funding will be inadequate or at unpredictable levels. These risks differ in nature from most of those discussed above in that they are largely external to the Census Bureau, since funding decisions are made by the administration and, ultimately, 6   Stovepipe systems are named such because they are vertically—but not horizontally—integrated. That is, they can be elaborate systems that perform one set of functions, but they are unable to interact and share information with other systems in an enterprise’s broader architecture.

OCR for page 23
Reengineering the 2010 Census: Risks and Challenges Table 2-2 Census Bureau Listing of Perceived Risks in 2010 Census Planning Risk Description Perceived Levela Strategyb Late changes to final census design High Tracking List Dress rehearsal experimental, not true rehearsal Medium Tracking List Alienate key stakeholders, including advisory groups and state, local, and tribal governments High Containment Change in or lack of funding High Contingency Low mail response Medium Contingency Outsourcing may not meet census needs Medium Tracking List Systems not performing or obsolete Low Tracking List Field infrastructure (PCDs) not ready Low Tracking List Lack of integration of design components (ACS, MAF/TIGER Enhancement, Early Planning) Low Tracking List Failure to maintain systems security High Containment Concern over privacy and confidentiality jeopardizes data collection Low Tracking List Lack of human, financial, schedule, or material resources High Contingency Staff attrition and corresponding loss of census knowledge Medium Tracking List Stovepipe systems that are not interoperable Low Tracking List a The Census Bureau derives this categorization by multiplying a score for likelihood of risk (1, 2, or 3, 3 being most likely) by a score for impact of risk if realized (1, 2, or 3, 3 being largest effect). Final scores of 1 or 2 are labeled “low” risk, 3 or 4 “medium” risk, 6 “high” risk, and 9 “very high” risk. b The Census Bureau identifies three possible risk management actions. “Tracking List” means that the risk is kept on a watch list, which is reviewed periodically “to assist the implementation team in ‘keeping their eyes on the horizon.’” A “Contingency Plan” “involve[s] preparation for actions to be taken in the event that a nonactionable risk becomes an issue.” Finally, a “Containment Plan” “involve[s] specific actions that will be taken to [r]educe the probability of the risk turning into an issue [and] reduce the negative impact to the project if the risk becomes an issue” (Decennial Management Division, 2003:20). SOURCE: Decennial Management Division (2003).

OCR for page 23
Reengineering the 2010 Census: Risks and Challenges Congress. Although the Census Bureau may not have control over funding decisions, it must take a more active role in informing the funding process in at least two ways. First, as we emphasize throughout this report, the Bureau must develop a sound research and evidentiary base for its 2010 census plan, thus making a stronger and more compelling case for sustained long-term funding. Second, the Bureau should be explicit in identifying, articulating, and quantifying the consequences associated with these broad risks—for instance, the impact of reduced funding on the quality of ACS estimates for small geographic areas and population groups. Failure to reach consensus on the role of the ACS in the census process raises the undesirable prospect of reversion to the long form, possibly late in the census process and therefore implemented in a rushed manner, which is likely to result in the same nonresponse and data quality problems as were experienced with the 2000 long form. More significantly, failure to reach closure on census design leaves open the possibility that the detailed socioeconomic and demographic characteristics measured by the current census long form may not be estimated at all in 2010, which would be an unacceptable outcome. 2–D.1 Specific Risk Areas Beyond these broad, systemic risks, the 2010 census planning process faces many risks of a more specific nature, some of which are acknowledged in the Bureau’s draft of a risk management plan and many of which are not. Based on the information known to us, we find that the 2010 census reengineering process may be seriously jeopardized in the following areas, among others (this list of risk areas is ordered for rhetorical flow, not by any estimated level of risk or urgency): Master Address File updating process: The panel believes this effort to be at severe risk. The Census Bureau’s current approach treats MAF updates as “routine maintenance,” relying principally on updates from U.S. Postal Service files. Detailed plans for review by local and tribal governments or for address input from ACS enumerators are either nonexistent or have not been shared with the panel. Absent a strong focus on enhancing the MAF in its own

OCR for page 23
Reengineering the 2010 Census: Risks and Challenges right, independent of presumed benefits from linkage to a realigned TIGER database, the 2010 census may be conducted with an address source that has unacceptable levels of housing unit duplication in some areas and coverage gaps in others. Development of census logical and technical architecture: The Census Bureau has begun efforts toward modeling the logical and technical infrastructure of the decennial census, a development the panel strongly endorses. However, early indications suggest that the modeling of the 2010 census process has focused on detail without any real reengineering taking place. Draft architecture documents are short on analysis and do not decompose census processes to the appropriate level of resolution needed to actually reengineer and to specify hardware/software systems. Should the full potential of architecture modeling not be realized, the consequences are dire: systems may be ill-suited to handle 2010 census process needs, and may fail during actual census operations due to lack of proper testing. Moreover, census hardware and software systems may not fit properly with each other at points of interface—guaranteeing the stovepipe systems that the Bureau’s draft risk management plan correctly suggests should be avoided. Realignment of TIGER features: The initial realignment of TIGER geographic features to be consistent with GPS coordinates may not be completed in time, or change detection for new features after the initial realignment may not be properly performed. These outcomes would have a negative impact on plans for PCD use by field enumerators and would lead to continued errors in geocoding addresses in the census and nonresponse follow-up operations. TIGER database conversion: The conversion of the MAF/ TIGER database from its current homegrown format to a modern, object-oriented computing environment may be slower or more difficult than anticipated. This is particularly dangerous if the Census Bureau attempts the conversion en masse, rather than via a more carefully designed software reengineering process with ample testing.

OCR for page 23
Reengineering the 2010 Census: Risks and Challenges Targeted methods in address list development or enumeration: Examples of these methods include targeting block canvass to verify address list entries to particular (e.g., high-growth) areas and expansion of update/leave enumeration (where a census enumerator drops questionnaires at housing units, which are then expected to mail them back) in urban areas where mail delivery may not be the most effective means. Failure to implement these methods may result in increased costs and continued problems of enumeration in high-density areas with structures containing multiple (and not well-listed or easily differentiated) housing units. Use of local knowledge: The Census Bureau’s draft risk management plan includes the risk of alienating key stakeholders, including local and tribal governments. Alienation of local authorities is a risk, to be certain, but a more fundamental risk is failure to fully involve them in census design and operations. In addition to serving a critical role as contributors and reviewers of address list information, local and tribal governments can offer guidance in enumerating group quarters and other special populations, tailoring enumeration techniques (e.g., update/leave rather than mailout/mailback) to specific areas within localities, and fostering acceptance and use of the ACS. Enumeration strategies for special, challenging populations: Special populations requiring specific, targeted approaches and methodologies include immigrant communities, irregular multiunit housing structures, gated communities, colonias along the U.S.-Mexico border, and the homeless. Efforts in this area may be compromised by failure to tailor enumeration techniques to these groups and to clarify the definition and presentation to respondents of the residence rules for the decennial census. Consequences associated with such a failure include poor quality data, failure to meet consumer needs, and continued differential undercount. Special place/group quarters enumeration process: In 2000, as in previous censuses, procedures for enumerating group quarters were conducted separate from the rest of the cen-

OCR for page 23
Reengineering the 2010 Census: Risks and Challenges sus process. Group quarters listings were not reconciled with the MAF for households, and little effort was given to the challenges of enumerating different types of group quarters. The enumeration processes for group quarters were not well controlled. Continuing with this approach incurs the risk of duplication, a repeat of the experience in 2000 when whole group quarters were geographically misplaced or miscounted,7 and ineffective coverage of this small but important population group. Census duplication (both housing unit and person): Concerns about duplication during the processing of 2000 census records prompted the Census Bureau to mount an ad hoc unduplication effort. Failure to research and improve techniques to identify and correct duplication of both housing units and person records could lead to a repeat of costly, spur-of-the-moment operations in 2010. Imputation: Though the U.S. Supreme Court upheld the use of imputation in its ruling on Utah’s challenge to the 2000 census counts, the debate suggests the need to revisit the techniques used to fill in missing questionnaire items and, in some cases, to impute household size when no information is available for a presumed-occupied unit. The costs and benefits of alternatives to current “hot-deck” methods should be evaluated. Research and development agenda for the American Community Survey: Though not an immediate threat to the integrity of the 2010 census, failure to implement a strong research and evaluation program for the ACS poses a longerterm risk to the quality and usefulness of the survey and will hinder the ability to fully exploit potential ties between 7   Many challenges to census counts filed by localities under the Census Bureau’s Count Question Resolution program involved geographic misplacement of such facilities as college dormitories and prisons. Most recently, the Census Bureau acknowledged that a 2,673-resident dormitory at the University of North Carolina at Chapel Hill had been double-counted—a highly contentious finding since North Carolina narrowly edged out Utah (by 857 residents) for the 435th and final seat in the U.S. House of Representatives. Utah had failed in two major legal challenges to the 2000 census totals in order to secure a fourth seat in the House (Baird, 2003).

OCR for page 23
Reengineering the 2010 Census: Risks and Challenges the ACS and programs for producing postcensal population and demographic analysis estimates. Coverage measurement: As we discuss in Chapter 7, the coverage measurement program in 2010 need not take the same shape as that of 2000, but it is essential that the Census Bureau have the means to determine the accuracy of its count, for the nation as a whole as well as for population subgroups. After the statistical adjustment battles preceding and following the 2000 census, the Census Bureau may be understandably reluctant to take up active debate on coverage techniques for 2010. However, this reluctance incurs the risk that a comprehensive plan for the measurement and assessment of census coverage in 2010 will be deferred until late in the census process. Portable computing devices: The Census Bureau’s plans for the use of portable computing devices (PCDs) in the 2010 census are a particularly exciting part of a reengineered census, but the plans also entail risks associated with the implementation of new technologies. Perhaps most significant is the risk that the Census Bureau may fail to fully understand the direction in which the technology is moving and thus may spend its resources testing devices that are inferior to those that will be available in 2010, in terms of both size and computing capacity. A consequence of this error is that wrong and misleading conclusions may be drawn about the real potential for portable computing devices to improve census data collection. A second risk inherent with the PCD technology lies in making the decision to purchase too early and without fully specified requirements, resulting in the possible selection of obsolete or inadequate devices. Third, and related, is the risk that the Census Bureau may not use the sheer size of its order (perhaps on the order of 500,000 devices) to obtain devices tailored to census needs, as opposed to buying only what is commercially available off the shelf. Finally, given the fact that the principal users of the devices will be the large corps of temporary enumerators (with limited training), there is the risk that Census Bureau PCD development will not take

OCR for page 23
Reengineering the 2010 Census: Risks and Challenges human factors into sufficient consideration. The full potential of PCDs will not be realized if they are either too simple or too complex. New response technologies: For example, a repeat of the 2000 experience concerning a second questionnaire mailing for nonresponding households would be highly undesirable. 2–D.2 Mitigating the Risks Although in this report we criticize certain aspects of the Census Bureau’s planning process, we do not want to appear unduly alarmist about it. We are not stating that the 2010 census is irrevocably headed toward a bleak outcome. Our comments are intended, however, to strongly emphasize the need for extensive research and evaluation so that the Census Bureau will be able to stay on course toward achieving its high-level goals for 2010. Throughout this report, we offer recommendations to improve 2010 census planning and mitigate risks. In brief, some of the steps that we believe will be crucial to the success of 2010 census planning include the following: Recast the 2006 census test as a proof of concept, not a technical test. The Census Bureau hopes to make its planned 2008 activity as close to a true dress rehearsal as possible, which is certainly desirable. However, that goal implies that the success of the 2006 test is of critical importance: it must provide the basis for definitively answering any remaining experimental questions in order to make the 2008 dress rehearsal a genuine preoperational rehearsal instead of just an extension of earlier tests. Focus on the Master Address File. Realignment of the TIGER database is undeniably important and long overdue, but it is the quality of the address list that is crucial to the quality of a census. As we argue in Chapter 3, the MAF is in need of much greater attention than it currently receives under the MAF/TIGER Enhancements Program. Use research to bolster the case for the American Community Survey. Making a stronger case for the utility of ACS-based

OCR for page 23
Reengineering the 2010 Census: Risks and Challenges estimates (including moving averages) in a variety of applications and further examining the relative quality of ACS and long-form estimates is essential to winning long-term support for the survey. Use logical and technical infrastructure modeling to its fullest extent. Reengineering to understand, simplify, and minimize inefficiencies and redundancies (both informational and functional) in census systems is essential to break up stovepipe systems and promote integration in census planning. Adapt testing of portable computing devices to anticipate future requirements. In particular, testing should be done on current high-end devices of the form that are certain to be available, and less expensive, by the time of procurement. In approaching PCDs, the Census Bureau should seek to exploit technology to guide new and better enumeration processes, rather than simply replicating old processes on new tools. Make research and evaluation a centerpiece of census operations, not a peripheral component. In addition to making better use of extant data resources, the Census Bureau should design its hardware and software systems to facilitate research and allow quick evaluation. In its architecture redesign efforts, the Census Bureau’s objective should be an integrated system that provides information for evaluation routinely and in real time—that is, a Master Trace System, rather than only a Sample.