CHAPTER 4
Assessment of 2000 Census Operations

WE BEGIN OUR EVALUATION OF THE 2000 CENSUS by describing two of its three most important achievements (4-A): first, halting the historical decline in mail response rates and, second, implementing data collection, processing, and delivery in a smooth and timely manner. (We describe and assess the third important achievement—the reduction in differential coverage errors among important population groups—in Chapters 5 and 6.) We then assess the contribution of seven major innovations in census operations, not only to the realization of the above-cited achievements, but also to data quality and cost containment. By quality, we mean that an estimate from the census accurately measures the intended concept, not only for the nation as a whole, but also for geographic areas and population groups. With regard to costs, although cost reduction drove the research and planning for the original 2000 design (see Chapter 3), it was not an explicit goal for the final 2000 design.

The seven innovations that we review include two that facilitated public response—namely, redesigned questionnaires and mailing materials and paid advertising and expanded outreach (4-B, see also Appendix C.2, C.4); three that facilitated timeliness—namely, contracting for data operations, improved data capture technology, and aggressive recruitment of enumerators and implementation of non-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 97
The 2000 Census: Counting Under Adversity CHAPTER 4 Assessment of 2000 Census Operations WE BEGIN OUR EVALUATION OF THE 2000 CENSUS by describing two of its three most important achievements (4-A): first, halting the historical decline in mail response rates and, second, implementing data collection, processing, and delivery in a smooth and timely manner. (We describe and assess the third important achievement—the reduction in differential coverage errors among important population groups—in Chapters 5 and 6.) We then assess the contribution of seven major innovations in census operations, not only to the realization of the above-cited achievements, but also to data quality and cost containment. By quality, we mean that an estimate from the census accurately measures the intended concept, not only for the nation as a whole, but also for geographic areas and population groups. With regard to costs, although cost reduction drove the research and planning for the original 2000 design (see Chapter 3), it was not an explicit goal for the final 2000 design. The seven innovations that we review include two that facilitated public response—namely, redesigned questionnaires and mailing materials and paid advertising and expanded outreach (4-B, see also Appendix C.2, C.4); three that facilitated timeliness—namely, contracting for data operations, improved data capture technology, and aggressive recruitment of enumerators and implementation of non-

OCR for page 97
The 2000 Census: Counting Under Adversity response follow-up (4-C, see also Appendix C.3, C.5); one that facilitated timeliness but may have had mixed effects on data quality—namely, greater reliance on computers to treat missing data (4-D, see also Appendix C.5); and one that was sound in concept but not well-implemented—namely, the use of multiple sources to develop the Master Address File, or MAF (4-E, see also Appendix C.1). We do not assess an eighth major innovation in 2000, which was the expanded use of the Internet for release of data products to users; see http://factfinder.census.gov [12/12/03]. The final section in this chapter (4-F) assesses the problem-plagued operations for enumerating residents of group quarters. Sections 4-B through 4-F end with a summary of findings and recommendations for research and development for 2010. 4–A TWO MAJOR OPERATIONAL ACHIEVEMENTS 4–A.1 Maintaining Public Cooperation The 2000 census, like censuses since 1970, was conducted primarily by delivering questionnaires to households and asking them to mail back a completed form. Procedures differed somewhat, depending on such factors as type of addresses in an area and accessibility; in all, there were nine types of enumeration areas in 2000 (see Box C.2 in Appendix C). The two largest types of enumeration areas in 2000—mailout/mailback and update/leave/mailback or update/leave—covered 99 percent of the household population; together, they constituted the mailback universe. The goal for this universe for the questionnaire delivery and mail return phase of the census was to deliver a questionnaire to every housing unit on the MAF and motivate people to fill it out and mail it back. The Census Bureau expected that mail response would continue to decline, as it had from 1970 to 1990, due to broad social and economic changes that have made the population more difficult to enumerate. These changes include rising numbers of new immigrants, both those who are legally in the country and those who are not, who may be less willing to fill out a census form or who may not be able to complete a form because of language difficulties; increasing amounts of junk mail, which may increase the likelihood

OCR for page 97
The 2000 Census: Counting Under Adversity that households will discard their census form without opening it;1 and larger numbers of households with multiple residences, making it unclear which form they should mail back. The Bureau budgeted for a decline in the mail response rate (mail returns as a percentage of all mailback addresses) to 61 percent in 2000, compared with a 70 percent budgeted rate in 1990. As the 2000 census got under way, the Census Bureau director initiated the ’90 Plus Five campaign, challenging state, local, and tribal government leaders to reach a mail response rate at least 5 percent higher than their 1990 mark. The Internet was used to post each jurisdiction’s goal and actual response rates day-by-day through April 11, 2000. Maintaining the budgeted mail response rate was key to the Bureau’s ability to complete nonresponse follow-up on time and within budget, and, if the rate improved over the budgeted figure, that would be beneficial. Estimates produced in conjunction with the 1990 census were that each 1 percentage point decline in the mail response rate increased the nonresponse follow-up workload and costs of that census by 0.67 percent (National Research Council, 1995b:48). In addition, evidence from the 1990 census, confirmed by analysis of 2000 data, indicated that returns obtained in the field, on balance, were less complete in coverage and content than mail returns (see Appendix D and Chapter 7). Success in the 2000 Mail Response A significant achievement of the 2000 census was that it halted the historical decline in the mail response rate (see Box 4.1). The rate achieved at the time when the workload for nonresponse follow-up was identified (April 18, 2000) was 64 percent—3 percentage points higher than the 61 percent rate for which the Bureau had budgeted follow-up operations. By the end of 2000 the mail response rate was 67 percent (Stackhouse and Brady, 2003a:2, Table 4). A significant proportion of the mail returns received after April 18, 2000, had arrived by April 30, so that the Census Bureau could perhaps have saved even more follow-up costs if it had set a later date to establish the nonresponse follow-up workload. However, the Bureau was not 1   A post-1990 census study of household behavior found that 4 percent of households that reported receiving a questionnaire in the mail discarded it without opening it (Kulka et al., 1991).

OCR for page 97
The 2000 Census: Counting Under Adversity Box 4.1 Mail Response and Return Rates The mail response rate is defined as the number of households returning a questionnaire by mail divided by the total number of questionnaires sent out in mailback areas. Achieving a high mail response rate is important for the cost and efficiency of the census because every returned questionnaire is one less household for an enumerator to follow up in the field. The mail return rate is defined as the number of households returning a questionnaire by mail divided by the number of occupied housing units that were sent questionnaires in the mailback areas. This rate is an indicator of public cooperation. Achieving a high mail return rate (at least to the level of 1990) is important because of evidence that mail returns are more complete than enumerator-obtained returns. In 2000, because of the alternative modes by which households could fill out their forms, the numerator of both “mail” responses and “mail” returns included responses submitted on the Internet, over the telephone, and on “Be Counted” forms. The denominator of the mail response rate included all addresses on the April 1, 2000, version of the MAF, covering both mailout/mailback and update/leave areas. The denominator of the mail return rate excluded addresses on the MAF that field follow-up determined were vacant, nonresidential, or nonexistent and included addresses for occupied units that were added to the MAF after April 1. Final Rates, 1970-2000 Censuses (as of end of census year)   1970 1980 1990 2000 Mail response rate: 78% 75% 65% 67% Mail return rate: 87% 81% 75% 78% Differences in Final Mail Return Rates: Short and Long Forms Return rates of long forms are typically below the return rates of short forms. This difference widened substantially in 2000.   1970 1980 1990 2000 Short-form rate: 88% 82% 76% 80% Long-form rate: 86% 80% 71% 71% Difference 2% 2% 5% 9% NOTES: 1980 and 1990 rates shown here differ slightly from those in Bureau of the Census (1995b:1-24). Mail response and return rates are not strictly comparable across censuses because of differences in procedures used to compile the address list and in the percentage of the population included in the mailback universe (about 60 percent in 1970 and 95 percent or more in 1980–2000). SOURCE: National Research Council (1995b:Table 3.1, App. A) for 1970 and 1980 rates; Stackhouse and Brady (2003a,b:v) for 1990 and 2000 rates.

OCR for page 97
The 2000 Census: Counting Under Adversity willing to risk possible delays in completing nonresponse follow-up by delaying the start of the effort (U.S. General Accounting Office, 2002a). In 1990, the mail response rate at the time when the workload for nonresponse follow-up was identified (late April) was 63 percent—1 percentage point lower than the corresponding 2000 rate and 7 percentage points lower than the 70 percent rate for which the Bureau had budgeted follow-up operations. By the end of 1990 the mail response rate was 65 percent, 2 percentage points lower than the final 2000 rate (Stackhouse and Brady, 2003a:2). Most of the improvement in mail response in 2000 was because of improved response to the short form. At the time when the nonresponse follow-up workload was specified (April 18, 2000), the long-form mail response rate was 12 percentage points below the rate for short forms (54 percent and 66 percent, respectively). When late mail returns are included, the gap between short- and long-form mail response rates was reduced from 12 to 10 percentage points (Stackhouse and Brady, 2003a:17,18). The 2000 census was also successful in stemming the historical decline in the mail return rate (mail returns as a percentage of occupied mailback addresses), which is a more refined measure of public cooperation than the mail response rate (see Box 4.1). At the end of 2000, the final mail return rate was 78 percent, compared with a final mail return rate of 75 percent in 1990 (Stackhouse and Brady, 2003b:v). Again, most of the improvement in mail return rates in 2000 was because of improved response to the short form: the final mail return rate for short forms was 80 percent in 2000 compared with 76 percent in 1990. For long forms, the final mail return rate was 71 percent in both years, so that the gap between short-form and long-form mail return rates in 2000 was 9 percentage points, compared with only 5 percentage points in 1990. Mail Return Patterns A tabular analysis of all 2000 mail returns (Stackhouse and Brady, 2003b:Tables 8, 10, 12, 14, 16) found considerable variation in mail return rates for population groups: Total mail return rates increased as a function of age of householder: the final mail return rate for householders ages 18–24

OCR for page 97
The 2000 Census: Counting Under Adversity was 57 percent, climbing to a rate of 89 percent for householders age 65 and older. White householders had the highest total final mail return rate of any race group—82 percent. Householders of other race groups had final mail return rates of 75 percent or lower; the lowest rates were for black householders (64 percent), householders of some other race (63 percent), and householders of two or more races (63 percent). Hispanic householders had a lower total final mail return rate (69 percent) than non-Hispanic householders (79 percent); Hispanic householders had a larger difference between short-form and long-form return rates (71 percent short-form rate, 57 percent long-form rate) than did non-Hispanic householders (81 percent short-form rate, 72 percent long-form rate). Total final mail return rates were similar for most household size categories, except that two-person households had a higher rate (82 percent) than other categories and households with seven or more members had a lower rate (72 percent) than other categories, largely because of differences in long-form return rates for these two household types (76 percent for two-person households and 58 percent for households with seven or more members). Owners had a higher total final mail return rate (85 percent) than renters (66 percent). Exploratory regression analysis conducted by the panel of preliminary total mail return rates for 2000 and 1990 for census tracts found a strong positive relationship in both years of a tract’s return rate to its 1990 percentage population over age 65 and a strong negative effect in both years of a tract’s return rate to its 1990 hard-to-count score, estimated percentage net undercount, percentage people in multiunit structures, and percentage people who were not high school graduates.2 Geographic region effects, which differed somewhat between 1990 and 2000, were also evident. While analysis found no variables that explained changes in mail return rates 2   See National Research Council (2001a:App.B). The data sets used were U.S. Census Bureau, Return Rate Summary File (U.S.), provided to the panel February 26, 2001, and 1990 Planning Database (U.S. Census Bureau, 1999a).

OCR for page 97
The 2000 Census: Counting Under Adversity for census tracts from 1990 to 2000, graphical displays identified clusters of tracts with unusually large increases and decreases in return rates between the two censuses, suggesting that local factors may have had an effect in those areas. Thus, large clusters of tracts that experienced 20 percent or greater declines in mail return rates from 1990 to 2000 were found in central Indiana; Brooklyn, New York; and throughout Kentucky, Tennessee, and the Carolinas. At the other extreme, tracts that experienced increases of 20 percent or more in mail return rates were concentrated in the Pacific census division (particularly around Los Angeles and the extended San Francisco Bay Area) and also in New England. In contrast, mail return rates for tracts in the Plains and Mountain states were very similar between 1990 and 2000. Further investigation of the characteristics of clusters of tracts that experienced unusually large increases or decreases in mail return rates and of local operations and outreach activities in those areas would be useful to identify possible problems and successes to consider for 2010 census planning. More generally, research is needed on patterns of mail response and the reasons for nonresponse. 4–A.2 Controlled Timing and Execution The Census Bureau met its overriding goal to produce data of acceptable quality from the 2000 census for congressional reapportionment and redistricting by the statutory deadlines. This outcome may appear unremarkable because the Bureau has never failed to produce basic population counts on time. However, as we saw in Chapter 3, success was by no means a foregone conclusion given the serious problems that hampered planning and preparations for the census, which included lack of agreement on the 2000 design until early 1999. Not only were the basic 2000 data provided on time, but most of the individual operations to produce the data, such as nonresponse follow-up and completion of edited data files, were completed on or ahead of schedule.3 Of the 520 census offices (excluding Puerto Rico), 68 completed nonresponse follow-up within 6 or fewer weeks, 290 finished within 7 weeks, 462 finished within 8 weeks, and all 3   The 2000 Accuracy and Coverage Evaluation operations were also carried out on or ahead of schedule; see Chapter 6.

OCR for page 97
The 2000 Census: Counting Under Adversity 520 finished within 9 weeks—a week ahead of the planned 10-week schedule for completion (U.S. General Accounting Office, 2002a:6).4 Few instances occurred in which operations had to be modified in major ways after Census Day, April 1, 2000. One exception was the ad hoc unduplication operation that was mounted in summer 2000 to reduce duplicate enumerations from duplicate addresses in the MAF (see Section 4-E). In contrast, the 1990 census experienced unexpected problems and delays in executing such key operations as nonresponse follow-up, which took 14 weeks to finish, and the Census Bureau had to return to Congress to obtain additional funding to complete all needed operations. The basic 1990 data were released on time, however, and delivery schedules for more detailed products were similar for 1990 and 2000 (see Bureau of the Census, 1995a:Ch.10; http://www.census.gov/population/www/censusdata/c2kproducts.html [1/10/04]). 4–B MAJOR CONTRIBUTORS TO PUBLIC RESPONSE The Census Bureau had three strategies to encourage response to the 2000 census. Two of the three strategies very likely contributed to the success in halting the historical decline in mail response and return rates—a redesigned questionnaire and mailing package (4-B.1) and extensive advertising and outreach (4-B.2). The third strategy, which we do not discuss further, was to allow multiple modes for response. This strategy had little effect partly because the Census Bureau did not actively promote alternative response modes. Specifically, the Bureau did not widely advertise the option of responding via the Internet (an option available only to respondents who had been mailed the census short form) because of a concern that it could not handle a large volume of responses. The Bureau also did not widely advertise the option of picking up a “Be Counted” form at a public place because of a concern about a possible increase in the number of responses that would require address verification and unduplication. Finally, telephone assistance was designed pri- 4   Subsequently, every housing unit in the workload of one district office, in Hialeah, Florida, was reenumerated because of problems that came to light in that office, and selected housing units were reenumerated in seven other offices for which problems were identified (see Appendix C.3.b).

OCR for page 97
The 2000 Census: Counting Under Adversity marily to answer callers’ questions and only secondarily to record responses. As it turned out, of 76 million questionnaires that were returned by households, 99 percent arrived by mail and only 1 percent by other modes (see Appendix C.2.b). In particular, only about 600,000 Be Counted and 200,000 telephone questionnaires were obtained. At the end of a process of geocoding, field verification, and unduplication, the Be Counted and telephone operations added about 100,000 housing units that would otherwise not have been included in the census count (see Vitrano et al., 2003a:29). 4–B.1 Redesigned Questionnaire and Mailing Package Strategy Based on its extensive research in the early 1990s (see Section 3-B.2), the Census Bureau redesigned the census short-form and long-form questionnaires and mailing materials for 2000 as part of its effort to encourage the public to fill out questionnaires and return them in the mail (or by an alternative response mode). The questionnaires were made easier to read and follow; the respondent burden of the short form was reduced by providing room for characteristics for six household members instead of seven as in 1990 and by moving questions to the long form; an advance letter was sent out in mailout/mailback areas; the envelope with the questionnaire carried a clear message that response was required by law; and a reminder postcard (also used in 1990) was sent out in mailout/mailback and update/leave areas. Evaluation The success of the 2000 census in maintaining mail response and return rates at 1990 levels and above was of major importance for its success in terms of timely, cost-effective completion of operations. It seems highly likely that the changes to the questionnaire and mailing package and the use of an advance letter—despite or perhaps even because of the publicity due to an addressing error in the letter (see Appendix C.2.a)—contributed to maintaining the response and return rates, although how large a role each of these elements played in this achievement is not known. It is also unknown whether the redesigned questionnaire and mailing package

OCR for page 97
The 2000 Census: Counting Under Adversity had greater effect on some population groups and geographic areas than others, or whether the effect was to boost return rates about equally across population groups and geographic areas and thereby maintain differences in response rates from past censuses. One disappointment of the mailing strategy and other initiatives to encourage response in 2000 was that they had much less effect on households that received the long form than on those that received the short form. As noted above, at the time when the nonresponse follow-up workload was specified (April 18, 2000), the long-form mail response rate was 12 percentage points below the rate for short forms. This difference was double the difference that the Bureau expected (U.S. General Accounting Office, 2000c:5) and far larger than differences between long-form and short-form response rates seen in previous censuses. One of the Bureau’s questionnaire experiments in the early 1990s, using an appealing form and multiple mailings, presaged this outcome: it found an 11 percentage point difference between short-form and long-form response rates (Treat, 1993), perhaps because the user-friendly long form tested was longer than the 1990 long form (28 versus 20 pages). The 2000 long-form had about the same number of questions as the 1990 long form but twice as many pages. When late mail returns are included, the gap between short- and long-form mail response and return rates in 2000 was somewhat reduced. In other words, some households that received the long form lagged in completing and mailing it back in. However, the wide gap at the time of nonresponse follow-up was important because it meant that a higher proportion of the workload comprised long forms, which experience in 1990 and 2000 demonstrated were more difficult to complete in the field than short forms. A drawback of the 2000 census mailing strategy was that the plan to mail a second questionnaire to nonresponding households had to be discarded, even though dress rehearsal results suggested that doing so might have increased mail response by 8 percentage points above what was achieved (Dimitri, 1999), or even by 10–11 percentage points as estimated from earlier tests (see Section 3-B.2). The Census Bureau did not work with vendors early in the decade to develop a feasible system for producing replacement questionnaires. Consequently, its plans for a targeted second mailing were stymied when the vendors selected for the dress rehearsal were not prepared

OCR for page 97
The 2000 Census: Counting Under Adversity to turn around the address list on the schedule required. Experience in the dress rehearsal suggested that mailing a second questionnaire to every address (instead of only to nonresponding households) would generate adverse publicity and increase the number of duplicate returns that would need to be weeded out from the census count, so no second mailing was made in 2000. 4–B.2 Paid Advertising and Expanded Outreach Expansion of Efforts in 2000 A second important element of the Census Bureau’s strategy in 2000 to reverse the historical decline in mail response rates and to encourage nonrespondents to cooperate with follow-up enumerators was to advertise much more extensively and expand local outreach and partnership programs well beyond what was done in the 1990 census. An integral part of the advertising strategy was to pay for advertisements instead of securing them on a pro bono basis. The advertising ran from November 1, 1999, to June 5, 2000, and included separate phases to alert people to the importance of the upcoming census, encourage them to fill out the forms when delivered, and motivate people who had not returned a form to cooperate with the follow-up enumerators. Advertisements were placed on TV (including a spot during the 2000 Super Bowl), radio, newspapers, and other media, using multiple languages. Using information from market research, the advertisements stressed the benefits to people and their communities from the census, such as better targeting of government funds to needy areas for schools, day care, and other services. The Census Bureau also hired 560 full-time-equivalent partnership and outreach specialists in local census offices (three times the number hired in 1990—see U.S. General Accounting Office, 2001a:11), who worked with community and public interest groups to develop special initiatives to encourage participation in the census. The Bureau signed partnership agreements with over 100,000 organizations, including federal agencies, state and local governments, business firms, and nonprofit groups. (Estimates vary as to the number of partnerships—see Westat [2002c], which reports on a survey of a sample of about 16,000 partners on their activities,

OCR for page 97
The 2000 Census: Counting Under Adversity Vitrano et al. (2003b:Table 13) estimate that 659,000 housing units were contributed uniquely by LUCA 98. The same study reports that another 512,000 housing units were contributed both by LUCA 98 and the block canvass, another 396,000 housing units were contributed by both LUCA 98 and the September 1998 DSF, and 289,000 housing units were contributed uniquely by LUCA 99. A thorough assessment of the contribution of LUCA to MAF is needed, including not only the effects of LUCA on the completeness of the census count in participating areas but also the possible effects on the counts in other areas from not having had a LUCA review. 4–E.4 2000 MAF Development: Summary of Findings The concept for building the 2000 MAF from many sources, including the previous census address file, was an important innovation that the Census Bureau plans to fully implement for the 2010 census. For 2000 the newness of many MAF-building operations led to significant problems of implementation. In turn, these problems not only contributed to errors in the MAF but also complicated a full evaluation. The variables and codes originally included on the MAF records made it difficult to trace address sources. To facilitate analysis, Census Bureau evaluation staff subsequently developed an “original source” code for each address record and used it to identify sources that contributed the highest numbers of addresses to the final 2000 MAF (Vitrano et al., 2003b). However, they could not always identify a single source or even any source for some addresses. In addition, there are no data with which to determine the contribution of each source to correct versus erroneous addresses on the final MAF. Other problems in the MAF coding, such as overstatement of the number of units at a basic street address in some instances and omission of this information for non-city-style addresses, hampered evaluation (see Vitrano et al., 2003a:5). Finally, very little analysis has been done to date of geographic variations in the effectiveness of various sources for building the 2000 MAF. Finding 4.4: The use of multiple sources to build a Master Address File—a major innovation in 2000—was appropriate in concept but not well executed. Problems included changes in schedules and operations, variabil-

OCR for page 97
The 2000 Census: Counting Under Adversity ity in the efforts to update the MAF among local areas, poor integration of the address list for households and group quarters, and difficulties in determining housing unit addresses in multiunit structures. Changes were made to the MAF development: a determination late in the decade that a costly complete block canvass was needed to complete the MAF in mailout/mailback areas and a determination as late as summer 2000 that an ad hoc operation was needed to weed out duplicate addresses. Problems with the MAF contributed to census enumeration errors, including a large number of duplicates. Finding 4.5: The problems in developing the 2000 Master Address File underscore the need for a thorough evaluation of the contribution of various sources, such as the U.S. Postal Service Delivery Sequence File and the Local Update of Census Addresses Program, to accuracy of MAF addresses. However, overlapping operations and unplanned changes in operations were not well reflected in the coding of address sources on the MAF, making it difficult to evaluate the contribution of each source to the completeness and accuracy of the MAF. 4–E.5 2010 MAF Development: Recommendation The Census Bureau plans to update the MAF over the 2000 decade. It has embarked on a major effort to reengineer its TIGER geocoding and mapping database to make it more accurate and easier to maintain.18 For the TIGER reengineering, the Bureau plans to the extent possible to work with local governments, many of which have geographic information systems that are of better quality than TIGER. In addition, although plans are not yet well developed, the Bureau intends to conduct a LUCA Program as part of constructing the 2010 census MAF. We discuss three areas for improvement in the 2010 MAF development process below. 18   TIGER—Topologically Integrated Geographic Encoding and Reference System—is used to assign MAF addresses to the proper geographic areas; see National Research Council (2003a) for an assessment of the MAF/TIGER reengineering project.

OCR for page 97
The 2000 Census: Counting Under Adversity Housing Unit Addresses The Bureau must recognize that obtaining an accurate address for a structure (single-family home, apartment building) is not sufficient for a complete, accurate MAF. It is also critical to develop accurate information on the number of housing units that exist within structures. Several reports, including the housing unit coverage study (Barrett et al., 2001, 2003) and the case studies prepared by the Working Group on LUCA (2001:Ch.4), document the problems in developing complete, correct address lists for small multiunit structures, which often have a single mail drop. For example, in neighborhoods with a stock of single-family or two-family residences and heavy inmigration since the last census, additional, hard-to-identify housing units may have been carved out of an apparently stable housing stock. Conversely, some structures with two or three units may have been turned into bigger single-family residences (e.g., as the result of gentrification). Indeed, in areas in which it is hard to keep track of housing units, as opposed to structures, it may be necessary to consider another enumeration procedure, such as update/enumerate, in place of mailout/mailback, in order to obtain an accurate count of households and people. LUCA Redesign The Bureau needs to redesign its cooperative programs with state and local governments for MAF/TIGER work. The LUCA effort in the late 1990s was one-sided. The Bureau offered each local government the opportunity to participate but did not actively encourage participation or provide financial or other support beyond training materials. Also, to protect confidentiality, the Bureau required participating localities to destroy all MAF-related materials after the review was completed. Consequently, localities could not benefit from the efforts they made, which were often substantial, to update the MAF. Because of TIGER inaccuracies, localities with better geographic information systems had to force their data to fit the TIGER database before they could begin their local review. We believe that a successful federal-state cooperative program involving the level of effort of LUCA must be a two-way street in which there are direct benefits not only to the Census Bureau but also

OCR for page 97
The 2000 Census: Counting Under Adversity to participating localities. The Census Bureau could adapt features of other federal-state cooperative statistical programs toward this end. For example, the Bureau should consider paying for a MAF/ TIGER/LUCA coordinator in each state, who would be a focal point for communication with localities. The Bureau should also give serious consideration to providing localities with updated MAF files, which would not only facilitate continuous updating of the MAF for the Bureau’s purposes but would also provide a useful tool for local planning and analysis. An issue for concern would be that sharing of MAF files might violate the confidentiality of individuals—for example, by disclosing overcrowding of housing units in violation of local codes. However, our view is that the confidentiality issues could be resolved; street addresses do not, of themselves, identify information about individual residents or even indicate whether an address is occupied. For structures that may have been divided into multiple housing units illegally, the Bureau could protect confidentiality through the use of data perturbation or masking techniques. For example, the Bureau could provide a street address for the structure and code the number of units as, say, 1 to 3, or 2 to 4, in a way that preserves confidentiality; alternatively, it could randomly assign a value for the number of units in a structure for a small sample of structure addresses. A further protection would be to provide the MAF to localities under a pledge to use it for statistical purposes only, not for enforcement purposes, and not to provide it to others. However, Title 13 of the U.S. Code would probably require amendment similar to the 1994 legislation that authorized LUCA, since U.S. Supreme Court precedent views the MAF as covered under Title 13 confidentiality provisions.19 There is national benefit in having an accurate address list for statistical uses that can be continuously maintained in a cost-effective manner. This national benefit should challenge the Census Bureau to think creatively about ways to share the MAF with localities while protecting the confidentiality of individual residents. 19   In Baldridge v. Shapiro, 455 U.S. 345 (1982), the U.S. Supreme Court ruled that the Census Bureau’s “address list … is part of the raw census data intended by Congress to be protected” under the confidentiality provisions of Title 13. Accordingly, the court concluded that the bureau’s address list is not subject to disclosure under the Freedom of Information Act or under the discovery process in civil court proceedings.

OCR for page 97
The 2000 Census: Counting Under Adversity Evaluation The Bureau needs to build a continuous evaluation program into its 2010 MAF/TIGER operations from the outset. That program should specify clear codes for tracing the source(s) of each address to facilitate evaluation. It should also include ongoing quality control and review procedures to determine what is working well and what is not so that corrective actions can be taken in a timely manner. Built into the MAF quality control and evaluation efforts should be a capability to analyze geographic variations in the effectiveness of address sources and in the completeness and accuracy of the list. Recommendation 4.3: Because a complete, accurate Master Address File is not only critical for the 2010 census but also important for the 2008 dress rehearsal, the new American Community Survey, and other Census Bureau surveys, the Bureau must develop more effective procedures for updating and correcting the MAF than were used in 2000. Improvements in at least three areas are essential: The Census Bureau must develop procedures for obtaining accurate information to identify housing units within multiunit structures. It is not enough to have an accurate structure address. To increase the benefit to the Census Bureau from its Local Update of Census Addresses (LUCA) and other partnership programs for development of the MAF and the TIGER geocoding system, the Bureau must redesign the program to benefit state and local governments that participate. In particular, the Bureau should devise ways to provide updated MAF files to participating governments for statistical uses and should consider funding a MAF/TIGER/LUCA coordinator position in each state government. To support adequate assessment of the MAF for the 2010 census, the Census Bureau must plan evaluations well in advance so that the MAF records can be assigned appropriate ad-

OCR for page 97
The 2000 Census: Counting Under Adversity dress source codes and other useful variables for evaluation. 4–F GROUP QUARTERS ENUMERATION 4–F.1 Strategy The population that resides in group quarters (e.g., college dormitories, prisons, nursing homes, juvenile institutions, long-term care hospitals and schools, military quarters, group homes, shelters, worker dormitories), and not in individual apartments or homes, is a small fraction of the total population—about 2.7 percent in 1990 and 2000.20 Individual group quarters often contain large numbers of people, however, so that group quarters residents can represent a significant component of the population of particular small areas (e.g., college students living in dormitories in a college town). The census is the only source of data on group quarters residents in addition to household members, so enumeration of this population, even though small, is important. Mailout/mailback techniques and the questionnaires used for household enumeration are not appropriate for many group quarters, so there was a separate operational plan for group quarters enumeration in 2000, as in censuses since 1970. This plan used procedures similar to those for previous censuses, with a few exceptions (see Citro, 2000c; U.S. Census Bureau, 1999b). Group quarters residents were asked to fill out individual (one-person) questionnaires called Individual Census Reports; somewhat different questionnaires were used for enumeration of four groups: armed forces personnel in group quarters, military and civilian shipboard residents, people enumerated at soup kitchens and mobile food vans (part of service-based enumeration), and all other group quarters residents. There was no effort to enumerate the homeless population not encountered at designated locations, which included some nonshel- 20   In Census Bureau terminology, one or more group quarters make up a special place. Such places (e.g., a university) are administrative units, the individual group quarters (e.g., dormitories) are where people sleep. A structure that houses a group quarters may also include one or more housing units (e.g., the apartment for a resident faculty member in a dormitory).

OCR for page 97
The 2000 Census: Counting Under Adversity tered outdoor locations in addition to shelters, soup kitchens, and regularly scheduled mobile food vans. The original design for 2000 included a plan to estimate the total homeless population by using information for people enumerated at shelters and soup kitchens on how many nights each of them used the service. The actual responses to questions on service usage exhibited high nonresponse and response bias, so plans were dropped to include adjusted counts for the homeless in any adjusted population totals (Griffin and Malec, 2001; see also U.S. General Accounting Office, 2003b). Such adjusted counts could not be used for reapportionment totals, in any case, because of the U.S. Supreme Court decision in January 1999 that precluded the use of sampling for census counts for reapportionment. 4–F.2 Implementation Problems For reasons that are not clear, the design and implementation of enumeration procedures for group quarters residents experienced numerous problems (see Jonas, 2002, 2003b): There was no definition of a group quarters as distinct from a large household of unrelated individuals. In 1980 and 1990 any residential unit with 10 or more unrelated people was tabulated as a group quarters; no such rule was set for 2000. Moreover, some group quarters (e.g., halfway-homes) are located in structures that appear to be housing units. Such ambiguities contributed to problems in developing an accurate, nonover-lapping MAF for housing unit and group quarters addresses. The development of an inventory of special places was handled by the Census Bureau’s Population Division staff,and special places addresses were not integrated with the main MAF (maintained by the Bureau’s Geography Division) until November 1999. Consequently, the Special Places LUCA operation was delayed by 18 months from the original schedule, and many localities could not participate given demands on their resources for census outreach and other activities close to Census Day. A number of group quarters, including long-established prisons and college dormitories, were assigned incorrect

OCR for page 97
The 2000 Census: Counting Under Adversity geographic codes. For example, a prison would be geocoded to a neighboring town or county. While not affecting the total population count, these errors significantly affected counts for some small geographic areas. Such errors account for a large fraction of the population counts that local jurisdictions challenged in the Census Bureau’s Count Question Resolution Program.21 There was no system, such as a preprinted group quarters identification code, for tracking individual questionnaires from group quarters residents. Instead, a total count of questionnaires received was recorded on a control sheet for each group quarters at several steps in the process, and discrepancies and errors crept into the processing as a result. The lack of a good tracking system also impeded evaluation. A small but undetermined number of group quarters questionnaires were never returned to a local census office and never included in the census. Also, some questionnaires were assigned to a different group quarters after receipt by a local office. In May 2000 the National Processing Center at Jeffersonville, Indiana—which was solely responsible for capturing the information on group questionnaires—reported that many questionnaires were not properly associated with a “control sheet” and therefore did not have a group quarters identification number on them. A team of census headquarters staff reviewed an estimated 700,000 group quarters questionnaires to resolve this problem (no official records were kept of this special operation). In July 2000 two special telephone operations were implemented to follow up group quarters with no recorded 21   This program occurred mainly in 2001. It permitted local governments to challenge their census population counts for geocoding errors and other errors that could be well documented. The Census Bureau reviewed such challenges and issued letters to local jurisdictions when their population count was increased (or decreased) after review. Localities could cite the letters for such uses as obtaining federal program funds; however, the revised counts could not be used for reapportionment or redistricting.

OCR for page 97
The 2000 Census: Counting Under Adversity population (presumably refusals) and group quarters for which the number of data-captured questionnaires fell far below the population count obtained in advance visits to special places conducted in February–March 2000. Population counts were obtained for these group quarters, and the results used to impute group quarters residents as needed. More than 200,000 group quarters records (almost 3 percent of the group quarters population) were wholly imputed as a consequence of the telephone follow-up and the reconciliation of multiple population counts on the group quarters control sheets. Some group quarters residents were mailed a housing unit questionnaire. If they returned it and the address was matched to a group quarters address, they were added to the appropriate group quarters count, but there was no provision to unduplicate such enumerations with enumerations obtained through the group quarters enumeration procedure. From a clerical review of a sample of cases in selected types of group quarters (excluding prisons, military bases, and service-based facilities such as soup kitchens), an estimated 56,000 group quarters enumerations were duplicates of other enumerations within the same group quarters. Residents at some types of group quarters (e.g., soup kitchens, group homes, worker dormitories) could declare a “usual home elsewhere” and be counted at that address if the address could be verified. This procedure was not implemented as designed, and the result was that a net of 150,000 people were moved erroneously from the group quarters to the housing unit universe. Of these, 31,000 people were erroneously omitted from the census altogether. A higher-than-expected proportion of group quarters enumerations were obtained from administrative records of the special place. Of the 83 percent of enumerations for which enumerators indicated the source of data, 59 percent were filled out from administrative data, 30 percent were filled out by the resident, and 12 percent were filled out by an enumerator interviewing the resident. Types of group quarters with high percentages of enumerations obtained from administrative data included

OCR for page 97
The 2000 Census: Counting Under Adversity nursing homes, hospitals, group homes, and prisons. These types of group quarters had especially high rates of missing data for long-form items (see Section 7-D). Because of poor results in the 1990 Post-Enumeration Survey Program, a deliberate decision was made to exclude group quarters residents from the 2000 A.C.E. Program (see Section 5-D.1 for the basis of this decision). As a consequence, there was no way to assess omissions of group quarters residents or the full extent of erroneous enumerations for this population. 4–F.3 Group Quarters: Summary of Findings and Recommendations Overall, we conclude that the procedures for enumerating group quarters residents and processing the information collected from them were not well controlled or carefully executed. There is evidence of errors of omission, duplication, and miscoding by geography and type of group quarters, although there are no data with which to conduct a definitive evaluation, partly due to the lack of procedures to track individual enumerations. The extent of imputation required for group quarters residents was high, not only in terms of the number of whole-person imputations required, but also in terms of item imputations, particularly for long-form-sample items (see Section 7-D). Finding 4.6: The enumeration of people in the 2000 census who resided in group quarters, such as prisons, nursing homes, college dormitories, group homes, and others, resulted in poor data quality for this growing population. In particular, missing data rates, especially for long-form-sample items, were much higher for group quarters residents than for household members in 2000 and considerably higher than the missing data rates for group quarters residents in 1990 (see Finding 7.3). Problems and deficiencies in the enumeration that undoubtedly contributed to poor data quality included: the lack of well-defined concepts of types of living arrangements to count as group quarters; failure to integrate the development of the group quarters address list with the devel-

OCR for page 97
The 2000 Census: Counting Under Adversity opment of the Master Address File; failure to plan effectively for the use of administrative records in enumerating group quarters residents; errors in assigning group quarters to the correct geographic areas; and poorly controlled tracking and case management for group quarters. In addition, there was no program to evaluate the completeness of population coverage in group quarters. Enumeration procedures for group quarters residents need re-thinking and redesign from top to bottom if the 2010 census is to improve on the poor performance in 2000. Questionnaire content and design also need to be rethought to obtain higher quality data (see Section 7-D), including the possibility of asking residents for alternate addresses to facilitate unduplication (e.g., home addresses for college students and prisoners). Given the increase in population of people in some types of group quarters and the lack of other data sources for group quarters residents, improvement in the enumeration of group quarters should be a high-priority goal for the 2010 census. Tracking systems should be built into the enumeration not only to facilitate a high-quality operation but also to support subsequent evaluation. Recommendation 4.4: The Census Bureau must thoroughly evaluate and completely redesign the processes related to group quarters populations for the 2010 census, adapting the design as needed for different types of group quarters. This effort should include consideration of clearer definitions for group quarters, redesign of questionnaires and data content as appropriate, and improvement of the address listing, enumeration, and coverage evaluation processes for group quarters.