Click for next page ( 104


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 103
5 The 2003 and 2004 Census Tests AN IMPORTANT PART OF PLANNING for the decennial census is testing trying out new procedures and techniques in or- der to finalize the census design before the count begins. A regular feature of the census process since the 1940 census, the Census Bureau's program of intercensal tests has pursued several major direc- tions (Bailer, 2000~: major changes in census methodology (most notably the conver- sion to mailout/mailback as the dominant mode of census collec- tion and the use of sampling); techniques to improve coverage and to better measure census coverage; optimal questionnaire wording and format; new technology; and . . Improved census processing. 103

OCR for page 103
104 PLANNING THE 2010 CENSUS From all indications, the Census Bureau is not eager to repeat the experience of the 2000 census, in which the lateness in reaching a general census design limited the effectiveness of operational testing. T T 1 .1 1 1- ``T 1 ~ 1 1 rr ~ 1 0 Under the heading Lessons Learned from Census 2000, Waite (2002) emphasized the importance of effective testing: "If we want to achieve our Census 2010 Goals, operational testing of design infrastructure must start early in the decade and continue through the Dress Re- hearsal. In particular, the census dress rehearsal typically held 2 years prior to census deployment should properly be a comprehen- sive run-through of census machinery to fine-tune the final census design. However, in 1998, the dress rehearsal had to serve as a feasi- bility test for three quite different general designs, involving different levels of sampling techniques (National Research Council, 2001a). The Census Bureau's proposed plans for the 2010 census partic- ularly the elimination of the census long form are sufficiently aiffer- ent from the plans for the 2000 census that all possible opportunities for testing design options must be tully exploited In order to Analyze an effective design for 2010. As depicted in Table 1-1, milestones in the 2010 planning process include major census tests roughly every other year leading up to 2010. The Census Bureau shared plans for the 2003 census test with the panel at its September 2002 meeting and in subsequent discussions. However, the plans were shared with us too late to allow us to suggest or for the bureau to effect any meaningful change in the 2003 test plan. Plans for 2004 and 2006 are still under development, and the panel looks forward to continued work with the Census Bureau on those plans. In this chapter, we offer comments on the 2003 test and initial comments on the 2004 test and will revisit the 2004 and 2006 tests in great detail in the final report. 1 1 1 1 ~ 1 1 1 1 1 ~ 1 - 1 1 1 r 1 _ A AN 2003 NATIONAL CENSUS TEST 1 1 AS presented to the panel, the 2003 National Census Test will be a nationwide test involving 250,000 households. It is strictly a mailout test; no field enumerators will be used to conduct nonresponse follow- up, thus distinguishing this test from the proposed 2004 Census Field Test, which will have a follow-up component. Households selected for

OCR for page 103
THE 2003 AND 2004 CENSUS TESTS . . 105 inclusion in the test were set to be notifier! by an advance mailing in late January (U.S. Census Bureau, Public Information Office, 2003a). The 2003 test focuses primarily on two issues: Multiple Self-Response Modes. Offering respondents the opportu- nity to respond by mailing the questionnaire back (the traditional method, filling out an Internet version of the questionnaire, or responding to the questionnaire via an automated TouchTone telephone system known as interactive voice response (IVR). Race and Ethnicity Question Wording. Altering the way in which the questions on race anct Hispanic origin are presented, incluct- ing the omission of the "some other race" category currently of- ferect as an alternative uncler race. Prior to the 2000 census, it was hoped that a seconct mailing of replace- ment questionnaires coulc! be usec! to increase responses to the initial mailed questionnaire; however, logistical concerns dictated that it be ciroppec! from the 2000 census plan. The 2003 census test is not directly a test of the use of targeted replacement questionnaires; however, the current plan is to use replacement questionnaires as part of the limited (not in-person) follow-up process. Hence it is thought that the test could yield some insight as to the effectiveness of replacement ques- . . . . . t1onna1res In Improving response. Plans call for the test to consist of 16 experimental groups, in each of which the households will be sent a letter in advance of the actual questionnaire delivery. A control group will be eligible for a replacement questionnaire in nonresponse follow-up but will have the race and ethnicity questions as worded in the 2000 census. Eight treatment groups will be used in the response mode portion of the test; each group will vary on whether IVR or Internet response is mentioned on the initial questionnaire or encouraged in a reminder postcard. An acictitional seven treatment groups on race anct ethnicity wording will round out the experimental groups. Each of these groups has slight variations on the wording of the questions: for example, whether "some other race" is allowocl as a possible response or whether examples of possible Hispanic origin groups are listed in the question. A particularly subtle variation involves whether respondents are askoct . .

OCR for page 103
106 PLANNING THE 2010 CENSUS "Is Person x Spanish/Hispanic/Latino?" versus the alternative "Is Person x of Spanish, Hispanic, or Latino origin?" From a purely conceptual point of view, developing a test strategy for a new decennial census design shoulc! flow from anc! build upon the experience of the previous census. In this case, evaluations from the 2000 census woulc! be usec! to identify problem areas parts of the cen- sus process in which procedures may not have performed as expected or in which problems occurrec! anc! tests woulc! be clevisec! to evalu- ate alternative routines. In this context, the selection of the preceding two topics for the first major census test leacling up to the 2010 census is intriguing but somewhat confusing. That the Census Bureau should investigate the effectiveness of accommodating responses by phone anc! Internet is clear, but whether the issue merits a large resource-intensive test this early in the clecacle is a valict point of clebate. With regard to both topics, it coulc! be argucc! that resources might better be channeled into completing evaluations of the 2000 census that could guicle more effective targeted testing. For example, one might naturally prefer to exploit data already in hand to answer key questions before launching a large-scale study: How many respondents submitted both Internet anct paper responses? What was the level of item nonresponse in the limited number of Internet responses relative to mail responses? Is there evidence that the restrictive security measures surrounding the 2000 census Internet option cleterrec! respondents from answering via the Internet? How many Internet respondents started to provide data but ctict not complete the instrument? How cloes the population that iclentifiec! itself as "some other race" compare to other subgroups, en c! why is dropping the possible response "some other race" a useful or meaningful possibility? Another consideration for timing the tests of Internet responses, in particular, is that this is an extremely dynamic area, with tremen- clous changes in the uptake of Internet-capable home computers, high- speecl connections, and other technologies. It would be risky to precli- cate a test of methods for 2010 on any predictions that could be made now other than that current state-of-the-art technologies will be more widely ctispersect anct new technologies will have appeared In fact, even without a test of Internet technologies, we probably could say more today about results that will be obtainer! in 2003 than we can about the relationship between those results and what will actually be found in

OCR for page 103
THE 2003 AND 2004 CENSUS TESTS 107 2010. Thus, data on Internet response collected in 2003 might con- tribute little to the planning process, relative to competing activities. A primary concern for the panel was the design of the race anct ethnicity component of the test. The plan for this portion involves seven treatment groups that like the control anct response mocle treatment groups are stratified only by their mail response rate in the 2000 census (specifically, they are grouped into "high" and "low" response groups based on a selectee! cutoff). The panel's concern is whether this design is sufficient to answer the primary questions of interest. The differences in the questions aciministerec! to the cliffer- ent treatment groups are often quite subtle the ctistinction between being "Hispanic" versus "of Hispanic origin," for example. Hence it is unclear whether the sample design will generate enough coverage in Hispanic communities to facilitate conclusive comparisons whether it will reach enough of a cross-section of the populace to gauge sensi- tivity to slight changes in question worcting anct whether it will reach a sufficiently heterogeneous mix of Hispanic nationalities anc! origins to clecicle whether including instructions or examples improves response. To this encI, there are two methods by which the effective number of respondents who are active on this question could be augmented. First, either through selection of test sites or through oversampling of blocks within test sites, the number of respondents who are likely to have rel- evant characteristics can be increasecl. Seconcl, given that the two major treatment factors being examiner! response mocle anc! questionnaire wording are likely to be relatively independent (orthogonal) in ac- tion, it seems reasonable to completely cross the two treatment factors in this experiment. By varying response mode and race/ethnicity worcl- ing for the complete sample, power to distinguish between alternatives may be gained relative to the current design using effectively separate experiments. Also, the alternative race and Hispanic origin questionnaire worcl- ings plannec! for the 2003 test are variations along three different fac- tors: identification as "Hispanic" versus "of Hispanic origin," inclusion of "some other race" as a valic! choice, en c! provision of examples of cletailect Hispanic origin. The panel is concerned that these alternatives are relatively narrow anct, as a result, may not provide sufficient infor- mation on question formats for use in 2010. It is also unclear that these particular selections are derived from an established research base. For 1 . . .

OCR for page 103
108 PLANNING THE 2010 CENSUS the purposes of a true experiment, our view that it may be useful to consider even more radical question alternatives, such as folding race and Hispanic origin into a single question in a "check one or more" format. Such an alternative may in fact not be tenable as a choice in 2010 it may force higher levels of imputation for groups who select Hispanic but no legally clefinec! race category, anc! it may be counter to Office of Management anct Budget guidelines on race anct ethnicity categorization. But from a research perspective, such might be the best way to determine how people prefer to categorize themselves, anct the results may inform strategies for analysis anc! tabulation of final census results. 2004 CENSUS FIELD TEST In October 2002, the Census Bureau announced the selection of sites for the 2004 Census Field Test. The selectee! sites are Colquitt, Thomas, anc! Tift counties in Georgia; Lake County in Illinois; en c! a portion of northwestern Queens County in New York. Original plans called for the 2004 test to involve approximately 450,000 housing units across the various sites; since the test involves a fielct work component, approximately 3,000 temporary jobs will be created to conduct the test (U.S. Census Bureau, Public Information Office, 2002~. As cliscussec! at the panel's September 2002 meeting, still-cleveloping plans for the 2004 test call for work on at least seven different topic areas. Test sites were selectee! to try to obtain a variety of geographic types (urban/suburban/rural) and racial groups (Waite, 2002~. Though fielc! work will be clone in each of the test sites, anti, in some re- spects, the activity will almost seem to be a census in miniature, the Census Bureau is not promising or even offering participating sites a population count at the enc! of the test. 1 Under the fiscal year 2004 budget proposed by President Bush in January 2003, the Census Bureau would scale down the 2004 test to drop Lake County, Illinois, as a test site and to reduce the planned workload in Queens County, New York (Lowenthal, 2003b).

OCR for page 103
THE 2003 AND 2004 CENSUS TESTS 109 Mobile Computing Devices A clear, primary thrust of the 2004 test is work with mobile com- puting crevices (MCDs) for fielc! work. To ciate, the Census Bureau's testing of MCDs have been small pilot tests of basic skills. For in- stance, small numbers of fielc! staff with different levels of local famil- iarity were assigned to locate an assigned set of acictresses on TIGER- basec! maps on a Pocket PC-class crevice in a pilot test in Gloucester County, Virginia. This test concentrated only on locating features us- ing a small-screen map and not on using the computer to calculate a route to those features (U.S. Census Bureau, Mobile Computing De- vice Working Group, 2002~. The Census Bureau's hope is that the 2004 test will be a more com- prehensive test of MCD capabilities, including use of global position- ing system (GPS) receivers and computer-assistect personal interview- ing software to administer short-form interviews in English or Spanish. As we commented in Chapter 2, prototyping of MCDs in 2004 is very important for getting a sense of current capabilities, but it is more important to clarify the requirements and information flows associated with the crevices. That said, we hope that sufficient information is gained about the process of MCD and GPS use cluring the 2004 test to inform final design decisions later in the clecacle. Race and Hispanic Origin Questionnaire Wording The Census Bureau also intends to follow up its work in the 2003 mailout-only test by including alternate wordings of the race en c! Hispanic origin census questions in the 2004 field test; these proposed wordings were discussed earlier. It is unclear whether the full range of alternatives will be workoct into the 2004 test; given the panel's concern that the 2003 test alone is unlikely to provide definitive eviclence in favor of any one of the subtle alternatives uncler consideration, we hope that the 2004 test is not used merely to test one "favored" alternative from the 2003 results. Foreign Language Questionnaires In the 2000 census, respondents requiring a questionnaire in a language other than English could request it from the Census Bureau,

OCR for page 103
110 PLANNING THE 2010 CENSUS from local census offices, or from follow-up enumerators. The 2004 test will be a first test for some proposer! improvements, including test deployment of a dual language (English and Spanish) question- naire. Delivery of the English/Spanish questionnaires is intenclec! to be clone in a targeted manner (e.g., based on high concentrations of Hispanic-origin responses in the 2000 census for a particular tract or block group). Other 2004 Test Topics Other topics have been identified for testing in 2004; however, the panel has not yet seen information on them or, more precisely, how exactly the Census Bureau plans to test them in fuller detail than bul- letec! lists of possible directions. Consequently, further comment on them awaits further interaction between the panel anct the Census Bu- reau as plans continue to take shape. These general topics include: Field-Based Coverage Improvement. Strategies for reclucing per- son duplication by clearer explication of residence rules and by better tracking of housing unit occupancy status cluring nonre- sponse follow-up. . Targeted Canvass to Update MAT: Use of administrative records and the Census Bureau's housing unit estimates program to iclen- tify localities with potential MAF coverage problems anct com- parison of aciciress canvasses in those areas with existing MAF coverage. . . Contact Strategy for Self-Response. Strategies for reminding and encouraging respondents to submit their questionnaires on their own (as opposed to having to be contacted by a field enumerator cluring nonresponse follow-up). One such strategy, a targeted mailing of a seconct questionnaire, is to be tested in both the 2003 Census Test and the 2004 Field Test. Special Place/Group Quarters. Development and testing of new definitions for group quarters en cl refinement of the techniques usec! to list group quarters for enumeration.

OCR for page 103
THE 2003 AND 2004 CENSUS TESTS 111 Counting Americans Overseas Distinct from the 2004 Census Field Test, the Census Bureau has announced plans for another test program for 2004. The Overseas Enu- meration Test would attempt to count American citizens resicting in France, Kuwait, anc! Mexico. The test will rely on a publicity campaign to be mounted using English-language media in the three countries. Po- tential respondents woulc! be urged to request that a questionnaire be mailed to them, to pick up a questionnaire at an embassy or consulate, to obtain a questionnaire from Census Bureau "partner organizations that serve Americans overseas," or to complete the questionnaire via the ~ % _% ~ ~ _ ~ ~ ~ ~ _ _ _ ~ ~ ~ ~~ lnternet I.. (census Bureau, Public lntormat~on Mince, 200Jb). the panel has not seen any more cletailec! information on this test, which is an apparent reaction to concerns raised in litigation shortly after the release of reapportionment totals.2 T ,- Imellne Uncler the basic timetable presented in Waite (2002), active prepara- tion for the 2004 Census Test will begin in April-Tune 2003 with the hir- ing anc! training of staff. Aciciress canvassing anc! the authoring of initial test evaluations are scheclulect for late 2003, as is questionnaire printing. Questionnaires are scheclulect for mailout in early 2004, with the refer- ence date (Census Day) set at April 1, 2004; questionnaire check-in, data capture, anct nonresponse follow-up continue through July 2004. Processing of the results en c! continucc! cirafting of evaluation reports are expected to continue through the rest of 2004 anct extenct into early 2005. 2Under the reapportionment counts issued by the Census Bureau in December 2000, North Carolina was allocated the 435th seat in the U.S. House of Represen- tatives, as prioritized by the method of "equal proportions" used to reapportion the House. A fourth congressional district for Utah was ranked 436th, falling short of the additional seat by less than 1,000 people. Consequently, Utah challenged the reappor- tionment counts on the basis that the Census Bureau's limited overseas enumeration omitted Mormon missionaries and other private citizens who should have been tallied in the census. However, Utah's case was rejected by a federal appeals court, and the U.S. Supreme Court declined to take the case on appeal (Lowenthal, 2003a). Utah would subsequently challenge the Census Bureau's imputation strategy but lost that challenge in a U.S. Supreme Court ruling (Utah v. Evans, 526 U.S. 452, 2002~.

OCR for page 103