–B–
2010 Census Program of Evaluations and Experiments

Since the issuance of the panel’s letter report in February 2009, some additional detail about the structure of the 2010 Census Program of Experiments and Evaluations (CPEX) has become available. Some revisions to specific CPEX components were indicated in a formal response by the Census Bureau to the panel’s letter report (U.S. Census Bureau, 2009a). Other information and detail on planned CPEX components have also been made available because of the Census Bureau’s filing for general approval of CPEX data collection with the U.S. Office of Management and Budget (OMB). Under law, OMB is responsible for reviewing and approving any information collection activity that will be administered to 10 or more respondents.1 The Census Bureau requested generic clearance of parts of the CPEX program involving original data collection,2 and the request was approved on May 12, 2009; subsequent specific changes and project plans have been submitted in addition to this generic clearance. The package (Information

1

As part of the clearance process, agencies must have also made two postings in the Federal Register to solicit public comment. Packages submitted by agencies must be accompanied by two supporting statements: Part A, giving a detailed justification of the collection and indicating how the data will be used, and—for statistical collections—Part B on methodology, sampling strategy, and preliminary testing.

2

“Generic clearance” seeks general authority to conduct data collection up to some set number of respondent-burden hours; though the content and questions of the data collection must still be specified, agencies have more latitude to make changes and revisions to specific forms and questions under a generic clearance.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 181
–B– 2010 Census Program of Evaluations and Experiments Since the issuance of the panel’s letter report in February 2009, some additional detail about the structure of the 2010 Census Program of Exper- iments and Evaluations (CPEX) has become available. Some revisions to specific CPEX components were indicated in a formal response by the Cen- sus Bureau to the panel’s letter report (U.S. Census Bureau, 2009a). Other information and detail on planned CPEX components have also been made available because of the Census Bureau’s filing for general approval of CPEX data collection with the U.S. Office of Management and Budget (OMB). Un- der law, OMB is responsible for reviewing and approving any information collection activity that will be administered to 10 or more respondents.1 The Census Bureau requested generic clearance of parts of the CPEX pro- gram involving original data collection,2 and the request was approved on May 12, 2009; subsequent specific changes and project plans have been submitted in addition to this generic clearance. The package (Information 1 As part of the clearance process, agencies must have also made two postings in the Federal Register to solicit public comment. Packages submitted by agencies must be accompanied by two supporting statements: Part A, giving a detailed justification of the collection and indicating how the data will be used, and—for statistical collections—Part B on methodology, sampling strategy, and preliminary testing. 2 “Generic clearance” seeks general authority to conduct data collection up to some set number of respondent-burden hours; though the content and questions of the data collection must still be specified, agencies have more latitude to make changes and revisions to specific forms and questions under a generic clearance. 181

OCR for page 181
182 ENVISIONING THE 2020 CENSUS Box B-1 Experiments in the 2010 CPEX • Alternative Questionnaire Experiment • Deadline Messaging Experiment • Nonresponse Follow-Up Contact Strategy Experiment • Privacy Notification Experiment • “Heavy-Up” Communications Experiment SOURCES: Presentations to the panel; “2010 CPEX Information Sheet: 2010 ICP Paid Media Heavy-up Experiment,” shared with panel in May 2008. Collection Review [ICR] 200902-0607-007) is accessible through OMB’s RegInfo web site (http://www.reginfo.gov). B–1 EXPERIMENTS As presented to the panel in early 2009, the Census Bureau’s 2010 CPEX program included four formal experiments. Since then, a fifth experiment has been added to the ranks. Box B-1 lists the experiments for ease of refer- ence; we provide additional description (and extend commentary from our letter report, as appropriate) on the experiments in the remainder of this section. B–1.a Alternative Questionnaire Experiment An Alternative Questionnaire Experiment (AQE) in which a sample of census respondents receives questionnaires that vary in content, layout, and question ordering and wording has been a staple of census experimentation since the 1950 census. In that census, 10 district offices in Ohio and Michi- gan were used as “experimental areas” in which—among other things—four census forms were oriented toward households as the unit of analysis and self-response by individuals, as opposed to the person-based ledgers then used by enumerators in conducting their interviewers (U.S. Census Bureau, 1955:5). The 2000 census AQE focused heavily on the effect of visual cues and narrative instructions to guide respondents through the census long- form questionnaire. It also included an experimental group that varied the instructions and formatting of the basic residence (household count) question on the census form; the National Research Council (2006:202– 203) observed that this single treatment constituted “a bundle of at least 10 changes,” some major and others extremely subtle, that rendered it impossi- ble to ferret out which features were more or less effective than others. The 2010 AQE is planned to include 19 panels, most of which (15) in- volve variations to the questions on race and Hispanic origin. The experi-

OCR for page 181
APPENDIX B 183 Figure B-1 2010 Alternative Questionnaire Experiment, control questionnaires

OCR for page 181
184 ENVISIONING THE 2020 CENSUS Figure B-2 2010 Alternative Questionnaire Experiment, structures of combined race and Hispanic origin question

OCR for page 181
APPENDIX B 185 Figure B-3 2010 Alternative Questionnaire Experiment, variations on race question

OCR for page 181
186 ENVISIONING THE 2020 CENSUS Figure B-4 2010 Alternative Questionnaire Experiment, variations on Hispanic question and hybrid approaches

OCR for page 181
APPENDIX B 187 Figure B-5 2010 Alternative Questionnaire Experiment, other residence panel

OCR for page 181
188 ENVISIONING THE 2020 CENSUS mental panels are as follows and are numbered in the accompanying figures using the Census Bureau’s scheme: • Controls: The AQE shares a planned control group of 30,000 hous- ing units with several of the other experiments described below; this group receives the standard 2010 census questionnaire with the only difference being that the phone number listed on the form for respon- dents to call if they have questions is a special CPEX line. However, the AQE includes a second control group that omits the overcount coverage probe question (i.e., “Does Person 1 sometimes live or stay somewhere else?”) on the 2010 census form because the space require- ments of the revised race and Hispanic-origin questions in the other experimental panels precluded the overcount question from fitting on the form.3 • Cumulative Changes from 2010: As shown in Figure B-1, one of the treatment groups uses the format and wording of the 2000 census questionnaire (save that it uses the 2010-standard blue color scheme rather than 2000’s yellow color). The 2000 census form did not in- clude either the overcount or undercount (“Were there any additional people staying here [on] April 1, 2010 that you did not include in Question 1?”) coverage probe questions that have been added for 2010, and so those questions are not included on the 2000-style form. This treatment group is meant for comparison with the control group in order to study the cumulative effect of design and format changes from 2000 to 2010, although it will not be able to shed light on which specific features were more or less effective than others. • Combined Race and Hispanic-Origin Question: Four panels test pos- sible structures for a “combined” race and Hispanic-origin question that lists Hispanic origin (and related subgroups) in line with tradi- tional race categories, as shown in Figure B-2. Two of the treatments (numbers 4 and 5) limit check-box choices to major race categories (permitting write-in of specific origins, nationalities, or tribal affilia- tions), and the others add specific check boxes for selected subgroups. • Variations on the Race Question: As illustrated in Figure B-3, five ex- perimental panels reflect suggestions from the Census Bureau’s Race and Ethnic Advisory Committees, varying the mix of listed examples for Other Asian (e.g., omitting “Thai” and adding “Mongolian”) and 3 A consequence of the overcount question not being included in most of the panels of the AQE, given that the overcount question is one of the ways households are flagged for inclusion in the Coverage Follow-Up (CFU) operation, is that the AQE households are less likely to enter into the CFU workload. The Census Bureau estimates that this creates “the possibility of about 1,600 individuals being double-counted in the Census” because their households are not entered into CFU.

OCR for page 181
APPENDIX B 189 Other Pacific Islander (adding “Marshallese”) groups. Two of the treat- ment groups (17 and 18) omit the word “race” from the wording of Question 9 and from the prefatory note, using the word only in the “Some other race” category.4 • Variations on the Hispanic-Origin Question: Also based on input from the Race and Ethnic Advisory Committee, one treatment group varies the listed examples of Hispanic subgroups (group 8) while another ex- plicitly permits respondents to check more than one Hispanic category, as is permitted for the race question (group 9; see Figure B-4). • Joint Variations of Race and Hispanic Origin: Also listed in Figure B-4, four treatment groups test combinations of the revised lists of race and Hispanic-origin examples with the “mark one or more” instruction on Hispanic origin. • Other Residence Information: As shown in Figure B-5, a final exper- imental group in the AQE expands on the overcount coverage probe question. If the respondent indicates that a person sometimes lives or stays somewhere else, he or she is prompted to provide address infor- mation for that other location. Two follow-up questions then attempt to determine which address is the “usual” residence (“live or stay most of the time”) and which was the “current” residence on Census Day. Originally planned for a target sample size of 560,000 households— 30,000 per treatment group except for the 2000-format panel 2, with 20,000 households—the exact numbers in each group may vary because of a change in approach to drawing the experimental sample. In previ- ous censuses, experimental groups were drawn once, from strata defined at the national level. However, the Census Bureau notes that the 2010 AQE will be different (ICR 200902-0607-007, individual request for clearance document for the AQE): For the 2010 processing system development, schedule and time lim- itations necessitate sampling each Local Census Office (LCO) before moving onto the next, rather than selecting our sample once at the na- tional level. This means that, since we will not have universe totals by stratum during the LCO sample selection, we have to fix our sampling intervals and let sample sizes vary. As a result of this process, the Census Bureau estimates that the actual sam- ple size of each 30,000-unit panel will be between 23,000 and 40,000 (and 4 In using the phrase “some other race”—even in an experimental treatment—the Census Bureau is complying with a congressional mandate. A passage in census appropriations language directs “that none of the funds provided in this or any other Act for any fiscal year may be used for the collection of census data on race identification that does not include ‘some other race’ as a category” (P.L. 110-161; 121 Stat. 1887). This provision was first inserted into the Census Bureau’s appropriations for fiscal year 2005, in an omnibus spending bill enacted in December 2004.

OCR for page 181
190 ENVISIONING THE 2020 CENSUS Figure B-6 2010 nonresponse follow-up enumerator questionnaire, record of contact box 15,000–26,000 for the 2000-format panel). The sample for the race and Hispanic-origin groups will be constructed hierarchically to try to ensure that the smallest demographic groups of interest are in the sample. In the ideal, fixed sample-size case, this would involve first drawing 9,000 house- holds per panel from tracts with 15 percent or more Asian or Pacific Islander people, then 9,000 from tracts with 25 percent or more black people, then from tracts with 40 percent or more Hispanic people, and finally 3,000 from all other tracts. For the other residence information panel, the Bureau indicated to OMB that it will attempt to target tracts with high densities of active-duty military personnel, seniors (possible nursing home residents), college-age students, and “areas more likely to have child custody coverage issues,” although how this will be done is not specified. The AQE is also a factor in one of the formal evaluations in the CPEX program—a reinterview study—as described below in Section B–2.a. B–1.b Nonresponse Follow-Up Contact Strategy Experiment The 2010 census will follow the example of recent censuses by calling for temporary enumerators to attempt up to six contacts with households in the nonresponse follow-up (NRFU) workload. Three of these visits are sup- posed to be in-person contact attempts, and the other three can be done by telephone if a phone number is available. If no contact is made by the sixth try, then proxy information (i.e., from a landlord or neighbor) can be col- lected as a last resort. The contact log section of the paper-based enumerator questionnaire planned for use in 2010 is illustrated by Figure B-6. The Nonresponse Follow-Up Contact Strategy Experiment of the 2010 CPEX proposes to test the effect of reducing the maximum number of con- tacts to either 4 or 5. As described to the panel in November 2008, the experiment was to take place in three local census offices and a total of about 52,000 housing units; all enumerators would use the same form with space for six contact attempts, but enumerators in sampled crew leader dis- tricts would receive different instructions on how many callbacks to make.

OCR for page 181
APPENDIX B 191 The Bureau argued that logistical challenges in varying field procedures (and training for fieldwork) precluded a larger sample of offices or housing units. We criticized the experiment in the letter report, principally on the grounds that it would be extremely difficult to generalize data from only three offices. Subsequently, the Bureau designed two alternative enumerator questionnaires—each of which uses the same amount of space for the contact log as in Figure B-6 but simply omits one or two entries (and adjusts the layout so that the remaining entries fill the space). As described in the Bureau’s reply to our letter report (U.S. Census Bureau, 2009a) and the Bureau’s filing with OMB, the design now calls for 1.2 million of these experimental enumerator forms (600,000 for each of the 4-contact- and 5-contact-maximum groups) to be randomly inserted with the standard enumerator forms, and hence for the experiment to take place in all 494 local census offices. The 1.2 million sample size is said to be large enough so that, on average, every enumerator will have approximately one experi- mental questionnaire in each of their assignment areas. Although the sample size is now massive, the redesigned experiment has the drawback that the enumerators will be acutely aware of the maximum number of attempts they can make; arguably, then, the test is less about the effect of simply cutting off NRFU after (say) four attempts than it is about whether enumerators will expend extra effort to try to resolve cases in four tries (in the same way that they may try particularly hard to make contact on the sixth time in normal cases). B–1.c Deadline Messaging/Compressed Schedule Experiment The Deadline Messaging/Compressed Schedule Experiment is intended to see whether mail response (and speed of response) improves when differ- ent messages urging a rapid response are included in four mailing pieces: 1. the advance letter sent prior to the main questionnaire mailout, 2. the envelope containing the census form, 3. the cover letter accompanying the census form, and 4. the reminder postcard sent after the main mailout but before beginning nonresponse follow-up. As originally described to the panel in November 2008, the experiment in- cluded three types of deadline messages; subsequently, a fourth has been added. The four message types are: 1. “Mild,” which simply suggests a date by which the form should be returned; 2. “Progressive Urgency,” which casts the date as a deadline for response and reminds the respondent that response is required by law;

OCR for page 181
192 ENVISIONING THE 2020 CENSUS 3. “Avoid NRFU Visit” (or, as the Bureau refers to it, “NRFU Motiva- tion”), which casts the date as a deadline and urges the respondent to avoid the trouble of having a nonresponse follow-up interviewer visit their home; and 4. “Cost Savings,” which notes that money is saved by simply mailing the census form rather than having an interviewer come to visit. It is useful to note that two of these message types repeat language used in 1990 census materials but not in 2000. The “Avoid NRFU Visit” message re- calls a statement made directly in the instructions on the 1990 census form: “Avoid the inconvenience of having a census taker visit your home.” Like- wise, the “Your Guide to the 1990 U.S. Census Form” brochure distributed with the 1990 census included a statement very similar to the “Cost Savings” message: “If you do not mail back your census form, a census taker will be sent out to assist you. But it saves time and your taxpayer dollars if you fill out the form yourself and mail it back.”5 The manner in which these deadline messages are rendered in the mailed items is shown in Table B-1. In the experiment, each of the deadline message strategies is used in combination with one of two schedules. Following the normal 2010 cen- sus schedule, advance letters are supposed to arrive between March 8 and March 10, the census questionnaire between March 15 and March 17, and the reminder postcards between March 22 and March 24. The alternative, “compressed” schedule shifts these dates by one week, closer to the April 1 Census Day: that is, advance letters arriving March 15–17, questionnaires March 22–24, and postcards March 29–31. Under the compressed sched- ule, the target or deadline date of April 5 referenced in Table B-1 remains the same. In our letter report, we offered little comment on the Deadline Messaging/Compressed Schedule Experiment, noting only that—like the other experiments—we were concerned about the lack of an analysis of the statistical power of the proposed test to discriminate between the alterna- tives. At the time, as presented to us in November 2008, each study panel was to include 10,000 households (drawn from sampling strata of expected high, medium, and low mail response based on the 2000 census). In replying to our letter (U.S. Census Bureau, 2009a), the Census Bureau cited two un- published Census Bureau internal memoranda (not shared with the panel) as justifying the sample size selection; the reply also indicated that the number of housing units per panel in the deadline experiment would be doubled to 20,000, without any indication of how this level was determined. 5 However, the 1990 census materials also explicitly and repeatedly directed respondents to “return the completed form by April 1, 1990”—that is, to report information on one’s household as of Census Day before that day had actually arrived—which the National Research Council (2006:Sec. 6–F) notes is a basic “violation by design” of the underlying census residence concept.

OCR for page 181
Table B-1 2010 Deadline Messaging/Compressed Schedule Experiment, deadline message treatments by form type APPENDIX B Panel Advance Letter Initial Mailing Envelope Cover Letter Reminder Postcard If you have not responded, Control (2010 When you receive your YOUR RESPONSE IS Please complete and mail please provide your Standard) form, please fill it out and REQUIRED BY LAW back the enclosed census form today. information as soon as mail it in promptly. possible. Please complete and mail If you have not responded, 1 (Mild) When you receive your YOUR RESPONSE IS back the enclosed census please provide your form, please fill it out and REQUIRED BY LAW ¶ information by April 5. mail it in by April 5. Mail by April 5 form by April 5. The deadline to complete If you have not responded, 2 (Progressive When you receive your YOUR RESPONSE IS and mail back the enclosed the deadline to provide Urgency) form, please fill it out and REQUIRED BY LAW ¶ census form is April 5. your information is April 5. mail it in by April 5. Deadline is April 5 Your response is required by law. If you have not responded, Please complete and mail 3 (Avoid NRFU When you receive your YOUR RESPONSE IS please provide your back the enclosed census Visit) form, please fill it out and REQUIRED BY LAW ¶ information by April 5 Mail by April 5 form by April 5 so that you mail it in by April 5. so that you can avoid a can avoid a personal visit personal visit from an from an interviewer. interviewer. If you have not responded, Please complete and mail 4 (Cost Savings) When you receive your YOUR RESPONSE IS please provide your back the enclosed census form, please fill it out and REQUIRED BY LAW ¶ information by April 5. form by April 5. Mailing mail it in by April 5. Mail by April 5 Mailing your census form your census form on time on time saves money that saves money that would would otherwise be used to otherwise be used to follow follow up with you. up with you. 193

OCR for page 181
194 ENVISIONING THE 2020 CENSUS B–1.d Confidentiality/Privacy Notification Experiment The cover letter accompanying a decennial census questionnaire typically is a letter or other printed message (often over the signature of the director of the Census Bureau) that assures respondents that the information they provide is kept confidential and is not disclosed to other government agen- cies. The planned cover letter for the 2010 census includes a short letter on one side and a short statement on confidentiality on the reverse side (the letter suggests that the document be turned over to read that infor- mation). The Confidentiality/Privacy Notification Experiment of the CPEX program involves two experimental treatments that make small changes to both the front and back of the cover letter, as shown in Figure B-7; these small changes are also made in the text of the letter that accompanies the second (replacement) questionnaire that the 2010 census will mail prior to the start of nonresponse follow-up. A control group—shared with the AQE and deadline messaging experiment—receives the standard 2010 cover let- ter in both the initial and second questionnaire mailings. A relatively small exercise as such census experiments go, the Confidentiality/Privacy Notifica- tion Experiment is planned to include 20,000 housing units in each of the two experimental panels. B–1.e Heavy-Up Publicity Experiment In May 2009, the panel was informed that a fifth experiment—a “heavy- up” test of increased media buys for census publicity materials in selected markets—had been added to the CPEX program. The experiment is appar- ently intended to study the effectiveness of saturated advertising (base mes- sages via local media buys or culturally/ethnically targeted media buys) in selected marketing areas. Because the only information the panel has heard or seen concerning this test is a short “fact sheet,” and the heavy-up experi- ment is not referenced in the Bureau’s CPEX clearance request to OMB, we cannot provide additional commentary on this experiment akin to what we provided for the other four CPEX experiments in our letter report. B–2 EVALUATIONS The Census Bureau’s supporting statement for its OMB submission for CPEX (ICR 200902-0607-007) describes it as including “over 20 evalua- tions.” Box B-2 lists the evaluation topic areas and the names of individual studies as they were presented to the panel in early 2009. The OMB submis- sion provides detail on four of the proposed evaluations that involve original data collection, as discussed below.

OCR for page 181
APPENDIX B Figure B-7 2010 Confidentiality/Privacy Notification Experiment, control and experimental treatments 195

OCR for page 181
196 ENVISIONING THE 2020 CENSUS Box B-2 Evaluations in the 2010 CPEX Coverage Improvement • Address frame accuracy—in preliminary design, said to involve (a) comparison of census coverage measurement (postenumeration survey) with census returns to study address errors and (b) additional field data collection in discrepant cases to establish “ground truth” • Address canvassing targeting—analysis of actual address canvassing results in a set of blocks identified before the operation as “growth” areas • Data-based extraction processes for address frame—use of data mining tech- niques to derive decision trees to predict address validity from address source(s), validated by comparison with address canvassing results • Small multiunit structures—in preliminary design, said to include assessment of revised training materials and use of a special field activity to study possible identifiers for such structures (i.e., unit labeling conventions, mail drop points) • Address list maintenance using supplemental data sources—comparison of Master Address File with updates from sources not currently used by the Census Bureau, possibly including updates from the Bureau’s Demographic Area Address Listing and American Community Survey operations as well as aerial imaging • Effectiveness of unduplication operations—in preliminary design, may include specific routing of some long-distance duplicates (cases where name and date of birth match but are not geographically proximate) through the Coverage Follow-Up (CFU) operation • Coverage of Group Quarters Population—based on use of an Alternative Group Quarters Report, as described in Section B–2.c • Alternative questions in CFU interview—use of questions at the end of CFU interview if responses conflict with answers to the undercount and overcount coverage probe questions on the census forms • Other proposed topics: Administrative records for studying coverage problems; ethnographic studies Coverage Measurement • Prospective Topics: Coverage of group quarters population; reverse record check; quality of Census Coverage Measurement (CCM) data collection and processing; comparison of operations history with CCM results; use of administrative records to augment CCM field work; comparison of household measurement in the census and the CCM survey • Planned to be completed by fall 2012 Field Operations • Automation for address canvassing—in preliminary design, said to involve comparison of cost and progress data for 2010 address canvassing with similar operations in 2000 as well as summaries of field personnel reactions to use of handheld computers (including Help Desk calls) • Geographic positioning system (GPS) technology for quality control—in prelimi- nary design, said to involve identification of unusual clusters of GPS coordinates (compared with manually drawn map spots) and use of imagery to examine veracity of clusters Language Program • Observation of enumerator interactions with non-English-speaking households Questionnaire Content • Behavior coding of enumerator interviews—analysis of audio tapes of field and telephone center interviews by survey methodologists to study how well interviewers ask, and how respondents answer, questions (continued)

OCR for page 181
APPENDIX B 197 Box B-2 (continued) • Prospective Topic: Comparison of 2010 census and American Community Survey data Marketing and Publicity • Evaluation of integrated communication program—evaluation to be conducted by National Opinion Research Council, making use of three surveys (including pre- and post-census) study awareness of census outreach Privacy and Confidentiality • Prospective Topic: Public concerns about privacy and confidentiality (possibly linked with communication program evaluation under “Marketing and Publicity”) SOURCES: Presentations and materials shared by the Census Bureau with the panel, particularly Jackson (2008) and Reichert (2009). B–2.a Alternative Questionnaire Experiment Reinterview The principal focus of the AQE is differences in response to revised forms of the census questions on race and Hispanic origin; the Census Bureau plans to conduct a reinterview study with about 4,000 households from each of the 15 race and Hispanic-origin treatment groups in the AQE (as well as from the two control groups) to assess response bias. Plans call for the reinterviews to be conducted exclusively by telephone, so the availability of phone numbers (either provided by respondents on the form or found through directory look-ups) will dictate final sample sizes. Because the in- terest is in response bias, reinterviews will be conducted with the household members who completed the mail census form whenever possible (accepting proxy responses only as a last resort). The reinterview walks through the other questions on the short-form-only census questionnaire but is meant to be more probing with regard to the race and ethnicity questions (i.e., seek- ing yes/no verification for every major category and subcategory); it is also meant to be more conversational by including an open-ended question on race and origin perceptions (“I’d like you to think about what you usually say when asked about your race and origin. . . . Keeping in mind that you can say more than one, what do you usually say when asked about your race and origin?”). B–2.b Content Reinterview On a much smaller basis than the AQE reinterview evaluation, the Cen- sus Bureau plans to conduct a general content reinterview study similar to those done in previous censuses. Like the AQE, plans call for the Content Reinterview to be performed by telephone, with an estimated sample size of 10,000 interview cases (drawn “proportionally across mail return ques-

OCR for page 181
198 ENVISIONING THE 2020 CENSUS tionnaires, update/enumerate interviews, nonresponse followup interviews, etc.”) in the United States and 860 in Puerto Rico. B–2.c Alternative Group Quarters Questionnaire The Census Bureau’s planned Alternative Group Quarters Questionnaire experiment builds from experience in the 2000 census, but it also portends to repeat a major lost opportunity from that census. In 2000, every Individual Census Report questionnaire filled by residents of group quarters (college housing, correctional facilities, health care facili- ties, etc.) included the question: “What is the address of the place where you live or stay MOST OF THE TIME?” (An instruction at the end of the first page of the form was intended to route respondents who live or stay at the group quarters location “most of the time” past this second address question and to the end of the questionnaire; still, the query for the second address dominated the second page of the form.) The same format (with slightly revised wording to fit the circumstances) held for the Military Census Re- ports and Shipboard Census Reports used to enumerate on-base military personnel and shipboard personnel. Although this “usual home elsewhere” (UHE) query was included on all group quarters questionnaires, the Census Bureau’s residence rules for the 2000 census considered UHE information from only certain group quarters types to be valid: valid for military per- sonnel and such small group quarters as temporary worker camps, carnival grounds, and monasteries and convents, but invalid for the most sizable of group quarters populations (e.g., the aforementioned college students, pris- oners, or persons in nursing homes). Like the 2000 census, the Individual Census Report for the 2010 census (Form D-20, filed with OMB in ICR 200808-0607-003) asks about a usual home elsewhere regardless of group quarters type: if a respondent answers “no” to the question “Do you live or stay in this facility MOST OF THE TIME?” he or she is asked “What is the full address of the place where you live or stay MOST OF THE TIME?” The Bureau’s planned Alterna- tive Group Quarters Questionnaire takes a different approach by asking for an “any residence elsewhere,” regardless of the respondent’s answer to the question about living or staying at the group quarters facility most of the time. The experimental question asks: “BESIDES THIS FACILITY, what is the full address of a place where you sometimes live or stay?” No follow- up question as to how frequently the person lives or stays at this alternate address is asked. As of its initial March 2009 filing with OMB, the Bureau planned to use this alternative group quarters questionnaire in only 60,000 cases, ad- ministering the experimental questions to whole group quarters facilities rather than mixing normal and experimental questionnaires at individual

OCR for page 181
APPENDIX B 199 sites. However, in a July 2009 updated filing, the Census Bureau indicated that the sample size had been increased to 125,000, and that this sample would be conducted in only three local census offices (selected in May 2009 “based on demographics and 2010 geography”). The National Research Council (2006) recommended that an “any res- idence elsewhere” question—similar to this experiment’s version but also including a follow-up on frequency of time spent at that address—should be asked of all group quarters residents in 2010 and of a large test sample of non-group-quarters, household respondents. Although this experiment goes a small part toward that recommendation, what is unknown at this time is whether the Bureau will make use of the alternative address informa- tion gathered on the standard group quarters form. In 2000, in principle, forms from non-UHE-eligible group quarters types should have been han- dled separately from those for which UHE was permissible. However, in practice, this prefiltering was not done, and all 2.9 million group quarters forms (including captured other-address responses) went through an initial geocoding operation when only 659,000 should have been eligible. Only when this extra work was done—and the resulting slow-down in other pro- cessing noticed—were the UHE-eligible and non-UHE-eligible groups sep- arated, with the other-address information for the non-UHE-eligible group quarters types being discarded. The National Research Council (2006:230) described this situation as a “highly regrettable lost research opportunity”— throwing out what could have been a “trove of information on the nature of potential census duplicates.” B–2.d Interactive Voice Response Customer Satisfaction Survey The Census Bureau plans to use interactive voice response (IVR) tech- nology in its Telephone Questionnaire Assistance (TQA) program in 2010. Respondents seeking clarification or information (e.g., requesting a foreign- language questionnaire) can call a number in the census mailing package, and an automated computer system will administer help based on spoken word commands from the respondent. As one of the formal CPEX evalu- ations, the Census Bureau plans to conduct a Customer Satisfaction Survey with an approximate 1 percent sample of persons calling the TQA lines to assess satisfaction with the IVR interface. The current plan is for this survey to be done at the end of a TQA call; the IVR system will say: If that’s all the information you needed, please hold for our Customer Satisfaction Survey. Otherwise, to hear the topic information again say “repeat that” or for help on another general question say “Census in- formation.” About two-thirds of the total estimated sample size (665,000 callers) will then be administered a five-item satisfaction survey via IVR, while the other

OCR for page 181
200 ENVISIONING THE 2020 CENSUS third will be transferred to a customer service representative to receive a seven-question survey. Perhaps recognizing that customers generally unsat- isfied with speaking to the IVR system will not be inclined to go through a follow-up survey on IVR (particularly one administered by IVR)—and pos- sibly accounting for people whose inquiries lead them to be diverted out of the IVR to a human operator—the Census Bureau currently projects a 7.6 percent response rate and an estimated 5,016 total respondents. B–3 ASSESSMENTS Box B-3 lists what is currently known about the content of the assess- ments portion of the CPEX program, in which “assessment” refers to opera- tional histories and descriptions akin to what were generally labeled “evalu- ations” in 2000. Aside from these major headings, we have no more detailed information about research plans for specific assessments.

OCR for page 181
APPENDIX B 201 Box B-3 Assessments in the 2010 CPEX Assessment Studies, Grouped by Name of Designated Staff Teams • Content and Forms Design Integrated Product Team—Content and forms design; item nonresponse and imputation rates • Field Infrastructure–Administrative Operational Integration Team—Field office ad- ministration and payroll; recruiting and hiring field staff; regional census cen- ter/local census office leasing, space, equipment, and information technology; Census Hiring and Employment Check (CHEC) system; Decennial Applicant, Per- sonnel, and Payroll System (DAPPS) • Field Infrastructure–Field Activities Operational Integration Team—Fingerprinting; kits • Language Program Integrated Product Team—Language fulfillment program; bilin- gual questionnaire • Integrated Communication Operational Integration Team—Integrated Communi- cations Program • Geographic Programs Operational Integration Team—Type of enumeration area delineation; Local Update of Census Addresses (LUCA) participation, review, and processing; LUCA feedback, appeals, and processing; service-based enumera- tion, group homes, and carnival locations address list update; new construction • Address List Development Operational Integration Team—Non-ID processing; field verification; update/leave; address canvassing; group quarters validation • Universe Control and Management System Integrated System Team—Universe control and management system • Forms Printing and Distribution Integrated Product Team—Forms and printing; re- moving undeliverable as addressed (UAAs) from the replacement questionnaire workload/postal tracking for optimal replacement questionnaire cutoff; mail re- sponse and return rates • Housing Unit Enumeration Operational Integration Team—Enumeration of tran- sitory locations; update/enumerate; remote Alaska; nonresponse follow-up and vacant/delete check (includes removing late mail returns); Coverage Follow-Up; Remote Update/Enumerate; Be Counted/Questionnaire Assistance Centers (co- sponsored with Integrated Communication Team) • Group Quarters Enumeration Operational Integration Team—Military enumer- ation; group quarters enumeration, including advance visit; service-based enumeration • Island Areas Operational Integration Team—Island Areas • Telephone Questionnaire Assistance and Fulfillment Operational Integration Team—Telephone Questionnaire Assistance program • Decennial Response Integration System—Data capture and integration • Response Processing System Integrated System Team—Response processing system; Census Unedited File; Census Edited File • Count Review Integrated Product Team—Count review • Archiving Integrated Product Team—Archiving • Census Coverage Measurement Operational Integration Team—Initial housing unit (listing, matching, and follow-up); person interview; final housing unit; sample design; person matching and follow-up • Demographic Analysis Research Operational Integration Team—Demographic analysis • Cost and Progress—Cost and Progress System • 2010 Planning and Coordination Staff —Program management processes (change control, schedule, risks, issues, and requirements) SOURCE: List of assessments shared by U.S. Census Bureau with the panel, May 15, 2009.

OCR for page 181