|
||||||||||||||||
|
||||||||||||||||
The following HTML text is provided to enhance online readability. Many aspects of typography translate only awkwardly to HTML. Please use the page image as the authoritative form to ensure accuracy. –A–
|
||||||||||||||||
 |
 |
Page 115 |
 |
 |
 |
|
| ||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 115
Envisioning the 2020 Census
–A–
Past Census Research Programs
The descriptions of census research in this section generally exclude operations and analyses directly related to census coverage measurement—matching of the results of an independent postenumeration survey to census records in order to estimate undercount and overcount. We do, however, try to describe some of the formative work along these lines in the 1950 and 1960 censuses, given the novelty of the approach in those counts. Furthermore, although these descriptions do describe experiment and evaluation work related to census content that was part of the long-form-sample questionnaire prior to 2010, we exclude some work related to areas that are now fully out of scope of the decennial census (in particular, the census of agriculture).
A–1
1950 CENSUS
A–1.a
Principal Pretests and Experiments Conducted Prior to the Census
Supplements to the Current Population Survey (CPS)
March 1946 (CPS areas, nationwide): test of collection of both current residence (where the interviewer found a respondent at time of interview) and usual residence (the CPS and decennial census standard) information; provided particular information on enumeration of non-residents staying with households and college students.
April 1948 (CPS areas, nationwide): tests of method of obtaining income data and of enumeration of people by both usual and current
OCR for page 116
Envisioning the 2020 Census
residence. The test resulted in determination of pattern for asking income questions in 1950 as well as decisions (confirming residence rules) on enumeration of nonresidents and college students.
May 1948 (CPS areas, nationwide): test of questions on physical characteristics of dwellings; led to revised definition of “dwelling unit.”
Experiments Conducted in Special Censuses or Other Surveys
April 1946 (Wilmington, NC): experiment conducted as part of special census, focusing on collection of both current and usual residence information. Changes to enumerator training and questions on general population characteristics were also tested. The test resulted in first draft of population questions for 1950.
February 1948 (Washington, DC): experiment conducted as part of survey conducted for National Park and Planning Commission, Bureau of Labor Statistics, and Housing and Home Finance Agency. The experiment focused on questions on income and led to revision in the format of the schedule, specific questions, and instructions.
May 1948 (Little Rock and North Little Rock, AR): experiment conducted as part of special census on self-enumeration techniques; test resulted in information on possible response rates and comparative costs.
June 1948 (Philadelphia, PA): experiment attached to survey conducted for Interdepartmental Subcommittee on Housing Adequacy on methods of measuring housing quality; test resulted in revised definition of “dilapidation.”
March 1949 (Chicago, IL, and adjacent counties): experiment conducted as part of Chicago Community Survey on rostering and obtaining complete enumeration of persons in households. Significantly, the experiment tested the use of a household questionnaire rather than either the master ledger-size schedule then used for census interviewing or individual person questionnaires.
June 1949 (Baltimore, MD): test to check the quality of reported housing data conducted as part of survey conducted for Baltimore Housing Authority; test led to some revision of housing questions.
Tests of Specific Phases or Operations of 1950 Censuses
May 1947 (Altoona, PA; Charlotte, NC; Cincinnati, OH; Louisville, KY): test on “document sensing,” or a format for the schedule to enable cards to be punched automatically; test indicated that technique was possible.
OCR for page 117
Envisioning the 2020 Census
January 1948 (6 southern counties): test of special Landlord-Tenant Operations Questionnaire led to revision of procedures.
August 1949 (33 field offices): test of alternative population and housing schedules provided input to determination of final schedule (questionnaire) for 1950.
September 1949 (Puerto Rico): test of population, housing, and agriculture questions for enumeration in Puerto Rico.
October 1949 (Raleigh and Roxboro, NC): test of training procedures led to determination of final training plan.
November 1949 (Raleigh, NC): test of questions on separate Survey of Residential Financing to be conducted simultaneously with the 1950 census.
January 1950 (Chicago, IL): test of Survey of Residential Financing questions helped determine final procedures.
Dress Rehearsals
April–May 1948 (Cape Girardeau and Perry Counties, MO): test intended to (1) compare quality of data obtained from a schedule pared down to very few questions to one with many questions; (2) collect both current and usual residence information; and (3) generally assess quality of data from new questions. The pretest led to the conclusion that the short schedule yielded no substantial gains in quality over the longer, more-questions instrument. It also helped refine residence rules for some census types and specification of the duties of enumeration crew leaders.
October 1948 (Oldham and Carroll Counties, KY; Putnam and Union Counties, IL; Minneapolis, MN): test focused on different enumeration procedures (self-enumeration, distribution of materials by post office, etc.) and their effects on data quality. Based on the test, the Census Bureau decided to use self-enumeration in the Census of Agriculture. The test also yielded cost, time, and quality data for the different approaches.
May 1949 (Anderson City, School District 17, and Edgefield County, SC; Atlanta, GA; rural areas near each of 64 CPS field offices): test of training methods and final questionnaires and training procedures led to some modification in procedures (including procedures for shipping supplies to local offices) and determination of procedures for the postenumeration survey.
SOURCE: Adapted from U.S. Census Bureau (1955:Table B, p. 6), with additional information from Appendix E of that document.
OCR for page 118
Envisioning the 2020 Census
A–1.b
Research, Experimentation, and Evaluation Program
Although the 1950 census was the first to include a structured experimental and evaluation program (Goldfield and Pemberton, 2000a), details of the precise evaluations conducted in 1950 are not generally available. Only slightly more information about the shape of specific experiments is detailed in the Census Bureau’s procedural history for the 1950 census. In a short section titled “Experimental Areas,” that history notes (U.S. Census Bureau, 1955:5):
A number of variations in the procedures for collecting data were introduced in ten District Offices. These variations made possible a comparison of procedures under actual census conditions. The experimental areas were located in Ohio and Michigan. In six of these districts, the alternative procedures involved the use of a household schedule (instead of a line schedule for a number of households), of the household as a sampling unit (instead of the person), and of self-enumeration (instead of direct enumeration). In four of the districts, assignments were made to enumerators in such manner that the variation in response could be studied in terms of enumerator differences.
Remarking on the self-enumeration portion of the 1950 experiments, the procedural history for the 1960 census clarifies that the test areas were in Columbus, OH, and Lansing, MI, and that part of the experiment requested that individual respondents complete and mail back (on Census Day) the questionnaires left with them by field enumerators (U.S. Census Bureau, 1966:292). The impact of the 1950 experiment on the later adoption of mailout-mailback methodology in the census is discussed by Bailar (2000).
A–2
1960 CENSUS
A–2.a
Principal Pretests and Experiments Conducted Prior to the Census
Supplements to the Current Population Survey (CPS)
November 1958: test of definitions for unit of enumeration.
April 1959: test of farm definitions.
Experiments Conducted in Special Censuses or Other Surveys
March–April 1957 (Yonkers, NY): major experiment conducted as part of special census. The experiment had two principal objectives: (1) obtaining data on the cost of a two-visit interview process (and effects on data quality) and (2) testing direct entry of interview information on machine-readable forms, either by enumerators or household respondents. In particular, a two-interview approach was tested in which—for roughly every fourth household—a limited amount of informa-
OCR for page 119
Envisioning the 2020 Census
tion was collected in a first interview and a long-form questionnaire was left for completion (and eventual pickup by an enumerator). The test demonstrated that enumerators could readily use the computer-readable schedules in the field but that respondents found them more difficult to follow. The two-visit approaches were found to be very expensive because of the difficulty of finding household members at home when enumerators attempted their visits. However, the idea of independent listing of dwelling units by crew leaders as well as enumerators was found to have potential value for improving coverage. The computer-readable forms generated by the test also helped with debugging the collection and tabulation routines. The Yonkers test also gave the Census Bureau the chance to test a new question on address of place of work, but the Bureau found the results to be unsatisfactory.
October 1957 (Indianapolis, IN): focused test of several possible coverage improvement techniques in one postal zone as part of a special census. Building on the Yonkers test, one approach compared results of a recheck of listed addresses by crew leaders with enumerators’ original results. Another technique involved preparing postcards based on enumerator interviews, with the postal carriers instructed to report boxes for which there were no cards for further verification (deliberately withholding a small sample of cards as a test of the carriers). Finally, the Bureau tested the completeness of lists of old-age assistance beneficiaries, juvenile delinquents, and other persons believed to live in low-income families obtained from local authorities, as well as the distribution of census forms in public schools for parents to complete. The tests provided evidence of coverage gains from the post office check and independent crew leader listings and suggested that the locally provided special lists were useful in indicating missed persons.
October 1957 (Philadelphia, PA): census experiment focused on wording and placement of questions on labor force status, address of place of work, and date of marriage. Three different schedules were used in the experiment, which deliberately sent inexperienced interviewers to do initial questioning and trained CPS interviewers for verification interviews. One tested approach for the place of work question asked respondents to identify the location on a map of the city. The test resulted in revisions of the occupation and industry questions in the final census schedule.
November 1957 (Hartford City, IN): test of mailout census methodology as part of a special census. A four-page Advance Census Report was mailed to individual household addresses; respondents were asked to complete the form but not to mail it back, holding it for an enumerator’s visit instead. In the test, about 40 percent of households had completed part or all of the questionnaire prior to the visit, and the
OCR for page 120
Envisioning the 2020 Census
advance questionnaire seemed to be particularly useful in improving the quality of data on value of home.
January 1958 (Memphis, TN): test of self-enumeration and mailback methodology as part of special census. Housing units were listed by enumerators and questionnaires distributed, with instructions to complete and mail on the census date. Mailed returns were chosen for follow-up verification, some by telephone and others by trained CPS interviewers; the post office check used in the Indianapolis test was also used for portions of Memphis. Part of the Memphis test also experimented with collection of information on visitors present in households on the census date and using reported usual residence information to try to allocate them to their usual home. Based on this test, a process for collecting “usual home elsewhere” information for some transient populations (enumerated where they are found on Census Day) was used in the 1960 census and in group quarters enumeration in subsequent censuses. Finally, the experiment asked some enumerators to query respondents for some information on neighboring living quarters (above or below, right or left, in back or in front); the technique was subsequently adopted as part of the evaluation process of the 1960 census.
February 1958 (Lynchburg, VA): further testing of mailout of Advance Census Reports as part of special census. As in the Memphis test, CPS interviewers and crew leaders were used to reinterview some households to verify the quality of mailback data. Attempts were also made to time interviews to determine whether the Advance Census Report reduced the time spent in enumerator interviewing. In May 1958 enumerators returned to Lynchburg to recheck and verify the quality of housing data collected in February.
March 1958 (Dallas, TX): census experiment on alternative questionnaire forms. Two small samples of households were interviewed using different schedules, specifically trying different methods for collecting information on income and place of work, on housing equipment items (e.g., type of heating and presence of air conditioning), and a 5-year versus 1-year migration question.
April, June 1958 (Ithaca, NY): test conducted as part of special census (with staff follow-up) on enumerators’ ability to classify types of living quarters.
October 1958 (Martinsburg, WV): test intended to determine which of three alternative population and housing questionnaires could be used most effectively in the field. The content of the questionnaires was identical, and essentially identical to the questions that would be used in the 1960 census, but the alternative schedules varied the size and structure of the schedules. The Martinsburg test also delivered
OCR for page 121
Envisioning the 2020 Census
Advance Census Reports prior to enumerator visits. Because the test was sufficiently close to the actual census, the test also permitted the Bureau to evaluate the training materials planned for use in 1960 as well as editing and coding routines.
December 1958 (Philadelphia, PA): limited pretest to compare two possible forms with different skip patterns, after a final decision to use some self-enumeration in the 1960 census.
Mid-1959 (800 households in 10 regional headquarters cities): small-scale test following the full-dress exercise in North Carolina (described below), testing variants of some questions to determine whether adding check boxes to routing questions (e.g., “If you do not live in a trailer, check here and continue with the next question.”) promoted fuller response. Questionnaires were also distributed with return envelopes for mailback.
“Informal” Experiments and Pretests for the Census of Housing
June 1957 (Washington, DC): test of procedure for listing structures.
October 1957 (12 standard metropolitan areas): test of collection of data on condition of housing unit in selected 1956 National Housing Inventory segments.
March 1958 (New York, NY): test of classification of living quarters; used to formalize 1960 census definition of housing unit based on separate entrance or separate cooking equipment.
March, May 1958 (Prince George’s County, MD): tests of questions on exterior materials of housing and on basement shelters.
June 1958 (Lynchburg, VA): recheck of data on condition and housing unit from May 1958 pretest.
September 1958 (Port Chester, NY): recheck of data on classification of living quarters.
Dress Rehearsal
February–March 1959 (Catawba and Rutherford Counties, NC): final full-scale test prior to the census, making use of the Advance Census Report delivery that would be used in the 1960 count. Households chosen for the long-form sample were asked to complete their questionnaires and mail them to their local census office.
SOURCE: Adapted from U.S. Census Bureau (1966:App. B), with supplemental information from Part III, Chapters 1 and 2 of that document.
OCR for page 122
Envisioning the 2020 Census
A–2.b
Research, Experimentation, and Evaluation Program
A long list of possible research studies—based, in part, on input from the Panel of Statistical Consultants (an advisory group to the Census Bureau’s assistant director for research and development)—for the 1960 census was refined to a final list of 22 studies. The Bureau divided these 22 studies into 8 “projects.”
Measurement of response variability
An expanded version of a study performed in 1950 to study variability in response due to enumerators and census staff; the 1960 version of the “Response Variance Study I” drew sample from all areas where questionnaires were mailed (with enumerators sent to pick them up or conduct interviews), rather than four selected areas as in 1950. The sample for which crew leaders and enumerators were assigned in order to estimate these effects included about 320,000 housing units and 1,000,000 persons.
A follow-up study, called “Response Variance Study II,” designed to measure variability in response due to the respondents themselves. A sample of 5,000 households from Response Variance Study I was drawn and enumerators sent to conduct interviews; a second sample of 1,000 housing units were asked to report again using a mailed, self-response questionnaire (with mailback, and interviewer follow-up if necessary).
“Response Variance Study III” looked at variability due to coding and data entry. For a one-fourth sample of households for which data was collected in Response Variance Study I, photocopies of the enumeration books were made; pairs of coders then independently coded and transcribed the data.
Reverse record checks (undercoverage in general population): a check to see whether persons in an independent sample were enumerated in the 1960 census, in which the independent sample was culled from a mix of past census records and administrative data. Specifically, the sample was constructed from samples from four sources: (1) persons enumerated in the 1950 census; (2) immigrants and aliens registered in January 1960 with the Immigration and Naturalization Service; (3) birth records for children born between the 1950 and 1960 censuses; and (4) persons who were found in the 1950 postenumeration survey but not in the 1950 census.
Reverse record checks II (undercoverage in specific groups): a check similar to Project B, except that the records-based sample consisted of Social Security beneficiaries and students enrolled at colleges or universities.
OCR for page 123
Envisioning the 2020 Census
Reenumerative studies of coverage error
Postenumeration survey based on an area sample: a sample of “segments” that had been independently canvassed by enumerators for the Survey of Components of Change and Residential Finance (SCARF), conducted alongside the census. Provided with both 1960 census and SCARF information, enumerators in 2,500 area segments were tasked to search for omitted housing units or structures mistakenly labeled as housing units. About 10,000 housing units were administered a detailed housing questionnaire.
Postenumeration survey based on a list sample: About 15,000 living quarters (both housing units and group quarters) already enumerated in the censuses was drawn, representing about 5,000 clusters of about 3 units each, dispersed across 2,400 enumeration districts. For each unit, a detailed reinterview was conducted in order to list persons within living quarters (this follow-up interview was conducted in early May 1960, so that not much time had passed since the April 1 Census Day). Enumerators were also asked to list “predecessor” and “successor” housing units or group quarters (e.g., neighboring units along a specified path of travel) in order to further check on missed housing units.
Measurement of content error in data collection
Reinterview of about 5,000 households in the long-form sample for the 1960 census, using specially trained enumerators: a first phase (about 1,500 households in July 1960) sent the enumerators to conduct blind reinterviews; in the second phase (October 1960), dependent interviewing using the household’s reported 1960 census information was conducted for half of the remaining sample and for the other half blind interviewing, as in the first phase.
Reenumerative study focusing on housing characteristics, administering a battery of housing characteristics: about half of the 10,000 housing units in the study received the short form in the 1960 census, and the others were in the long-form sample (and hence had already reported some of the housing items).
Match of records from the 1960 census long-form sample and the Current Population Survey’s March or April 1960 samples.
Match of respondent-provided occupation and industry information with data collected directly from employers.
Match of about 10,000 sampled Internal Revenue Service returns to census records (although only about one-fourth of these were studied in depth regarding the consistency of reported income).
OCR for page 124
Envisioning the 2020 Census
Studies of processing error
A September 1963 detailed review of a sample of individually filled Advance Census Reports, Household Questionnaires (enumerator schedules), and the coded computer-readable forms in order to estimate transcription and other response errors.
Coding error study in which a set of long-form questionnaires were separately coded by three coding clerks but only one was the designated “census coder” whose work went onto the final questionnaire.
Study of the accuracy of automated editing rules in the microfilm–computer system for data collection and tabulation.
Analytical studies: general studies of census quality and coverage, including comparison of census counts with demographic analysis estimates.
Post Office coverage improvement study: postcensus postal check making use of postal employees and resources. A sample area of 10,000–15,000 housing units was selected in each of the 15 postal regions in the continental United States. These areas were matched to census enumeration districts. Postal carriers were asked to review name and address cards completed by enumerators for every counted household, making new cards for households on their routes that were not included in the census.
SOURCE: Adapted from U.S. Census Bureau (1963).
A–3
1970 CENSUS
A–3.a
Principal Pretests and Experiments Conducted Prior to the Census
Tests Conducted as Part of Special Census or Other Survey
August 1961 (Fort Smith, AR): test conducted as part of special census to compare an address register compiled through enumerator visits with a register based on 1960 census records, new building permits, and a postal check. The separate listings were “found to be just as complete.” Enumerator-visited households were also left with a brief questionnaire and asked to return the form by mail.
Pretests of Census Operations and Questionnaires
June 1962 (Fort Smith, AR, and Skokie, IL): tests of address list updating using building permits and postal checks (updating the Fort Smith list from the August 1961 special census and repeating the methodology in Skokie). Short questionnaires were mailed to all households for response by mail, yielding 71–72 percent return rates.
OCR for page 125
Envisioning the 2020 Census
April 1963 (Huntington, NY): further testing of address listing and mailout-mailback methods, now focused on a larger city—deemed to be a rapidly growing area and including a mix of urban, suburban, and rural housing types. The test also included administering a long-form questionnaire to approximately 25 percent of households. In nonmail delivery areas, address lists were built through enumerator canvass and questionnaires deposited at the households for mailback.
May–June 1964 (Louisville, KY, Standard Metropolitan Statistical Area [SMSA]): based on the Fort Smith, Skokie, and Huntington experiences, the Census Bureau requested and received funds for larger-scale testing of mailout-mailback methods (experimental censuses) in 1964 and 1965. The first of these was to be conducted in an area of about 750,000 population with a large central city; the Louisville area was selected (with the test being conducted from a special census office in Louisville, rather than the Bureau’s nearby processing center in Jeffersonville, IN). The Louisville test added separate listing and contact strategies for “special places” (mainly group quarters), based on input from a task force on difficult-to-enumerate areas that had been convened in Louisville in late 1963. In this test, the Census Bureau concluded that a computer-generated address register based on previous census records and building permits was more complete than listing books completed by enumerators. In particular, “using a list based on enumerator canvass did not seem the ideal way to insure complete coverage in multiunit structures in city delivery areas.” The Bureau further concluded “that at the current state of development of procedures the cost of a mail census, including certain important improvements over the 1960 approach, would not exceed the cost of a census by enumerator canvass.”
April 1965 (Cleveland, OH): Cleveland was chosen as the site for the second experimental mailout-mailback census both for its big-city nature and for its perceived enumeration problems in the 1960 census. For this test, the Census Bureau experimented with using a commercial mailing list as the base for the address register; the test was also the first to be geocoded by computer based on an address coding guide (i.e., coded information on address ranges on odd and even sides of street segments). As in Louisville, both short- and long-form questionnaires were distributed. The Cleveland test also centralized editing procedures, assigning completeness checks of returned questionnaires to district office clerks rather than individual enumerators. However, the Bureau’s attempts to predesignate hard-to-enumerate areas were deemed to be less successful than was the case in Louisville, and probe questions added to the questionnaire on other households at the same street address were generally found to be unclear and confusing. Fur-
OCR for page 170
Envisioning the 2020 Census
determining final housing unit inventory, done by merging information on the processed Decennial Response File (census returns) with the Decennial MAF.
Beta Site: operational analysis of software evaluation facility within the Census Bureau that is responsible for integrating software systems of the census as well as to conduct security testing.
Cancelled: Invalid Return Detection.
Quality Assurance Evaluations (Series M)
Evaluation of Quality Assurance Philosophy and Approach Used for the Address List Development and Enumeration Operations: document operational experiences with quality assurance approach.
Effectiveness of Existing Variables in the Model Used to Detect Discrepancies During Reinterview, and the Identification of New Variables: document specific quality assurance measure used in cases in which enumerators’ work suggested discrepancies.
A.C.E. Survey Operations (Series N)
Contamination of Census Data Collected in A.C.E. Blocks: study intended to determine success in keeping census and A.C.E. operations independent.
Discrepant Results in A.C.E.: examine how quality assurance steps identified interviewers who entered discrepant data in the A.C.E. interview.
Evaluation of Matching Error: assess error in the matching process used to identify missed or erroneously enumerated persons between the census and the A.C.E.
Targeted Extended Search Block Cluster Analysis: study result of expending search for possible matches from adjoining blocks for all sample clusters (as in 1990) to a broader search (still targeted to clusters judged most likely to benefit from additional searching.
Field Operations and Instruments for A.C.E.: assess quality of housing unit and person coverage in A.C.E. operations by examining the quality of the (independent) address listing, effect of follow-up interviewing, and noninterview rates.
Cancelled (16, most converted to separate A.C.E. evaluation program): Analysis of Listing Future Construction and Multi-Units in Special Places; Analysis of Relisted Blocks; Analysis of Blocks With No Housing Unit Matching; Analysis of Blocks Sent Directly for Housing Unit Follow-Up; Analysis of Person Interview With Unresolved Housing Unit Status; Analysis on the Effects of Census Questionnaire Data Capture in A.C.E.; Analysis of
OCR for page 171
Envisioning the 2020 Census
the Census Residence Questions Used in A.C.E.; Analysis of the Person Interview Process; Extended Roster Analysis; Matching Stages Analysis; Analysis of Unresolved Codes in Person Matching; Outlier Analysis in the A.C.E.; Impact of Targeted Extended Search; Effect of Late Census Data on Final Estimates; Group Quarters Analysis; Analysis of Mobile Homes.
Coverage Evaluations of the Census and of the A.C.E. Survey (Series O)
Housing Unit Coverage Study: examine net coverage rate, gross omission rate, and erroneous enumeration rate of housing units, at various geographic levels as well as by A.C.E. poststrata.
Analysis of Conflicting Households: report on resolution of cases in A.C.E. housing unit matching when the census and the A.C.E. listed two entirely different families for the same unit.
Analysis of Proxy Data in the A.C.E.: study accuracy (match rates and erroneous enumeration rates) of data collected from proxy respondents: persons who are not members of the household, such as neighbors or landlords.
Housing Unit Duplication in Census 2000: study characteristics of duplicate housing units, attempting to identify census operations most likely to produce housing unit duplication.
Analysis of Deleted and Added Housing Units in the Census Measured by the A.C.E.: evaluate housing unit coverage on the early Decennial MAF.
Consistency of Census Estimates with Demographic Benchmarks: comparison of census results with demographic analysis benchmarks.
Cancelled (20, most converted to separate A.C.E. evaluation program or combined with other evaluations): Type of Enumeration Area Summary; Coverage of Housing Units in the Early Decennial MAF; P-Sample Nonmatches Analysis; Person Coverage in Puerto Rico; Housing Unit Coverage in Puerto Rico; Geocoding Error Analysis; E-Sample Erroneous Enumeration Analysis; Analysis of Nonmatches and Erroneous Enumerations Using Logistic Regression; Analysis of Various Household Types and Long-Form Variables; Measurement Error Reinterview Analysis; Impact of Housing Unit Coverage on Person Coverage Analysis; Person Duplication; Analysis of Households Removed Because Everyone in the Household Is Under 16 Years of Age; Synthesis of What We Know About Missed Census People; Implications of Net Census Undercount on Demographic Measures and Program Uses; Evaluation of Housing Units Coded as Erroneous Enumerations; Analysis of Insufficient Information for Matching and Follow-Up;
OCR for page 172
Envisioning the 2020 Census
Evaluation of Lack of Balance and Geographic Errors Affecting Person Estimates; Mover Analysis; Analysis of Balancing in the Targeted Extended Search.
A.C.E. Survey Statistical Design and Estimation (Series P)
Cancelled (5, most converted to separate A.C.E. evaluation program): Measurement of Bias and Uncertainty Associated with Application of the Missing Data Procedures; Synthetic Design Research/Correlation Bias; Variance of Dual System Estimates and Adjustment Factors; Overall Measures of A.C.E. Quality; Total Error Analysis.
Organization, Budget, and Management Information System (Series Q)
Management Processes and Systems: study the Census Bureau’s organizational structure and decision-making processes, as well as its interaction with the Census Monitoring Board, Congress, the General Accounting Office, and other outside interests.
Automation of Census Processes (Series R); reviews of technical systems conducted by Titan Corporation
Telephone Questionnaire Assistance: toll-free service provided by a commercial phone center to answer questions about the census and the questionnaire.
Coverage Edit Follow-Up: program to resolve count discrepancies and obtain missing data for large households.
Internet Questionnaire Assistance: system that allowed respondents to use the Census Bureau’s Internet site to ask questions and receive answers about the census questionnaire or other census-related information.
Internet Data Collection: system that offered census short-form respondents the opportunity to respond via the Internet, using the 22-digit ID number found on their mailed census form.
Operations Control System 2000: system for control, tracking, and progress reporting for all field operations conducted for the census.
Laptop Computers for A.C.E.: systems used in A.C.E. follow-up interviewing.
A.C.E. Control System: tracking and control system behind the A.C.E. field operations.
Matching and Review Coding System for A.C.E.: system used to match A.C.E. returns to census records.
Pre-Appointment Management System/Automated Decennial Administrative Management System: system used for administrative management in the census, including tracking and processing of temporary enumerators, payroll, and background checks.
OCR for page 173
Envisioning the 2020 Census
American FactFinder: systems developed to provide access to census data via the Internet at http://factfinder.census.gov.
Management Information System 2000: systems used to manage census operations, including tracking of dates and budgets and creation of progress reports of current status during census operations.
Data Capture: systems developed for full electronic data capture and imaging of census questionnaires, using optical mark and optical character recognition.
SOURCE: Adapted from National Research Council (2004a:App. I).
A–7
PRINCIPAL PRETESTS AND EXPERIMENTS CONDUCTED PRIOR TO THE 2010 CENSUS
A–7.a
Pretests of Census Operations and Questionnaires
2002 (Gloucester County, VA): pilot testing of use of Census TIGER maps on a handheld computer (Pocket PC-class) by census interviewers. The tests involved only locating particular address features on the small-screen map, not using the computer map to navigate a route or the collection of GPS coordinates.
February–April 2003 (national sample): the 2003 National Census Test was administered to a national sample of about 250,000 households, drawn from the set of households that was enumerated using mailout-mailback methodology in the 2000 census. Strictly mail-based, the 2003 test involved no field follow-up component. The test focused primarily on two issues:
Response mode and contact strategies: different experimental groups were offered the opportunity to reply by mail (traditional method), Internet, or interactive voice response (IVR, an automated telephone system). Groups also varied as to whether these response modes were offered as a choice or whether they were “pushed” (e.g., providing Internet directions but no actual paper questionnaire in the mailing). Finally, contact strategies (including targeted replacement questionnaires and reminder postcards) were also varied. This component of the test involved eight experimental groups, one with 20,000 households and the other seven with 10,000 households each.
Race and ethnicity (Hispanic-origin) question wording: seven treatment groups of 20,000 households each received different variations on the wording and arrangement of questions on race and Hispanic origin. Experimental settings included whether “some other race” was offered as a choice in the categories for
OCR for page 174
Envisioning the 2020 Census
race, whether wording was slightly revised to ask respondents if they are “Hispanic” or if they are “of Hispanic origin,” and whether instructions explicitly directed respondents to answer both questions.
The test was rounded out by a control group of 20,000 households; this group’s questionnaire included the race and Hispanic-origin questions worded as they were in the 2000 census (unlike the 2000 census context, the 2003 test control group households were eligible for a replacement questionnaire in lieu of nonresponse follow-up).
The samples for all groups were stratified by response rate in the 2000 census, in which the classification was a grouping into “high” and “low” response groups based on a selected cut-off. Martin et al. (2003:11) comment that the low-response strata “included areas with high proportions of Blacks and Hispanics and renter-occupied housing units” and further comment that addresses in low-response areas were oversampled. Still, it is unclear whether the sample design generated enough coverage in Hispanic communities to facilitate conclusive comparisons—that is, whether it reached enough of a cross-section of the populace and a sufficiently heterogeneous mix of Hispanic nationalities and origins to gauge sensitivity to very slight and subtle changes in question wording.
With regard to the response mode and contact strategy portion of the test, results reported by Treat et al. (2003) suggest that multiple response mode options may change the distribution of responses by mode—shifting some would-be mail responses to Internet, for example. However, the addition of choices does not generally increase cooperation overall. The experience of the 2003 test suggests serious difficulties with the interactive voice response option; 17–22 percent of IVR attempts had to be transferred to a human agent when the system detected that the respondent was having difficulty progressing through the IVR questionnaire. Moreover, rates of item nonresponse were greater for IVR returns than for the (paper response) control group. Internet returns, by comparison, experienced higher item response rates than the control. As indicated in past research, reminder postcards and replacement questionnaires had a positive effect on response.
Martin et al. (2003) report that the race and Hispanic-origin question segment of the test showed mixed results. Predictably, elimination of “some other race” as a response category reduced “some other race” responses considerably, by 17.6 percent (i.e., Hispanic respondents apparently declined to write in a generic response like “Hispanic” or “other” if “some other race” was not a formal choice). The Bu-
OCR for page 175
Envisioning the 2020 Census
reau concluded that the 17.6 percent decline in generic race reporting “more than offset” the impact of a 6.4 percent increase in the estimated number of Hispanics declining to answer the race question altogether (Martin et al., 2003:15). Adding examples of ancestry groups (e.g., Salvadoran, Mexican, Japanese, Korean) boosted the reporting of detailed origins among Hispanics, Asians, and Native Hawaiian or Other Pacific Islanders. Treatment groups for which instructions were revised, instructing respondents to answer both the race and Hispanic-origin questions, produced the most puzzling results; levels of missing data on one or both questions increased, as did the percentage reporting themselves as Native Hawaiian or Other Pacific Islanders (relative to the control group).
February–July 2004 (7 neighborhoods of Queens County, NY [Astoria, Corona, Elmhurst, East Elmhurst, Jackson Heights, Long Island City, and part of Woodside]; Colquitt, Thomas, and Tift Counties, GA): the 2004 test was intended to test a package of new procedures and technologies in both a high-density urban site and a rural site. Lake County, IL, was originally designated a test site but was dropped due to constraints in funding for fiscal year 2004. Intended to include approximately 200,000 housing units, the test centered around a few major topics:
Handheld devices: the test marked the Census Bureau’s first attempt to use handheld computers, equipped with GPS receivers, for nonresponse follow-up interviewing. The handhelds developed for use in the 2004 test were pieced together from commercial, off-the-shelf components; although the test included some advanced workflows (e.g., transmitting questionnaire data directly from enumerators’ devices to headquarters and pushing enumerator assignments directly to individual handhelds, without filtering through regional or local offices), some parts of case management and assignment were still done using paper reports.
Further testing of race and Hispanic-origin question wording: the 2004 test permitted continuation of work from the 2003 test.
Special place/group quarters definition: the 2004 test field exercises allowed Bureau staff to test the aptness of revised definitions of group quarters (nonhousehold) populations.
In addition to dropping the Illinois test site, other planned parts of the 2004 test were eliminated from the test plan, including an attempt to target a mailing of dual-language (English and Spanish) questionnaires to certain households and to target canvassing methods for updating the MAF.
OCR for page 176
Envisioning the 2020 Census
The 2004 field test also provided the first chance to test a version of the Bureau’s planned Coverage Follow-Up (CFU) operation, a combination of coverage improvement programs from previous censuses and computerized matching to detect potential census duplicates. Given the relatively small size of the test, the Bureau was able to perform follow-up on all eligible cases and to conduct a post hoc clerical review of cases to specify the likely source of duplication. Follow-up interviews were conducted both by telephone (experienced interviewers) and field staff (temporary, relatively novice interviewers).
July 2004 (France, Kuwait, and Mexico): Overseas Enumeration Test meant to assess the feasibility of a count of all Americans living overseas, motivated by congressional interest. Interest in the issue had been heightened by one of Utah’s legal challenges to the 2000 census (having been edged for the 435th seat in the U.S. House of Representatives by North Carolina) concerning the counting of missionaries of the Church of Jesus Christ of Latter-day Saints. To carry out the test, the Bureau adopted techniques similar to those used to elicit overseas resident counts in the 1960 and 1970 censuses: persons living overseas had to make contact with a U.S. embassy or consular office in order to obtain the census questionnaire. Some publicity about the test in the selected countries (France, Kuwait, and Mexico) was made in English-language newspapers and media. Costing approximately $8 million, the test yielded very low response: 5,390 questionnaires total, compared to 520,000 questionnaires printed for the test and rough estimates of on the order of 1.15 million American citizens in the test countries. Although the 2004 effort was originally intended to be repeated in 2006, no funding for the overseas test was included in the Bureau’s appropriations for that year.
August–September 2005 (national sample): a second, mail-only National Census Test in 2005 involved only variations in questionnaire design. The test reached a sample of about 420,000 households and was intended to simultaneously address several major objectives: (1) test revised questions on tenure, relationship, age, date of birth, race, and Hispanic origin; (2) test respondent friendliness of new designs of mail and Internet questionnaires; (3) compare revised versions of the Question 1 household count item, including residence rules instructions, and further test coverage probe questions selected from those tested in 2003; (4) test use of a replacement questionnaire; and (5) test use of a bilingual English-Spanish questionnaire. All of these objectives were intended to be covered by a set of 20 experimental treatment groups within a single-panel test; the control group included some items worded as they had been in the 2000 census, but other items used versions tested in 2003 or 2004. Although there was no field
OCR for page 177
Envisioning the 2020 Census
follow-up, a sample of respondents was reinterviewed by telephone to assess the consistency of household rostering. Logistically, the test encountered problems as it coincided with the impact of Hurricanes Katrina and Rita and the disruption or suspension of mail service in the Gulf Coast area; the test was also complicated by the decision to keep the Internet questionnaire English-only and in a single format that lacked the other experimental features, preventing Internet returns from being used in studying some treatments.
Early 2006 (national sample): ad hoc Short Form Mail Experiment with a planned sample size of about 24,000 addresses from the U.S. Postal Service’s Delivery Sequence File. Not originally part of the testing plan for the 2010 census, the Census Bureau sought authority for the test in a Federal Register notice in October 2005. As described in that notice (Federal Register, October 5, 2005, p. 58181), this special mailout test was developed based on three objectives:
“Evaluate the effects of the wording of the instruction about who to list as Person 1” on the questionnaire (the householder, with whom the reported relationships of other household members are defined);
“Evaluate the proportion of respondents who forget to enumerate themselves by asking them to provide their personal information at the end of the form” (with “personal information” described as name, phone number, and proxy status—that is, whether the respondent is completing a form for someone else); and
“Evaluate how a compressed schedule with a fixed due date impacts unit response patterns.” (The notice specified only that the “compressed schedule” would change from questionnaires being “mailed 2 weeks before ‘Census Day’ ” to “households receiv[ing] the questionnaires a few days before ‘Census Day.’ ”)
The Census Bureau divided the sample into four treatment groups (Federal Register, October 5, 2005, p. 58181):
Group 1. Housing units in this treatment group will receive questionnaires with the same wording for the Person 1 instruction that we used in the Census 2000 questionnaire. In the Final Question, respondents will be asked to provide their name, telephone number and proxy information. The mail out schedule will be the conventional schedule. The questionnaire will be mailed two weeks before “Census Day”, and there will be no explicit deadline.
Group 2. Housing units in this treatment group will receive questionnaires with the revised wording for the Person 1 in-
OCR for page 178
Envisioning the 2020 Census
struction. In the Final Question, respondents will be asked to provide their name, telephone number and proxy information. The mailout schedule will be the conventional schedule. The questionnaire will be mailed two weeks before “Census Day” and there will be no explicit deadline.
Group 3. Housing units in this treatment group will receive questionnaires with the revised wording for the Person 1 instruction. In the Final Question, respondents will be asked to check over their answers before considering the survey complete. The mailout schedule will be the conventional schedule. The questionnaire will be mailed two weeks before “Census Day” and there will be no explicit deadline.
Group 4. Housing units in this treatment group will receive questionnaires with the revised wording for the Person 1 instruction. In the Final Question, respondents will be asked to check over their answers before considering the survey complete. The mailout schedule will be compressed, so that the survey is received closer to “Census Day” and an explicit due date will be provided.
April 2006 (part of Travis County, TX, including the cities of Austin and Pflugerville; Cheyenne River Indian Reservation, SD): a second operational test involving field work, the 2006 Census Test used mailout-mailback (with field nonresponse follow-up) in the Texas site and enumerator visits (no mail) in the South Dakota site. As in 2004, handheld computers were used for enumerator interviews and were assembled from off-the-shelf components; in 2006, the new aspect of handhelds being tested was the delivery of maps to enumerators via the devices. The Census Bureau’s list of definitions of group quarters was expanded following the 2004 test, and the usefulness of this revised set was assessed in Travis County.
A–7.b
Dress Rehearsal
April 2008 (San Joaquin County, CA; nine-county area surrounding Fayetteville, NC [Chatham, Cumberland, Harnett, Hoke, Lee, Montgomery, Moore, Richmond, and Scotland Counties, including Fort Bragg and Pope Air Force Base]): as a full rehearsal of census activities, the 2008 dress rehearsal actually began operations in spring 2007 with address canvassing. This activity was the first operational trial of custom handheld devices designed by Harris Corporation and its subcontractors under the Field Data Collection Automation (FDCA) contract; problems encountered in use of the handhelds precipitated a crisis in funding and a “replan” of FDCA and core census operations in the first half of 2008.
OCR for page 179
Envisioning the 2020 Census
SOURCES: Adapted from “2010 Census: How We Prepare for 2010” (http://2010.census.gov/2010census/about_2010_census/007623.html); for the 2006 Short Form Mail Experiment, Federal Register, October 5, 2005, pp. 58180–58182; Hill et al. (2006); Karl et al. (2005); Knight et al. (2005); National Research Council (2004b:Boxes 9.1 and 9.2); Pennington (2005); Tancreto (2006).
OCR for page 180
Envisioning the 2020 Census
This page intentionally left blank.