Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page R1
Envisioning the 2020 Census
ENVISIONING THE 2020 CENSUS
Panel on the Design of the 2010 Census Program of Evaluations and Experiments
Lawrence D. Brown, Michael L. Cohen, Daniel L. Cork, and Constance F. Citro, Editors
Committee on National Statistics
Division of Behavioral and Social Sciences and Education
NATIONAL RESEARCH COUNCIL
OF THE NATIONAL ACADEMIES
THE NATIONAL ACADEMIES PRESS
Washington, DC
www.nap.edu
OCR for page R2
Envisioning the 2020 Census
THE NATIONAL ACADEMIES PRESS
500 Fifth Street, NW
Washington, DC 20001
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.
The project that is the subject of this report was supported by contract no. YA1323-06-CN-0031 between the U.S. Census Bureau and the National Academy of Sciences. Support of the work of the Committee on National Statistics is provided by a consortium of federal agencies through a grant from the National Science Foundation (No. SES-0453930). Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the organizations or agencies that provided support for the project.
International Standard Book Number 13: 978-0-309-15115-3
International Standard Book Number 10: 0-309-15115-5
Additional copies of this report are available from the
National Academies Press,
500 Fifth Street, NW, Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area); Internet, http://www.nap.edu.
Copyright 2010 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
Suggested citation: National Research Council. (2010). Envisioning the 2020 Census. Panel on the Design of the 2010 Census Program of Evaluations and Experiments. Lawrence D. Brown, Michael L. Cohen, Daniel L. Cork, and Constance F. Citro, eds. Committee on National Statistics, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.
OCR for page R3
Envisioning the 2020 Census
THE NATIONAL ACADEMIES
Advisers to the Nation on Science, Engineering, and Medicine
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council.
www.national-academies.org
OCR for page R4
Envisioning the 2020 Census
This page intentionally left blank.
OCR for page R5
Envisioning the 2020 Census
PANEL ON THE DESIGN OF THE 2010 CENSUS PROGRAM OF EVALUATIONS AND EXPERIMENTS
LAWRENCE D. BROWN (Chair),
Department of Statistics, Wharton School of Business, University of Pennsylvania
RICHARD A. BERK,
Department of Criminology, University of Pennsylvania
ERIC T. BRADLOW,
Wharton School of Business, University of Pennsylvania
IVAN P. FELLEGI,
Statistics Canada (retired), Ottawa
LINDA GAGE,
California Department of Finance, Sacramento
VIJAY NAIR,
Department of Statistics, University of Michigan
JESSE H. POORE, JR.,
Department of Electrical Engineering and Computer Science, University of Tennessee, Knoxville
NORA CATE SCHAEFFER,
Department of Sociology, University of Wisconsin–Madison
ALLEN L. SCHIRM,
Mathematica Policy Research, Inc., Washington, DC
JUDITH A. SELTZER,
Department of Sociology, University of California, Los Angeles
STANLEY K. SMITH,
Department of Economics, University of Florida, Gainesville
JOHN H. THOMPSON,
National Opinion Research Center, University of Chicago
ROGER TOURANGEAU,
Survey Research Center, University of Michigan, Ann Arbor, and Joint Program in Survey Methodology, University of Maryland, College Park
KIRK WOLTER,
National Opinion Research Center, University of Chicago
MICHAEL L. COHEN, Co-Study Director
DANIEL L. CORK, Co-Study Director
AGNES E. GASKIN, Administrative Assistant
MEYER ZITTER, Consultant
OCR for page R6
Envisioning the 2020 Census
COMMITTEE ON NATIONAL STATISTICS 2008–2009
WILLIAM F. EDDY (Chair),
Department of Statistics, Carnegie Mellon University
KATHARINE G. ABRAHAM,
Department of Economics and Joint Program in Survey Methodology, University of Maryland
ALICIA CARRIQUIRY,
Department of Statistics, Iowa State University
WILLIAM DUMOUCHEL,
Phase Forward, Inc., Waltham, Massachusetts
JOHN C. HALTIWANGER,
Department of Economics, University of Maryland
V. JOSEPH HOTZ,
Department of Economics, Duke University
KAREN KAFADAR,
Department of Statistics, Indiana University, Bloomington
DOUGLAS S. MASSEY,
Department of Sociology, Princeton University
SALLY MORTON,
Statistics and Epidemiology, RTI International, Research Triangle Park, North Carolina
JOSEPH NEWHOUSE,
Division of Health Policy Research and Education, Harvard University
SAMUEL H. PRESTON,
Population Studies Center, University of Pennsylvania
HAL STERN,
Department of Statistics, University of California, Irvine
ROGER TOURANGEAU,
Joint Program in Survey Methodology, University of Maryland, and Survey Research Center, University of Michigan
ALAN ZASLAVSKY,
Department of Health Care Policy, Harvard Medical School
CONSTANCE F. CITRO, Director
OCR for page R7
Envisioning the 2020 Census
Acknowledgments
The Panel on the Design of the 2010 Census Program of Evaluations and Experiments (CPEX) wishes to thank the many people who have contributed to our work. The panel is indebted to former Deputy Director Preston Jay Waite’s support during the developmental stage of the study and to the continued support from Daniel Weinberg, assistant director for decennial census and American Community Survey. Deborah Bolton and Randall Neugebauer ably served as principal liaisons between the Census Bureau and the panel, assisted by Gary Chappell, Joyce Price, Jennifer Reichert, and Donna Souders. All of these Census Bureau staff were responsive to requests from the panel for help in the development of agendas, the collection of supporting materials, and assistance with other logistical details. They have been a pleasure to work with.
Also, a number of Census Bureau personnel provided extremely useful presentations and supporting materials during the panel’s meetings in April 2007, July 2007, April 2008, July 2008, November 2008, and February 2009. In this regard, we would like to thank Teresa Angueira, Michael Bentley, Larry Cahoon, Joan Hill, Elizabeth Martin, Mary Mulry, Manuel de la Puente, Jennifer Reichert, Courtney Reiser, Annetta Clark Smith, and Frank Vitrano. Many of these individuals, along with others, also contributed to three very productive small group meetings of the panel (held in July 2007), and here we would like to mention interactions with Nancy Bates, William Bell, Sharon Boyer, Mary Destasio, Donna Kostanich, Laurel Schwede, Jennifer Tancreto, and Jim Treat. We would like to single out Jim Treat to thank him for his help in learning about a number of disparate areas of census planning, experimentation, and evaluation.
Our panel also benefited from the contributions of a number of outside speakers at our plenary meetings who gave generously of their talents. We thank Mick Couper, University of Michigan, and John Czajka, Mathe-
OCR for page R8
Envisioning the 2020 Census
matica Policy Research, Inc., for their thoughtful presentations on Internet data collection and administrative records, respectively. Rosemary Bender of Statistics Canada provided useful insight on census research in Canada, contributing to a discussion organized by panel member Ivan Fellegi. Finally, Marios Hadjieleftheriou of AT&T spoke about the possible structure of a master trace system and the use of such systems in industry.
We would like to thank Constance Citro, along with Jay Waite, for developing the study. In addition, Connie participated actively in our panel meetings and helped with writing and editing this report, providing her usual insightful comments and suggestions. Her knowledge and experience of census history and procedures proved to be an essential and invaluable addition to our deliberations. We would also like to thank Meyer Zitter, consultant to the panel, for also providing important, experienced perspectives on the census and in particular for his advice on the potential use of administrative records in census taking. We express our gratitude to Agnes Gaskin, who has seen to it that the meetings of the panel run as smoothly as possible, facilitating the travel and other needs of the panel members, and formatting this report. The panel is also indebted to Christine McShane, who provided expert technical editing of the draft report.
Michael Cohen and Daniel Cork served as extraordinarily effective and essential co-directors for the work of the panel. They admirably fulfilled their responsibility for compiling, organizing and presenting most of the background material and basic research appearing in the report, and for skillfully organizing and drafting the report so as to integrate the panel’s opinions and concerns along with our individual contributions. They were responsible from the side of the panel in arranging our cooperative interactions with the Census Bureau, which have been essential in our deliberations. And their experience and common sense helped keep us on track and focused as a panel during meetings and during the process of writing our interim and final reports.
Finally, it has been a pleasure interacting with a very talented, energetic, and collaborative panel as we considered plans for evaluation of the 2010 census and for research and development to be carried out in the next decade to achieve as cost effective as possible a census in 2020.
This final report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the Report Review Committee of the National Research Council. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making the published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We thank the
OCR for page R9
Envisioning the 2020 Census
following individuals for their participation in the review of this report: Betsy Ancker-Johnson, vice president (retired), General Motors Corporation; John L. Czajka, Mathematica Policy Research, Inc., Washington, DC; Shoreh Elhami, GIS director, Delaware County Auditor’s Office, Delaware, OH; Benjamin King, statistical consultant, Durham, NC; Daniel B. Levine, Statistics, Westat, Inc., Rockville, MD; Betsy Martin, consultant, Alexandria, VA; C. Matthew Snipp, Department of Sociology, Stanford University; and Alan M. Zaslavsky, Department of Health Care Policy, Harvard Medical School.
In addition, we thank the following individuals who served as reviewers for Experimentation and Evaluation Plans for the 2010 Census: Interim Report, which is reprinted in Part II of this volume: C.A. (Al) Irvine, consultant, San Diego, CA; Benjamin King, statistical consultant, Durham, NC; J. Michael Oakes, Division of Epidemiology, University of Minnesota, Minneapolis; Joseph Salvo, Population Division, New York City Department of City Planning, New York City, NY; Robert Scardamalia, Center for Research and Information Analysis, Department of Economic Development, Albany, NY; Frederick J. Scheuren, consultant, Alexandria, VA; and Judith M. Tanur, Department of Sociology and Behavior, State University of New York, Stony Brook. Finally, we thank the reviewers of the panel’s February 2009 letter report, which is reprinted as Part III of this volume: Barbara A. Bailar, independent consultant, Washington, DC; John L. Czajka, Mathematica Policy Research, Inc., Washington, DC; C.A. (Al) Irvine, consultant, San Diego, CA; Benjamin King, statistical consultant, Durham, NC; and Colm A. O’Muircheartaigh, Harris Graduate School of Public Policy Studies, The University of Chicago.
Although the reviewers listed above provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations nor did they see the final draft of the report before its release. The review of this report was overseen by Samuel H. Preston, Population Studies Center, University of Pennsylvania, and William F. Eddy, Department of Statistics, Carnegie Mellon University. Appointed by the National Research Council, they were responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution.
Lawrence D. Brown, Chair
Panel on the Design of the 2010 Census
Program of Evaluations and Experiments
OCR for page R10
Envisioning the 2020 Census
This page intentionally left blank.
OCR for page R11
Envisioning the 2020 Census
Contents
I
Final Report
1
Summary
3
1
Introduction
15
1–A The Panel, Its Charge, and Previous Reports
17
1–B Overview of This Report
19
2
Planning the 2020 Census: Cost and Quality
21
2–A Development of the Modern Census
22
2–A.1 1940 and 1950: Sampling and R&D
22
2–A.2 1960: Mailout and the “Long Form”
23
2–A.3 1970–1980: Mailout-Mailback, Computerized Address List, Coverage Improvement, Dual-System Estimation
23
2–A.4 1990–2000: Controversy Over Adjustment; Incremental Change
25
2–A.5 2010: Recovery from Near Disaster?
26
2–B Census Quality
28
2–B.1 Definition and Measures
28
2–B.2 Net Coverage Error, 1940–2000
29
2–B.3 Another Metric: Gross Coverage Errors
30
2–C Census Costs
31
2–C.1 Census Costs Over Time
32
2–C.2 Life-Cycle Cost of the 2010 Census
34
2–C.3 Comparison with Other Censuses
38
2–C.4 Coverage and Costs
39
OCR for page R12
Envisioning the 2020 Census
2–D Assessment: Time to Rethink the Census and Census Research
40
3
Census Bureau Research, Past and Present
59
3–A Current Research: Unfocused and Ineffective
60
3–A.1 Legacy of Research
61
3–A.2 Flaws in Current Census Research and the 2010 CPEX
63
3–B Key Steps in Rethinking the Census by Rethinking Research
75
3–B.1 Identify Visions for Next Census and Focus on a Limited Set of Goals
75
3–B.2 Build Capacity to Evaluate Costs of Alternative Visions
78
3–B.3 Build from 2010 Experience and Data (If Not the 2010 CPEX)
79
3–B.4 Examples of Research Directions: Strategic Issues for the 2020 Census
80
4
Revitalizing Census Research and Development
95
4–A In-House R&D—Why and What
97
4–B Properties of a Successful R&D Program
98
4–C Structuring a Successful R&D Program
103
4–C.1 Leadership
103
4–C.2 Organization
104
4–C.3 Project Teams
107
4–C.4 Funding
108
4–C.5 Training
108
4–C.6 Advisory Committees
109
4–C.7 Opportunities to Participate in Research
109
4–D A New Census Research and Development Program
109
4–D.1 Organization and Leadership
109
4–D.2 Integration
112
4–D.3 Fostering Outside Collaboration
113
4–D.4 Budgeting for Research
113
A
Past Census Research Programs
115
A–1 1950 Census
115
A–1.a Principal Pretests and Experiments Conducted Prior to the Census
115
A–1.b Research, Experimentation, and Evaluation Program
118
A–2 1960 Census
118
A–2.a Principal Pretests and Experiments Conducted Prior to the Census
118
OCR for page R13
Envisioning the 2020 Census
A–2.b Research, Experimentation, and Evaluation Program
122
A–3 1970 Census
124
A–3.a Principal Pretests and Experiments Conducted Prior to the Census
124
A–3.b Research, Experimentation, and Evaluation Program
128
A–4 1980 Census
133
A–4.a Principal Pretests and Experiments Conducted Prior to the Census
133
A–4.b Research, Experimentation, and Evaluation Program
138
A–5 1990 Census
148
A–5.a Principal Pretests and Experiments Conducted Prior to the Census
148
A–5.b Research, Experimentation, and Evaluation Program
153
A–6 2000 Census
157
A–6.a Principal Pretests and Experiments Conducted Prior to the Census
157
A–6.b Research, Experimentation, and Evaluation Program
160
A–7 Principal Pretests and Experiments Conducted Prior to the 2010 Census
173
A–7.a Pretests of Census Operations and Questionnaires
173
A–7.b Dress Rehearsal
178
B
2010 Census Program of Evaluations and Experiments
181
B–1 Experiments
182
B–1.a Alternative Questionnaire Experiment
182
B–1.b Nonresponse Follow-Up Contact Strategy Experiment
190
B–1.c Deadline Messaging/Compressed Schedule Experiment
191
B–1.d Confidentiality/Privacy Notification Experiment
194
B–1.e Heavy-Up Publicity Experiment
194
B–2 Evaluations
194
B–2.a Alternative Questionnaire Experiment Reinterview
197
B–2.b Content Reinterview
197
B–2.c Alternative Group Quarters Questionnaire
198
B–2.d Interactive Voice Response Customer Satisfaction Survey
199
B–3 Assessments
200
II
Interim Report: Experimentation and Evaluation in the 2010 Census (December 7, 2007)
203
Executive Summary
205
OCR for page R14
Envisioning the 2020 Census
1
Introduction
213
1–A Charge to the Panel
214
1–B Background: Experiments and Evaluations in the 2000 Census
215
1–B.1 Experiments
215
1–B.2 Evaluations
218
1–C Post Hoc Assessment of the 2000 Experiments and Evaluations
218
1–D Overview of the 2010 Census
221
1–E The CPEX Planning Document
222
1–F Guide to the Report
223
2
Initial Views on 2010 Census Experiments
225
2–A A General Approach to the Selection of Census Experiments
225
2–B Priority Topics for Experimentation in 2010
227
2–B.1 Internet Data Collection
228
2–B.2 Use of Administrative Records to Assist in Component Census Operations
230
2–B.3 Alternative Questionnaire Experiment
234
2–B.4 Race/Ethnicity as a Single Question
235
2–B.5 Representation of Residence Concepts
239
2–B.6 Other Content Issues
241
2–B.7 Deadline Messaging and Other Presentation Issues
243
2–B.8 A Possible Additional Experiment: Comparison of Telephone to Personal Interview for Coverage Follow-Up Interview
244
2–C Conclusion
245
3
Initial Views on 2010 Census Evaluations
247
3–A Suggestions for the 2010 Census Evaluations
247
3–A.1 Address List Improvement
247
3–A.2 Master Trace Sample
249
3–A.3 Reverse Record Check
251
3–A.4 Edit Protocols
252
3–A.5 Coverage Assessment of Group Quarters
253
3–A.6 Training of Field Enumerators
254
3–B A General Approach to Census Evaluation
254
3–C Initial Considerations Regarding a General Approach to Census Research
256
4
Considerations for the 2010 Census
259
4–A Technology
259
4–B Data Retention by Census Contractors
261
OCR for page R15
Envisioning the 2020 Census
4–C Census Enumeration as Part of Telephone Questionnaire Assistance
262
A
The Census Bureau’s Suggested Topics for Research
265
B
Internet Response Options in Selected Population Censuses
283
B–1 The Internet and the U.S. Census
283
B–2 Use of the Internet in Foreign Censuses
290
III
Letter Report (February 19, 2009)
303
References
319
Biographical Sketches of Panel Members and Staff
333
OCR for page R16
Envisioning the 2020 Census
This page intentionally left blank.
OCR for page R17
Envisioning the 2020 Census
List of Figures
Part I:
Final Report
B-1
2010 Alternative Questionnaire Experiment, control questionnaires
183
B-2
2010 Alternative Questionnaire Experiment, structures of combined race and Hispanic origin question
184
B-3
2010 Alternative Questionnaire Experiment, variations on race question
185
B-4
2010 Alternative Questionnaire Experiment, variations on Hispanic question and hybrid approaches
186
B-5
2010 Alternative Questionnaire Experiment, other residence panel
187
B-6
2010 nonresponse follow-up enumerator questionnaire, record of contact box
190
B-7
2010 Confidentiality/Privacy Notification Experiment, control and experimental treatments
195
Part II:
Interim Report
2-1
First page (Person 1), draft 2008 dress rehearsal questionnaire
236
2-2
Person 2 panel, draft 2008 dress rehearsal questionnaire
237
B-1
Housing Unit ID log-in screen and race response screen, Internet questionnaire, 2005 census test
287
OCR for page R18
Envisioning the 2020 Census
List of Tables
Part I:
Final Report
2-1
Estimates of Percentage Net Undercount, by Race, from Demographic Analysis, 1940–2000 (in Percent)
30
2-2
Per Housing Unit Costs, 1960–2010 U.S. Censuses
33
2-3
Census Bureau Life-Cycle Cost Estimates for the 2010 Census (covering fiscal years 2002–2013)
36
2-4
Coverage Improvement Programs and Procedures, 1970–2010 Censuses—Address List Development
44
2-5
Coverage Improvement Programs and Procedures, 1970–2010 Censuses—Publicity/Outreach
47
2-6
Coverage Improvement Programs and Procedures, 1970–2010 Censuses—Initial Enumeration Methods
51
2-7
Coverage Improvement Programs and Procedures, 1970–2010 Censuses—Follow-Up of Mail Returns
55
2-8
Coverage Improvement Programs and Procedures, 1970–2010 Censuses—NRFU (Nonresponse Follow-Up) and Post-NRFU
56
B-1
2010 Deadline Messaging/Compressed Schedule Experiment, deadline message treatments by form type
193
Part II:
Interim Report
1-1
Topic Headings, 2010 CPEX Research Proposals and 2000 Census Evaluation Program
219
OCR for page R19
Envisioning the 2020 Census
List of Boxes
Part I:
Final Report
3-1
Census Bureau’s Tentative Goals and Objectives for the 2020 Census
76
4-1
Historical Overview of the Census Bureau’s Organization of R&D
105
B-1
Experiments in the 2010 CPEX
182
B-2
Evaluations in the 2010 CPEX
196
B-3
Assessments in the 2010 CPEX
201
Part II:
Interim Report
2-1
Situations Generating a Coverage Follow-Up Interview
245
B-1
United Nations Economic Commission for Europe Comments on Internet Data Collection in the 2010 Round of Censuses
291
OCR for page R20
Envisioning the 2020 Census
This page intentionally left blank.
