Reengineering the
Census Bureau’s
Annual Economic Surveys
Panel on Reengineering the Census Bureau’s Annual Economic Surveys
Katharine G. Abraham, Constance F. Citro, Glenn D. White, Jr., and
Nancy K. Kirkendall, Editors
Committee on National Statistics
Division of Behavioral and Social Sciences and Education
A Consensus Study Report of
THE NATIONAL ACADEMIES PRESS
Washington, DC
www.nap.edu
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001
This activity was supported by Contract No. YA1323-14-CN-0033 between the National Academy of Sciences and the U.S. Census Bureau. Support for the work of the Committee on National Statistics is provided by a consortium of federal agencies through a grant from the National Science Foundation, a National Agricultural Statistics Service cooperative agreement, and several individual contracts. Any opinions, findings, conclusions, or recommendations expressed in this publication do not necessarily reflect the views of any organization or agency that provided support for the project.
International Standard Book Number-13: 978-0-309-47536-5
International Standard Book Number-10: 0-309-47536-8
Digital Object Identifier: https://doi.org/10.17226/25098
Additional copies of this publication are available for sale from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.
Copyright 2018 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
Suggested citation: National Academies of Sciences, Engineering, and Medicine. (2018). Reengineering the Census Bureau’s Annual Economic Surveys. Washington, DC: The National Academies Press. doi: https://doi.org/10.17226/25098.
The National Academy of Sciences was established in 1863 by an Act of Congress, signed by President Lincoln, as a private, nongovernmental institution to advise the nation on issues related to science and technology. Members are elected by their peers for outstanding contributions to research. Dr. Marcia McNutt is president.
The National Academy of Engineering was established in 1964 under the charter of the National Academy of Sciences to bring the practices of engineering to advising the nation. Members are elected by their peers for extraordinary contributions to engineering. Dr. C. D. Mote, Jr., is president.
The National Academy of Medicine (formerly the Institute of Medicine) was established in 1970 under the charter of the National Academy of Sciences to advise the nation on medical and health issues. Members are elected by their peers for distinguished contributions to medicine and health. Dr. Victor J. Dzau is president.
The three Academies work together as the National Academies of Sciences, Engineering, and Medicine to provide independent, objective analysis and advice to the nation and conduct other activities to solve complex problems and inform public policy decisions. The National Academies also encourage education and research, recognize outstanding contributions to knowledge, and increase public understanding in matters of science, engineering, and medicine.
Learn more about the National Academies of Sciences, Engineering, and Medicine at www.nationalacademies.org.
Consensus Study Reports published by the National Academies of Sciences, Engineering, and Medicine document the evidence-based consensus on the study’s statement of task by an authoring committee of experts. Reports typically include findings, conclusions, and recommendations based on information gathered by the committee and the committee’s deliberations. Each report has been subjected to a rigorous and independent peer-review process and it represents the position of the National Academies on the statement of task.
Proceedings published by the National Academies of Sciences, Engineering, and Medicine chronicle the presentations and discussions at a workshop, symposium, or other event convened by the National Academies. The statements and opinions contained in proceedings are those of the participants and are not endorsed by other participants, the planning committee, or the National Academies.
For information about other products and activities of the National Academies, please visit www.nationalacademies.org/about/whatwedo.
PANEL ON REENGINEERING THE CENSUS BUREAU’S ANNUAL ECONOMIC SURVEYS
KATHARINE G. ABRAHAM (Chair), Department of Economics and Joint Program in Survey Methodology, University of Maryland
MARY BATCHER, BDS Data Analytics, Bethesda, MD
MARIE BRODEUR, Statistics Canada, Ottawa (retired)
CYNTHIA Z.F. CLARK, National Agricultural Statistics Service, U.S. Department of Agriculture (retired)
WILLIAM DUNKELBERG, National Federation of Independent Businesses, Nashville, TN
ROBERT E. HALL, Hoover Institution and Department of Economics, Stanford University
KAYE HUSBANDS FEALING, School of Public Policy, Georgia Institute of Technology
J. STEVEN LANDEFELD, Economics/Statistical Division, United Nations, and U.S. Naval Academy
DAVID MARKER, Westat, Rockville, MD
JOAN GENTILI NAYMARK, Minnesotans for the American Community Survey and JG Naymark Demographics, Minneapolis, MN
KRISTEN OLSON, Department of Sociology, University of Nebraska–Lincoln
GER SNIJKERS, Department of Methodology, Statistics Netherlands
CHAD SYVERSON, Booth School of Business, University of Chicago
RICHARD VALLIANT, Joint Program in Survey Methodology, University of Michigan and University of Maryland
GLENN D. WHITE, JR., Study Director
NANCY J. KIRKENDALL, Senior Program Officer
CONSTANCE F. CITRO, Senior Scholar
MARY ANN KASPER, Senior Program Assistant
COMMITTEE ON NATIONAL STATISTICS
ROBERT M. GROVES (Chair), Department of Mathematics and Statistics and Department of Sociology, Georgetown University
FRANCINE BLAU, Department of Economics, Cornell University
MARY ELLEN BOCK, Department of Statistics (emerita), Purdue University
ANNE C. CASE, Woodrow Wilson School of Public and International Affairs, Princeton University
MICHAEL E. CHERNEW, Department of Health Care Policy, Harvard Medical School
JANET CURRIE, Woodrow Wilson School of Public and International Affairs, Princeton University
DONALD A. DILLMAN, Department of Sociology, Washington State University
CONSTANTINE GATSONIS, Center for Statistical Sciences, Brown University
JAMES S. HOUSE, Survey Research Center, Institute for Social Research, University of Michigan
THOMAS L. MESENBOURG, U.S. Census Bureau (retired)
SARAH M. NUSSER, Department of Statistics, Iowa State University
COLM O’MUIRCHEARTAIGH, Harris School of Public Policy Studies, University of Chicago
JEROME P. REITER, Department of Statistical Science, Duke University
ROBERTO RIGOBON, Sloan School of Management, Massachusetts Institute of Technology
JUDITH A. SELZTER, Department of Sociology, University of California, Los Angeles
EDWARD H. SHORTLIFFE, Department of Biomedical Informatics, Columbia University/Arizona State University
BRIAN HARRIS-KOJETIN, Director
CONSTANCE F. CITRO, Senior Scholar
Acknowledgments
This project was sponsored by the U.S. Census Bureau, which asked the Committee on National Statistics to convene a panel to review the Bureau’s portfolio of annual economic surveys. The Panel on Reengineering the Census Bureau’s Annual Economic Surveys could not have conducted its work without the contributions from Census Bureau staff and many other colleagues, whom we thank for their time, generosity, and expert guidance in helping the panel develop a broader and deeper understanding of key issues for the future of these surveys. The panel especially thanks Kevin Deardorff, Maria Iseman, and Jessica Wellwood at the Census Bureau for their time and background materials in response to our many requests.
The panel thanks the following individuals who attended open meetings and generously gave of their time to present material to inform the panel’s deliberations: Kim Bayard, Board of Governors, Federal Reserve System; Aaron Catlin, Office of the Actuary, Centers for Medicare & Medicaid Services; Jim Diffley, IHS Economics; Dennis Fixler, Bureau of Economic Analysis; Alexa Ghagouri, Accenture; Vanessa Goeschl, Charlotte Regional Partnership; Maurine Haver, Haver Analytics, Inc.; Eric Long, Greater Oklahoma City Chamber of Commerce; Nancy McCrea, Maryland Department of Commerce; Thomas Mesenbourg, Census Bureau (retired) and Key Concepts Knowledgebase, LLC; Daniela Ravindra, Statistics Canada; Anthony Scriffignano, Dun & Bradstreet; Chris Sparks, Bureau of Labor Statistics; Gaétan St-Louis, Statistics Canada; David Talan, Bureau of Labor Statistics; and Neal Young, Minnesota Department of Employment and Economic Development. In addition, the panel thanks Jose Plehn,
Powerlytics, and Jack Kleinhenz, National Retail Federation, who were unable to attend an open meeting but provided written statements.
We also thank the following presenters from the U.S. Census Bureau, many of whom generously gave of the their time to answer questions and provide additional materials to the panel: Andrew Baer, Erika Becker Medina, Naomi Blackman, William G. Bostic, Jr., Chuck Brady, Carol Caldwell, Rob Chestnut, Jeffrey Dalzell, Bill Davie, Kevin Deardorff, Aneta Erdie, Meghan Harrison, Carrie Hill, Jim Jamski, Ron Jarmin, Susanne Johnson, Mike Kornbau, Jim Liu, Erica Marquette, Valerie Mastalski, Blynda Metcalf, Stephanie Morales Garcia, John Murphy, Patrice Norman, Nick Orsini, Steve Roman, Anne Russell, Chris Savage, Scott Scheleur, Julius Smith, Robert Struble, Julie Vesely, Katrina Washington, Deanna Weidenhammer, Jessica Wellwood, Jeremy Wiedemann, Diane Willimack, and Brandy Yarbrough.
The panel could not have conducted its work efficiently without the capable staff of the National Academies of Sciences, Engineering, and Medicine. Brian Harris-Kojetin, director of the Committee on National Statistics (CNSTAT), provided institutional leadership, and Nancy Kirkendall, CNSTAT senior program officer, provided invaluable support to the study director throughout the project. Mary Ann Kasper provided logistical support for the panel’s meetings. We also thank Eugenia Grohman, Kirsten Sampson-Snyder, and Yvonne Wise of the Division of Behavioral and Social Sciences and Education, who, respectively, edited this Consensus Study Report, oversaw the report review process, and managed the production of this Report.
A special note of appreciation is due to Constance F. Citro, CNSTAT senior scholar and former director, for her significant contributions to improving the structure of this Report and the clarity of presentation throughout the document, as well as contributing in other ways to the panel’s work and preparation of this Report.
On behalf of the panel, I express our gratitude to our study director, Glenn D. White, Jr. Over a span of 30 months, he facilitated communication among panel members, identified relevant studies and reports, coordinated all of our activities, and kept the panel on track. He organized our meetings, communicated our questions to the Census Bureau, masterfully organized the material the Bureau provided, and helped to ensure that the panel’s thinking was fully reflected in the final report.
A note of appreciation is in order for my fellow panel members. Despite their many professional commitments, every panel member donated considerable time and contributed a wealth of expertise to make this Consensus Study possible. All of the panel members helped to draft one or more chapters of this Consensus Study Report in their particular areas of expertise, in addition to providing input on this Report as a whole, so that it reflects our
collective expertise and commitment. This group—deliberately chosen for its varied perspectives, diverse statistical, economic, and business backgrounds, and deep subject-matter knowledge—displayed rigor and creativity, and also patience when dealing with one another, throughout the project. I also thank Michael D. Larsen, Department of Statistics and Survey Design and Data Analysis, The George Washington University, who served on the panel in its beginning but was unable to continue for the entirety of this study.
This Consensus Study Report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise. The purpose of this independent review is to provide candid and critical comments that will assist the National Academies in making each published report as sound as possible and to ensure that it meets the institutional standards for quality, objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process.
We thank the following individuals for their reviews of this Report: Thomas Abt, Center for International Development, Harvard Law School; Gale A. Boyd, Triangle Research Data Center, Duke University; Jill A. Dever, Division for Statistical and Data Sciences, RTI International; Bonnie Ghosh-Dastidar, Statistical Advisory Center, RAND; Gustav Haraldsen, Division for Methods, Statistics Norway; Jacqui Jones, Macroeconomic Statistics Division, Australian Bureau of Statistics; Jack Kleinhenz, Kleinhenz Associates; William D. Nordhaus, Department of Economics, Yale University; and Polly Phipps, Office of Survey Methods Research, Bureau of Labor Statistics.
Although the reviewers listed above provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations of this Report, nor did they see the final draft before its release. The review of this Report was overseen by Keith F. Rust, Westat, Rockville, Maryland, and Roderick J.A. Little, Department of Biostatistics, School of Public Health, University of Michigan. They were responsible for making certain that an independent examination of this Report was carried out in accordance with the standards of the National Academies and that all review comments were carefully considered. Responsibility for the final content rests entirely with the authoring panel and the National Academies.
Katharine G. Abraham, Chair
Panel on Reengineering the Census Bureau’s
Annual Economic Surveys
This page intentionally left blank.
2.4.2 More Detailed Data by Industry and Other Characteristics
2.4.3 Gaps in Subject Coverage
2.5 Consistency across Domains and over Time
2.7 Consultation with Data Users on an ABSS
3.1 The Current Census Bureau Business Register
3.2 Improving the Business Register
3.2.1 Statistical Unit Definition
3.2.4 Sample Selection Information
3.2.5 Recommendations on Business Register Redesign
4 HARMONIZATION OF QUESTIONNAIRES AND DATA COLLECTION PROCESSES
4.2 Key Concepts for Data Collection
4.2.1 Key Concepts in Current Surveys
4.2.3 Respondents’ Perspective
4.2.4 Methodological Perspective
4.2.5 A Concept Harmonization Team
4.4.1 Currently Used Data Collection Modes
4.4.2 Benefits of Electronic Data Collection
4.5 Questionnaire Design and Development
4.6 Survey Recruitment and Participation
5.1 Overview of Current Approaches
5.2 Industry Coverage and Sampling Units
5.4 Identification of Certainty Units
5.5 Rotation and Synchronization of Samples
5.9 Use of Alternative Data Sources in Estimation
6 EDITING, IMPUTATION, DISCLOSURE CONTROL, AND QUALITY STANDARDS
6.2 Imputation for Missing Data
6.2.1 Use of Administrative Data for Imputation
6.2.2 Use of Models in Item Imputation
6.2.3 Imputing for Large Units that Do Not Respond
6.2.4 Reflecting the Effects of Imputation on Estimates
7.1 Current Dissemination Products and Practices
7.2 Toward Improved Dissemination
7.4 Scheduling of Data Releases
7.5 Publicizing Data and Data Discovery
7.7 Monitoring of Dissemination Practices
8 TOWARD AN INTEGRATED ANNUAL BUSINESS SURVEY SYSTEM
8.2 Planning an Integrated System
8.3 Initial Steps in Harmonizing the Annual Economic Surveys
8.3.4 Editing, Imputation, Disclosure Control, and Quality Standards
8.4 Reengineering Initiatives in Other Statistical Agencies
8.5 Lessons Learned for Implementing an ABSS
8.5.5 Lessons Learned in Implementation
A Observations from Users of the Annual Economic Surveys
B Surveys Covered in this Report
C Characteristics of the Annual Economic Surveys (extensive table online only)
Acronyms
ABS | Annual Business Survey |
ABSS | Annual Business Survey System |
ACES | Annual Capital Expenditures Survey |
ACS | American Community Survey |
AFF | American Fact Finder |
API | application programming interface |
ARTS | Annual Retail Trade Survey |
ASE | Annual Survey of Entrepreneurs |
ASM | Annual Survey of Manufactures |
AWTS | Annual Wholesale Trade Survey |
BEA | Bureau of Economic Analysis |
BES | Business Expenses Supplement |
BLS | Bureau of Labor Statistics |
BMF | Business Master File |
BR | Business Register |
BRDI-M | Business R&D and Innovation for Microbusinesses |
BRDIS | Business R&D and Innovation Survey |
CBP | County Business Patterns |
CES | Center for Economic Studies |
CMS | Centers for Medicare & Medicaid Services |
CNSTAT | Committee on National Statistics |
COS | Company Organization Survey |
CPS | Current Population Survey |
CV | coefficient of variation |
EDI | Electronic Data Interchange |
EID | Economic Indicator Division |
EIN | Employer Identification Number |
EPM | Enterprise Portfolio Management |
ESMD | Economic Statistical Methods Division |
ESP | Enterprise Statistics Program |
EWD | Economy-Wide Statistics Division |
FAQ | frequently asked questions |
FESAC | Federal Economic Statistics Advisory Committee |
FRB | Federal Reserve Board |
FRED | Federal Reserve Economic Data |
FSRDC | Federal Statistical Research Data Center |
FTE | full-time equivalent |
FY | fiscal year |
GDP | gross domestic product |
HT | Horvitz-Thompson |
IBSP | Integrated Business Statistics Program |
ICTS | Information and Communication Technology Survey |
IMPECT | IMPlementation EConomic Transformation program |
IRS | Internal Revenue Service |
IT | information technology |
LEHD | Longitudinal Employer-Household Dynamics Program |
MARTS | Advanced Monthly Retail Trade Survey |
MBDA | Minority Business Development Agency |
MOPS | Management and Organizational Practices Survey |
MSA | Metropolitan Statistical Area |
M3UFO | Manufacturers’ Unfilled Orders Survey |
NAICS | North American Industry Classification System |
NAPCS | North American Product Classification System |
NASS | National Agricultural Statistics Service |
NCSES | National Center for Science and Engineering Statistics |
NSF | National Science Foundation |
OECD | Organisation for Economic Co-operation and Development |
OMB | Office of Management and Budget |
PPS | probability proportional to size |
PRN | permanent random number |
QSS | Quarterly Services Survey |
R&D | research and development |
RDC | Research Data Center |
SAE | small-area estimation |
SAS | Service Annual Survey |
SBO | Survey of Business Owners |
SEC | Securities and Exchange Commission |
SMART | specific, measurable, assignable, realistic, and time-bound |
SQ-CLASS | Business and Professional Classification Survey |
SRS | simple random sampling |
SSN | Social Security number |
STSRS | stratified simple random sampling |
STSYS | stratified systematic random sampling |
TQRR | total quantity response rate |
UN | United Nations |
URR | unit response rate |