Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
PREPUBLICATION COPY, UNCORRECTED PROOFS Reengineering the Census Bureauâs Annual Economic Surveys Panel on Reengineering the Census Bureauâs Annual Economic Surveys Katharine Abraham, Constance F. Citro, Glenn D. White, Jr., and Nancy K. Kirkendall, Editors Committee on National Statistics Division of Behavioral and Social Sciences and Education A Consensus Study Report of ADVANCE COPY NOT FOR PUBLIC RELEASE BEFORE Thursday, May 3, 2018 11:00 am EDT
PREPUBLICATION COPY, UNCORRECTED PROOFS THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001 This activity was supported by Contract No. YA1323-14-CN-0033 between the National Academy of Sciences and the U.S. Census Bureau. Support for the work of the Committee on National Statistics is provided by a consortium of federal agencies through a grant from the National Science Foundation (award number SES-1560294), a cooperative agreement from the U.S. Department of Agriculture, to which several agencies contribute, and individual agreements with agencies in the U.S. Departments of Commerce, Health and Human Services, Housing and Urban Development, and Treasury. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of any organization or agency that provided support for the project. International Standard Book Number-13: 978-0-309 International Standard Book Number-10: Digital Object Identifier: https://doi.org/10.17226/25098 Additional copies of this report are available for sale from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu. Copyright 2018 by the National Academy of Sciences. All rights reserved. Printed in the United States of America Suggested citation: The National Academies of Sciences, Engineering, and Medicine. (2018). Reengineering the Census Bureauâs Annual Economic Surveys. Washington, DC: The National Academies Press. doi: https://doi.org/10.17226/25098.
PREPUBLICATION COPY, UNCORRECTED PROOFS The National Academy of Sciences was established in 1863 by an Act of Congress, signed by President Lincoln, as a private, nongovernmental institution to advise the nation on issues related to science and technology. Members are elected by their peers for outstanding contributions to research. Dr. Marcia McNutt is president. The National Academy of Engineering was established in 1964 under the charter of the National Academy of Sciences to bring the practices of engineering to advising the nation. Members are elected by their peers for extraordinary contributions to engineering. Dr. C. D. Mote, Jr., is president. The National Academy of Medicine (formerly the Institute of Medicine) was established in 1970 under the charter of the National Academy of Sciences to advise the nation on medical and health issues. Members are elected by their peers for distinguished contributions to medicine and health. Dr. Victor J. Dzau is president. The three Academies work together as the National Academies of Sciences, Engineering, and Medicine to provide independent, objective analysis and advice to the nation and conduct other activities to solve complex problems and inform public policy decisions. The National Academies also encourage education and research, recognize outstanding contributions to knowledge, and increase public understanding in matters of science, engineering, and medicine. Learn more about the National Academies of Sciences, Engineering, and Medicine at www.national- academies.org.
PREPUBLICATION COPY, UNCORRECTED PROOFS Consensus Study Reports published by the National Academies of Sciences, Engineering, and Medicine document the evidence-based consensus on the studyâs statement of task by an authoring committee of experts. Reports typically include findings, conclusions, and recommendations based on information gathered by the committee and the committeeâs deliberations. Each report has been subjected to a rigorous and independent peer-review process and it represents the position of the National Academies on the statement of task. Proceedings published by the National Academies of Sciences, Engineering, and Medicine chronicle the presentations and discussions at a workshop, symposium, or other event convened by the National Academies. The statements and opinions contained in proceedings are those of the participants and are not endorsed by other participants, the planning committee, or the National Academies. For information about other products and activities of the National Academies, please visit www.nationalacademies.org/about/whatwedo.
PREPUBLICATION COPY, UNCORRECTED PROOFS PANEL ON REENGINEERING THE CENSUS BUREAUâS ANNUAL ECONOMIC SURVEYS KATHARINE G. ABRAHAM (Chair), Department of Economics and Joint Program in Survey Methodology, University of Maryland MARY BATCHER, BDS Data Analytics, Bethesda, MD MARIE BRODEUR, Statistics Canada, Ottawa (retired) CYNTHIA Z.F. CLARK, National Agricultural Statistics Service, U.S. Department of Agriculture (retired) WILLIAM DUNKELBERG, National Federation of Independent Businesses, Nashville, TN ROBERT E. HALL, Hoover Institution and Department of Economics, Stanford University KAYE HUSBANDS FEALING, School of Public Policy, Georgia Institute of Technology J. STEVEN LANDEFELD, Economics/Statistical Division, United Nations, and U.S. Naval Academy DAVID MARKER, Westat, Rockville, MD JOAN GENTILI NAYMARK, Minnesotans for the American Community Survey and JG Naymark Demographics, Minneapolis, MN KRISTEN OLSON, Department of Sociology, University of Nebraska-Lincoln GER SNIJKERS, Department of Methodology, Statistics Netherlands CHAD SYVERSON, Booth School of Business, University of Chicago RICHARD VALLIANT, Joint Program in Survey Methodology, University of Michigan and University of Maryland GLENN D. WHITE, JR., Study Director NANCY J. KIRKENDALL, Senior Program Officer CONSTANCE F. CITRO, Senior Scholar MARY ANN KASPER, Senior Program Assistant v
PREPUBLICATION COPY, UNCORRECTED PROOFS COMMITTEE ON NATIONAL STATISTICS ROBERT M. GROVES (Chair), Department of Mathematics and Statistics and Department of Sociology, Georgetown University FRANCINE BLAU, Department of Economics, Cornell University MARY ELLEN BOCK, Department of Statistics (emerita), Purdue University ANNE C. CASE, Woodrow Wilson School of Public and International Affairs, Princeton University MICHAEL E. CHERNEW, Department of Health Care Policy, Harvard Medical School JANET CURRIE, Woodrow Wilson School of Public and International Affairs, Princeton University DONALD A. DILLMAN, Department of Sociology, Washington State University CONSTANTINE GATSONIS, Center for Statistical Sciences, Brown University JAMES S. HOUSE, Survey Research Center, Institute for Social Research, University of Michigan THOMAS L. MESENBOURG, U.S. Census Bureau (retired) SUSAN A. MURPHY, Department of Statistics, University of Michigan SARAH M. NUSSER, Department of Statistics, Iowa State University COLM OâMUIRCHEARTAIGH, Harris School of Public Policy Studies, University of Chicago JEROME P. REITER, Department of Statistical Science, Duke University ROBERTO RIGOBON, Sloan School of Management, Massachusetts Institute of Technology JUDITH A. SELZTER, Department of Sociology, University of California, Los Angeles EDWARD H. SHORTLIFFE, Department of Biomedical Informatics, Columbia University and Arizona State University BRIAN HARRIS-KOJETIN, Director vi
PREPUBLICATION COPY, UNCORRECTED PROOFS ACKNOWLEDGEMENTS This project was sponsored by the U.S. Census Bureau, which asked the Committee on National Statistics to convene a panel to review the Bureauâs portfolio of annual economic surveys. The Panel on Reengineering the Census Bureauâs Annual Economic Surveys could not have conducted its work without the contributions from Census Bureau staff and many other colleagues, whom we thank for their time, generosity, and expert guidance in helping the panel develop a broader and deeper understanding of key issues for the future of these surveys. The panel especially thanks Kevin Deardorff, Jessica Wellwood, and Maria Iseman at the U.S. Census Bureau for their time and background materials in response to our many requests. The panel thanks the following individuals who attended open meetings and generously gave of their time to present material to inform the panelâs deliberations: Kim Bayard, Federal Reserve; Aaron Catlin, Office of the Actuary, Centers for Medicare and Medicaid Services; Jim Diffley, IHS Economics; Dennis Fixler, Bureau of Economic Analysis; Alexa Ghagouri, Accenture; Vanessa Goeschl, Charlotte Regional Partnership; Maurine Haver, Haver Analytics, Inc.; Eric Long, Greater Oklahoma City Chamber of Commerce; Nancy McCrea, Maryland Department of Commerce; Thomas Mesenbourg, U.S. Census Bureau (retired) and Key Concepts Knowledgebase, LLC; Daniela Ravindra, Statistics Canada; Anthony Scriffignano, Dun & Bradstreet; Chris Sparks, Bureau of Labor Statistics; GaÃ©tan St-Louis, Statistics Canada; David Talan, Bureau of Labor Statistics; and Neal Young, Minnesota Department of Employment and Economic Development. In addition, the panel thanks Jose Plehn, Powerlytics, and Jack Kleinhenz, chief economist for the National Retail Federation, who were unable to attend an open meeting but provided written statements. We also thank the following presenters from the U.S. Census Bureau, many of whom generously gave of the their time to answer questions and provide additional materials to the panel: Andrew Baer, Erika Becker Medina, Naomi Blackman, William G. Bostic, Jr., Chuck Brady, Carol Caldwell, Rob Chestnut, Jeffrey Dalzell, Bill Davie, Kevin Deardorff, Aneta Erdie, Meghan Harrison, Carrie Hill, Jim Jamski, Ron Jarmin, Susanne Johnson, Mike Kornbau, Jim Liu, Valerie Mastalski, Erica Marquette, Blynda Metcalf, Stephanie Morales Garcia, John Murphy, Patrice Norman, Nick Orisini, Steve Roman, Anne Russell, Chris Savage, Scott Scheleur, Julius Smith, Robert Struble, Julie Vesely, Katrina Washington, Deanna Weidenhammer, Jessica Wellwood, Jeremy Wiedemann, Diane Willimack, and Brandy Yarbrough. The panel could not have conducted its work efficiently without the capable staff of the National Academies of Sciences, Engineering, and Medicine: Brian Harris-Kojetin, director of the Committee on National Statistics (CNSTAT), provided institutional leadership, and Nancy Kirkendall, CNSTAT senior program officer, provided invaluable support to the study director throughout the project. Mary Ann Kasper provided logistical support for the panelâs meetings. We also thank Eugenia Grohman, Kirsten Sampson-Snyder, and Yvonne Wise of the Division of Behavioral and Social Sciences and Education, who, respectively, edited the report, oversaw the report review process, and managed the production of the report. vii
PREPUBLICATION COPY, UNCORRECTED PROOFS A special note of appreciation is due to Constance F. Citro, CNSTAT senior scholar and former director, for her significant contributions to improving the structure of the report and the clarity of presentation throughout the document, as well as contributing in other ways to the panelâs work and preparation of its report. On behalf of the panel, I express our gratitude to our study director, Glenn D. White, Jr. Over a span of 30 months, he facilitated communication among panel members, identified relevant studies and reports, coordinated all of our activities, and kept the panel on track. He organized our meetings, communicated our questions to the Census Bureau, masterfully organized the material the Bureau provided, and helped to ensure that the panelâs thinking was fully reflected in the final report. A note of appreciation is in order for my fellow panel members. Despite their many professional commitments, every panel member donated considerable time and contributed a wealth of expertise to make this report possible. All of the panel members helped to draft one or more sections of the report in their particular areas of expertise, in addition to providing input on the report as a whole, so that the final report reflects our collective expertise and commitment. This groupâdeliberately chosen for its varied perspectives, diverse statistical, economic, and business backgrounds, and deep subject-matter knowledgeâdisplayed rigor and creativity, and also patience when dealing with one another, throughout the project. I also thank Michael D. Larsen, Department of Statistics and Survey Design & Data Analysis, George Washington University, who served on the panel in its beginning but was unable to continue for the entirety of the study. This Consensus Study Report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise. The purpose of this independent review is to provide candid and critical comments that will assist the National Academies of Sciences, Engineering, and Medicine in making each published report as sound as possible and to ensure that it meets the institutional standards for quality, objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We thank the following individuals for their reviews of this report: Thomas Abt, Center for International Development, Harvard Law School; Gale A. Boyd, Triangle Research Data Center, Duke University; Jill A. Dever, Division for Statistical and Data Sciences, RTI International; Bonnie Ghosh-Dastidar, Statistical Advisory Center, RAND; Gustav Haraldsen, Division for Methods, Statistics Norway; Jacqui Jones, Macroeconomic Statistics Division, Australian Bureau of Statistics; Jack Kleinhenz, Principal and Chief Economist, Kleinhenz Associates; Polly Phipps, Office of Survey Methods Research, U.S. Bureau of Labor Statistics; William D. Nordhaus, Department of Economics, Yale University. Although the reviewers listed above provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations of this report, nor did they see the final draft before its release. The review of this report was overseen by Keith F. Rust, Associate Director, Westat, Inc., Rockville, MD and Roderick J.A. Little, Department of Biostatistics, School of Public Health, University of University of Michigan. They were responsible for making certain that an independent examination of this report was carried out in accordance with the standards of the National Academies and that all review comments were carefully considered. Responsibility for the final content rests entirely with the authoring panel and the National Academies. viii
PREPUBLICATION COPY, UNCORRECTED PROOFS Katharine G. Abraham, Chair Panel on Reengineering the Census Bureauâs Annual Economic Surveys ix
PREPUBLICATION COPY, UNCORRECTED PROOFS x
PREPUBLICATION COPY, UNCORRECTED PROOFS CONTENTS EXECUTIVE SUMMARY 1 1 Introduction 4 1.1 Economic Statistics at the Census Bureau 5 1.1.1 Economic Censuses 5 1.1.2 Economic Surveys 6 1.1.3 Lack of Integration of Economic Censuses and Surveys 7 1.2 Study Scope 7 1.3 The Panelâs Work 10 1.3.1 Understanding the Landscape 10 1.3.2 A New Approach for the FutureâAn Integrated System 11 1.3.3 Report Structure 12 1.4 References 14 2 What Data Users Want, Need, and Use 16 2.1 Framework 17 2.2 Accuracy 18 2.3 Timeliness 21 2.4 Relevance 23 2.4.1 Geographically Disaggregated Statistics 23 2.4.2 More Detailed Data by Industry and Other Characteristics 23 2.4.3 Gaps in Subject Coverage 24 2.5 Consistency across Domains and over Time 26 2.6 Access to Survey Data 28 2.7 Consultation with Data Users on an ABSS 28 2.8 References 29 3 Business Register 32 3.1 The Current Census Bureau Business Register 32 3.1.1 Uses and Units Included 32 3.1.2 Input Data Sources 34 3.1.3 Content 36 3.2 Improving the Business Register 36 3.2.1 Statistical Unit Definition 37 3.2.2 Frame Content 37 3.2.3 Frame Maintenance 38 3.2.4 Sample Selection Information 39 3.2.5 Recommendations on Business Register Redesign 39 3.3 Account Manager Program 40 xi
PREPUBLICATION COPY, UNCORRECTED PROOFS 3.4 References 41 4 Harmonization of Questionnaires and Data Collection Processes 44 4.1 Data Sources 45 4.2 Key Concepts for Data Collection 47 4.2.1 Key Concepts in Current Surveys 48 4.2.2 Usersâ Perspective 50 4.2.3 Respondentsâ Perspective 51 4.2.4 Methodological Perspective 51 4.2.5 A Concept Harmonization Team 53 4.3 Data Gaps 55 4.4 Data Collection Modes 57 4.4.1 Currently Used Data Collection Modes 57 4.4.2 Benefits of Electronic Data Collection 58 4.5 Questionnaire Design and Development 59 4.5.1 Questionnaire Elements 60 4.5.2 Pretesting 62 4.6 Survey Recruitment and Participation 63 4.6.1 Communication Methods 64 4.6.2 Timing 65 4.6.3 Assessing a Communications Strategy 66 4.7 References 68 5 Sampling and Estimation 72 5.1 Overview of Current Approaches 73 5.2 Industry Coverage and Sampling Units 74 5.3 Sample Selection Methods 79 5.4 Identification of Certainty Units 80 5.5 Rotation and Synchronization of Samples 84 5.6 Estimation Methods 86 5.7 Small-Area Estimation 91 5.8 Preliminary Estimates 92 5.9 Use of Alternative Data Sources in Estimation 92 5.10 References 93 6 Editing, Imputation, Disclosure Control, and Quality Standards 96 6.1 Current Editing Practices 96 6.2 Imputation for Missing Data 98 6.2.1 Use of Administrative Data for Imputation 99 6.2.2 Use of Models in Item Imputation 100 6.2.3 Imputing for Large Units that Do Not Respond 101 6.2.4 Reflecting the Effects of Imputation on Estimates 101 6.2.5 Informing Users 101 6.3 Disclosure Control 102 6.4 Quality Standards 104 6.5 References 110 xii
PREPUBLICATION COPY, UNCORRECTED PROOFS 7 Dissemination 112 7.1 Current Dissemination Products and Practices 113 7.2 Toward Improved Dissemination 115 7.3 Data Manipulation 117 7.4 Scheduling of Data Releases 118 7.5 Publicizing Data and Data Discovery 118 7.6 Archiving 119 7.7 Monitoring of Dissemination Practices 120 7.8 References 120 8 Toward an Integrated Annual Business Survey System 121 8.1 The Concept of an ABSS 122 8.2 Planning an Integrated System 125 8.3 Initial Steps in Harmonizing the Annual Economic Surveys 126 8.3.1 The Business Register 133 8.3.2 Content and Data Collection 133 8.3.3 Sampling and Estimation 134 8.3.4 Editing, Imputation, Disclosure Control, and Quality Standards 134 8.3.5 Dissemination 135 8.4 Reengineering Initiatives in Other Statistical Agencies 135 8.4.1 Statistics Canada 135 8.4.2 Statistics Netherlands 136 8.5 Lessons Learned for Implementing an ABSS 138 8.5.1 Vision 139 8.5.2 Communication 140 8.5.3 Governance 140 8.5.4 Consultation with Users 141 8.5.5 Lessons Learned in Implementation 141 8.6 References 143 Appendixes A Observations from Users of the Annual Economic Surveys 145 B Surveys Covered in this Report 153 C Characteristics of the Annual Economic Surveys 166 D Biographical Sketches of Panel Members and Staff 253 xiii
PREPUBLICATION COPY, UNCORRECTED PROOFS Acronyms ABSS Annual Business Survey System ACES Annual Capital Expenditure Survey ACS American Community Survey AFF American Fact Finder API Application Program Interface ARTS Annual Retail Trade Survey ARU alternate reporting unit ASE Annual Survey of Entrepreneurs ASM Annual Survey of Manufactures AWTS Annual Wholesale Trade Survey BEA Bureau of Economic Analysis BES Business Expense Supplement BLS Bureau of Labor Statistics BMF Business Master File BR Business Register CAPI computer-assisted personal interviewing CART classification and regression trees CATI computer-assisted telephone interviewing CAWI computer-assisted web interviewing CBP County Business Patterns CES Center for Economic Studies CFS Commodity Flow Survey CIPSEA Confidential Information Protection and Statistical Efficiency Act CMS Center for Medicare and Medicaid Services CNSTAT Committee on National Statistics COS Company Organization Survey CPS Current Population Survey CV coefficient of variation EDI Electronic data interchange EID Economic Indicator Division EIN Employer Identification Number EPM Enterprise Portfolio Management ESMD Economic Statistics Methods Division ESP Enterprise Statistics Program xiv
PREPUBLICATION COPY, UNCORRECTED PROOFS EWD Economy-Wide Statistics Division EWKS Economy-Wide Key Statistics FAQ frequently asked question FESAC Federal Economic Statistics Advisory Committee FRB Federal Reserve Board FRED Federal Reserve Economic Data FSRDCs Federal Statistical Research Data Centers FTE full-time equivalent FY fiscal year GDP gross domestic product GSBPM Generic Statistical Business Process Model HT Horvitz-Thompson IBSP Integrated Business Statistics Program ICTS Information and Communication Technology Survey IMPECT IMPlementation EConomic Transformation program IRS Internal Revenue Service IT information technology LDB Longitudinal Data Base LEHD Longitudinal Employer-Household Dynamics program MARTS Advance Monthly Retail Trade Survey MBDA Minority Business Development Agency MOPS Management and Organizational Practices Survey MOS measure of size M3UFO Manufacturers' Unfilled Orders Survey NAICS North American Industry Classification System NAPCS North American Product Classification System NASS National Agricultural Statistics Service NSF National Science Foundation OECD Organization for Economic Cooperation and Development OMB Office of Management and Budget PPS probability proportional to size PRN permanent random number QSS Quarterly Services Survey R&D research and development RDCs Regional Data Centers SAE small-area estimation SAS Service Annual Survey SBO Survey of Business Owners SBR Standardized Business Reporting SC Statistics Canada SEC Securities and Exchange Commission xv
PREPUBLICATION COPY, UNCORRECTED PROOFS SIC Standard Industrial Classification SMART specific, measurable, assignable, realistic, and time-bound SN Statistics Netherlands SQ-CLASS Business and Professional Classification Survey SRS simple random sampling SSN Social Security Number STSRS stratified simple random sampling SU single unit SUSB Statistics of United States Businesses TQRR total quantity response rate UN United Nations UO Unfilled Orders URR unit response rate VS value of shipments XBRL Extended Business Reporting Language xvi