Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Prepublication Copy--Uncorrected Proofs PRINCIPLES and PRACTICES For a Federal Statistical Agency SEVENTH EDITION Brian A. Harris-Kojetin and Constance F. Citro, Editors Committee on National Statistics Division of Behavioral and Social Sciences and Education A Consensus Study Report of
Prepublication Copy--Uncorrected Proofs THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001 This activity was supported by Grant No. SES-1560294 between the National Acad- emies of Sciences, Engineering, and Medicine and the National Science Foundation, which provides support for the work of the Committee on National Statistics from a consortium of federal agencies. Also supporting the Committeeâs work are a cooperative agreement with the U.S. Department of Agriculture, to which several agencies contribute, and individ- ual agreements with agencies in the U.S. Departments of Commerce, Health and Human Services, Housing and Urban Development, and Treasury. Any opinions, findings, con- clusions, or recommendations expressed in this publication do not necessarily reflect the views of any organization or agency that provided support for the project. International Standard Book Number-13: 978-0-309-XXXXX-X International Standard Book Number-10: 0-309-XXXXX-X Digital Object Identifier: https://doi.org/10.17226/25885 Additional copies of this publication are available from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu. Copyright 2021 by the National Academy of Sciences. All rights reserved. Printed in the United States of America Suggested citation: National Academies of Sciences, Engineering, and Medicine. 2021. Principles and Practices for a Federal Statistical Agency, Seventh Edition. Washington, DC: The National Academies Press. https://doi.org/10.17226/25885.
Prepublication Copy--Uncorrected Proofs The National Academy of Sciences was established in 1863 by an Act of Con- gress, signed by President Lincoln, as a private, nongovernmental institution to advise the nation on issues related to science and technology. Members are elected by their peers for outstanding contributions to research. Dr. Marcia Mc- Nutt is president. The National Academy of Engineering was established in 1964 under the charter of the National Academy of Sciences to bring the practices of engineering to ad- vising the nation. Members are elected by their peers for extraordinary contribu- tions to engineering. Dr. John L. Anderson is president. The National Academy of Medicine (formerly the Institute of Medicine) was established in 1970 under the charter of the National Academy of Sciences to advise the nation on medical and health issues. Members are elected by their peers for distinguished contributions to medicine and health. Dr. Victor J. Dzau is president. The three Academies work together as the National Academies of Sciences, En- gineering, and Medicine to provide independent, objective analysis and advice to the nation and conduct other activities to solve complex problems and inform public policy decisions. The National Academies also encourage education and research, recognize outstanding contributions to knowledge, and increase public understanding in matters of science, engineering, and medicine. Learn more about the National Academies of Sciences, Engineering, and Medi- cine at www.nationalacademies.org.
Prepublication Copy--Uncorrected Proofs Consensus Study Reports published by the National Academies of Sciences, En- gineering, and Medicine document the evidence-based consensus on the studyâs statement of task by an authoring committee of experts. Reports typically in- clude findings, conclusions, and recommendations based on information gath- ered by the committee and the committeeâs deliberations. Each report has been subjected to a rigorous and independent peer-review process and it represents the position of the National Academies on the statement of task. Proceedings published by the National Academies of Sciences, Engineering, and Medicine chronicle the presentations and discussions at a workshop, symposium, or other event convened by the National Academies. The statements and opin- ions contained in proceedings are those of the participants and are not endorsed by other participants, the planning committee, or the National Academies. For information about other products and activities of the National Academies, please visit www.nationalacademies.org/about/whatwedo.
Prepublication Copy--Uncorrected Proofs COMMITTEE ON THE 7TH EDITION OF PRINCIPLES AND PRACTICES FOR A STATISTICAL AGENCY ROBERT M. GROVES, (CHAIR), Office of the Provost, Georgetown University ANNE C. CASE, Woodrow Wilson School of Public and International Affairs, Princeton University, emeritus JANET M. CURRIE, Woodrow Wilson School of Public and International Affairs, Princeton University DONALD A. DILLMAN, Social Science and Economic Research Center, Washington State University DIANA FARRELL, JPMorgan Chase Institute, Washington, DC ROBERT GOERGE, Chapin Hall at The University of Chicago HILARY HOYNES, Goldman School of Public Policy, University of California, Berkeley DANIEL KIFER, Department of Computer Science and Engineering, The Pennsylvania State University SHARON LOHR, School of Mathematical and Statistical Sciences, Arizona State University, emerita THOMAS L. MESENBOURG, Retired, U.S. Census Bureau SARAH M. NUSSER, Center for Survey Statistics and Methodology, Iowa State University JEROME P. REITER, Department of Statistical Science, Duke University JUDITH A. SELZTER, Department of Sociology, University of California, Los Angeles C. MATTHEW SNIPP, School of the Humanities and Sciences, Stanford University JEANNETTE WING, Data Science Institute, Columbia University BRIAN A. HARRIS-KOJETIN, Director CONSTANCE F. CITRO, Senior Scholar â v
Prepublication Copy--Uncorrected Proofs COMMITTEE ON NATIONAL STATISTICS ROBERT M. GROVES, (CHAIR), Office of the Provost, Georgetown University LAWRENCE D. BOBO, Department of Sociology, Harvard University ANNE C. CASE, Woodrow Wilson School of Public and International Affairs, Princeton University, emeritus MICK P. COUPER, Institute for Social Research, University of Michigan JANET M. CURRIE, Woodrow Wilson School of Public and International Affairs, Princeton University DIANA FARRELL, JPMorgan Chase Institute, Washington, DC ROBERT GOERGE, Chapin Hall at The University of Chicago ERICA L. GROSHEN, The ILR School, Cornell University HILARY HOYNES, Goldman School of Public Policy, University of California, Berkeley DANIEL KIFER, Department of Computer Science and Engineering, The Pennsylvania State University SHARON LOHR, School of Mathematical and Statistical Sciences, Arizona State University, emerita JEROME P. REITER, Department of Statistical Science, Duke University JUDITH A. SELZTER, Department of Sociology, University of California, Los Angeles C. MATTHEW SNIPP, School of the Humanities and Sciences, Stanford University ELIZABETH A. STUART, Johns Hopkins Bloomberg School of Public Health JEANNETTE WING, Data Science Institute, Columbia University BRIAN A. HARRIS-KOJETIN, Director CONSTANCE F. CITRO, Senior Scholar â vi
Prepublication Copy--Uncorrected Proofs Dedication This seventh edition of Principles and Practices for a Federal Sta- tistical Agency is dedicated to Lauren D. Harris-Kojetin, beloved wife of Brian A. Harris-Kojetin, and Joseph F. Citro, beloved husband of Constance F. Citro. Lauren and Joe both passed away in 2020 while this edition of P&P was being prepared. They provided the love, support, and light that enriched our lives beyond measure and enabled us to fulfill our careers. vii
Prepublication Copy--Uncorrected Proofs
Prepublication Copy--Uncorrected Proofs Preface The Committee on National Statistics (CNSTAT) is a standing unit of the National Academies of Sciences, Engineering, and Medicine, established in 1972 to provide an independent, objective resource for evaluating and improving the work of the decentralized federal statistical system. Under the terms of the 1863 congressional charter to the National Academy of Sciences to provide advice to the government on scientific and technical matters, CNSTAT has assessed a wide range of statistical methods and data sources for information that the public and policy mak- ers need. It has evaluated numerous federal censuses, surveys, and other statistical programs and addressed a range of statistical policy issues, such as the independence required for statistical agencies to be credible with policy makers, data users, and the public, regardless of persuasion or party. The origins of Principles and Practices for a Federal Statistical Agency (commonly known as P&P or the âpurple bookâ) date back to the late 1980s and early 1990s. During legislative debates over the (un- successful) establishment of a Bureau of Environmental Statistics and the (successful) establishment of a Bureau of Transportation Statistics, congressional staff asked CNSTAT for advice on what constitutes an ef- fective federal statistical agency. CNSTAT prepared a document of high- level guidance, launching the first edition of P&P in 1992. It defined and discussed reasons for the establishment of a statistical agency, identified three fundamental principles for an effective statistical agency (relevance ix
Prepublication Copy--Uncorrected Proofs to policy, credibility with data users, and trust of data providers), and identified 11 practices to enable a statistical agency to put these principles into action and adhere to them. The 1992 document served a useful purpose for the federal statisti- cal community. However, it has also needed to be updated to respond to changes in the political, economic, social, and technological environment for statistical agency work. CNSTAT released the second edition of P&P in 2001, and it has released subsequent editions every four years to be available to new appointees and others at the beginning of a presidential term of office. CNSTAT adopted this schedule recognizing that officials in the various agencies that house federal statistical agencies are not al- ways cognizant about how to be trusted as credible sources of objective, relevant, accurate, and timely statistics. P&P is designed to assist them, as well as the statistical agenciesâ leadership and staff, to be fully aware of the standards and ideals that are fundamental to the agenciesâ work. Stakeholders, Congress, the Government Accountability Office, and the Office of Management and Budget have also found P&P useful for such purposes as reviewing agency programs and setting standards. CNSTAT has made some changes to the principles and practices over time. The first three editions included three principles. In light of threats to independence, the fourth edition elevated statistical agency indepen- dence from a practice to a fourth principle. The number of practices rose from 11 to 13 across the first six editions: conclusions and recommenda- tions in CNSTAT study reports led to adding or rewording some practices. This seventh edition contains five principles: CNSTAT added a new principle on Continual Improvement and Innovation, which has been a strong theme in a number of practices, to recognize its importance for the effective functioning of statistical agencies in the 21st century. We also streamlined the list of practices, reducing their number to 10 (from 13) by combining a few that were closely related. In Part I, we discuss the value of national statistics, uses of statistics for the public good, and the role of federal statistical agencies. In Part II we explicate and comment on each principle, and in Part III we do the same for each practice. Three appen- dixes (available online) follow: two provide updated information on leg- islation and regulations that govern federal statistics and the organization of the federal statistical system; a new appendix provides information on international frameworks relevant for U.S. statistical agencies. x
Prepublication Copy--Uncorrected Proofs We thank the many people who contributed their time and expertise to the preparation of this report, including all the current members of CNSTAT. We are most appreciative of their cooperation and assistance. We are particularly grateful to the CNSTAT staff, including director Brian Harris-Kojetin, senior scholar Constance F. Citro, and program associate Rebecca Krone. We are also indebted to many others who offered valu- able comments and suggestions, too numerous to mention. This Consensus Study Report was reviewed in draft form by indi- viduals chosen for their diverse perspectives and technical expertise. The purpose of this independent review is to provide candid and critical com- ments that will assist the National Academies of Sciences, Engineering, and Medicine in making each published report as sound as possible and to ensure that it meets the institutional standards for quality, objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We thank the following individuals for their review of this report: Mary Jo Hoeksema, Government and Public Affairs, Population Association of America/Association of Population Centers Thomas A. Louis, Department of Biostatistics (retired), Bloomberg School of Public Health, Johns Hopkins University Colm A. O'Muircheartaigh, Harris School of Public Policy Studies, The University of Chicago Steven Pierson, Science Policy, American Statistical Association Nancy A. Potok, Chief Statistician (retired), Office of Management and Budget Katherine K. Wallman, Chief Statistician (retired), Office of Management and Budget Although the reviewers listed above provided many constructive com- ments and suggestions, they were not asked to endorse the conclusions or recommendations of this report nor did they see the final draft before its release. The review of this report was overseen by Alicia Carriquiry, Department of Statistics, Iowa State University. She was responsible for making certain that an independent examination of this report was carried out in accordance with the standards of the National Academies and that all review comments were carefully considered. xi
Prepublication Copy--Uncorrected Proofs Responsibility for the final content rests entirely with the authoring com- mittee and the National Academies. Finally, we thank the following federal agencies, which support the Committee on National Statistics directly and through a grant from the National Science Foundation, a cooperative agreement from the National Agricultural Statistics Service, and several individual contracts: â¢ National Science Foundation: Methodology, Measurement, and Statistics Program; National Center for Science and Engineering Statisticsâ â¢ Social Security Administration: Office of Research, Evaluation, and Statistics â¢ U.S. Department of Agriculture: Economic Research Service, National Agricultural Statistics Serviceâ â¢ U.S. Department of Commerce: Bureau of Economic Analysis, U.S. Census Bureauâ â¢ U.S. Department of Education: National Center for Education Statisticsââ â¢ U.S. Department of Energy: Energy Information Administrationâ â¢ U.S. Department of Health and Human Services: Agency for Healthcare Research and Quality, National Center for Health Statistics, National Institute on Agingâ â¢ U.S. Department of Housing and Urban Development: Office of Policy Development and Researchâ â¢ U.S. Department of Justice: Bureau of Justice Statistics â¢ U.S. Department of Labor: Bureau of Labor Statisticsâ â¢ U.S. Department of Transportation: Bureau of Transportation Statistics â¢ U.S. Department of the Treasury: Statistics of Income Division, Internal Revenue Service. Without their support and their commitment to improving the national statistical system, the committee work that is the basis of this report would not have been possible. Robert M. Groves, Chair Committee on National Statistics xii
Prepublication Copy--Uncorrected Proofs Contents SUMMARY I. INTRODUCTION The Value of National Statistics in the United States Federal Statistical Agencies and Units The Committee on National Statistics and This Report II. PRINCIPLES Principle 1: Relevance to Policy Issues and Society Principle 2: Credibility Among Data Users and Stakeholders Principle 3: Trust Among the Public and Data Providers Principle 4: Independence from Political and Other Undue External Influence Principle 5: Continual Improvement and Innovation III. PRACTICES 1. A Clearly Defined and Well-Accepted Mission 2. Necessary Authority and Procedures to Protect Independence 3. Commitment to Quality and Professional Standards of Practice 4. Professional Advancement of Staff 5. An Active Research Program 6. Strong Internal and External Evaluation Processes for an Agencyâs Statistical Programs 7. Coordination and Collaboration with Other Statistical Agencies 8. Respect for Data Providers and Protection of Their Data 9. Dissemination of Statistical Products that Meet Usersâ Needs 10. Openness about Sources and Limitations of the Data Provided REFERENCES xiii
Prepublication Copy--Uncorrected Proofs APPENDIXES (available in online version only) A. Legislation and Regulations That Govern Federal Statistics LEGAL AUTHORITY OF OMB OVER FEDERAL STATISTICS Background, 1933â1980 Paperwork Reduction Act, 1980âPresent The Foundations of Evidence-Based Policymaking Act of 2018 OMB STATISTICAL POLICY DIRECTIVES Statistical Policy Directive No. 1âFundamental Responsibilities of Federal Statistical Agencies and Recognized Statistical Units Statistical Policy Directive No. 2âStandards and Guidelines for Statistical Surveys Statistical Policy Directive No. 3âCompilation, Release, and Evaluation of Principal Federal Economic Indicators Statistical Policy Directive No. 4âRelease and Dissemination of Statistical Products Produced by Federal Statistical Agencies Statistical Policy Directive No. 7âMetropolitan Statistical Areas Statistical Policy Directive No. 8âNorth American Industry Classification System Statistical Policy Directive No. 10âStandard Occupational Classification Statistical Policy Directive No. 14âDefinition of Poverty for Statistical Purposes Statistical Policy Directive No. 15âStandards for Maintaining, Collecting, and Presenting Federal Data on Race and Ethnicity OMB GUIDANCE FOR SURVEYS, FEDERAL STATISTICS, AND EVIDENCE BUILDING PRA Implementation Guidance for Surveys, and Other Agency Information Collection Activities North American Product Classification System 2014 Guidance for Providing and Using Administrative Data for Statistical Purposes (M-14-06) 2015 Guidance on Improving Statistical Activities through Interagency Collaboration (M-15-15) Phase 1 Implementation of the Foundations for Evidence-Based xiv
Prepublication Copy--Uncorrected Proofs Policymaking Act of 2018: Learning Agendas, Personnel, and Planning Guidance (M-19-23) Phase 4 Implementation of the Foundations for Evidence-Based Policymaking Act of 2018: Program Evaluation Standards and Practices (M-20-14) OMB GUIDANCE ON CONFIDENTIALITY AND PRIVACY PROTECTION Privacy Act of 1974 Federal Policy for the Protection of Human Subjects, 45 Code of Federal Regulations (CFR) 46, Subpart A (âCommon Ruleâ), as Revised in 2017 1997 Order Providing for the Confidentiality of Statistical Information Confidential Information Protection and Statistical Efficiency Act Privacy Impact Assessments: E-Government Act of 2002, Section 208 Federal Information Security Management Act of 2002 Federal Information Technology Acquisition Reform Act of 2014 Federal Cybersecurity Enhancement Act of 2015 OMB GUIDANCE RELATED TO INFORMATION QUALITY AND THE FEDERAL DATA STRATEGY The Information Quality Act of 2000 2004 OMB Final Information Quality Bulletin for Peer Review Government Performance and Results Modernization Act of 2010 2010 Office of Science and Technology Policy Memorandum on Scientific Integrity 2013 OSTP Memorandum on Increasing Access to the Results of Scientific Research Improving Implementation of the Information Quality Act (M-19-15) Federal Data StrategyâA Framework for Consistency (M-19-18) Federal Data Strategy Mission Federal Data Strategy Principles Federal Data Strategy Practices Federal Data Strategy Action Plan 2020 xv
Prepublication Copy--Uncorrected Proofs INTERAGENCY COUNCIL ON STATISTICAL POLICY DOCUMENTS 2002 Federal Statistical Agency Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Disseminated Information Principles for Modernizing Production of Federal Statistics by Interagency Council on Statistical Policy Principles for Using Non-Statistical Data for Statistical Purposes B. Organization of the Federal Statistical System OVERVIEW Brief History and Structure of the U.S. Federal Statistical System Budget U.S. OFFICE OF MANAGEMENT AND BUDGET PRINCIPAL STATISTICAL AGENCIES Bureau of Economic Analysis Bureau of Justice Statistics Bureau of Labor Statistics Bureau of Transportation Statistics Census Bureau Economic Research Service Energy Information Administration National Agricultural Statistics Service National Center for Education Statistics National Center for Health Statistics National Center for Science and Engineering Statistics Office of Research, Evaluation, and Statistics, Social Security Administration Statistics of Income Division, Internal Revenue Service OTHER AGENCIES WITH STATISTICAL OFFICIALS Department of Defense Department of Homeland Security Department of Housing and Urban Development Department of the Interior Department of State xvi
Prepublication Copy--Uncorrected Proofs Department of Veterans Affairs Environmental Protection Agency General Services Administration National Aeronautics and Space Administration Nuclear Regulatory Commission Office of Personnel Management Small Business Administration U.S. Agency for International Development OMB-RECOGNIZED STATISTICAL UNITS UNDER CIPSEA Center for Behavioral Health Statistics and Quality Microeconomic Surveys Section National Animal Health Monitoring System Program Unit C. Some International Frameworks Relevant for U.S. Federal Statistics UNITED NATIONS FUNDAMENTAL PRINCIPLES OF OFFICIAL STATISTICS THE COMMON QUALITY FRAMEWORK OF THE EUROPEAN STATISTICAL SYSTEM Institutional Environment Statistical Processes Statistical Output QUALITY FRAMEWORK FOR OECD STATISTICAL ACTIVITIES Quality Dimensions Core Values for OECD Statistics UK CODE OF PRACTICE FOR STATISTICS Trustworthiness: Confidence in the people and organisations that produce statistics and data Quality: Data and methods that produce assured statistics Value: Statistics that support societyâs needs for information GENERIC STATISTICAL BUSINESS PROCESS MODEL xvii
Prepublication Copy--Uncorrected Proofs