Definition of a Federal Statistical Agency
Establishment of a Federal Statistical Agency
Principles for a Federal Statistical Agency
• Relevance to Policy Issues
• Credibility Among Data Users
• Trust Among Data Providers
• Independence from Political and Other Undue External Influence
Practices for a Federal Statistical Agency
• A Clearly Defined and Well-Accepted Mission
• Necessary Authority to Protect Independence
• Continual Development of More Useful Data
• Openness About Sources and Limitations of the Data Provided
• Wide Dissemination of Data
• Cooperation with Data Users
• Respect for the Privacy and Autonomy of Data Providers
• Protection of the Confidentiality of Data Providers’ Information
• Commitment to Quality and Professional Standards of Practice
• An Active Research Program
• Professional Advancement of Staff
• A Strong Internal and External Evaluation Program
• Coordination and Collaboration with Other Statistical Agencies
DEFINITION OF A FEDERAL STATISTICAL AGENCY
A federal statistical agency is a unit of the federal government whose principal function is the compilation and analysis of data and the dissemination of information for statistical purposes.1
• The unit is generally recognized as a distinct entity. It may be located within a cabinet-level department or an independent agency, or it may be an independent agency.
• Compilation may include direct collection of data from individuals organizations, or establishments through surveys or the acquisition of information from other sources, such as administrative records maintained by government agencies to operate a program, datasets available from the private sector, or data gleaned from selected Internet websites.
• Analysis may take various forms. It includes methodological research to improve the quality and usefulness of data. It also includes substantive analysis—for example, developing indicators from one or more data series, developing models to produce estimates, making projections, interpreting data, and explaining differences among statistics obtained by different methods, such as surveys and administrative records. An analysis by a statistical agency does not advocate policies or take partisan positions.
• Dissemination means making information available to the public, to the executive branch, and to Congress.
• Statistical purposes include description, evaluation, analysis, inference, and research. Statistical purposes relate to descriptions of groups and exclude any interest in or identification of an individual person or economic unit. For these purposes, a statistical agency may collect data directly from providers, or it may obtain data from other sources, but it does not use these data for administrative, regulatory, or law enforcement purposes. The data are used solely to describe and analyze statistical patterns, trends, and relationships involving groups of individuals or other units.
1 The U.S. Office of Management and Budget (2007:33364) provides a similar definition of a statistical agency: “An agency or organizational unit of the executive branch whose activities are predominantly the collection, compilation, processing, or analysis of information for statistical purposes.”
ESTABLISHMENT OF A FEDERAL STATISTICAL AGENCY
Statistics that are publicly available from government agencies are essential for a nation to advance the economic well-being and quality of life of its people. Public policy makers are best served by statistics that are relevant for policy decisions, accurate, timely, and credible. Individuals, households, businesses, academic institutions, and other organizations rely on high-quality, publicly available data as the basis for informed decisions on a wide variety of issues. Moreover, the effective operation of a democratic system of government depends on the unhindered flow of statistical information that citizens can use to assess government actions and for other purposes. Federal statistical agencies are established to be a credible source of relevant, accurate, and timely statistics in one or more subject areas that are available to the public and policy makers.
“Relevant statistics” are statistics that measure things that matter to policy making, program implementation, monitoring, and evaluation, and public understanding. Relevance requires concern for providing data that help users meet their current needs for decision making and analysis, as well as anticipating future needs. “Accurate statistics” are statistics that match the phenomena being measured and do so in repeated measurements. Accuracy requires proper concern for consistency across geographic areas and across time, as well as for statistical measures of errors in the data. “Timely satistics” are those that are known close in time to the phenomena they measure. Timeliness requires concern for issuing data as frequently as is needed to reflect important changes in what is being studied, as well as disseminating data as soon as practicable after they are collected. “Credibility” requires concern for both the reality and appearance of impartiality and independence from political and other undue external influence. Credibility also requires that agencies follow such practices as making their data and the information that users need to work with the data readily available to all. It is the primary mission of agencies in the federal statistical system to strive to ensure the relevance, accuracy, timeliness, and credibility of statistical information.
Reasons to establish a statistical agency include:
• the opportunity to achieve higher data quality and greater efficiency of statistical production through a consolidated and professional activity,
• the need for ongoing, up-to-date information in a subject area that extends beyond the scope of individual operating units, possibly involving other departments or agencies,
• the need to protect the confidentiality of responses of data providers,both individuals and organizations, and
• the need for data series that are independent—not subject to control by policy makers or regulatory or enforcement agencies and readily available on an equal basis to all users.
The principles and practices for a federal statistical agency that are reviewed in this report pertain to individual agencies as separate organizational entities in the context of a decentralized system for providing federal statistics. Historically, the response of the U.S. government to needs for information to support new federal responsibilities in such areas as agriculture, education, labor, health, science, energy, criminal justice, and transportation has been to create a separate statistical unit in the relevant cabinet department or independent agency. As a consequence, the United States now has one of the most decentralized statistical systems of any modern nation. This report does not comment on the advantages or disadvantages of the U.S. system nor compare it with other models for organizing government statistics. It discusses the critical importance of ensuring that federal statistical agencies coordinate and collaborate with each other and with other agencies on a range of activities, describes the coordinating role of the U.S. Office of Management and Budget (OMB), and reviews some mechanisms for interagency collaboration.
PRINCIPLES FOR A FEDERAL STATISTICAL AGENCY
Principle 1: Relevance to Policy Issues
A federal statistical agency must be in a position to provide objective, accurate, and timely information that is relevant to issues of public policy.
A statistical agency must be knowledgeable about the issues and requirements of public policy and federal programs and able to provide information that is relevant to policy and program needs. In establishing priorities for statistical programs for this purpose, a statistical agency must work closely with the users of such information in the executive branch, Congress, and elsewhere.
Statistical agencies must also provide objective, accurate, and timely information on the subject area(s) in their purview that is useful to a broad
range of private- and public-sector users as well as the public. To establish priorities for such information, a statistical agency must engage with a broad spectrum of users in state and local governments, businesses, academia, and other sectors.
Principle 2: Credibility Among Data Users
A federal statistical agency must have credibility with those who use its data and information.
It is essential that a statistical agency strive to maintain credibility for itself and for its data. Few data users are in a position to verify the completeness and accuracy of statistical information; they must rely on an agency’s reputation as a credible source of accurate and useful statistics.
Credibility derives from the respect and trust of users in the statistical agency and its data. Such respect results not only from an agency’s production of data that merit acceptance as relevant, accurate, timely, and free from political and other undue external influence, but also from many aspects of an agency’s policies and practices. Key among these are wide dissemination of data on an equal basis to all users; openness about the sources and processes used to produce data and the limitations of the data; commitment to quality and professional practice; a strong internal and external evaluation program to assess and improve an agency’s data systems; a willingness to understand and strive to meet user needs, even though users may not clearly articulate their needs; and a posture of respect and trust in the users of an agency’s data.
Principle 3: Trust Among Data Providers
A federal statistical agency must have the trust of those whose information it obtains.
Data providers, such as respondents to surveys and custodians of administrative records, must be able to rely on the word of a statistical agency that the information they provide about themselves or others needs to be collected and will be used only for the purposes that the agency has described. Importantly, data providers must be able to trust that a statistical agency will honor its pledges to protect the confidentiality of their responses. Such protection, in particular, precludes the use of individually
identifiable information collected and maintained by a statistical agency under a pledge of confidentiality—whatever its source—for any administrative, regulatory, or law enforcement purpose.
The trust of data providers is also achieved by respecting their privacy and autonomy.2 Such respect requires that an agency minimize the intrusiveness of questions and the time and effort to respond to them to the extent that is compatible with the agency’s requirements for information. An agency’s data collection staff should take care to treat respondents with courtesy and appreciation for their time.
Respect also requires that an agency provide sufficient information for the provider to make an informed decision about whether to supply the requested data, including the intended uses of the data being collected, their relevance for important public purposes, and the extent of confidentiality protection that will be provided.
Principle 4: Independence from Political and Other Undue External Influence
A federal statistical agency must be independent from political and other undue external influence in developing, producing, and disseminating statistics.
To fulfill its mission to provide objective, useful, accurate, and timely information, a statistical agency must not only be distinct from those parts of a department that carry out administrative, regulatory, law enforcement, or policy-making activities, but it also must have a widely acknowledged position of independence from political and other undue external influences and the necessary authority to protect independence.3 It must be able to execute its mission without being subject to pressures to advance a political agenda. It must be impartial and avoid even the appearance that its collection, analysis, and reporting processes might be manipulated for political
2 We take the term “autonomy” from the 1978 report of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, known as the Belmont Report: “To respect autonomy is to give weight to autonomous persons’ considered opinions and choices.” Available: http://archive.org/details/belmontreporteth00unit [February 2013].
3 A statistical agency actively works to obtain a broad range of external input to develop its programs; “undue external influences” are those that seek to undermine an agency’s impartiality and professional judgment.
purposes or that individually identifiable data collected under a pledge of confidentiality might be turned over for administrative, regulatory, or law enforcement purposes. Independence from any undue outside influence is an essential element of credibility with data users and the public so that they maintain confidence in the accuracy and objectivity of a statistical agency’s data. It is also essential for trust among data providers so that they continue to be willing to cooperate with agency requests.
PRACTICES FOR A FEDERAL STATISTICAL AGENCY
The effective operation of a federal statistical agency must begin with a clearly defined and well-accepted mission. With this prerequisite, effective operation involves a wide range of practices: necessary authority to protect independence, continual development of more useful data, openness about sources and limitations of the data provided, wide dissemination with ample documentation of data, cooperation with data users, respect for privacy and autonomy of data providers, protection of confidentiality of providers’ information, commitment to quality and professional standards of practice, an active research program, professional advancement of staff, a strong internal and external evaluation program, and coordination and collaboration with other statistical agencies.
Practice 1: A Clearly Defined and Well-Accepted Mission
An agency’s mission should include responsibility for all elements of its programs for providing statistical information—determining sources of data, measurement methods, efficient methods of data collection and processing, and appropriate methods of analysis—and ensuring the public availability not only of the data, but also of documentation and explanation of the methods used to obtain and process the data and their quality. The mission should include the responsibility for continually assessing information needs and priorities through proactive engagement with policy makers and other users of its data. The mission should also include the responsibility for identifying, evaluating, implementing, documenting, and explaining new ways to meet user needs, such as by the establishment, modification, or discontinuance of a survey or census or by the implementation of another method of data collection, such as extracting information from administrative records, private-sector data, or selected relevant Internet sources that meet quality standards.
Practice 2: Necessary Authority to Protect Independence
Protection from political or other undue outside influence requires that a statistical agency have the necessary authority for professional decisions on the scope, content, and frequency of data compiled, analyzed, and disseminated within the limits of budgetary resources, departmental requirements, review by OMB, and congressional mandates. It should also have authority over selection and promotion of professional, technical, and operational staff; processing, storage, and maintenance of the data that it collects; and the timing and content of data releases, including accompanying public announcements and documentation, without prior external clearance. An agency’s leaders and qualified technical staff should have authority to speak about the agency’s statistics before Congress, with congressional staff, and before public bodies. Such authority may come from legislation, OMB directives (which carry over from one administration to another), or policies and practices that are communicated by agency leadership to political appointees.4
An agency’s independence is enhanced by adhering to fixed schedules that are announced in advance for the public release of important statistical indicators to prevent even the appearance of manipulation of release dates for political purposes.5 Independence is also fostered by an agency’s maintaining a clear distinction between statistical information and policy interpretations of such information by executive branch officials and having dissemination policies that foster regular, frequent release of statistical findings and any data limitations to the public through the traditional media, the Internet, and other appropriate means. To bolster public credibility with regard to an agency’s independence, an agency’s website should include a clear description of the procedures it follows to protect against undue external influence in such matters as data dissemination.
4 Thirteen agencies have adopted and made accessible a “Statement of Commitment to Scientific Integrity by Principal Statistical Agencies” (see Appendix A and http://bls.gov/bls/integrity.htm [February 2013]), which affirms their commitment to the principles and practices enunciated in National Research Council (2009c), OMB statistical policy directives, and guidelines required by the Information Quality Act. This statement and these supporting documents uphold the necessity for a statistical agency to have authority to protect its independence from undue outside influence.
Practice 3: Continual Development of More Useful Data
Statistical agencies should continually look to improve their data systems to provide information that is accurate, timely, and relevant for changing public policy and data user needs. They should also continually seek to improve the cost-effectiveness of their programs for collecting, analyzing, and disseminating statistical information.
There are many ways for an agency to achieve these goals:
• Establish a multifaceted program of data collection from individuals, households, businesses, and other organizations to provide relevant information to meet different data needs. Such a program could include one-time surveys on special topics; repeated surveys of cross-sections of the population that provide regularly updated statistics; and longitudinal surveys that track people, firms, and institutions over time in order to analyze the antecedents and consequences of changes in their circumstances.
• Seek opportunities to combine data from multiple surveys or to integrate survey data with other kinds of data, with appropriate safeguards for protection of confidentiality and the maintenance of quality standards. When separate datasets are collected and analyzed in such a manner that they may be used together, the value of the resulting information and the efficiency of obtaining it may be greatly enhanced.
• Use administrative records that are maintained by government agencies for program operations to improve the quality and cost-effectiveness of some kinds of statistics. Such uses could include producing data series derived from one or more administrative datasets, using administrative records to improve the quality of imputations for missing data in surveys or to adjust survey responses for misreporting or population undercoverage, and combining administrative and survey data in models to produce estimates with improved accuracy for small geographic areas or small population groups.
• Explore the use of other sources (e.g., data from selected Internet sources or private-sector transactions) to improve the relevance and timeliness of some information. For example, an Internet source might provide the basis for timely information, which is later revised on the basis of data from a survey or administrative records, or it might provide the basis for an additional indicator that is not otherwise readily available. Care must always be taken to evaluate a source of data before deciding to use it, perhaps initially on an experimental basis, and to fully explain the source and its limitations.
• Share technical information and ideas with other statistical agencies. Such sharing can stimulate the development of innovative data collection, analysis, and dissemination methods that improve the accuracy and timeliness of information and the efficiency of data operations.
Practice 4: Openness About Sources and Limitations of the Data Provided
A statistical agency should be open about the strengths and limitations of its data, taking as much care to understand and explain how its statistics may fall short of accuracy as it does to produce accurate data. Data releases from a statistical program should be accompanied by a full description of the purpose of the program; the methods and assumptions used for data collection, processing, and reporting; what is known and not known about the quality and relevance of the data; sufficient information for estimating variability in the data; appropriate methods for analysis that take account of variability and other sources of error; and the results of research on the methods and data.
When problems are found in a previously released statistic that could affect its use, an agency should issue a correction promptly and publicly. It should also consider maintaining an online list of previous corrections to assist both new and long-standing users. Generally, an agency should be proactive in seeking ways to alert known and likely users of the data about the nature of a problem and the appropriate corrective action that it is taking or that users should take.
Practice 5: Wide Dissemination of Data
A statistical agency should strive continually for the widest possible dissemination of the data it compiles in formats that are widely accessible. Data dissemination should be timely, and information should be made readily available on an equal basis to all users. Also, measures should be taken to ensure that data are preserved and accessible for use in future years.
There are many elements of an effective dissemination program:
• An established publications policy should describe, for a data collection program, the types of reports and other data releases to be made available, the formats to be used, the audience to be served, and the frequency of release.
• A variety of avenues for data dissemination should be chosen to reach as broad a public as reasonably possible—including, but not limited to, an agency’s Internet website, government depository libraries, conference exhibits and programs, newsletters and journals, email address lists, social media and blogs, and the traditional media for regular communication of major findings.
• Data should be released in a variety of forms, including printed reports, easily accessible website displays and databases, public-use microdata, and other publicly available computer-readable files, so that the information can be accessed by users with varying skills and needs for data retrieval and analysis. All data releases should be suitably processed to protect confidentiality and accompanied by careful and complete documentation, including explanatory material to assist users in appropriate interpretation. Particularly for complex databases, user training should be provided through such forums as webinars, online tutorials, and sessions at appropriate conferences.
• For research and other statistical purposes, an agency should provide access to relevant information that is not publicly available through restricted access modes that protect confidentiality. Such modes include protected research data centers, remote monitored online access systems, and licensing of datasets to individual researchers.
• Policies should be in place for the preservation of data that guide what data to retain and how they are to be archived and made accessible for future secondary analysis.
Practice 6: Cooperation with Data Users
A statistical agency shows cooperation with data users by facilitating their access to and ability to use data through well-designed websites and other dissemination vehicles, careful and complete documentation, and user training adapted to varying skills and needs. In addition, a statistical agency should seek input from users on every aspect of its programs. The goal is to make its data as relevant, accurate, timely, and accessible as possible to a broad range of users. It should:
• seek advice on data concepts, content, processing, estimation, products, and documentation from a wide spectrum of data users, as well as from professional and technical subject-matter and methodological experts, using a variety of formal and informal means of communication that are appropriate to the types of input sought;
• seek advice on its statistical programs and priorities from external groups, including those with relevant subject-matter and technical expertise; and
• widely disseminate its responses to those who have provided input.
In developing and implementing new methods or data sources to produce statistical information, it is particularly important to reach out to policy makers and other key data users so that they understand an agency’s criteria and decision process for the new methods or data. Statistics that are based on models (for example, for small geographic areas) or that use nontraditional data sources will likely require an explanation of their benefits and limitations that is more extensive than is usually provided. Reaching out to policy makers and other key data users when new data sources or methods are in a developmental stage can help in identifying and responding to users’ concerns and earning their acceptance of the resulting data products.
Practice 7: Respect for the Privacy and Autonomy of Data Providers
To maintain a relationship of respect and trust with survey participants and other data providers, a statistical agency should respect their privacy and minimize the burden imposed on them. Two key data collection practices demonstrate an agency’s respect for and fair treatment of data providers:
• Prior to collection of information, data providers should be informed of the purposes of data collection and the anticipated uses of the information, the expected burden of participation, whether their participation is mandatory or voluntary, and, if voluntary, using appropriate informed consent procedures to obtain their participation.
• The data collection method should minimize—to the extent possible and consistent with the need for the data—the intrusiveness of questions and the time and effort needed to respond. For items that may be perceived as burdensome, an agency should provide an explanation of their purpose.
In addition, agencies can recognize the value of respondents’ participation in data collection programs by accurately representing the statistical information they provide and by making it widely available on an equal basis to all.
Practice 8: Protection of the Confidentiality of Data Providers’ Information
To earn the respect and trust of data subjects and other data providers, it is essential for a statistical agency to protect the confidentiality of the information it collects for statistical purposes. An agency should have policies and procedures to maintain the confidentiality of data—whether collected directly in surveys or obtained from administrative records or other sources—so that individual data collected under a pledge of confidentiality cannot be used for administrative, regulatory, law enforcement, or any other nonstatistical purpose. As part of confidentiality protection, an agency should have the authority to manage the storage of confidential microdata on secure servers that are controlled by the agency and not by a department-wide information technology system. A statistical agency should also have policies and procedures to inform data providers of the manner and level of confidentiality protection and the kinds of research and analysis that will be allowed with the data.
Practice 9: Commitment to Quality and Professional Standards of Practice
A statistical agency should:
• keep abreast of and use modern statistical theory and sound statistical practice in all technical work;
• document concepts, definitions, data collection methodologies, and measures of uncertainty and discuss possible sources of error in reports and other data releases to the public;
• develop strong staff expertise in the disciplines relevant to its mission, in the theory and practice of statistics, and in data collection, processing, analysis, and dissemination techniques;
• develop an understanding of the validity and accuracy of its data and convey the resulting measures of quality to users in ways that are comprehensible to nonexperts;
• maintain quality assurance programs to improve data quality and to improve the processes of compiling, editing, and analyzing data; and
• develop a strong and continuous relationship with appropriate professional organizations in the fields of statistics and relevant subject-matter areas.
Practice 10: An Active Research Program
A statistical agency should have a research program that is relevant to its activities. Because a small agency may not be able to afford an appropriate research program, agencies should collaborate and share research results and methods. Agencies can also augment their staff resources for research by using outside experts through consulting or other arrangements as appropriate.
Several elements should be part of a statistical agency’s research program:
• Research should be conducted on the substantive issues for which the agency’s data are compiled. Such research should be conducted not only to provide useful objective analytical results, but also as a means to identify potential improvements to the content of the data, suggest improvements in the design and operation of the data collection, and provide fuller understanding of the limitations of the data.
• An agency’s program should include research to evaluate and improve statistical methods, including the identification and creation of new statistical measures; improved methods for analyzing reporting and other errors in the data; ways to reduce the time and effort requested of respondents; and means to improve the timeliness, accuracy, and efficiency of data collection, analysis, and dissemination procedures.
• Research should be conducted to understand and estimate new sources of risk to confidentiality protection and to enhance mechanisms for access to data in ways that guard against disclosure.
• Research should be conducted to understand how the agency’s information is used, in order to make the data more relevant to policy concerns and more useful for policy research and decision making.
Practice 11: Professional Advancement of Staff
A statistical agency should recruit, develop, and support professional staff who are committed to the highest standards of quality work, professional practice, and professional ethics. To develop and maintain a high-caliber staff, a statistical agency must recruit qualified people with relevant skills for efficient and effective operations, including analysts in fields relevant to its mission (e.g., demographers, economists), statistical methodologists who specialize in data collection and analysis, and other skilled
staff (e.g., computer specialists). To retain and make the most effective use of its staff, an agency should provide opportunities for work on challenging projects in addition to more routine, production-oriented assignments. An agency’s personnel policies, supported with significant resources, should enable staff to extend their technical capabilities through appropriate professional and developmental activities, such as attendance and participation in professional meetings, participation in relevant training programs, rotation of assignments, and involvement in collaborative activities with other statistical agencies.
An agency should also seek opportunities to reinforce the commitment of its staff to ethical standards of practice. Such standards are the foundation of an agency’s credibility as a source of relevant, accurate, and timely information obtained through fair treatment of data providers and data users.
Practice 12: A Strong Internal and External Evaluation Program
Statistical agencies should have regular, ongoing programs of evaluation for major statistical programs and program components and for their overall portfolio of programs. Regular formal reviews of major data collection programs and their components should consider, among other topics, how to produce relevant, accurate, and timely data in the most cost-effective manner possible and evaluate whether there are ways to improve cost-effectiveness by combining data from multiple sources. Regular formal reviews of an agency’s portfolio should consider ways to reduce duplication, fill gaps, and adjust priorities so that the suite of programs remains as relevant as possible to the information needs of policy makers and the public given the available resources. Such evaluations should include internal reviews by staff and external reviews by independent groups.
Practice 13: Coordination and Collaboration with Other Statistical Agencies
A statistical agency should actively seek opportunities to collaborate with other statistical agencies to enhance the value of its own information and that of other agencies in the federal statistical system. Although agencies differ in their subject-matter focus, there is overlap in their missions and a common interest in serving the public need for credible, relevant, accurate, and timely statistics gathered as efficiently and fairly as possible.
When possible and appropriate, federal statistical agencies should collaborate not only with each other, but also with state and local statistical agencies in providing data for subnational areas. Federal statistical agencies should also collaborate with foreign and international statistical agencies to exchange information on both data and methods and to develop appropriate common classifications and procedures to promote international comparability of information.
Such collaborative activities as integrating data compiled by different statistical agencies, standardizing concepts and measures, sharing data among agencies, and identifying ways to reduce unneeded duplication invariably require effort to overcome differences in agency missions and operations. Yet with constrained budgets and increasing demand for more relevant, accurate, and timely statistical information, the importance of proactive collaboration and coordination among statistical agencies cannot be overstated. To achieve the most effective integration of their work for the public good, agencies must be willing to take a long view and to strive to accommodate other agencies.
The rewards can be data that are more efficiently obtained and more relevant to policy concerns. Another reward can be a stronger, more effective statistical system as a whole. To achieve these rewards, statistical agencies need to act as partners, not only in the development of statistical information for public use, but also for the entire panoply of statistical activities, including the definition and updating of concepts and classifications and the continual improvement of measurement methods, analytical tools, means for confidentiality protection, and modes of data dissemination. Statistical agencies, working with OMB, also need to be continually vigilant to refine, disseminate, and inculcate the highest standards of professional practice and policies in such areas as privacy and confidentiality protection, data release schedules, and scientific integrity—standards that are critical for credibility with the providers and users of their information.