Principles and Practices for a Federal Statistical Agency: Fourth Edition (2009)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Part I: Principles and Practices for a Federal Statistical Agency Definition of a Federal Statistical Agency Establishment of a Federal Statistical Agency Principles for a Federal Statistical Agency • Relevance to Policy Issues • Credibility Among Data Users • Trust Among Data Providers • A Strong Position of Independence Practices for a Federal Statistical Agency • A Clearly Defined and Well-Accepted Mission • Continual Development of More Useful Data •  Openness About Sources and Limitations of the Data Provided • Wide Dissemination of Data • Cooperation with Data Users • Fair Treatment of Data Providers •  Commitment to Quality and Professional Standards of Practice • An Active Research Program • Professional Advancement of Staff • A Strong Internal and External Evaluation Program •  Coordination and Cooperation with Other Statistical Agencies NOTE: Part I is a summary statement of principles and practices for an effective statistical agency. Part II, Com- mentary, further explains, defines, and illustrates the topics in Part I.  

 principles and practices for a federal statistical agency DEFINITION OF A FEDERAL STATISTICAL AGENCY A federal statistical agency is a unit of the federal government whose principal function is the compilation and analysis of data and the dissemination of information for statistical purposes. The theory and methods of the discipline of statistics and related fields and the practice of the profession of statistics are brought to bear on the compilation of data, on producing information from the data, and on dis- seminating that information. • The unit is generally recognized as a distinct entity. It may be located within either a cabinet-level department or an independent agency, or it could itself be an independent agency. • Compilation may include direct collection of data from individuals, organizations, or establishments or the acquisition of information from administrative records. It may also include assembling information from a variety of sources, including other statistical agencies, in order to produce an integrated data series, such as the national income and product accounts. • Analysis may take various forms. It includes methodological research to improve the quality and usefulness of data. It also includes substantive analysis—for example, developing indicators, modeling, making projec- tions, interpreting data, and explaining relationships among survey statistics at various levels of aggregation and other variables. Analysis by a statistical agency does not advocate policies or take partisan positions. • Dissemination means making information available to the public, to the executive branch, and to Congress. • Statistical purposes include description, evaluation, analysis, infer- ence, and research. For these purposes, a statistical agency may collect data from individuals, establishments, or other organizations directly, or it may obtain data from administrative records, but it does not use these data for administrative, regulatory, or law enforcement purposes. Statistical purposes relate to descriptions of groups and exclude any interest in or identifica- tion of an individual person or economic unit. The data are used solely to The U.S. Office of Management and Budget (2007:33364) provides a similar defini- tion of a statistical agency: “an agency or organizational unit of the executive branch whose activities are predominantly the collection, compilation, processing, or analysis of information for statistical purposes.” 

part i: principles and practices  describe and analyze statistical patterns, trends, and relationships involving groups of persons or other units. ESTABLISHMENT OF A FEDERAL STATISTICAL AGENCY Statistics that are publicly available from government agencies are es- sential for a nation to advance the economic well‑being and quality of life of its people. Public policy makers are best served by statistics that are relevant for policy decisions, accurate, timely, and credible. Individuals, households, corporations, academic institutions, and other organizations rely on high- quality, publicly available data as the basis for informed decisions on a wide variety of issues. Even more, the operation of a democratic system of government depends on the unhindered flow of statistical information that citizens can use to assess government actions and for other purposes. Federal statistical agencies are established to be a credible source of relevant, accurate, and timely statistics in one or more subject areas that are available to the public and policy makers. “Relevant statistics” are statistics that measure things that matter to policy making and public understanding. Relevance requires concern for providing data that help users meet their current needs for decision making and analysis, as well as anticipating future needs. “Accurate statistics” are statistics that match the phenomena being measured and do so in repeated measurements. Accuracy requires proper concern for consistency across geographic areas and across time, as well as for statistical measures of errors in the data. “Timely statistics” are those that are known close in time to the phenomena they measure. Timeliness requires concern for issuing data as frequently as is needed to reflect important changes in what is being studied, as well as disseminating data as soon as practicable after they are collected. “Credibility” requires concern for both the reality and appearance of impar- tiality and independence from political control. It is the primary mission of agencies in the federal statistical system to work to ensure the relevance, accuracy, timeliness, and credibility of statistical information. There are four major reasons to establish a statistical agency: (1) The opportunity to achieve higher data quality and greater ef- ficiency of statistical production through a consolidated and more highly professional activity. (2) The need for ongoing, up-to-date information in a subject area 

 principles and practices for a federal statistical agency that extends beyond the scope of individual operating units, possibly involv- ing other departments or agencies. (3) The need to protect the confidentiality of responses of data pro- viders, both individuals and organizations. (4) The need for data series that are independent—not subject to control by policy makers or regulatory or enforcement agencies and equally available to all users. The principles and practices for a federal statistical agency that are reviewed in this report pertain to individual agencies as separate orga- nizational entities in the context of a decentralized system for providing federal statistics. Historically, the response of the U.S. government to new information needs has been to create separate statistical units, so that the United States now has one of the most decentralized statistical systems of any modern nation. This report does not comment on the advantages or disadvantages of the U.S. system nor compare it with other models for organizing government statistics. It discusses the need for federal statisti- cal agencies to coordinate and cooperate with other agencies on a range of activities, describes the coordinating role of the U.S. Office of Management and Budget (OMB), and reviews some mechanisms for interagency col- laboration. Appendixes A and B provide detailed information on the U.S. statistical system. PRINCIPLES FOR A FEDERAL STATISTICAL AGENCY Principle 1: Relevance to Policy Issues A federal statistical agency must be in a position to provide objec- tive information that is relevant to issues of public policy. A statistical agency must be knowledgeable about the issues and requirements of public policy and federal programs and able to provide objective information that is relevant to policy and program needs. Objec- tive information is information that is as accurate and comprehensible as possible and is not designed to promote a particular policy position or group interest. In establishing priorities for statistical programs for this purpose, a statistical agency must work closely with the users of such information in the executive branch, the Congress, and interested nongovernmental groups. 

part i: principles and practices  Statistical agencies must also provide objective information on the subject area(s) in their purview that is useful to a broad range of private- and public-sector users as well as the public. To establish priorities for such information, a statistical agency must maintain contact with a broad spec- trum of users in the business sector, academia, state and local governments, and elsewhere. Principle 2: Credibility Among Data Users A federal statistical agency must have credibility with those who use its data and information. It is essential that a statistical agency strive to maintain credibility for itself and for its data. Few data users are in a position to verify the complete- ness and accuracy of statistical information; they must rely on an agency’s reputation as a credible source of accurate and useful statistics. Credibility derives from the respect and trust of users for the statistical agency and its data. These qualities come from more than producing data that merit acceptance as objective, accurate, and timely. Respect and trust are earned when an agency exhibits openness about sources and limitations of the data provided, a willingness to understand and strive to meet user needs, even those users may not well articulate, and a posture of respect and trust in the users of its data. To have credibility, an agency must be free—and must be perceived to be free—of political interference and policy advocacy. Also important for credibility is for an agency to follow such practices as wide dissemination of data on an equal basis to all users, openness about the data provided, and commitment to quality and professional practice, as well as a strong internal and external evaluation program to assess and improve its data systems. Principle 3: Trust Among Data Providers A federal statistical agency must have the trust of those whose in- formation it obtains. Data providers, such as respondents to surveys and custodians of ad- ministrative records, must be able to rely on the word of a statistical agency that the information they provide about themselves or others will be used only for statistical purposes. An agency earns the trust of its data providers 

 principles and practices for a federal statistical agency by appropriately protecting the confidentiality of responses. Such protec- tion, in particular, precludes the use of individually identifiable information maintained by a statistical agency—whether derived from survey responses or another agency’s administrative records—for any administrative, regula- tory, or law enforcement purpose. Trust of data providers is also achieved by respecting their privacy. Such respect requires that an agency minimize the time and effort of people who provide information and inform them of the intended uses of the informa- tion. Trust of data providers is also achieved by successfully conveying to them the relevance of the data being collected for important public pur- poses. Data providers must be convinced not only that the information they provide will be kept confidential, but also that the information is intended for effective, beneficial public use. Principle 4: A Strong Position of Independence A federal statistical agency must have a strong position of indepen- dence within the government. To be credible and unhindered in its mission to provide objective, useful, high-quality information, a statistical agency must not only be dis- tinct from those parts of a department that carry out law enforcement and policy-making activities, but also have a widely acknowledged position of independence. It must be able to execute its mission without being subject to pressures to advance a political agenda. It must be impartial and avoid even the appearance that its collection, analysis, and reporting processes might be manipulated for political purposes or that individually identifiable data might be turned over for administrative, regulatory, or law enforce- ment purposes. A strong degree of independence is reflected in such practices as adher- ence to predetermined schedules in the public release of important statistical indicators; control over statistical press releases; control over processing of the data that an agency collects; authority for professional decisions over the scope, content, and frequency of data compiled, analyzed, and dissemi- nated; and maintaining a clear distinction between statistical information and policy interpretation. Without the credibility that comes from a strong degree of independence, users may lose confidence in the accuracy and objectivity of a statistical agency’s data, and data providers may become less willing to cooperate with agency requests. 

part i: principles and practices  PRACTICES FOR A FEDERAL STATISTICAL AGENCY The effective operation of a federal statistical agency must begin with a clearly defined and well-accepted mission. With this prerequisite, effective operation involves a wide range of practices: continual development of more useful data, openness about sources and limitations of the data provided, wide dissemination of data, cooperation with data users, fair treatment of data providers, commitment to quality and professional standards of prac- tice, an active research program, professional advancement of staff, a strong internal and external evaluation program, and coordination and coopera- tion with other statistical agencies. Practice 1: A Clearly Defined and Well-Accepted Mission An agency’s mission should include responsibility for all elements of its programs for providing statistical information—determining sources of data, measurement methods, efficient methods of data collection and processing, and appropriate methods of analysis—and ensuring the public availability not only of the data, but also of documentation of the methods used to obtain the data and their quality. The mission should include the re- sponsibility for assessing information needs and priorities and ways to meet those needs, such as by the establishment, modification, or discontinuance of a survey, census, or other method of data collection, such as extracting information from administrative records. Practice 2: Continual Development of More Useful Data Statistical agencies should continually look to improve their data systems to provide information that is accurate, timely, and relevant for changing public policy needs. They should also continually seek to improve the efficiency of their programs for collecting, analyzing, and disseminating statistical information. Ways for an agency to achieve these goals include the following: • Seeking opportunities to combine data from multiple surveys or to integrate data from surveys with data from administrative records, with appropriate safeguards for confidentiality. When separate data sets are col- lected and analyzed in such a manner that they may be used together, the 

 principles and practices for a federal statistical agency value of the resulting information and the efficiency of obtaining it may be greatly enhanced. • Sharing technical information and ideas with other statistical agen- cies. Such sharing can stimulate the development of innovative data collec- tion, analysis, and dissemination methods that improve the accuracy and timeliness of information and the efficiency of data operations. • Establishing a multifaceted data collection program to provide rel- evant information for different types of data needs. Such a program could include one-time surveys on special topics, repeated surveys of cross-sections of the population that provide regularly updated statistics, and longitudinal surveys that track people, firms, and institutions over time and make it pos- sible to analyze the causes and effects of changes in their circumstances. • Using administrative records as part of an agency’s data collection program. Practice 3: Openness About Sources and Limitations of the Data Provided A statistical agency should be open about its data and their strengths and limitations, taking as much care to understand and explain how its statistics may fall short of accuracy as it does to produce accurate data in the first place. Data releases from a statistical program should be accom- panied by a full description of the purpose of the program; the methods and assumptions used for data collection, processing, and reporting; what is known and not known about the quality and relevance of the data; suffi- cient information for estimating variability in the data; appropriate methods for analysis that take account of variability and other sources of error; and the results of research on the methods and data. When problems are found in a previously released statistic that could affect its use, an agency should issue a correction promptly and publicly. An agency should be proactive in seeking ways to alert known and likely users of the data about the nature of the problem and the appropriate cor- rective action. Practice 4: Wide Dissemination of Data A statistical agency should strive for the widest possible dissemination of the data it compiles. Data dissemination should be timely and public. 

part i: principles and practices  Also, measures should be taken to ensure that data are preserved and acces- sible for use in future years. Elements of an effective dissemination program include the following: • An established publications policy that describes, for a data collec- tion program, the types of reports and other data releases to be made avail- able, the audience to be served, and the frequency of release. • A variety of avenues for data dissemination, chosen to reach as broad a public as reasonably possible. Channels of dissemination include, but are not limited to, an agency’s Internet web site, government depository libraries, conference exhibits and programs, newsletters and journals, e-mail address lists, and the media for regular communication of major findings. • Release of data in a variety of formats, including printed reports, easily accessible web site displays and databases, public-use microdata and other publicly available computer-readable files, so that the information can be accessed by users with varying skills and needs for data retrieval and analysis. All data releases should be suitably processed to protect confiden- tiality, with careful and complete documentation. • For research and other statistical purposes, access to relevant infor- mation that is not publicly available through restricted access modes that protect confidentiality. Such modes include protected research data centers, remote monitored online access for special tabulations and analyses, and licensing of individual researchers to allow them to use confidential data on their desktop computers under stringent arrangements to ensure that no one else can access the information. • Procedures for release of information that preclude actual or per- ceived political interference. In particular, the content and timing of the public release of data should be the responsibility of the statistical agency, and the agency or unit that produces the data should publish in advance and meet release schedules for important indicators to prevent even the ap- pearance of manipulation of release dates for political purposes. • Policies for the preservation of data that guide what data to retain and how they are to be archived for future secondary analysis. Practice 5: Cooperation with Data Users A statistical agency should consult with a broad spectrum of users of its data in order to make its products more useful. It should 

10 principles and practices for a federal statistical agency • seek advice on data concepts, statistical methods, and data products from data users as well as from other professional and technical subject- matter and methodological experts, using a variety of formal and infor- mal means of communication that are appropriate to the types of input sought; • seek advice on its statistical programs and priorities from exter- nal groups, including those with relevant subject-matter and technical expertise; • provide wide access to data while maintaining appropriate safe- guards for the confidentiality of individual responses; and • provide equal access to data to all users. Practice 6: Fair Treatment of Data Providers To maintain a relationship of respect and trust with data subjects and other data providers, a statistical agency should observe fair information practices. Such practices include the following: • Policies and procedures to maintain the confidentiality of data, whether collected directly or obtained from administrative record sources, and to inform data providers of the manner and level of protection. • Policies and procedures to inform data providers of the purposes of data collection and the anticipated uses of the information, whether their participation is mandatory or voluntary, and, if voluntary, using appropriate informed consent procedures to obtain their information. • Respecting the privacy of respondents by minimizing the contribu- tion of time and effort asked of them, consistent with the purposes of the data collection activity. • Recognizing the value of respondents’ participation in data collec- tion programs by accurately representing the statistical information they provide and by making it widely and equally available to all. • Seeking, to the extent practicable, input from respondents, as well as others, on useful information to collect and disseminate and on the best ways to obtain information. 

part i: principles and practices 11 Practice 7: Commitment to Quality and Professional Standards of Practice A statistical agency should • use modern statistical theory and sound statistical practice in all technical work; • develop strong staff expertise in the disciplines relevant to its mis- sion, in the theory and practice of statistics, and in data collection, process- ing, analysis, and dissemination techniques; • develop an understanding of the validity and accuracy of its data and convey the resulting measures of quality to users in ways that are com- prehensible to nonexperts; • maintain quality assurance programs to improve data quality and to improve the processes of compiling, editing, and analyzing data; • develop a strong and continuous relationship with appropriate pro- fessional organizations in the fields of statistics and relevant subject-matter areas; and • document concepts, definitions, data collection methodologies, and measures of uncertainty and discuss possible sources of error in reports and other data releases. Practice 8: An Active Research Program A statistical agency should have a research program that is integral to its activities. Because smaller agencies may not be able to afford as extensive a research program as larger agencies, agencies should share research results and methods. Agencies can also augment their staff resources for research by obtaining the services of experts not on the agency’s staff through consulting or other arrangements as appropriate. The research program of a statistical agency should include the following: • Research on the substantive issues for which the data were com- piled. Such research should be conducted not only to provide useful objective analytical results, but also as a means to identify potential im- provements to the content of the data, suggest improvements in the design and operation of the data collection, and provide fuller understanding of the limitations of the data. 

12 principles and practices for a federal statistical agency • Research to evaluate and improve statistical methods, in particular the identification and creation of new statistical measures and the develop- ment of improved methods for analyzing errors in data that are due not only to sampling variability, but also to other sources. Research should also be conducted on ways to reduce the time and effort requested of respondents and to improve the timeliness, accuracy, and efficiency of data collection, analysis, and dissemination procedures. • Research to understand how the agency’s information is used, in order to make the data more relevant to policy concerns and more useful for policy research and decision making. Practice 9: Professional Advancement of Staff A statistical agency should recruit, develop, and support professional staff who are committed to the highest standards of quality work and profes- sional practice. An agency’s staff should also be committed to the highest standards of professional ethics with regard to maintaining the agency’s credibility as an objective, independent source of accurate and useful infor- mation obtained through fair information practices. To develop and maintain a high-caliber staff, a statistical agency must recruit qualified people with the relevant skills for its efficient and effective operation, including analysts in fields relevant to its mission (e.g., demog- raphers, economists), statistical methodologists who specialize in data col- lection and analysis, and other specialized staff (e.g., computer specialists). To retain and make the most effective use of its staff, an agency should provide opportunities for work on challenging projects in addition to more routine, production-oriented assignments. An agency’s personnel policies, supported with significant resources, should enable staff to extend their technical capabilities through appropriate professional and developmental activities, such as attendance and participation in professional meetings, participation in relevant training programs, and rotation of assignments. An agency should also seek opportunities to reinforce the commitment of its staff to ethical standards of practice. Practice 10: A Strong Internal and External Evaluation Program Statistical agencies should have regular, ongoing programs of evaluation for major statistical programs and program components and for the agency’s portfolio of programs as a whole. Regular formal reviews of major data 

part i: principles and practices 13 collection programs and their components should consider, among other topics, how to produce the highest quality data possible given the available resources. Regular formal reviews of an agency’s portfolio should consider how to produce the most relevant information possible for policy makers and the public. Such evaluations should include internal reviews by staff and external reviews by independent bodies. Practice 11: Coordination and Cooperation with Other Statistical Agencies A statistical agency should seek opportunities to cooperate with other statistical agencies to enhance the value of its own information and that of other agencies in the federal statistical system. Although agencies differ in their subject-matter focus, there is overlap in their missions and a common interest in serving the public need for credible, high-quality statistics gath- ered as efficiently and fairly as possible. When possible and appropriate, federal statistical agencies should coop- erate not only with each other, but also with state and local statistical agen- cies in the provision of data for subnational areas. Federal statistical agencies should also cooperate with foreign and international statistical agencies to exchange information on both data and methods and to develop appropriate common classifications and procedures to promote international comparabil- ity of information. Such cooperative activities as integrating data compiled by different statistical agencies invariably require effort to overcome differences in agency missions and operations. But the rewards are data more relevant to policy concerns and a stronger statistical system as a whole. For these reasons, statistical agencies must act as partners to one another, not only in the development of data, but also for the entire panoply of statistical activities, including definitions, concepts, measurement methods, analyti- cal tools, professional practice, dissemination modes, means to protect the confidentiality of responses, and ways to advance the effective use of statisti- cal information.