This section comments on most of the topics in the principles and practices; the comments are offered to explain, illustrate, or further define the statement of principle in Part I.
DEFINITION OF A FEDERAL STATISTICAL AGENCY
A federal statistical agency is a unit of the federal government whose principal function is the compilation and analysis of data and the dissemination of information for statistical purposes.
A statistical agency may be labeled a bureau, center, division, or office or similar title, so long as it is recognized as a distinct entity. Statistical agencies have been established for several reasons: (1) to develop new information for an area of public concern (e.g., the Bureau of Labor Statistics, the National Center for Health Statistics); (2) to conduct large statistical collection and dissemination operations specified by law (e.g., the U.S. Census Bureau); (3) to compile and analyze statistics from sets of administrative records for policy purposes and public use (e.g., the Statistics of Income Division in the Internal Revenue Service); and (4) to develop broad and consistent estimates from a variety of statistical and administrative sources in accordance with a prespecified conceptual framework (e.g., the Bureau of Economic Analysis in the U.S. Department of Commerce). Once
established, many statistical agencies engage in all of these functions to varying degrees.
This definition of a federal statistical agency does not include many statistical activities of the federal government because they are not performed by distinct units, or because they do not result in the dissemination of statistics to others—for example, statistics compiled by the U.S. Postal Service to set rates or by the U.S. Department of Defense to test weapons (see National Research Council, 1998b, 2002b, 2003b, on statistics and testing for defense acquisition). Nor does it include agencies whose primary functions are the conduct or support of problem-oriented research, although their research may be based on information gathered by statistical means, and they may also sponsor important surveys, as do, for example, the National Institutes of Health, the Agency for Healthcare Research and Quality, and other agencies in the U.S. Department of Health and Human Services.
Finally, this definition of a statistical agency does not usually include agencies whose primary function is policy analysis and planning (e.g., the Office of Tax Analysis in the U.S. Department of the Treasury, the Office of the Assistant Secretary for Planning and Evaluation in the U.S. Department of Health and Human Services). Such agencies may collect and analyze statistical information, and statistical agencies, in turn, may perform some policy-related analysis (e.g., produce reports on trends in after-tax income or child care arrangements of families). However, to maintain credibility as an objective source of accurate, useful information, statistical agencies must be separate from units that are involved in developing policy and assessing policy alternatives.
The work of federal statistical agencies is coordinated through the Interagency Council on Statistical Policy (ICSP), created by the U.S. Office of Management and Budget (OMB) in the 1980s and authorized in statute in the 1995 reauthorization of the Paperwork Reduction Act. The ICSP is chaired by OMB and currently includes representation from 10 principal statistical agencies and from the statistical units in the Environmental Protection Agency, the Social Security Administration, the National Science Foundation, and the Internal Revenue Service (see Box 1).
Throughout the federal government, OMB recognizes more than 70 units and agencies that are not statistical agencies but that have annual budgets of $500,000 or more for statistical activities (U.S. Office of Management and Budget, 2004c:4-7). The principles for federal statistical agencies presented here should apply to other federal agencies that carry
Office of Management and Budget (OMB), Chair
Bureau of Economic Analysis (BEA), U.S. Department of Commerce
Bureau of Justice Statistics (BJS), U.S. Department of Justice
Bureau of Labor Statistics (BLS), U.S. Department of Labor
Bureau of Transportation Statistics (BTS), U.S. Department of Transportation
Census Bureau, U.S. Department of Commerce
Economic Research Service (ERS), U.S. Department of Agriculture
Energy Information Administration (EIA), U.S. Department of Energy
National Agricultural Statistics Service (NASS), U.S. Department of Agriculture
National Center for Education Statistics (NCES), U.S. Department of Education
National Center for Health Statistics (NCHS), U.S. Department of Health and Human Services
Office of Environmental Information, Environmental Protection Agency (EPA)
Office of Research, Evaluation, and Statistics, Social Security Administration (SSA)
Science Resources Statistics Division, National Science Foundation (NSF)
Statistics of Income Division (SOI), Internal Revenue Service, U.S. Department of the Treasury
out statistical activities, and they may find many of the detailed practices pertinent as well. Similarly, the principles and practices may be relevant to statistical units in state and local government agencies, and international audiences may also find them useful.
ESTABLISHMENT OF A FEDERAL STATISTICAL AGENCY
One of the most important reasons for establishing a statistical agency is to provide information that will allow for an informed citizenry. A democracy depends on an informed electorate. A citizen has a right to information that comes from a trustworthy, credible source and is relevant, accurate, and timely. Timely information of high quality is also critical to
policy analysts and decision makers in both the public and private sectors. (For more information on the purposes of official statistics, see the Fundamental Principles of Official Statistics of the United Nations Statistical Commission in Appendix A; see also United Nations Economic Commission for Europe, 2003; United Nations Statistical Commission, 2003.) Federal statistical agencies serve the key functions of providing a broad array of information to the public and policy makers and of ensuring the necessary quality and credibility of the data.
Commercial, nonprofit, and academic organizations in the private sector also provide useful statistical information, including data they collect themselves and data they acquire from government agencies and other data sources to which they add value. However, because the benefits of statistical information are shared widely throughout society and because it is often difficult to collect payments for these benefits, private markets are not likely to provide all of the data that are needed for public and private decision making or to make data as widely available as needed for important public purposes. Government statistical agencies are established to ensure that a broad range of information is publicly available. (See National Research Council, 1999b, for a discussion of the governmental role in providing public goods, or near public goods, such as research and data.)
The United States government collected and published statistics long before any distinct federal statistical agency was formed (see Duncan and Shelton, 1978; Norwood, 1995). The U.S. Constitution mandated the conduct of a decennial census of population beginning in 1790, and the census enumeration was originally conducted by U.S. marshals as one of their many duties. Legislation providing for the compilation of statistics on agriculture, education, and income was enacted by Congress in the 1860s. The Bureau of Labor (forerunner of the Bureau of Labor Statistics) was established by law in 1884 as a separate agency with a general mandate to respond to widespread public demand for information on the conditions of industrial workers. The Census Bureau was established as a permanent agency in 1902 to conduct the decennial census and related statistical activities.
Many federal statistical agencies that can trace their roots back to the 19th or early 20th century, such as the National Center for Education Statistics and the National Center for Health Statistics, were organized in their current form following World War II. Several relatively new agencies have since been established, including the Energy Information Administration, the Bureau of Justice Statistics, and, most recently (in 1991), the Bureau of Transportation Statistics. In every case, the agency itself, in
consultation with users of its information, has major responsibility for determining its specific statistical programs and for setting priorities. Initially, many of these agencies also had responsibilities for certain policy analysis functions for their department heads. More recently, policy analysis has generally been located in separate units that are not themselves considered to be statistical agencies, a separation that helps establish and maintain the credibility of statistical agencies as providers of data and analyses that are not designed for particular policy alternatives.
A statistical agency has at least two roles: (1) provider of the statistical information and analysis needed to inform policy making and program assessment by its own department and (2) source of national statistics for the public in its area of concern. It is sometimes difficult to keep these two roles distinct on policy-relevant statistics. An effective statistical agency, nevertheless, will frequently play a creative, not just reactive, role in the development of data needed for policy analysis. Sometimes federal statistical agencies play additional roles, such as monitor and consultant on statistical matters to other units within the same department (see e.g., National Research Council, 1985a) and collector of data on a reimbursable basis for other agencies.
There is no set rule or guideline for when it is appropriate to establish a separate federal statistical agency, carry on statistical activities within the operating units of departments and independent agencies, or contract for statistical services from existing federal statistical agencies or other organizations. Establishment of a federal statistical agency should be considered when one or more of the following conditions prevails:1
There is a need for information on an ongoing basis beyond the capacity of existing operating units, possibly involving other departments and agencies. Such needs may require coordinating data from various sources, initiating new data collection programs to fill gaps, or developing regularly updated time series of estimates.
There is a need, as a matter of credibility, to ensure that major data series are independent of policy makers’ control.
There is a need to establish the functional separation of statistical data that are collected for statistical purposes from identifiable data that may be used for administrative, regulatory, or law enforcement uses. Such separation, recommended by the Privacy Protection Study Commission (1977), bolsters a culture and practice of respect for privacy and protection of confidentiality. Functional separation is easier to maintain when the statistical data are compiled by a unit that is separate from operating units. The Confidential Information Protection and Statistical Efficiency Act of 2002 (CIPSEA) extended legal confidentiality protection to every statistical data collection by a federal agency, whether a statistical agency or other type of agency (see Appendix B). Nonetheless, functional separation of statistical data from other kinds of data is important because it makes promises of confidentiality protection more credible.
There is a need to emphasize the principles and practices of an effective statistical agency, for example, professional practice, openness about the data provided, and wide dissemination of data.
There is a need to encourage research and development of a broad range of statistics in a particular area of public interest or of government activity or responsibility.
There is a need to consolidate compilation, analysis, and dissemination of statistics in one unit to encourage high-quality performance, eliminate duplication, and streamline operations.
PRINCIPLES FOR A FEDERAL STATISTICAL AGENCY
Principle 1: A federal statistical agency must be in a position to provide information relevant to issues of public policy.
A statistical agency supplies information not only for the use of managers and policy makers in the executive branch and for legislative designers and overseers in Congress, but also for all those who require statistical information on public issues, whether the information is needed for purposes of production, trade, consumption, or participation in civic affairs. Just as a free enterprise economic system depends on the availability of economic information to all participants, a democratic political system depends on—and has a fundamental duty to provide—wide access to information on education, health, transportation, the economy, the environment, criminal justice, and other social concerns.
Federal statistical agencies are responsible for providing statistics on
conditions in a variety of areas. The resulting information is used both inside and outside the government not only to delineate problems and sometimes to suggest courses of action, but also to evaluate the results of government activity or lack of activity. The statistics provide much of the basis on which the government itself is judged. This role places a heavy responsibility on federal statistical agencies for impartiality and objectivity.
In order to provide information that is relevant to public issues, statistical agencies need to reach out to users of the data. Federal statistical agencies usually are in touch with the primary users in their own departments. Considerable energy and initiative are required to open avenues of communication more broadly to other current and potential users, including analysts and policy makers in other federal departments, state and local government agencies, academic researchers, private-sector organizations, organized constituent groups, the media, and Congress. Advisory committees are recommended as a means to obtain the views of users, as well as people with relevant technical expertise (see, e.g., National Research Council, 1993a). Many agencies have traditionally had such committees to advise them—examples include the American Statistical Association’s Committee on Energy for the Energy Information Administration, the Board of Scientific Counselors for the National Center for Health Statistics, and the Census Advisory Committee of Professional Associations for the Census Bureau. The Federal Economic Statistics Advisory Committee (FESAC), chartered in November 1999, provides substantive and technical advice to three agencies—the Bureau of Economic Analysis, the Bureau of Labor Statistics, and the Census Bureau—thereby providing an important crosscutting perspective on major economic statistics programs (Bureau of Labor Statistics, 2005a).
One frequently recommended method for alerting statistical agencies to emerging statistical information needs is for the agency’s own staff to engage in analysis of its data (Martin, 1981; Norwood, 1975; Triplett, 1991). For example, relevant analysis may use the agency’s data to examine correlates of key social or economic phenomena or to study the statistical error properties of the data. Such in-house analysis can lead to improvements in the quality of the statistics, to identification of new needs, to a reordering of priorities, and to closer cooperation and mutual understanding with policy analysis units. In its work for a policy analysis unit, a statistical agency describes conditions and possibly measures progress toward some previously identified goal, but it refrains from making policy recommendations. The distinction between statistical analysis and policy analysis
is not always clear, and a statistical agency will need to consider carefully the extent of policy-related activities that are appropriate for it to undertake.
Principle 2: A federal statistical agency must have credibility based on a relationship of mutual respect and trust with those who use its data and information.
Users of a statistical agency’s data must be able to trust that the data were collected and analyzed in an objective, impartial manner and that they are as reliable as the agency can make them. An agency should make every effort to provide accurate and credible statistics that will permit policy debates to be concerned about policy, not about the credibility of the data. Credibility is enhanced when an agency fully informs users of the strengths and weaknesses of the data, makes data available widely, and consults with users about priorities for data collection and analysis.
Principle 3: A federal statistical agency must have a relationship of mutual respect and trust with respondents who provide data and all data subjects whose information it obtains.
The statistics programs of the federal government rely in large part on information supplied by individuals and by organizations outside the federal government, such as state and local governments, businesses, and other organizations. Some of this information is required by law or regulation (such as employers’ wage reports), some of it is related to administration of government programs (such as information provided by benefit recipients), but much of it is obtained through the voluntary cooperation of respondents in statistical surveys. Even when response is mandatory, the cooperation of respondents reduces costs and likely promotes accuracy (see National Research Council, 1995b, 2004e). Important elements in encouraging such cooperation are that respondents believe that the data requested are important and legitimate for the government to collect, that they are being collected in an impartial, competent manner, and that the confidentiality of their responses will be protected.
In brief, trust in a statistical agency must be maintained. The agency must not be perceived as being swayed by political considerations. It must be perceived as working in the national interest, not the interest of a particular administration, and as taking a long view, balancing new data needs against the need for consistency with past data (Ryten, 1990). Respondent
trust also depends on providing respondents with realistic promises of confidentiality that the agency can reasonably expect to honor and then scrupulously honoring those promises. Finally, respondent trust depends on adopting practices that respect personal privacy, such as taking steps to minimize the intrusiveness of questions and the time and effort required to participate in a survey.
PRACTICES FOR A FEDERAL STATISTICAL AGENCY
Practice 1: A Clearly Defined and Well-Accepted Mission
A clear understanding of the mission of an agency, the scope of its statistical programs, and its authority and responsibilities is basic to planning and evaluating its programs and to maintaining credibility and independence from political control (National Research Council, 1986, 1997b). Some agency missions are clearly spelled out in legislation; other agencies have only very general legislative authority. On occasion, very specific requirements may be set by legislation or regulation.
Agencies should communicate their mission clearly to others. The use of the Internet is one means to publicize an agency’s mission to a broad audience and to provide related information, including enabling legislation, the scope of the agency’s statistical program, confidentiality provisions, operating procedures, and data quality guidelines. An agency’s mission should focus on the compilation, evaluation, analysis, and dissemination of statistical information. In addition, considerable and formal attention must be paid to setting statistical priorities (National Research Council, 1976). Advice from outside groups should be sought on the agency’s statistical program, on setting statistical priorities, on the statistical methods used, and on data products. Such advice may be sought in a variety of formal and informal ways, but it should be obtained from data users and providers as well as professional or technical experts in the subject-matter area and in statistical methods and procedures. A strong research program in the agency’s subject-matter field can assist in setting priorities and identifying ways to improve an agency’s statistical programs (Triplett, 1991).
Practice 2: A Strong Position of Independence
A statistical agency must be able to provide credible information that may be used to evaluate the program and policies of its own department or
the government as a whole. More broadly, a statistical agency must be a trustworthy source of objective, reliable information for decision makers, analysts, and others inside and outside the government who want to use statistics to understand present conditions, draw comparisons with the past, and help guide plans for the future. For these purposes, a strong position of independence for a statistical agency is essential. (See the Fundamental Principles of Official Statistics of the United Nations Statistical Commission in Appendix A.)
Statistical agency independence must be exercised in a broader framework. Legislative authority usually gives ultimate responsibility to the department rather than the statistical agency head. In addition, an agency is subject to the normal budgetary processes and to various coordinating and review functions of OMB, as well as the legislative mandates, oversight, and informal guidance of Congress.
Within this broader framework, a statistical agency must work to maintain its credibility as an impartial purveyor of information. In the long run, the effectiveness of an agency depends on its maintaining a reputation for impartiality; thus, an agency must be continually alert to possible infringements on its credibility and be prepared to argue strenuously against such infringements.
Independence of an agency head can be strengthened by the head’s being appointed for a fixed term by the president, with approval by the Senate, as is the case with the Bureau of Labor Statistics and the National Center for Education Statistics. The heads of the Bureau of Justice Statistics, the Census Bureau, and the Energy Information Administration are presidential appointees, but their terms are not fixed and usually end with a change of administration. In the case of a fixed term, it is desirable that it not coincide with the presidential term so that incumbents need not end their leadership with changes of administration and professional considerations may more easily predominate over political aims in the appointment process.
It is also desirable that a statistical agency head have direct access to the secretary of the department or the head of the independent agency in which the statistical agency is located. Such access allows the head to inform new secretaries about the appropriate role of a statistical agency and present the case for new statistical initiatives to the secretary directly. Among the agency heads with presidential appointments, such direct access currently is provided by legislation only for the Bureau of Labor Statistics.
It is desirable for a statistical agency to have its own funding appropria-
tion from Congress and not be dependent on allocations from budgets of other agencies that may be subject to reallocation.
These organizational aspects—appointment of the agency head by the president with approval by the Senate for a fixed term not coincident with that of the administration, direct access to the secretary of the agency’s department, and separate budgetary authority—are neither necessary nor sufficient for a strong position of independence for a statistical agency, but they facilitate such independence.2
Control over personnel actions, especially the selection and appointment of qualified professional staff, including senior executive career staff, is another aspect of independence. Agency staff reporting directly to the agency head should have formal education and deep experience in the substantive, methodological, operational, or management issues facing the agency as appropriate for their positions. In addition, professional qualifications are of the utmost importance for statistical agency heads, whether the profession is that of statistician or the subject-matter field of the statistical agency (National Research Council, 1997b). Relevant professional associations can be a source of valuable input on suitable candidates.
The authority to ensure that information technology systems fulfill the specialized needs of the statistical agency is also an aspect of independence. A statistical agency must be able to vouch for the integrity, confidentiality, and impartiality of the information collected and maintained under its authority so that it retains the trust of its data providers and data users. Such trust is fostered when a statistical agency has control over its information technology resources, and there is no opportunity or perception that policy, program, or regulatory agencies could gain access to records of individual respondents. A statistical agency also needs control over its information technology resources to support timely and accurate release of official statistics, which are often produced under stringent deadlines.
Authority to decide the scope and specific content of the data collected or compiled is yet another important element of independence. Most
statistical agencies have broad authority, limited by budgetary constraints, departmental interests, OMB review, and congressional mandates. In addition, the courts sometimes become involved in interpreting laws and regulations that affect statistical agencies, as in a number of issues concerning data confidentiality and Freedom of Information Act requests and in the issue of the use of sampling in the population census. Congress frequently specifies particular data that it wishes to be collected (e.g., by the National Agricultural Statistics Service in the U.S. Department of Agriculture, the National Center for Health Statistics in the U.S. Department of Health and Human Services) and, in the case of the decennial census, requires an opportunity to review the proposed questions before the forms are printed. The OMB Office of Information and Regulatory Affairs, under the Paperwork Reduction Act (and under the preceding Federal Reports Act), has the responsibility for designating a single data collection instrument for information wanted by two or more agencies. It also has the responsibility under the same act for reviewing all questionnaires and other instruments for the collection of data from 10 or more respondents.
The budgetary constraints on statistical agencies and OMB review of data collections are ongoing; the other pressures depend, in part at least, on the relations between a statistical agency and those who have supervisory or oversight functions. Agencies need to develop skills in communicating to oversight groups the need for statistical series and credibility in assessing the costs of statistical work. In turn, although it is standard practice for the secretary of a department or the head of an independent agency to have ultimate responsibility for all matters within the department or agency, the head of a statistical agency, for credibility, should be allowed full authority in professional and technical matters. For example, decisions to revise the methodology for calculating the consumer price index (CPI) or the gross domestic product (GDP) have been and are properly made by the relevant statistical agency heads.
Other aspects of independence that underscore a statistical agency’s credibility are important as well. Authority to release statistical information and accompanying materials (including press releases) without prior clearance by department policy officials is important so that there is no opportunity for or perception of political manipulation of any of the information. Authority for the statistical agency head and qualified staff to speak about the agency’s statistics before Congress, with congressional staff, and before public bodies is also important to bolster the agency’s standing.
When a statistical agency releases information publicly, a clear distinc-
tion should be made between the statistical information and any policy interpretations of such. Not even the appearance of manipulation for political purposes should be allowed. This is one reason that statistical agencies are required by OMB statistical policy directive number 3 to adhere to predetermined schedules for the public release of key economic indicators and take steps to ensure that no person outside the agency can gain access to such indicators before the official release time (see U.S. Office of Management and Budget, 1985). Predetermined schedules for the public release of other important social and economic indicators can also contribute to independence. When an agency modifies a customary release schedule for statistical purposes, it should announce and explain the change as far in advance as possible.
Practice 3: Continual Development of More Useful Data
Federal statistical agencies cannot be static. To provide information of continued relevance for public and policy use, they must continually anticipate data needs for future policy considerations and look for ways to develop data systems that can serve broad purposes. To improve the quality and timeliness of their information, they must keep abreast of methodological and technological advances and be prepared to implement new procedures in a timely manner. They must also continually seek ways to make their operations more efficient. Preparing for the future requires that agencies reevaluate existing data series, plan new data series as required, and be innovative and open in their consideration of ways to improve their programs. Because of the decentralized nature of the federal statistical system, innovation often requires cross-agency collaboration. Innovation also implies a willingness to implement different kinds of data collection efforts to answer different needs.
Integration of Data Sources
One way to increase the usefulness of survey data is to integrate them with data from other surveys or with data from administrative records, such as social program records. Such integration typically requires that several agencies work together.
For example, in the area of health care provider statistics, a study by a panel of the Committee on National Statistics (CNSTAT) concluded that no single survey was likely ever to meet all the criteria, address all the tech-
nical problems, or meet all users’ needs for data. In order to provide adequate information on the availability, financing, and quality of health care, a coordinated and integrated system of data collection activities involving several organizational entities was required (National Research Council and Institute of Medicine, 1992).
Similarly, a CNSTAT study on retirement income statistics concluded that some of the information that is essential for analysis of savings and retirement decisions and the effect of medical care use and expenditures on retirement income security is most efficiently and accurately obtained from existing administrative records (National Research Council, 1997a). To be useful for estimation, this information (e.g., Social Security earnings histories, Medicare and Medicaid benefits) must be linked to individual data that are available from such panel surveys as the Health and Retirement Study sponsored by the National Institute on Aging, the National Longitudinal Surveys of Labor Market Experience sponsored by the Bureau of Labor Statistics, and the Census Bureau’s Survey of Program Dynamics and Survey of Income and Program Participation. Similarly, linkage of employer survey data with administrative records could provide enhanced analysis and modeling capability at low marginal cost. In both instances, careful attention must be paid to the means by which the confidentiality of the linked data can be protected while allowing access for research use.
Sharing of Microdata
Another way to improve data quality and develop new kinds of information is for statistical agencies that collect similar information to share microdata records. For example, the sharing of business data would make it possible to evaluate reporting errors and the completeness of coverage of business firms in different surveys. Such sharing would also make it possible to develop more useful and accurate statistics on the nation’s economy while decreasing the reporting burden on business data providers. The Confidential Information Protection and Statistical Efficiency Act of 2002 (CIPSEA), subtitle B, for the first time in the nation’s history authorizes the sharing of business data among the three principal statistical agencies that produce the nation’s key economic statistics—the Bureau of Economic Analysis (BEA), the Bureau of Labor Statistics (BLS), and the U.S. Census Bureau. June 2003 saw the first formal proposal for data sharing under CIPSEA, which involved matching data from BEA’s international investment surveys with data from the Census Bureau’s Survey of Industrial Research and De-
velopment conducted for the National Science Foundation. The results helped BEA improve its survey sample frames and enabled the Census Bureau to identify companies that were not previously known to engage in research and development activities (U.S. Office of Management and Budget, 2004c:44-45).
The need to understand temporal changes in important social or economic events may call for the development of longitudinal surveys that track people, institutions, or firms over time. Developing longitudinal data (and general purpose cross-sectional data, as well) usually requires much coordination with policy research agencies, other statistical agencies, and academic researchers. Longitudinal data generally require more sophisticated methods for collection and analysis than data from repeated or one-time cross-sectional surveys. In addition, considerable time may be needed to produce useful data products for analyzing transitions and other dynamic characteristics of longitudinal samples (although production of useful cross-sectional products from longitudinal surveys need not take long). Yet data from longitudinal surveys are potentially very useful—sometimes they are the only means to answer important policy questions (see, e.g., National Research Council, 1997a, on data needs to inform retirement income policy, and National Research Council, 2001b, on data needs to evaluate welfare reform in an era of transition).
Historically, because statistical agencies are oriented toward the mission of their particular department, the longitudinal surveys they developed (and cross-sectional data activities as well) typically focused on subject matter and population groups (or other entities) that the department serves. For example, separate data sets are available on health characteristics of infants and children (and other age groups), educational characteristics for children and teenagers, and work force characteristics for adults. Yet surveys that follow individuals across such key transitions as from early childhood to school and from school to the labor force are important to consider (National Research Council, 1998a; National Research Council and Institute of Medicine, 2004). Examples of surveys that are designed for analysis of some kinds of transitions include the Early Childhood Longitudinal Study (ECLS), sponsored by the National Center for Education Statistics in collaboration with other agencies, and the National Longitudinal Surveys of Youth (NLSY79, NLSY97), sponsored by the Bureau of Labor Statis-
tics. The ECLS includes two cohorts of children, one of kindergartners in 1998 who will be followed through twelfth grade and another of babies born in 2001 who will be followed through first grade (National Center for Education Statistics, 2005). The NLSY includes two cohorts of young people, one of people aged 14-22 in 1979, who are being interviewed every other year, and the other of people aged 12-17 in 1997, who are being interviewed annually (Bureau of Labor Statistics, 2005b).
Another area in which it is important for statistical agencies to be innovative concerns the methods used for data collection, processing, estimation, analysis, and dissemination. Agencies need to investigate new or modified methods that have the potential to improve the accuracy and timeliness of their data and the efficiency of their operations. Careful evaluation of new methods is required to assess their benefits and costs in comparison with current methods and to determine effective implementation strategies, including the development of methods for bridging time series before and after a change in procedures.
For example, experience with the use of computer-assisted interviewing techniques, which many agencies have adopted for data collection, has identified benefits. It has also identified challenges for the timely provision of data and documentation that require continued research to develop solutions that maximize the gains from these techniques (see National Research Council, 2003e).
Similarly, agencies need to carefully evaluate their growing use of the Internet, which has become a standard vehicle for data dissemination and is increasingly being used for data collection. Internet dissemination facilitates the timely availability of data to a broad audience and provides a valuable tool for users to learn of related data sets from other agencies. However, it poses challenges in several areas, such as how best to provide information on data quality and appropriate use of the data to an audience that spans a wide range of analytical skills and understanding. Internet data collection poses challenges in such areas as sample design, questionnaire design, and protecting data confidentiality. It also requires careful evaluation of the effects on the quality of responses in comparison with traditional data collection modes (telephone, mail, personal interview).
Practice 4: Openness About Sources and Limitations of the Data Provided
An important means to instill credibility and trust among data users and data providers is for an agency to operate in an open manner with regard to the sources and the limitations of its data. Openness requires that an agency provide a full description of its data with acknowledgment of any uncertainty and a description of the methods used and assumptions made. Agencies should provide to users reliable indications of the kinds and amounts of statistical error to which the data are subject (see Brackstone, 1999; Federal Committee on Statistical Methodology, 2001a; see also President’s Commission on Federal Statistics, 1971). Some statistical agencies have developed detailed quality profiles for some of their major series, such as those developed for the American Housing Survey (Chakrabarty, 1996), the Residential Energy Consumption Survey (Energy Information Administration, 1996), the Schools and Staffing Survey (Kalton et al., 2000), and the Survey of Income and Program Participation (U.S. Census Bureau, 1998). Earlier, the Federal Committee on Statistical Methodology (1978c) developed a quality profile for employment as measured in the Current Population Survey. These profiles have proved helpful to experienced users and agency personnel responsible for the design and operation of major surveys and data series (see National Research Council, 1993a).
Openness about data limitations requires much more than providing estimates of sampling error. In addition to a discussion of aspects that statisticians recognize as nonsampling errors, such as coverage errors, nonresponse, measurement errors, and processing errors, a description of the concepts used and how they relate to the major uses of the data is desirable. Descriptions of the shortcomings of and problems with the data should be provided in sufficient detail to permit the user to take them into account in the analysis and interpretation of the data.
Openness means that a statistical agency should describe how decisions on methods and procedures were made for a data collection program. It is important to be open about research conducted on methods and data and other factors that were weighed in a decision.
Openness also means when mistakes are discovered after a statistic is released, the agency has an obligation to issue a correction publicly and in a timely manner. It should not only use the same dissemination vehicles to announce corrections that it used to release the original statistic, but also
use additional vehicles, as appropriate, to alert the widest possible audience of current and potential users.
In summary, agencies should make an effort to provide information on the quality, limitations, and appropriate use of their data that is as frank and complete as possible. Such information, which is sometimes termed “metadata,” should be made available in ways that are easy for users to access and understand, recognizing that users differ in their level of understanding of statistical data (see National Research Council, 1993a, 1997b). Agencies need to work to educate users that all data contain some uncertainty and error, which does not mean that the data are wrong but that they must be used with care.
The Information Quality Act of 2000 (see Appendix B) stimulated all federal agencies to develop written guidelines for maintaining and documenting the quality of their information programs and activities. Using a framework developed collaboratively by the members of the Interagency Council on Statistical Policy, individual statistical agencies have developed guidelines for their own data collection programs, which are available on the Internet (see Appendix B).
Practice 5: Wide Dissemination of Data
A statistical agency must have vigorous and well-planned dissemination programs to get information into the hands of users who need it on a timely basis. Planning should be undertaken from the viewpoint that the public has contributed the data elements, has paid for the data collection and processing, and should in return have the information accessible in ways that make it as useful as possible to the largest number of users.
A good dissemination program provides data to users in forms that are suited to their needs. Data release may take the form of regularly updated time series, cross-tabulations of aggregate characteristics of respondents, and analytical reports that are made available in printed publications, on computer-readable media (e.g., CD-ROM), and on the Internet. (See Appendix C for a number of federal statistical agency web sites, which are accessible from a single source: http://www.fedstats.gov.)
Yet another form of dissemination involves access to microdata files, which make it possible to conduct in-depth research in ways that are not possible with aggregate data. Public-use microdata files may be developed for general release. Such files contain data for individual respondents that have been processed to protect confidentiality by deleting, aggregating, or
modifying any information that might permit individual identification. Alternatively, an agency may provide a facility on the Internet to allow users to aggregate individual microdata to suit their purposes, with safeguards so that the data cannot be retabulated in ways that could identify individual respondents. Another alternative is to grant a license to researchers to analyze restricted microdata (that is, data that have not been processed for general release) at their own sites by agreeing to follow strict procedures for protecting confidentiality and accepting liability for penalties if confidentiality is breached. A third alternative is to allow researchers to analyze restricted microdata at secure sites maintained by a statistical agency, such as one of the Census Bureau’s Research Data Centers located at several universities and research organizations around the country or the National Center for Health Statistics’ Research Data Center at its headquarters (see Doyle et al., 2001). Agencies should consider all forms of dissemination in order to gain the most use of their data consistent with protecting the confidentiality of responses.
The stunning improvements over the past two decades in computing speed, power, and storage capacity, the growing availability of information from a wide range of public and private sources on the Internet, and the increasing richness of statistical agency data collections have increased the risks that individually identifiable information can be obtained (see National Research Council, 2003d:Ch.5). Statistical agencies must be vigilant in their efforts to protect against the increased threats to disclosure from their summary data and microdata products while honoring their obligation to be proactive in seeking ways to provide data to users. When statistical data are not disseminated in useful forms, there is a loss to the public, not only of wasted taxpayer dollars but also of research findings that could have informed public policy and served other important societal purposes.
A good dissemination program for statistical data uses a variety of channels to inform the broadest possible audience of potential users about available data products and how to obtain them. Such channels may include providing direct access to data on the Internet, depositing data products in libraries, establishing a network of data centers (such as the Census Bureau’s state data centers), holding exhibits and making presentations at conferences, and maintaining lists of individuals and organizations to notify of new data. Agencies should also arrange for archiving of data with the National Archives and Records Administration and other data archives, as appropriate, so that data are available for historical research in future years.
An effective dissemination program provides not only the data, but also information about the strengths and weaknesses of the data in ways that can be comprehended by diverse audiences. Information about the limitations of the data should be included in every form of data release, whether in a printed report, on a computer-readable data file, or on the Internet.
On occasion, the objective of presenting the most accurate data possible may require more time than is consistent with the needs of users for the information. The tension between frequency and promptness of release on one hand and accuracy on the other should be explicitly considered. When concerns for timeliness prompt the release of preliminary estimates (as in some economic indicators), consideration should be given to the frequency of revisions and the mode of presentation of revised figures from the point of view of the users as well as the issuers of the data. Agencies that release preliminary estimates must educate the public about differences among preliminary, revised, and final estimates.
Practice 6: Cooperation with Data Users
Users of federal statistical data span a broad spectrum of interests and needs. They include policy makers, planners, administrators, and researchers in federal agencies, state and local governments, the business sector, and academia. They also include activists, citizens, students, and media representatives. An effective statistical agency endeavors to learn about its data users and to obtain input from them on the agency’s statistical programs.
The needs of users can be explored by forming advisory committees, holding focus groups, analyzing requests and Internet activity, or undertaking formal surveys of users. The task requires continual alertness to the changing composition and needs of users and the existence of potential users. An agency should cooperate with professional associations, institutes, universities, and scholars in the relevant fields to determine the needs of the research community and obtain their insight on potential uses. An agency should also work with relevant associations and other organizations to determine the needs of business and industry for its data.
Within the limitations of its confidentiality procedures as noted above, an agency should seek to provide maximum access to its data, including making the data available to external researchers for secondary analysis (National Research Council, 1985b). Having data accessible for a wide range of analyses increases the return on the investment in data collection
and provides support for an agency’s program. Once statistical data are made public, they may be used in numerous ways not originally envisaged. An agency should attempt to monitor the major uses of its data as part of its efforts to keep abreast of user needs. In 2002, OMB introduced requirements for performance assessment of federal agencies: for statistical agencies, the requirements emphasize assessment of how well the agency understands and serves its users (see Appendix B).
Researchers and other users of data frequently request data from statistical agencies for specific purposes. The agency should have procedures in place for referring users to professionals within the agency who can comprehend the user’s purposes and needs and who have a thorough knowledge of the agency’s data. Statistical agencies should view these services as a part of their dissemination activities.
Ensuring equal access requires avoiding release of data to selected individuals, organizations, or news media in advance of other users. Agencies that prepare special tabulations of their data on request for external groups must be alert to the proposed uses. If the data are to be used in court cases, administrative proceedings, or collective bargaining negotiations, it is wise to have a known policy ensuring that all sides may receive the special tabulations, regardless of which side requested them or paid the cost of the tabulation.
Practice 7: Fair Treatment of Data Providers
Clear policies and effective procedures for protecting data confidentiality, respecting the privacy of respondents, and, more broadly, protecting the rights of human research participants are critical to maintaining the quality and comprehensiveness of the data that federal statistical agencies provide to policy makers and the public. Part of the challenge for statistical agencies is to develop effective means of communicating not only the agency’s protection procedures and policies, but also the importance of the data being collected for the public good.
Data providers must believe that the data they give to a statistical agency will not be used by the agency to harm them. For statistical data collection programs, protecting the confidentiality of individual responses is considered essential to encourage high response rates and accuracy of
response. (For reviews of research on the relationship of concerns about confidentiality protection to response rates, see National Research Council, 2004e:Ch.4; Nie et al., in press.) Furthermore, if participants have been assured of confidentiality, then under federal policy for the protection of human subjects, disclosure of identifiable information about them would violate the principle of respect for persons even if the information is not sensitive and would not result in any social, economic, legal, or other harm (National Research Council, 2003d:Ch.5).
Historically, some agencies had legislative mandates supporting promises of confidentiality (e.g., for the U.S. Census Bureau, Title 13 of the U.S. Code, first enacted in 1929); other agencies (e.g., the Bureau of Labor Statistics) relied on strong statements of policy, legal precedents in court cases, or custom (see Gates, 2000; Norwood, 1995). The latter agencies risked having their policies overturned by judicial interpretations of legislation or executive decisions that might have required the agency to disclose identifiable data collected under a pledge of confidentiality (for an example involving the Energy Information Administration, see National Research Council, 1993b:185-186).
To give additional weight and stature to policies that statistical agencies had pursued for decades, OMB issued a Federal Statistical Confidentiality Order on June 27, 1997. This order assured respondents who provided statistical information to specified agencies that their responses would be held in confidence and would not be used against them in any government action, “unless otherwise compelled by law” (U.S. Office of Management and Budget, 1997).
In 2002, Congress passed the E-Government Act of 2002, which was signed into law on December 17. Title V, the Confidential Information Protection and Statistical Efficiency Act (CIPSEA), subtitle A, provides a statutory basis for protecting the confidentiality of all federal data collected for statistical purposes under a confidentiality pledge, including but not limited to data collected by statistical agencies (see Appendix B). Subtitle A places strict limits on the disclosure of individually identified information collected with a pledge of confidentiality: such disclosure to persons other than the employees or agents of the agency collecting the data can occur only with the informed consent of the respondent and the authorization of the agency head and only when the disclosure is not prohibited by any other law (e.g., Title 13). It also provides penalties for employees or agents who knowingly or willfully disclose statistical information (up to 5 years in prison, up to $250,000 in fines, or both).
Although confidentiality protection for statistical data is now on a much firmer legal footing across the federal government than prior to CIPSEA, there is an exception for some data from the National Center for Education Statistics (NCES) that could have an adverse effect on survey response. The USA Patriot Act of 2001, Section 508, amended the National Center for Education Statistics Act of 1994 by allowing the U.S. Attorney General (or an assistant attorney general) to apply to a court to obtain any “reports, records, and information (including individually identifiable information) in the possession” of NCES that are considered relevant to an authorized investigation or prosecution of domestic or international terrorism. Section 508 also removed the penalties for NCES employees who furnish individual records under this section.
Statistical agencies continually strive to avoid inadvertent disclosure of confidential information in disseminating data. Recently, the widespread dissemination of statistical data via the Internet has heightened attention by agencies to ensuring that effective safeguards to protect confidential information are in place. Risks are increased when data for small groups are tabulated, when the same data are tabulated in a variety of ways, or when public-use microdata files (samples of records for unidentified individuals or units) are released with highly detailed content. Longitudinal surveys, for example, particularly newer ones, typically have richly detailed content for multiple domains (e.g., health, education, labor force participation) or multiple respondents (e.g., parents, students, teachers) or both. Risks may also be increased when surveys include linked administrative data or collect biomarkers from blood samples or other physiological measures (National Research Council, 2001a).
Because of the disclosure risks associated with detailed tabulations and rich public-use microdata files, there is always a tension between the desire to safeguard confidentiality and the desire to provide public access to data. This dilemma is an important one to federal statistical agencies, and it has stimulated ongoing efforts to develop new statistical and administrative procedures to safeguard confidentiality while permitting more extensive access. An effective federal statistical agency will exercise judgment in determining which of these procedures are best suited to its requirements to serve data users while protecting confidentiality. (Several Committee on National Statistics’ study panels have discussed these issues and alternative procedures for providing data access while maintaining confidentiality protection; see National Research Council, 1993b, 2000, 2003d.)
To promote trust and encourage accurate response, it is important that statistical agencies respect the privacy of respondents. When data providers are asked to participate in a survey, they should be told whether the survey is mandatory or voluntary, how the data will be used, and who will have access to the data. In the case of voluntary surveys, information on these matters is necessary in order for data providers to give their informed consent to participate.
Respondents invest time and effort in replying to surveys. The amount of effort or burden varies considerably from survey to survey, depending on such factors as the complexity of the information that is requested. Statistical agencies should attempt to minimize such effort, to the extent possible, by using concepts and definitions that fit respondents’ common understanding; by simplifying questionnaires; by allowing alternative modes of response (e.g., via the Internet) when appropriate; and by using administrative records or other data sources if they are sufficiently complete and accurate to provide some or all of the needed information. In surveys of businesses or other institutions, agencies should seek innovative ways to obtain information from the institution’s records and minimize the need for respondents to reprocess and reclassify information. It is also the responsibility of agencies to use qualified, well-trained interviewers. Respondents should be informed of the likely duration of a survey interview and, if the survey involves more than one interview, how many times they will be contacted over the life of the survey. This information is particularly important when respondents are asked to cooperate in extensive interviews, search for records, or participate in longitudinal surveys.
Ways in which participation in surveys can be made easier for respondents and result in more accurate data can be explored by such means as focus group discussions or surveys. Many agencies apply the principles of cognitive psychology to questionnaire design, not only to make the resulting data more accurate, but also to make the time and effort of respondents more efficient (National Research Council, 1984). Some agencies thank respondents for their cooperation by providing them with brief summaries of the information after the survey is compiled.
Increasing privacy concerns may contribute to observed declines in survey response rates. In a time when individuals are inundated with requests for information from public and private sources, when there are documented instances of identity theft and other abuses of confidential
information on the Internet, and when individual information is being used for terrorism-related investigatory or law enforcement purposes, it may not be surprising that individuals decline to respond to censuses and surveys, even when the questions appear non-invasive and the data are collected for statistical purposes under a pledge of confidentiality.
The E-Government Act of 2002 requires agencies to develop privacy impact assessments (PIAs) whenever “… initiating a new collection of information … in an identifiable form ….” (see Appendix B; see also U.S. Office of Management and Budget, 2003). The purpose of a privacy impact assessment is to ensure that there is no collection, storage, access, use, or dissemination of identifiable information that is not both needed and permitted. In response, statistical agencies have begun conducting and releasing PIAs for statistical programs (see, e.g., U.S. Census Bureau, 2003), and, in the process, rethinking how to respect individual privacy in order to maintain trust with data providers.
Statistical agencies should devote resources to understanding the privacy and confidentiality concerns of individuals (and organizations). They should also devote resources to devising effective strategies for communicating privacy and confidentiality policies and practices to respondents. Such strategies appear to be more necessary—and more challenging—than ever before.
Finally, a reason that respondents reply to statistical surveys is that they believe that their answers will be useful to the government or to society generally. Statistical agencies should respect this contribution by compiling the data and making them accessible to users in convenient forms. A statistical agency has an obligation to publish statistical information from the data it has collected unless it finds the results invalid.
Protecting Human Research Participants
Collecting data from individuals as part of a research study or a statistical information program is a form of research involving human participants, for which the federal government has developed regulations, principles, and best practices over a period of 50 years (National Research Council, 2003d). The pertinent regulations, which have been adopted by ten departments and seven agencies, are known as the “Common Rule” (Title 45 Code of Federal Regulations, section 46). The Common Rule regulations require that researchers protect the privacy of human participants and maintain the confidentiality of data collected from them, minimize the
risks to participants from the data collection and analysis, select participants equitably with regard to the benefits and risks of the research, and seek informed consent from participants. Under the regulations, most federally funded research involving human participants must be reviewed by an independent institutional review board (IRB) to determine that the design meets the ethical requirements for protection. (See Office for Human Research Protections  for information about the Common Rule and procedures for the certification of IRBs by the Office for Human Research Protections in the U.S. Department of Health and Human Services.)
Data collections of federal statistical agencies are subject to IRB review within some departments. The Census Bureau, citing Title 13, has maintained an exemption from IRB review for its data collection programs under section 46.101(b.3), which permits exemption if “federal statute(s) require(s) without exception that the confidentiality of the personally identifiable information will be maintained throughout the research and thereafter.”
Whether or not a statistical agency is subject to formal IRB review, it should strive to incorporate the spirit of the Common Rule regulations in the design and operation of its data collection programs. An agency that is required to obtain IRB approval for data collection should work proactively with the IRB to determine how best to apply the regulations in ways that do not unnecessarily inhibit response. For example, signed written consent is not necessary for mail surveys and is hardly ever necessary for telephone surveys of the general population: such documentation does not provide any added protection to the respondent, and it is likely to reduce participation. As noted above, an effective statistical agency will seek ways—such as sending an advance letter—to furnish information to potential respondents that will help them make an informed decision about whether to participate. Such information should include the planned uses of the data and the benefits to individuals and the public.
Practice 8: Commitment to Quality and Professional Standards of Practice
The best guarantee of high-quality data is a strong professional staff that includes experts in the subject-matter fields covered by the agency’s program, experts in statistical methods and techniques, and experts in data collection, processing, and other operations. A major function of an agency’s managers is to strike a balance among these groups and promote working
relationships that make the agency’s program as productive as possible, with each group of experts contributing to the work of the others.
An effective statistical agency devotes resources to developing, implementing, and inculcating standards for data quality and professional practice. Although a long-standing culture of data quality contributes to professional practice, an agency should also seek to develop and document standards through an explicit process. The Information Quality Act of 2000 requires agencies to develop and publicize quality guidelines of a fairly general nature (see Appendix B).3 In addition, it is in an agency’s interest to develop detailed guidelines for use by its staff, contractors, and even outside researchers. The existence of explicit standards and guidelines, regularly reviewed and updated, facilitates training of new in-house staff and contractors’ staffs.
An effective statistical agency keeps up to date on developments in theory and practice that may be relevant to its program, such as new techniques for imputing missing data (see, for example, National Research Council, 2004e:App. F). An effective agency is also alert to changes in the economy or in society that may call for changes in the concepts or methods used in particular data sets (see, for example, National Research Council, 1995a, on concepts of poverty, and National Research Council, 2002a, on cost-of-living concepts). Often the need for change conflicts with the need for comparability with past data series, and this issue can easily dominate consideration of proposals for change. Agencies have the responsibility to manage this conflict by initiating more relevant data series or revising existing series to improve quality while providing information to compare old and new series, such as was done when the Bureau of Labor Statistics revised the treatment of owner-occupied housing in the consumer price index.
To ensure the quality of its data collection programs and reports, an effective statistical agency has mechanisms and processes for obtaining both inside and outside review of such aspects as the soundness of the data collection and estimation methods and the completeness of the documentation of the methods used and the error properties of the data. At a program or agency-wide level, mechanisms for outside review include standing advisory committees of technical experts and periodic assessments by
ad hoc committees (see, e.g., National Research Council, 1985a, 1986, 1993a, 1997b, 2003c, 2004c, 2004d). For individual publications and reports, formal processes are needed that incorporate review by agency technical experts and, as appropriate, by technical experts in other agencies and outside the government. (See Appendix B for a description of recent OMB guidelines for peer review of scientific information.)
Practice 9: An Active Research Program
Substantive Research and Analysis
There are strong arguments for a statistical agency to have staff whose responsibility is to conduct objective substantive analyses of the data that the agency compiles, such as analyses that assess trends over time or compare population groups:
Agency analysts are in a position to understand the need for and purposes of the data and know how the statistics will be used. Such information must be available to the agency and understood thoroughly if the survey design is to produce the data required.
Those involved in analysis can best articulate the concepts that should form the basic framework of a statistical series. Agency analysts are well situated to understand and transmit the views of external users and researchers; at the same time, close working relationships between analysts and data producers are needed for the translation of the conceptual framework into the design and operation of the survey.
Agency analysts have access to the complete microdata and so are in a better position than analysts outside the agency to understand and describe the limitations of the data for analysis purposes and to identify errors or shortcomings in the data that can lead to subsequent improvements.
Substantive research by analysts on an agency’s staff will have credibility because of the agency’s commitment to openness about the data provided and maintaining independence from political control.
Substantive research by analysts on an agency’s staff can assist in formulating the agency’s data program, suggesting changes in priorities, concepts, and needs for new data or discontinuance of outmoded or little-used series.
As with descriptive analyses provided by the agency, substantive analyses must be designed to be relevant to policy but not take positions on policy options or be designed with any particular policy agenda in mind. These issues are discussed in Martin (1981), Norwood (1975), and Triplett (1991).
Research on Methodology and Operations
For statistical agencies to be innovative in methods for data collection, analysis, and dissemination, research on methodology and operational procedures must be ongoing. Methodological research may be directed toward improving survey design, measuring error and, when possible, reducing it from such sources as nonresponse and reporting errors, reducing the time and effort asked of respondents, developing new and improved summary measures and estimation techniques, and developing innovative statistical methods for confidentiality protection. Research on operational procedures may be directed toward facilitating data collection in the field, improving the efficiency and reproducibility of data capture and processing, and enhancing the usability of Internet-based data dissemination systems.
Much of current practice in statistical agencies was developed through research they conducted or obtained from other agencies. Federal statistical agencies, frequently in partnership with academic researchers, pioneered the applications of statistical probability sampling, the national economic accounts, input-output models, and other analytic methods. The U.S. Census Bureau pioneered the use of computers for processing the census, and research on data collection, processing, and dissemination operations continues to lead to creative uses of automated procedures and equipment in these areas. Several federal statistical agencies sponsor research using academic principles of cognitive psychology to improve the design of questionnaires, the clarity of data presentation, and the ease of use of electronic data collection and dissemination tools such as the Internet. The history of the statistical agencies has shown repeatedly that methodological and operations research can lead to large productivity gains in statistical activities at relatively low cost.
An effective statistical agency actively partners with the academic community for methodological research. It also seeks out academic and industry expertise for improving data collection, processing, and dissemination operations. For example, a statistical agency can learn techniques and
best practices for improving software development processes from computer scientists (see National Research Council, 2003e, 2004d).
Research on Policy Uses
Much more needs to be known about how statistics are actually used in the policy-making process, both inside and outside the government. Research about how the information produced by a statistical agency is used in practice can contribute to future improvements in design, concepts, and format of data products. For example, public-use files of statistical microdata were developed in response to the growing analytic needs of government and academic researchers.
Gaining an understanding of the variety of uses and users of an agency’s data is only a first step. More in-depth research on the policy uses of an agency’s information might, for example, explore the use of data in microsimulation or other economic models, or go further to examine how the information from such models and other sources is used in decision-making (see National Research Council, 1991a, 1991b, 1997a, 2001b, 2003a).
Practice 10: Professional Advancement of Staff
An effective federal statistical agency has personnel policies that encourage the development and retention of a strong professional staff who are committed to the highest standards of quality work. There are several key elements of such a policy:
The required levels of technical and professional qualifications for positions in the agency are identified, and the agency adheres to these requirements in recruitment and professional development of staff. Position requirements take account of the different kinds of technical and other skills, such as supervisory skills, that are necessary for an agency to have a full range of qualified staff, including not only statisticians, but also experts in relevant subject-matter areas, data collection, processing, dissemination processes, and management of complex, technical operations.
Continuing technical education and training of staff, appropriate to the needs of their positions, is provided by sponsoring in-house training programs and providing opportunities for external education and training.
Professional activities, such as publication in refereed journals and presentations at conferences, are encouraged and recognized, including
presentations of technical work in progress with appropriate disclaimers. Participation in relevant statistical and other scientific associations is encouraged to promote interactions with academic researchers and other data users. Such participation is also a mechanism for openness about the data provided.
Interaction with other professionals is increased through technical advisory committees, supervision of contract research and research consultants, fellowship programs of visiting researchers, exchange of staff with relevant statistical, policy, or research organizations, and opportunities for new assignments within the agency.
Accomplishment is rewarded by appropriate recognition and by affording opportunity for further professional development. The prestige and credibility of a statistical agency is enhanced by the professional visibility of its staff, which may include establishing high-level non-management positions for highly qualified technical experts.
An effective statistical agency considers carefully the costs and benefits—monetary and nonmonetary—of using contractor organizations, not only for data collection as most agencies do, but also to supplement inhouse staff in other areas.4 Outsourcing of functions can have benefits, such as providing experts in areas in which the agency is unlikely to be able to attract highly qualified in-house staff (e.g., some information technology functions), enabling an agency to handle an increase in its workload that is expected to be temporary or that requires specialized skills, and allowing an agency to learn from best industry practices. However, outsourcing can also have costs, including that agency staff become primarily contract managers and less qualified as technical experts and leaders in their fields. An effective statistical agency maintains and develops a sufficiently large number of in-house staff, including mathematical statisticians, who are qualified to analyze the agency’s data and to plan, design, carry out, and evaluate its core operations so that the agency maintains the integrity of its data and its credibility in planning and fulfilling its mission. Statistical agencies should also maintain and develop staff with the expertise necessary for effective management of contractor resources.
An effective statistical agency has policies and practices to instill the highest possible commitment to professional ethics among its staff, as well as procedures for monitoring contractor compliance with ethical standards. When an agency comes under pressure to act against its principles—for example, if it is asked to disclose confidential information for an enforcement purpose or to support an inaccurate interpretation of its data—it must be able to rely on its staff to resist such actions as contrary to the ethical principles of their profession. An effective agency refers its staff to such statements of professional practice as the guidelines published by the American Statistical Association (2005) and the International Statistical Institute (2005), as well as to the agency’s own statements about protection of confidentiality, respect for privacy, standards for data quality, and similar matters. It endeavors in other ways to ensure that its staff are fully cognizant of the ethics that must guide their actions in order for the agency to maintain its credibility as a source of objective, reliable information for use by all.
Practice 11: Coordination and Cooperation with Other Statistical Agencies
The U.S. federal statistical system consists of many agencies in different departments, each with its own mission. Nonetheless, statistical agencies do not and should not conduct their activities in isolation. An effective statistical agency actively explores ways to work with other agencies to meet current information needs, for example, by seeking ways to integrate the designs of existing data systems to provide new or more useful data than a single system can provide. An effective agency is also alert for occasions when it can provide technical assistance to other agencies—including not only other statistical agencies, but also program agencies in its department—as well as occasions when it can receive such assistance in turn. Efforts to standardize concepts and definitions, such as those for industries and occupations, further contribute to effective coordination of statistical agency endeavors, as does the development of broad macro models, such as the system of national accounts (see, e.g., National Research Council, 2004a). Initiatives for sharing data among statistical agencies (including individual data and address lists when permitted by law and when sharing does not violate confidentiality promises) can be helpful for such purposes as achieving greater efficiency in drawing samples, evaluating completeness
of population coverage, and reducing duplication among statistical programs, as well as reducing respondent burden.
The responsibility for coordinating statistical work in the federal government is specifically assigned to the Office of Information and Regulatory Affairs (OIRA) in the OMB by the Paperwork Reduction Act (previously, by the Federal Reports Act and the Budget and Accounting Procedures Act). The Statistical and Science Policy Office in OIRA, often working with the assistance of interagency committees, reviews concepts of interest to more than one agency; issues standard classification systems (of industries, metropolitan areas, etc.) and oversees their periodic revision; consults with other parts of OMB on statistical budgets; and, by reviewing the statistical program of the government as a whole, identifies gaps in statistical data, programs that may be duplicative, and areas in which interagency cooperation might lead to greater efficiency and added utility of data. The Statistical and Science Policy Office also is responsible for coordinating U.S. participation in international statistical activities.5
The Statistical and Science Policy Office encourages the use of administrative data for statistical purposes, when feasible, and works to establish common goals and norms on major statistical issues such as confidentiality. It sponsors and heads the interagency Federal Committee on Statistical Methodology (FCSM), which issues guidelines and recommendations on statistical issues common to a number of agencies (see Federal Committee on Statistical Methodology, 1978a-2004; see also http://www.fcsm.gov). It encourages the Committee on National Statistics at the National Academies to serve as an independent adviser and reviewer of federal statistical activities. The 1995 reauthorization of the Paperwork Reduction Act created a statutory basis for the Interagency Council on Statistical Policy (ICSP), formalizing an arrangement whereby statistical agency heads participate with OMB in activities to coordinate federal statistical activities. (See Box 1 for a list of agencies represented on the ICSP.)
There are many forms of interagency cooperation and coordination. Some efforts are multilateral, some bilateral. Many result from common interests in specific subject areas, such as economic statistics, statistics on
people with disabilities, or statistics on children or the elderly. (See U.S. Office of Management and Budget, 2004c:Ch.3, for a description of several interagency collaborative efforts, such as joint support for research that fosters new and innovative approaches to surveys, expansion and improvement of FedStats coverage and features, and development of the American Community Survey.)
A common type of bilateral arrangement is the agreement of a program agency to provide administrative data to a statistical agency to be used as a sampling frame, a source of classification information, or a summary compilation to check (and possibly revise) preliminary sample results. The Bureau of Labor Statistics, for example, benchmarks its monthly establishment employment reports to data supplied by state employment security agencies. Such practices improve statistical estimates, reduce costs, and eliminate duplicate requests for information from the same respondents. In other cases, federal statistical agencies engage in cooperative data collection with state counterparts to let one collection system satisfy the needs of both. A number of such joint systems have been developed, notably by the Bureau of Labor Statistics, the National Agricultural Statistics Service, the National Center for Education Statistics, and the National Center for Health Statistics.
Another example of a joint arrangement is the case in which one statistical agency contracts with another to conduct a survey, compile special tabulations, or develop models. Such arrangements make use of the special skills of the supplying agency and facilitate use of common concepts and methods. The Census Bureau conducts many surveys for other agencies, as do the National Center for Health Statistics and the National Agricultural Statistics Service (see U.S. Office of Management and Budget, 2004c:App. A).
The major federal statistics agencies are also concerned with international comparability of statistics. Under the leadership of OMB’s Statistical and Science Policy Office, they contribute to the deliberations of the United Nations Statistical Commission, the Organization for Economic Co-operation and Development, and other international organizations; participate in the development of international standard classifications and systems; and support educational activities that promote improved statistics in developing countries. Statistical agencies also learn from and contribute to the work of established statistical agencies in other countries in such areas as survey methodology, record linkage, confidentiality protection techniques, and data quality standards. Several statistical agencies run educational programs for government statisticians in developing countries.
Some statistical agencies have long-term cooperative relationships with international groups, for example, the Bureau of Labor Statistics with the International Labor Organization, the National Agricultural Statistics Service with the Food and Agriculture Organization, and the National Center for Health Statistics with the World Health Organization.
To be of most value, the efforts of statistical agencies to cooperate as partners with one another should involve the full range of their activities, including definitions, concepts, measurement methods, analytical tools, dissemination modes, and disclosure limitation techniques. Such efforts should also extend to policies and professional practices so that agencies can respond effectively and with a coordinated voice to such governmentwide initiatives as data quality guidelines, privacy impact assessments, performance rating criteria, institutional review board requirements, and others.
Finally, coordination efforts should encompass the development of data, especially for emerging policy issues (National Research Council, 1999a). In some cases, it may be not only more efficient, but also productive of needed new data for agencies to fully integrate the designs of existing data systems, such as when one survey provides the sampling frame for a related survey. In other instances, cooperative efforts may identify ways for agencies to improve their individual data systems so that they are more useful for a wide range of purposes.
Two of the more effective continuing cooperative efforts in this regard have been the Federal Interagency Forum on Aging-Related Statistics and the Federal Interagency Forum on Child and Family Statistics. The former was established in the mid-1980s by the National Institute on Aging, in cooperation with the National Center for Health Statistics and the Census Bureau. The forum’s goals include coordinating the development and use of statistical databases among federal agencies, identifying information gaps and data inconsistencies, and encouraging cross-national research and data collection for the aging population. The forum was reorganized in 1998 to include six new member agencies, and the reconfigured forum decided at its first meeting in March 1999 to focus on developing a periodic indicators chart book, which was first published the following year (the latest publication is Federal Interagency Forum on Aging-Related Statistics, 2004).
The Federal Interagency Forum on Child and Family Statistics was formalized in a 1994 executive order to foster coordination and collaboration in the collection and reporting of federal data on children and families. It includes many relevant statistical and program agencies. Its reports (e.g.,
Federal Interagency Forum on Child and Family Statistics, 2003, 2004) describe the condition of America’s children, including changing population and family characteristics, the environment in which children are living, and indicators of well-being in the areas of economic security, health, behavior, social environment, and education.
No single agency, whether a statistical or program agency, could have produced the forum reports alone. Working together in this way, federal statistical agencies contribute to presenting data in a form that is more relevant to policy concerns and to a stronger statistical system overall.