National Academies Press: OpenBook

Principles and Practices for a Federal Statistical Agency: Sixth Edition (2017)

Chapter: Practice 9: Commitment to Quality and Professional Standards of Practice

« Previous: Practice 8: Protection of the Confidentiality of Data Providers' Information
Suggested Citation:"Practice 9: Commitment to Quality and Professional Standards of Practice." National Academies of Sciences, Engineering, and Medicine. 2017. Principles and Practices for a Federal Statistical Agency: Sixth Edition. Washington, DC: The National Academies Press. doi: 10.17226/24810.
×

Practice 9: Commitment to Quality and Professional Standards of Practice

THERE IS A WIDE RANGE of actions a statistical agency should undertake to ensure the quality of its products and practices:

  • develop strong staff expertise in the subject areas relevant to the agency’s mission, in the theory and practice of statistics, and in data collection, processing, analysis, and dissemination techniques;
  • keep abreast of and use modern statistical theory and sound statistical and computational practice in all technical work;
  • publish and implement formal quality standards;
  • maintain quality assurance programs to improve data quality and the processes of compiling, editing, and analyzing data;
  • develop an understanding of the validity and accuracy of the agency’s data and how to convey the resulting measures of quality (both uncertainty and bias) in comprehensible ways to users;
  • document concepts, definitions, and data collection methods and discuss possible sources of error in data releases to the public (see Practice 4); and
  • develop continuing relationships with appropriate professional organizations in statistics and relevant subject-matter areas.

The best guarantee of high-quality data is a strong professional staff, which includes experts in the subject-matter fields covered by the agency’s program, experts in statistical methods and techniques, and experts in data collection, processing, and other operations (see Practice 11). A major function of an agency’s leadership is to strike a balance among these staff

Suggested Citation:"Practice 9: Commitment to Quality and Professional Standards of Practice." National Academies of Sciences, Engineering, and Medicine. 2017. Principles and Practices for a Federal Statistical Agency: Sixth Edition. Washington, DC: The National Academies Press. doi: 10.17226/24810.
×

and promote working relationships that make the agency’s program as productive as possible, with each group of experts contributing to the work of the others.

An effective statistical agency keeps up to date on developments in theory and practice that may be relevant to its program—for example, new techniques for imputing missing data (see, e.g., National Research Council, 2004a:App. F, 2010e) or for combining data from more than one source and estimating error in the resulting statistics (see National Academies of Sciences, Engineering, and Medicine, 2017b:Ch. 5); new technologies for data collection, processing, and dissemination; and new kinds of and uses for paradata (see, e.g., National Research Council, 2013a). Paradata, not only from data collection processes, but also from tracking how users work with an agency’s data products on its website, can help improve methods in each of these important areas.

An effective statistical agency is also alert to social and economic changes that may call for changes in the concepts or methods used in particular datasets.81 The need for change often conflicts with the need for comparability with past data series, and the latter need can easily dominate consideration of proposals for change. Agencies have the responsibility to manage this conflict by initiating more relevant series or revising existing series to improve quality while providing information to compare old and new series, such as was done when BLS revised the treatment of owner-occupied housing in the Consumer Price Index.82

An effective statistical agency devotes resources to developing, implementing, and inculcating standards for data quality and professional practice. Although a long-standing culture of data quality contributes to professional practice, an agency should also seek to develop and document standards through an explicit process. The existence of explicit standards and guidelines, regularly reviewed and updated, facilitates training of new in-house staff and contractors’ staffs. Statistical Policy Directive No. 2 (U.S. Office of Management and Budget, 2016a), on survey standards and guidelines, is helpful in that it covers every aspect of a survey from planning

__________________

81 Reviews of concepts underlying important statistical data series, which have identified areas in which change is needed, include: poverty (National Research Council, 1995, 2005c); cost-of-living and price indexes (National Research Council, 2002); “satellite” accounts for nonmarket activities—e.g., home production, volunteerism (National Research Council, 2005a); food insecurity and hunger (National Research Council, 2006a); usual residence in the decennial census and surveys (National Research Council, 2006c); disability (National Research Council, 2009b); health satellite accounts (National Research Council, 2010a); medical care economic risk and burden (National Research Council and Institute of Medicine, 2012); happiness, suffering, and other dimensions of experience (National Research Council, 2013b); civic engagement and social cohesion (National Research Council, 2014b); and innovation (National Research Council, 2014a:Ch. 4; National Academies of Sciences, Engineering, and Medicine, 2017a).

82 See, for example, Gillingham and Lane (1982).

Suggested Citation:"Practice 9: Commitment to Quality and Professional Standards of Practice." National Academies of Sciences, Engineering, and Medicine. 2017. Principles and Practices for a Federal Statistical Agency: Sixth Edition. Washington, DC: The National Academies Press. doi: 10.17226/24810.
×

through data release (see Appendix A).83 It recommends that agencies develop additional, more detailed standards that focus on their specific statistical activities.84

To ensure the quality of its data collection programs and data releases, an effective statistical agency has not only formal quality assurance programs (e.g., well-developed methods for detecting outliers and other errors in raw data and errors from editing and other data processing steps), but also mechanisms and processes for obtaining both inside and outside reviews (see Practice 12). Such reviews should address various aspects of an agency’s operations, including the soundness of the data collection and estimation methods and the completeness of the documentation of the methods used and the error properties of the data. For individual publications and reports, formal processes are needed that incorporate review by agency technical experts and, as appropriate, by technical experts in other agencies and outside the government.85

Finally, an effective statistical agency builds strong ties with relevant professional associations. It encourages professional staff to participate in relevant associations to refresh their human capital and develop networks of experts from other statistical agencies, academia, and the private sector. It also uses professional associations as one source of advice on ways to keep its data collection programs as relevant, accurate, timely, and cost-effective as possible.

__________________

83 Data quality guidelines of statistical agencies in other countries are also helpful (see, e.g., Statistics Canada, 2009; United Kingdom Office for National Statistics, 2007).

84 For examples, see National Center for Education Statistics (2012) and Bureau of Transportation Statistics (2005). These standards work within but go well beyond the broad data quality guidelines adopted by statistical agencies in response to the 2000 Information Quality Act (see Appendix A).

85 See Appendix A for a description of OMB guidelines for peer review of scientific information; reviews at a program or agency-wide level are considered in Practice 12.

Suggested Citation:"Practice 9: Commitment to Quality and Professional Standards of Practice." National Academies of Sciences, Engineering, and Medicine. 2017. Principles and Practices for a Federal Statistical Agency: Sixth Edition. Washington, DC: The National Academies Press. doi: 10.17226/24810.
×

This page intentionally left blank.

Suggested Citation:"Practice 9: Commitment to Quality and Professional Standards of Practice." National Academies of Sciences, Engineering, and Medicine. 2017. Principles and Practices for a Federal Statistical Agency: Sixth Edition. Washington, DC: The National Academies Press. doi: 10.17226/24810.
×
Page 101
Suggested Citation:"Practice 9: Commitment to Quality and Professional Standards of Practice." National Academies of Sciences, Engineering, and Medicine. 2017. Principles and Practices for a Federal Statistical Agency: Sixth Edition. Washington, DC: The National Academies Press. doi: 10.17226/24810.
×
Page 102
Suggested Citation:"Practice 9: Commitment to Quality and Professional Standards of Practice." National Academies of Sciences, Engineering, and Medicine. 2017. Principles and Practices for a Federal Statistical Agency: Sixth Edition. Washington, DC: The National Academies Press. doi: 10.17226/24810.
×
Page 103
Suggested Citation:"Practice 9: Commitment to Quality and Professional Standards of Practice." National Academies of Sciences, Engineering, and Medicine. 2017. Principles and Practices for a Federal Statistical Agency: Sixth Edition. Washington, DC: The National Academies Press. doi: 10.17226/24810.
×
Page 104
Next: Practice 10: An Active Research Program »
Principles and Practices for a Federal Statistical Agency: Sixth Edition Get This Book
×
Buy Paperback | $54.00 Buy Ebook | $43.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Publicly available statistics from government agencies that are credible, relevant, accurate, and timely are essential for policy makers, individuals, households, businesses, academic institutions, and other organizations to make informed decisions. Even more, the effective operation of a democratic system of government depends on the unhindered flow of statistical information to its citizens.

In the United States, federal statistical agencies in cabinet departments and independent agencies are the governmental units whose principal function is to compile, analyze, and disseminate information for such statistical purposes as describing population characteristics and trends, planning and monitoring programs, and conducting research and evaluation. The work of these agencies is coordinated by the U.S. Office of Management and Budget. Statistical agencies may acquire information not only from surveys or censuses of people and organizations, but also from such sources as government administrative records, private-sector datasets, and Internet sources that are judged of suitable quality and relevance for statistical use. They may conduct analyses, but they do not advocate policies or take partisan positions. Statistical purposes for which they provide information relate to descriptions of groups and exclude any interest in or identification of an individual person, institution, or economic unit.

Four principles are fundamental for a federal statistical agency: relevance to policy issues, credibility among data users, trust among data providers, and independence from political and other undue external influence. Principles and Practices for a Federal Statistical Agency: Sixth Edition presents and comments on these principles as they’ve been impacted by changes in laws, regulations, and other aspects of the environment of federal statistical agencies over the past 4 years.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!