National Academies Press: OpenBook
« Previous: Front Matter
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Methods to Foster Transparency and Reproducibility of Federal Statistics: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25305.
×

1

Introduction

In 2014, the National Science Foundation (NSF) provided support to the National Academies of Sciences, Engineering, and Medicine for a series of Forums on Open Science in response to a government-wide directive to support increased public access to the results of research funded by the federal government. The forums were successful in raising a number of important issues concerning the advantages of open science and presenting ideas to support greater openness in scientific enterprises. However, the breadth of the ideas precluded a focus on any specific topic or discussion about how to improve public access.

With continuing support from NSF, the Committee on National Statistics (CNSTAT) organized the Workshop on Transparency and Reproducibility of Federal Statistics with the following statement of task:

An ad hoc steering committee will organize and conduct a public workshop on key aspects of transparency and reproducibility in federal statistics, including data access, archiving, and documentation, as a follow-on activity to the NSF-funded Forum on Open Science at the National Academies of Sciences, Engineering, and Medicine. The workshop will help focus discussion of issues surrounding the credibility and transparency of federally funded scientific digital data in a manner that will help not only federal statistical agencies, but also other federal agencies that fund original data collection for scientific use. A rapporteur will prepare a proceedings that summarizes the workshop presentations and discussions.

For the workshop, CNSTAT defined transparency to mean that one can generally know how something was done and reproducibility to mean

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Methods to Foster Transparency and Reproducibility of Federal Statistics: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25305.
×

that one could independently recreate the process that led to the creation of a given statistical data product. The overall goal of the workshop was to develop some understanding of what principles and practices are, or would be, supportive of making federal statistics more understandable and reviewable by both agency staff and the public.

The workshop was organized around eight broad questions:

  1. What official federal guidance or standards currently exist to provide assistance to the federal statistical system?
  2. What guidance or standards are used by foreign national statistical agencies that the U.S. federal statistical system could learn from?
  3. What are the benefits and costs of greater or lesser degrees of transparency for a federal statistical agency, and how is this tradeoff affected by the growing number of nonfederal entities that are attempting to produce federal statistical system-type data products?
  4. For complicated statistical data products, what should and should not be documented, given that some details may be of little interest to outside analysts or the public?
  5. What data, metadata, paradata, and assessments of data quality should be archived so that they can be retrieved by others?
  6. How do these answers change given that there are now various nonsurvey sources of data, including administrative records and Internet transaction data sources?
  7. How do these answers change for agency staff and for people outside an agency?
  8. What tools currently exist, either domestic or foreign, that can expedite either the documentation of methods or the archiving of data sources?

These questions come at a time when various changes are exerting pressure on the federal statistical system, as detailed in several recent CNSTAT reports.1 First, survey response rates are decreasing for all types of survey

___________________

1 See National Academies of Sciences, Medicine, and Engineering. (2017). Innovations in Federal Statistics: Combining Data Sources While Protecting Privacy. Robert M. Groves and Brian A. Harris-Kojetin, Editors. Panel on Improving Federal Statistics for Policy and Social Science Research Using Multiple Data Sources and State-of-the-Art Estimation Methods, Committee on National Statistics, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Methods to Foster Transparency and Reproducibility of Federal Statistics: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25305.
×

instruments, so there is greater interest in the use of alternative sources of information, especially administrative records. Second, while statistical agencies were once primarily interested in producing totals, means, percentages, and cross-tabulations for various geographic or demographic subsets of the population, statistical agencies now make much greater use of statistical modeling. Third, private entities are increasingly producing estimates of quantities of national interest. And fourth, there is increasing interest in a number of tools that assist in documentation and archiving of statistical data along with associated metadata and paradata.

In the workshop’s opening session, Amy Friedlander (National Science Foundation) described NSF’s work in the area. The agency has been hosting a series of events concerning data, data management, public access to data, reproducibility, and questions that derive from a concern about the integrity of the scientific research process. She noted that this is by necessity a cross-disciplinary issue and that the federal statistical agencies could learn a great deal by including computer scientists in the discussions.

Friedlander further noted that the reliability of the information provided, which is supported by transparency, is crucially important because the data products of the federal statistical system are used for important policy programs. She added that another key aspect of this issue is the need to be aggressive toward maintaining data confidentiality and other aspects of data security and data integrity. In addition, Friedlander pointed out, the reuse of historical methods necessitates the proper understanding of the context of the data and, therefore, including metadata in a data archive is important. Furthermore, she said, documentation of methods gets into the issue of retention of the form of instrumentation (and mode bias) and methodology workflow.

This Proceedings has been prepared by the workshop rapporteur as a factual summary of what occurred at the workshop. The planning committee’s role was limited to planning and convening the workshop. The views contained in this Proceedings are those of individual workshop participants and do not necessarily represent the views of all workshop participants, the planning committee, or the National Academies.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Methods to Foster Transparency and Reproducibility of Federal Statistics: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25305.
×

This page intentionally left blank.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Methods to Foster Transparency and Reproducibility of Federal Statistics: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25305.
×
Page 1
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Methods to Foster Transparency and Reproducibility of Federal Statistics: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25305.
×
Page 2
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Methods to Foster Transparency and Reproducibility of Federal Statistics: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25305.
×
Page 3
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Methods to Foster Transparency and Reproducibility of Federal Statistics: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25305.
×
Page 4
Next: 2 Existing Guidelines Related to Transparency »
Methods to Foster Transparency and Reproducibility of Federal Statistics: Proceedings of a Workshop Get This Book
×
Buy Paperback | $60.00 Buy Ebook | $48.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

In 2014 the National Science Foundation (NSF) provided support to the National Academies of Sciences, Engineering, and Medicine for a series of Forums on Open Science in response to a government-wide directive to support increased public access to the results of research funded by the federal government. However, the breadth of the work resulting from the series precluded a focus on any specific topic or discussion about how to improve public access. Thus, the main goal of the Workshop on Transparency and Reproducibility in Federal Statistics was to develop some understanding of what principles and practices are, or would be, supportive of making federal statistics more understandable and reviewable, both by agency staff and the public. This publication summarizes the presentations and discussions from the workshop.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!