National Academies Press: OpenBook
« Previous: Summary
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Assuring Data Quality at U.S. Geological Survey Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/25524.
×

1

Introduction

The U.S. Geological Survey (USGS) mission is to provide reliable and impartial scientific information “to describe and understand the Earth; minimize loss of life and property from natural disasters; manage water, biological, energy, and mineral resources; and enhance and protect our quality of life.”1 Data collection, analysis, interpretation, and dissemination are central to everything the USGS does.

The USGS operates some 250 laboratories to analyze physical or biological samples, including water, sediment, rock, plants, invertebrates, fish, and wildlife. The data generated in the laboratories help answer pressing scientific and societal questions associated with energy and mineral resources, natural hazards, surface and groundwater quality and quantity, biology and ecology, environmental health, and other topics. A growing number of these topics touch on controversial issues, such as those concerning environmental pollutants or climate change, and USGS scientific results are coming under increasing scrutiny. For these issues especially, the USGS reputation and the self-correction of science through peer-reviewed publications may no longer suffice to maintain public trust in USGS research results.

The importance of providing reliable information came to the fore in 2016, when an Inspector General report found scientific misconduct and data manipulation at a USGS laboratory in Lakewood, Colorado (IG, 2016). Two laboratory analysts had adjusted values outside of protocols over two extended periods. Reports of such incidents are rare, but they highlight the importance of data quality assurance and quality control procedures. To restore confidence in USGS data, the USGS has begun implementing a quality management system (QMS)—a structured and documented system that establishes the requirements for how work is to be managed, conducted, and monitored to assure data quality. A QMS is a paradigm shift for the USGS because instead of each laboratory developing its own quality assurance procedures, all USGS laboratories will be required to implement a centrally defined quality standard in a similar and consistent way, as applicable to the work performed (USGS, 2018). The laboratory elements typically managed in a QMS are illustrated in Figure 1.1.

___________________

1 See https://www.usgs.gov/about/about-us/who-we-are.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Assuring Data Quality at U.S. Geological Survey Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/25524.
×
Image
FIGURE 1.1 Laboratory elements typically coordinated and managed using a QMS developed to ensure compliance with a quality standard.

In addition to developing a QMS, the USGS has commissioned external reviews of its QMS implementation, including this report.

COMMITTEE TASKS AND APPROACH

At the request of William Werkheiser, then USGS Deputy Director, the National Academies of Sciences, Engineering, and Medicine established a committee to review a selection

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Assuring Data Quality at U.S. Geological Survey Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/25524.
×

of USGS analytical laboratories and identify best practices and procedures for assuring the integrity and reliability of laboratory results. The specific tasks to the committee are given in Box 1.1.

When the tasks in Box 1.1 were developed, USGS managers had only begun collecting information about its laboratories and their quality assurance practices. As their knowledge grew, it became clear that some of the information needed to address the tasks would not be

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Assuring Data Quality at U.S. Geological Survey Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/25524.
×

available for the study. Consequently, some tasks could not be addressed exactly as written. At the first committee meeting, the USGS deputy director suggested ways to address the intention, rather than the exact wording, of these tasks. With those guidelines, the committee addressed the tasks as described below.

Task 1 was to provide an overview of all USGS analytical laboratories, including their science and applications objectives, budget, staff and user profiles, and history of sample throughput. The USGS collected and analyzed this and other information (e.g., data quality procedures) in 2016 (see Phillips et al., 2018). Consequently, the committee used the 2016 USGS laboratory inventory as a baseline and supplemented it with information gathered at site visits to USGS laboratories. Site visits and associated committee meetings were held in four cities: Reston, Virginia; Denver, Colorado; Menlo Park, California; and Kearneysville, West Virginia (see Table 1.1). Only 17 laboratories (6 percent), 16 active and 1 inactive at the time, could be visited in the confines of the study, and they were chosen to sample a diversity of sizes and activities. Visits to a small number of diverse laboratories cannot yield a comprehensive picture of the USGS laboratory landscape. However, a number of common themes emerged that informed the committee’s response to Task 1.

TABLE 1.1 USGS Laboratories Visited by the Committee

Date Location Laboratories Visited
June 2018 Reston, Virginia
  • Energy Environmental Labs
  • Environmental Organic Geochemistry Lab (inactive)
  • Microbiology Lab
  • Reston Stable Isotope Lab
  • Wetland Ecosystems Ecology and Biogeochemistry Laboratory
August 2018 Denver, Colorado
  • Analytical Chemistry Project
  • Gas Chromatography/Mass Spectrometry
  • National Water Quality Laboratory
  • Plasma Laboratory
  • Radiogenic Isotope Laboratory
  • Spectroscopy Laboratory
November 2018 Menlo Park, California
  • Benthic Lab
  • Rock Physics Laboratory
  • Tephrochronology Project Laboratory
  • Unsaturated Zone Flow Processes Lab
February 2019 Kearneysville, West Virginia
  • Fish Health Laboratory—Fish Culture and Pathology
  • Functional Genomic, Biomarker, and Environmental Contaminant Bioanalysis
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Assuring Data Quality at U.S. Geological Survey Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/25524.
×

Task 2 was to describe analytical procedures, standards, and data management procedures for laboratories at the USGS, other federal agencies, and other geological surveys. Describing analytical and data management procedures in hundreds of laboratories was not feasible in the confines of this study. Consequently, the USGS asked the committee to focus on laboratory QMS procedures in order to provide a benchmark for its QMS effort. The committee addressed the task by asking a selection of U.S. federal agencies and academic institutions and geological surveys in other countries to discuss a set of questions on QMS and alternative approaches for assuring data quality at committee meetings.

Task 3 was to assess the extent to which laboratory resources are sufficient to meet their objectives. The USGS-collected data were not sufficiently detailed to carry out this assessment. Laboratory budget information was available in ranges, staff (principal investigators, laboratory managers, analysts, and students) were grouped into total full-time equivalents, and research and applications objectives were general. Consequently, the committee used the opinions of laboratory staff and its own observations at the laboratory visits to draw high-level conclusions on the adequacy of resources.

Tasks 4, 5, and 6 concern the identification of best practices for assuring data quality in USGS laboratories. Best practices and data quality terms used in this report are defined in Box 1.2. The USGS distinguishes two types of laboratories:

  1. Research laboratories support innovation or scientific discovery and are typically led by a principal investigator (senior scientist). Some research laboratories also develop methods that are needed to answer a scientific question or that other laboratories can use for routine analyses.
  2. Production laboratories carry out routine or repeated analyses for USGS or external customers, are sometimes supported by user fees, and are generally led by laboratory managers.

The USGS data do not indicate which laboratories do routine analyses or are led by laboratory managers, but they do specify which have external users, such as regulators and resource managers, which is an indication of production activities.

Task 4 was to develop criteria for assessing laboratory protocols and procedures used by the organizations mentioned in Task 2 and to use them to identify best practices for USGS production laboratories. Numerous assessments of laboratory procedures and recommended best practices have been published, and the committee drew from these to address Task 4.

Task 5 was to recommend best practices and procedures for achieving objectives and assuring the integrity and reliability of results for USGS production laboratories, and Task 6 was to comment on best practices and procedures for research and method development laboratories. The committee used what it learned from the laboratory site visits, meeting presentations, and the published literature to address Tasks 5 and 6.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Assuring Data Quality at U.S. Geological Survey Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/25524.
×

U.S. GEOLOGICAL SURVEY

The USGS has a federally funded budget of roughly $1 billion and has some 8,000 employees. The USGS is organized in a matrix structure that involves seven mission areas chosen to reflect USGS science and applications objectives and seven regional offices chosen to reflect geographic variations in geological, biological, and hydrological characteristics

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Assuring Data Quality at U.S. Geological Survey Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/25524.
×

(e.g., northeast, southwest, and Alaska). The mission areas are core science systems, ecosystems, energy and minerals, environmental health, land resources, natural hazards, and water resources. The USGS is headed by a director, under whom are a deputy director and seven associate directors corresponding to the mission areas (Figure 1.2). The directors of the seven regional offices report to the USGS deputy director. Each regional office has several science centers, which manage the science activities of the region and also oversee most of the laboratories.

Line management, personnel management, and science center management are organized through the regional offices. However, the QMS is being implemented by mission area. The reporting structure is complex. Although QMS implementation plans are being developed by the mission areas, QMS reporting lines flow through the Deputy Director. The Bureau QMS Coordinator and staff—who oversee implementation of the QMS—report to the Deputy Director through the director of the Office of Science Quality and Integrity (Figure 1.3). The other QMS staff, including QMS champions, report to their Deputy/Associate Science Center Director, and not to the Bureau QMS Coordinator.2

ORGANIZATION OF THIS REPORT

This report examines approaches to managing laboratory data quality and recommends best practices and procedures for assuring the integrity of USGS laboratory results. The report is organized to emphasize the information that USGS most needs to develop its quality assurance program. Chapter 2 summarizes the literature on systematic approaches for assuring data quality, including assessments of laboratory procedures and best practices (Task 4), and describes example approaches that are relevant for USGS laboratories. Chapter 3 provides an overview of the approximately 250 USGS laboratories being examined, including the adequacy of resources to meet their science and applications objectives, and their approach to assuring data quality (Tasks 1, 2, and 3). Chapter 4 highlights experiences with quality management systems at the USGS, other federal agencies, other geological surveys, and research institutions (Task 2). Chapter 5 provides the committee’s recommendations for the USGS, including the development and implementation of its QMS, and best practices and procedures for USGS laboratories (Tasks 5 and 6). The report concludes with a list of references and biographical sketches of committee members (Appendix).

___________________

2 The language of this sentence was modified after release of the publication version to reflect current USGS staffing plans.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Assuring Data Quality at U.S. Geological Survey Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/25524.
×
Image
FIGURE 1.2 Simplified representation of key organizational elements and reporting relationships of USGS upper-level management.
SOURCE: Modified from USGS.
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Assuring Data Quality at U.S. Geological Survey Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/25524.
×
Image
FIGURE 1.3 Proposed QMS staffing and reporting structure for the USGS.3
SOURCE: USGS.

___________________

3 This figure was replaced with a current organizational chart after release of the publication version.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Assuring Data Quality at U.S. Geological Survey Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/25524.
×

This page intentionally left blank.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Assuring Data Quality at U.S. Geological Survey Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/25524.
×
Page 11
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Assuring Data Quality at U.S. Geological Survey Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/25524.
×
Page 12
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Assuring Data Quality at U.S. Geological Survey Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/25524.
×
Page 13
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Assuring Data Quality at U.S. Geological Survey Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/25524.
×
Page 14
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Assuring Data Quality at U.S. Geological Survey Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/25524.
×
Page 15
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Assuring Data Quality at U.S. Geological Survey Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/25524.
×
Page 16
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Assuring Data Quality at U.S. Geological Survey Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/25524.
×
Page 17
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Assuring Data Quality at U.S. Geological Survey Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/25524.
×
Page 18
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Assuring Data Quality at U.S. Geological Survey Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/25524.
×
Page 19
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2019. Assuring Data Quality at U.S. Geological Survey Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/25524.
×
Page 20
Next: 2 Improving Data Quality and Integrity »
Assuring Data Quality at U.S. Geological Survey Laboratories Get This Book
×
Buy Paperback | $60.00 Buy Ebook | $48.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The U.S. Geological Survey (USGS) mission is to provide reliable and impartial scientific information to understand Earth, minimize loss of life and property from natural disasters, and manage water, biological, energy, and mineral resources. Data collection, analysis, interpretation, and dissemination are central to everything the USGS does. Among other activities, the USGS operates some 250 laboratories across the country to analyze physical and biological samples, including water, sediment, rock, plants, invertebrates, fish, and wildlife. The data generated in the laboratories help answer pressing scientific and societal questions or support regulation, resource management, or commercial applications.

At the request of the USGS, this study reviews a representative sample of USGS laboratories to examine quality management systems and other approaches for assuring the quality of laboratory results and recommends best practices and procedures for USGS laboratories.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!