Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Preserving Scientific Data On Our Physical Universe A New Strategy for Archiving the Nation's Scientific Information Resources Steering Committee for the Study on the Long-term Retention of Selected Scientific and Technical Records of the Federal Government Commission on Physical Sciences, Mathematics, and Applications National Research Council NATIONAL ACADEMY PRESS Washington, D.C. 1995
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. This report has been reviewed by a group other than the authors according to procedures approved by a Report Review Committee consisting of members of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Bruce Alberts is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Robert M. White is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine. The National Research Council was established by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy's purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Bruce Alberts and Dr. Robert M. White are chairman and vice chairman, respectively, of the National Research Council. Support for this project was provided by the National Archives and Records Administration (under Contract No. NAMA-S-92-0019), the National Oceanic and Atmospheric Administration (under Contract No. 50-DGNE-3-00105), and the National Aeronautics and Space Administration (under Contract No. S-54040-Z). The views expressed in this report are those of the authors and do not necessarily reflect the views of the sponsoring agencies or subagencies. Library of Congress Catalog Card Number 94-68991 International Standard Book Number 0-309-05186-X Additional copies of this report are available from: National Academy Press 2101 Constitution Ave., NW Box 285 Washington, DC 20055 800-624-6242 202-334-3313 (in the Washington Metropolitan Area) B-499 Copyright 1995 by the National Academy of Sciences. All rights reserved. Printed in the United States of America
STEERING COMMITTEE FOR THE STUDY ON THE LONG-TERM RETENTION OF SELECTED SCIENTIFIC AND TECHNICAL RECORDS OF THE FEDERAL GOVERNMENT JEFF DOZIER, University of California, Santa Barbara, Chair SHELTON ALEXANDER, Pennsylvania State University MARJORIE COURAIN, Consultant (deceased, January 14, 1994) JOHN A. DUTTON, Pennsylvania State University WILLIAM EMERY, University of Colorado BRUCE GRITTON, Monterey Bay Aquarium Research Institute ROY lENNE, National Center for Atmospheric Research WILLIAM KURTH, University of [owe DAVID LIDE, Consultant, Gaithersburg, Maryland B.K. RICHARD, TRW JOAN WARNOW-BLEVVETT, American Institute of Physics . air ~- 1 ·17_ ~ . National Research Council Staff Paul F. Uhlir, Associate Executive Director, Commission on Physical Sciences, Mathematics, and Applications Mark David Handel, Program Officer, Board on Atmospheric Sciences and Climate Alice Killian, Research Associate, Commission on Geosciences, Environment, and Resources lames E. Mallory, Staff Officer, Computer Science and Telecommunications Board Scott T. Weidman, Senior Program Officer, Board on Chemical Sciences and Technology Julie M. Esanu, Research Assistant, Commission on Physical Sciences, Mathematics, and Applications David I. Baskin, Project Assistant, Commission on Physical Sciences, Mathematics, and Applications . . .
COMMISSION ON PHYSICAL SCIENCES, MATHEMATICS, AND APPLICATIONS RICHARD N. ZARE, Stanford University, Chair RICHARD S. NICHOLSON, American Association for the Advancement of Science, Vice Chair STEPHEN L. ADLER, Institute for Advanced Study SYLVIA T. CEYER, Massachusetts Institute of Technology SUSAN L. GRAHAM, University of California at Berkeley ROBERT I. HERMANN, United Technologies Corporation RHONDA I. HUGHES, Bryn Mawr College SHIRLEY A. JACKSON, Department of Physics KENNETH I. KELLERMANN, National Radio Astronomy Observatory HANS MARK, University of Texas at Austin THOMAS A. PRINCE, California Institute of Technology JEROME SACKS, National Institute of Statistical Sciences L.E. SCRIVEN, University of Minnesota A. RICHARD SEEBASS IIT, University of Colorado LEON T. SILVER, California Institute of Technology CHARLES P. SLIGHTER, University of Illinois at Urbana-Champaign ALVIN W. TRIVELPIECE, Oak Ridge National Laboratory SHMUEL WINOGRAD, IBM T.~. Watson Research Center CHARLES A. ZRAKET, MITRE Corporation (retired) NORMAN METZGER, Executive Director PAUL F. UHLIR, Associate Executive Director IV
Preface In January 1992 the National Archives and Records Administration (NARA) sponsored a three-day planning meeting at the National Research Council (NRC) to review the issues related to the long-term retention of the federal government's scientific and technical data in the physical sciences. The planning meeting was organized by the NRC's Commission on Physical Sciences, Mathematics, and Applications and provided the basis for this study, which was initiated in the fall of 1992 at the request of NARA. The National Oceanic and Atmospheric Administration (NOAA) and the National Aeronautics and Space Administration (NASA) subsequently provided additional support. The study's steering committee, in consultation with the sponsors, developed the following charge to guide the writing of this report: . Describe the status and plans for the government's archiving of observational and experimental data in the physical sciences. Identify the principal scientific, technical, information management, and institutional issues regarding the permanent archiving of such data. · Assess the commonalities and differences among the case studies provided by the panels organized under this study (see below) in order to determine the extent to which common long-term retention policies and appraisal guidelines can be applied to disciplines that collect observational and experimental data in the physical sciences. · Establish a set of goals, principles, and priorities, as well as generic retention criteria and appraisal guidelines that NARA can incorporate into its mission, program, and budget planning. · Suggest mechanisms and processes for NARA and NOAA to use in implementing a program of data appraisal, retention, and preservation, and later in evaluating the effectiveness of the program. · Provide a summary of findings, conclusions, and recommendations. The steering committee formed five panels in space sciences, atmospheric sciences, ocean scienc- es, geosciences, and physics, chemistry, and materials sciences- to provide their views on the key data retention issues from different disciplinary perspectives in the physical sciences. These panels each met twice and produced a set of working papers, which are published separately in Study on the Long-term Retention of Selected Scientific and Technical Records of the Federal Government: Working Papers (National Academy Press, Washington, D.C., 19951. The work of the panels was invaluable to the v
Vl Preface steering committee in framing the issues, in forming its conclusions and recommendations, and in producing its final report. There are several aspects regarding the scope and focus of this report that should be mentioned. The committee devoted most of its attention to data stored on electronic media, rather than on paper or on other media. Almost all data are now acquired, stored, and distributed electronically. Thus, the preponderance of data archiving problems and their solutions must be considered in this context. Nevertheless, much of the advice offered here is equally relevant to data in other formats. The principal focus of this report is on the Tong-term retention of data in the physical sciences. Much of the discussion, however, includes near-term data management issues, because effective archiving begins when the plans for acquiring a data set are made and extends throughout the life cycle of the data. Although the focus is exclusively on data in the physical sciences, the committee believes that the distinctions it has drawn between the experimental and the observational data, as well as the data management principles it has provided, are broadly applicable to most data in the other natural sciences. In addition, the strategic approach adopted by the committee necessarily involves all federal agencies that acquire and manage physical science data, and not simply the three agencies that sponsored this study. Finally, it is necessary to point out that the committee was unable to achieve consensus on one major recommendation of the study, namely, the proposal to establish the National Scientific Information Resource (NSIR) Federation. Appendix B contains the minority opinion of the dissenting committee member, Roy Jenne. The rest of the committee members, who strongly support the NSIR Federation recommendation, are disappointed by this lack of unanimity and consider many of the assertions in the minority opinion to be based on an erroneous interpretation of what the report actually states or recommends. We leave that to the reader to judge. Nevertheless, we believe that the minority opinion can perhaps serve a useful purpose by drawing greater attention to these issues and by broadening the discussion of them among the sponsors of the study, the other science agencies, and the research community. In conclusion, the committee hopes that its advice will help bring about the changes necessary to effectively preserve the valuable scientific data on our physical universe. Jeff Dozier Steering Committee Chair Paul F. Uhlir Study Director
Acknowledgments The steering committee is very grateful to the many individuals who played a significant role in the completion of this study, including the members of the five ad hoc panels that provided conclusions and recommendations on data archiving from the different physical science disciplines; the individuals who briefed the steering committee and panels; and members of the National Research Council (NRC) staff who worked on various aspects of this study. The steering committee also extends its thanks to Trudy Peterson and Kenneth Thibodeau of the National Archives and Records Administration (NARA), William Turnbull and Helen Wood of the National Oceanic and Atmospheric Administration (NOAA), and Joseph King of the National Aeronautics and Space Administration (NASA), from the study's . . sponsoring agencies. Gerd Rosenblatt, of Lawrence Berkeley Laboratory, chaired the Physics, Chemistry, and Materials Sciences Data Panel. The members were R. Stephen Berry, University of Chicago; Edward Galvin, The Aerospace Corporation; J.G. Kaufman, The Aluminum Association; Kirby Kemper, Florida State University; David R. Lide, Ir., consultant; and Edgar Westrum, Ir., University of Michigan. The steering committee gratefully acknowledges the detailed briefings and information provided to this pane] by Donald AIderson, Department of Defense Nuclear Information Analysis Center; Frank Biggs, Sandia National Laboratories; Robert Billingsley, Defense Technical Information Center; Mark Conrad, NARA; Suzanne Leech, Bionetics, Inc.; Victoria McLane, Brookhaven National Laboratory; and Patricia Schuette, Battelle Pacific Northwest Laboratory. The Space Sciences Data Pane] was chaired by Christopher Russell of the University of California at Los Angeles. The pane] members were Guiseppina Fabbiano, Harvard-Smithsonian Center for Astro- physics; Sarah Kadec, consultant; William Kurth, University of Iowa; Steven Lee, University of Colorado; and R. Stephen Saunders, let Propulsion Laboratory. The steering committee extends its thanks for the assistance of the following individuals, who provided briefings and other information to the Space Sciences Data Panel: Joe Allen, National Geophysical Data Center; Steven Blair, Los Alamos National Laboratory; Joseph Bredekamp, NASA; Dean Bundy, Naval Research Laboratory; David deYoung, National Optical Astronomy Observatories; Robert Frederick, Air Force Space Forecast Center; Joseph King, National Space Science Data Center; Knox Long, Space Science Telescope Institute; Guenther Riegler, NASA Astrophysics Division; Thomas Smith and Jud Stailey, Air Force Environmental Technical Applications Center; Ear] Tech, Los Alamos National Laboratory; Raymond Walker, University of California at Los Angeles; and lames Willet, NASA Space Physics Division. . . Vll
~ . . V111 Acknowledgments Werner Baum, of Florida State University, was the chair of the Atmospheric Sciences Data Panel. The members were Marjorie Courain, consultant (deceased, January 14, 1994~; William Haggard, CTimatological Consulting Corporation; Roy lenne, National Center for Atmospheric Research; Kelly Redmond, Desert Research Institute; and Thomas yonder Haar, Colorado State University. The steering committee gratefully acknowledges the diverse and substantial inputs provided by the following individ- uals to the Atmospheric Sciences Data Panel: Larry Baume, NARA; Thomas Boden, Carbon Dioxide Information and Analysis Center; Dean Bundy, Naval Research Laboratory; Donald Collins, NASA; Richard Davis, National Climatic Data Center, P.C. Hariharan, Johns Hopkins University; and Gerald Stokes, Pacific Northwest Laboratories. The Ocean Sciences Data Pane! was chaired by Bruce Gritton, Monterey Bay Aquarium Research Institute. The members were Richard DugdaTe, University of Southern California; Thomas Duncan, University of California at Berkeley; Robert Evans, Rosenstiel School of Marine and Atmospheric Science; Terrence Joyce, Woods Hole Oceanographic Institution; and Victor ZIotnicki, let Propulsion Laboratory. The steering committee extends its thanks for the briefings and other information provided to the Ocean Sciences Data Pane] by Larry Baume, NARA; Donald Collins and Susan Digby, let Propulsion Laboratory; Ronald Fauquet, NOAA; Ted Tsui, Naval Research Laboratory; and R.S. Winokur, Office of Naval Research. The Geoscience Data Pane} was chaired by Theodore Albert, a private consultant. The members were Shelton Alexander, Pennsylvania State University; Sara Graves, University of Alabama in Hunts- ville; David Landgrebe, Purdue University; and Soroosh Sorooshian, University of Arizona. The steering committee gratefully acknowledges the information provided at the meetings of the Geosciences Data Pane] by the following individuals: Roger Barry, National Snow and Ice Data Center; Daniel Cavanaugh, U.S. Geological Survey; Donald Collins, let Propulsion Laboratory; Katrin Douglass, Southern California Earthquake Center Data Center; William Draegar, U.S. Geological Survey; John Dwyer, NARA; Claire Henson, National Snow and Ice Data Center; Herb Meyers, National Geophysical Data Center; Ron Weaver, National Snow and Ice Data Center; and Thomas Yorke, U.S. Geological Survey. Finally, the steering committee is grateful to the staff of the National Research Council: Paul F. Uhlir, associate executive director of the Commission on Physical Sciences, Mathematics, and Applica- tions, who served as study director; Mark David Handel and Theresa Fisher (Board on Atmospheric Sciences and Climate), Alice Killian (Commission on Geosciences, Environment, and Resources), lames E. Mallory (Computer Science and Telecommunications Board), and Scott T. Weidman and Tana Spencer (Board on Chemical Sciences and Technology), who provided staff support for the five panels; Julie M. Esanu, for the program assistance provided to the steering committee and panels and for the preparation of the final manuscript; David Baskin, for his work on preparing the final manuscript; Liz Panos, for coordinating the report review; and Roseanne Price, who edited the final manuscript.
Contents SUMMARY 1 INTRODUCTION Imperatives for Preserving Data on Our Physical Universe, 11 A New Future for Scientific Data, 12 2 THE CHALLENGE: PRESERVATION AND USE OF SCIENTIFIC DATA Experimental Laboratory Data, 13 Observational Data in the Physical Sciences, 15 Summary of Major Issues, 29 RETENTION CRITERIA AND THE APPRAISAL PROCESS Retention Criteria, 33 Other Elements of the Appraisal Process, 39 Recommendations, 40 4 THE OPPORTUNITIES: THE RELATIONSHIP OF TECHNOLOGICAL ADVANCES TO NEW DATA USE AND RETENTION STRATEGIES Enabling Technologies and Related Developments, 43 Opportunities for New Organizational Structures, 47 A NEW STRATEGY FOR ARCHIVING THE NATION'S SCIENTIFIC AND TECHNICAL DATA Fundamental Principles for Long-term Data Retention, 50 The Proposed National Scientific Information Resource Federation, 51 Recommendations for the Creation of the NSIR Federation, 55 Recommendations Specifically for NARA, 57 Recommendations Specifically for NOAA, 59 REFERENCES 1 10 13 33 42 49 62 APPENDIX A List of Acronyms 64 APPENDIX B Minority Opinion 66 IX
This shady is dedicated in fond memory of Marjorie Contain.
