An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories

FISCAL YEARS 2004–2005

Board on Assessment of NIST Programs

Division on Engineering and Physical Sciences

NATIONAL RESEARCH COUNCIL OF THE NATIONAL ACADEMIES

THE NATIONAL ACADEMIES PRESS
Washington, D.C. www.nap.edu



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page R1
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Years 2004 – 2005 An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories FISCAL YEARS 2004–2005 Board on Assessment of NIST Programs Division on Engineering and Physical Sciences NATIONAL RESEARCH COUNCIL OF THE NATIONAL ACADEMIES THE NATIONAL ACADEMIES PRESS Washington, D.C. www.nap.edu

OCR for page R1
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Years 2004 – 2005 THE NATIONAL ACADEMIES PRESS 500 Fifth Street, N.W. Washington, DC 20001 NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the board responsible for the report were chosen for their special competences and with regard for appropriate balance. This study was supported by Contract No. SB1341-04-C-0013 between the National Academy of Sciences and the National Institute of Standards and Technology, an agency of the U.S. Department of Commerce. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of the agency that provided support for the project. International Standard Book Number 0-309-10036-4 Copies of this report are available from: Board on Assessment of NIST Programs National Research Council 500 Fifth Street, N.W. Washington, D.C. 20001 Internet, http://www.nap.edu Additional copies of this report are available from the National Academies Press, 500 Fifth Street, N.W., Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area); Internet, http://www.nap.edu. Copyright 2005 by the National Academy of Sciences. All rights reserved. Printed in the United States of America

OCR for page R1
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Years 2004 – 2005 THE NATIONAL ACADEMIES Advisers to the Nation on Science, Engineering, and Medicine The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Wm. A. Wulf is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Wm. A. Wulf are chair and vice chair, respectively, of the National Research Council. www.national-academies.org

OCR for page R1
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Years 2004 – 2005 This page intentionally left blank.

OCR for page R1
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Years 2004 – 2005 BOARD ON ASSESSMENT OF NIST PROGRAMS KENNETH H. KELLER, University of Minnesota, Minneapolis, Chair DAVID C. BONNER, Pretium Consulting Services, LLC OSCAR N. GARCIA, University of North Texas ANTHONY M. JOHNSON, University of Maryland, Baltimore County MARK B. MYERS, Consultant THOMAS A. SAPONAS, Agilent Technologies (retired) (resigned 2004) EUGENE SEVIN, Consultant Ex Officio Members ALAN CAMPION, University of Texas at Austin CONSTANCE J. CHANG-HASNAIN, University of California, Berkeley ROSS B. COROTIS, University of Colorado at Boulder RICHARD A. CURLESS, Cincinnati Lamb ROBERT R. DOERING, Texas Instruments, Inc. NEIL A. DUFFIE, University of Wisconsin, Madison ALBERT M. ERISMAN, Institute for Business, Technology, and Ethics KATHARINE G. FRASE, IBM Microelectronics Division C. WILLIAM GEAR, NEC Research Institute, Inc. ARTHUR H. GUENTHER, University of New Mexico ROBERT J. HITCHCOCK, Lawrence Berkeley National Laboratory DAVID W. JOHNSON, JR., Agere Systems (retired) ERIC W. KALER, University of Delaware JAMES R. KATZER, Exxon Mobil Research and Engineering DUNCAN T. MOORE, University of Rochester National Research Council Staff JAMES P. McGEE, Director PATRICIA P. PAULETTE, Senior Program Officer RADHIKA S. CHARI, Administrative Coordinator ALLISON E. SHOUP, Senior Program Assistant

OCR for page R1
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Years 2004 – 2005 This page intentionally left blank.

OCR for page R1
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Years 2004 – 2005 Preface The mission of the National Institute of Standards and Technology (NIST) Measurement and Standards Laboratories is to promote the U.S. economy and public welfare by providing technical leadership for the nation’s measurement and standards infrastructure and assuring the availability of essential reference data and measurement capabilities. The charge of the National Research Council’s (NRC’s) Board on Assessment of NIST Programs is to provide biennial assessments of the scientific and technical quality, relevance, and effectiveness of the NIST Measurement and Standards Laboratories. This assessment focuses on the technical merit of the laboratory programs relative to the current state of the art worldwide, the effectiveness with which the laboratory programs are carried out and the results disseminated, and the degree to which the laboratory programs are meeting the needs for which they are intended. In addition, the Board was specifically asked by the NIST Director to consider the adequacy of the laboratories’ facilities, equipment, and human resources, insofar as they affect the quality of the technical programs and the effectiveness with which the laboratories meet their customers’ needs. The Board currently consists of 22 leading scientists and engineers whose experience collectively spans the major topics within the scope of NIST. Seven panels, one for each of NIST’s laboratories,1 report to the Board; 15 of the Board members also serve on panels, 7 of them as panel chairs and 8 as panel vice chairs. The panels range in size from 17 to 24 members, whose expertise is tailored to the technical fields covered by the laboratories that they review. In total, 153 experts participated, without compensation, in the 2-year process that led to this report. The Board and panels are appointed by the NRC with an eye to assembling balanced slates of experts without conflicts of interest and with balanced perspectives. The 153 experts include current and 1   The seven NIST laboratories are the Building and Fire Research Laboratory, the Chemical Science and Technology Laboratory, the Electronics and Electrical Engineering Laboratory, the Information Technology Laboratory, the Manufacturing Engineering Laboratory, the Materials Science and Engineering Laboratory (which includes the NIST Center for Neutron Research), and the Physics Laboratory.

OCR for page R1
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Years 2004 – 2005 former executives and research staff from industrial research and development laboratories, leading academic researchers, and staff from other national government laboratories. Fifteen of them are members of the National Academy of Engineering, two are members of the National Academy of Sciences, and a number have been leaders in relevant professional societies. Biographical information on the Board members, along with a listing of the panel membership, appears in Appendix B. The current report is the first biennial assessment by the Board. In the past, since 1959 when the assessment process commenced, annual reports had been produced. Like the earlier reviews, this report contains the Board’s judgments about the quality of NIST’s work. The amount of information that is funneled to the Board, including the consensus evaluations of the recognized experts who make up the Board’s panels, provides a solid foundation for a thorough peer review. A rich set of interactions supports the judgments of the Board, which are summarized in this report. This peer review of NIST is based on a large amount of information received from NIST staff and on interactions with them. Most of the information exchange occurs during annual visits by the panels to the respective NIST sites. This information, gathered during collegial and generally unstructured exchanges between panel members and NIST scientific and technical staff, is supplemented by written material provided by the NIST laboratories to panel members prior to or during the site visits. Additionally, each panel convenes biennially to deliberate on its impressions and findings and to develop formal conclusions and recommendations that are subsequently communicated to the Board. Overview briefings by NIST management keep the panels up to date on NIST’s technical planning, significant events, highlighted accomplishments, and resources. Technical briefings focus on the scientific and technical goals, strategies, methodologies, and results of selected projects at each laboratory. Briefings are targeted with an eye toward coverage of a representative sample of each laboratory’s work over the 2-year assessment cycle. Ample time during both overview and technical briefings is devoted to discussion, both to clarify a panel’s understanding and to convey the observations and impressions of individual panel members to NIST’s scientists and technical staff. The Board and panels applied assessment criteria organized by four categories: Technical quality and merit: criteria that include the extent to which the work demonstrates high technical quality and merit; compares to current state-of-the-art capabilities worldwide; reflects a broad understanding of comparable work being done elsewhere (other government laboratories, universities, and industry); demonstrates links between NIST researchers and the external community; and balances anticipatory, longer-term research and activities that respond to immediate customer needs. Relevance: criteria that include the extent to which the laboratories are pursuing unique measurement and standards activities that clearly support high-priority national needs; are performing work that is clearly tied to NIST objectives; are performing work that is focused on clear and compelling industry/customer needs; are scaling programs appropriately to meet the technical problems being addressed; and are moving programs at a pace and in a direction that is well matched to current and emerging customer needs. Effectiveness: criteria that include the extent to which the laboratories are regularly implementing sound and effective techniques and practices for delivering products and services; make the results of the laboratory work program readily available to customers; and conduct work that is likely to have consequential, long-term impact. Adequacy of facilities, equipment, and human resources: criteria that include the extent to which the available scientific and technical competencies are adequate to achieve success; the state of the equipment and facilities is adequate to meet program objectives and customer needs; and the laboratory is sustaining the technical competencies and capacity to respond quickly to critical issues as they arise.

OCR for page R1
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Years 2004 – 2005 In addition to the primary assessment criteria outlined above, in certain circumstances the Board may judge it useful to explore other issues, either to garner more information relevant to the primary criteria or to deal with specific projects or lines of investigation under way within a laboratory. These considerations include collaboration and crosscutting work within a given NIST laboratory, project-specific methodological considerations underlying the quality and merit of products and results, and project-level technical planning. The Board and its panels are supported by NRC staff, who interact with NIST, the Board, and the panels on an ongoing basis to ensure that the Board and panels receive the information they need to carry out their assessments. Board and panel members serve for finite terms, generally between 2 and 6 years, staggered so that there is regular turnover and a refreshing of viewpoints. In May 2005, the Board met for 2 days to share members’ summaries of their panels’ findings; this report represents the Board’s consensus findings and recommendations. The Board’s aim with this report is to provide guidance to the NIST Director that will help NIST sustain its process of continuous improvement. To that end, the Board examined its extensive and detailed notes from the many Board, panel, and individual interactions with NIST over the fiscal year 2004-2005 period and distilled from them a short list of the main trends, opportunities, and challenges that merit attention at the level of the NIST Director. Specific NIST projects are used to illustrate these points when it is helpful to do so, but the Board did not aim to present the Director with a detailed account of 2 years’ worth of interactions with bench scientists. The draft of this report was subsequently honed and reviewed according to NRC procedures before being released. The report begins with a synopsis intended for NIST stakeholders and an introductory chapter that summarizes the Board’s institution-wide assessment of the NIST Measurement and Standards Laboratories. The synopsis and introductory chapter focus on key themes that apply across the seven laboratories; they are not intended as a distillation or highlighting of the findings described in the subsequent seven chapters, each of which addresses a specific laboratory. The reader is encouraged to peruse those chapters for key conclusions and recommendations specific to each laboratory. Throughout this thorough 2-year assessment, the 153 expert volunteers have devoted considerable attention, time, and energy to attending annual visits to NIST laboratories; carefully examining materials presented to them by NIST; very actively interacting with the NIST staff; meeting with one another to deliberate diligently and to develop formal findings, conclusions, and recommendations; formulating written material documenting their findings; and reviewing one another’s written drafts. The panel chairs and vice chairs, as well as the at-large members of the Board, have also attended Board meetings and worked in a dedicated fashion to develop this report; their untiring efforts are gratefully acknowledged. NIST management and staff have, at all levels, graciously and openly discussed their work with these expert visitors. The Board acknowledges with gratitude their hospitality and their willing and consistent support of its efforts. Kenneth H. Keller, Chair Board on Assessment of NIST Programs

OCR for page R1
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Years 2004 – 2005 This page intentionally left blank.

OCR for page R1
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Years 2004 – 2005 Acknowledgment of Reviewers This report has been reviewed by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the National Research Council’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report: William B. Bridges, California Institute of Technology, Gregory R. Choppin, Florida State University, T. Dixon Dudderar, Chatham, New Jersey, Theodore V. Galambos, University of Minnesota, James N. Gray, Microsoft Corporation, John W. Lyons, National Defense University, Stephen M. Pollock, University of Michigan, Donald E. Ross, Laurel Hollow, New York, and Richard S. Stein, University of Massachusetts, Amherst. Although the individuals listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations, nor did they see the final draft of the report before its release. The review of this report was overseen by General Alton D. Slay, Slay Enterprises, Inc. Appointed by the National Research Council, he was responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution.

OCR for page R1
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Years 2004 – 2005 This page intentionally left blank.

OCR for page R1
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Years 2004 – 2005 Contents     SYNOPSIS OF THE 2004-2005 ASSESSMENT   1 1   INSTITUTIONAL ASSESSMENT   5      Introduction,   5      Strategic Focus Areas,   7      Interinstitutional Collaborations,   9      NIST at Boulder,   10      Balancing New Strategic Foci, New Science, and Traditional Roles,   11      Effective Dissemination Strategies,   12      Adequacy of Staff Resources,   12      Conclusions and Recommendations,   14 2   BUILDING AND FIRE RESEARCH LABORATORY   17      Introduction,   17      Major Observations,   18      Technical Quality and Merit,   19      Relevance,   21      Effectiveness,   23      Resources,   25 3   CHEMICAL SCIENCE AND TECHNOLOGY LABORATORY   29      Introduction,   29      Major Observations,   29      Technical Quality and Merit,   31      Relevance and Effectiveness,   36      Resources,   38

OCR for page R1
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Years 2004 – 2005 4   ELECTRONICS AND ELECTRICAL ENGINEERING LABORATORY   41      Introduction,   41      Major Observations,   41      Technical Quality and Merit,   42      Relevance,   44      Effectiveness,   46      Resources,   48 5   INFORMATION TECHNOLOGY LABORATORY   51      Introduction,   51      Major Observations,   51      Technical Quality and Merit,   53      Relevance,   56      Effectiveness,   57      Resources,   58 6   MANUFACTURING ENGINEERING LABORATORY   59      Introduction,   59      Major Observations,   59      Technical Quality and Merit,   61      Relevance,   63      Effectiveness,   65      Resources,   67 7   MATERIALS SCIENCE AND ENGINEERING LABORATORY   69      Introduction,   69      Major Observations,   69      Technical Quality and Merit,   70      Relevance and Effectiveness,   72      Resources,   75      NIST Center for Neutron Research,   76 8   PHYSICS LABORATORY   79      Introduction,   79      Major Observations,   79      Technical Quality and Merit,   80      Relevance,   84      Effectiveness,   85      Resources,   86     REFERENCES   89     APPENDIXES         A  NIST Organizational Structure and Staffing Trends   93     B  Biographical Sketches of Board and Staff Members and Panel Rosters   103     C  Acronyms and Abbreviations   115