The Charge to the Panel and the Assessment Process

At the request of the National Institute of Standards and Technology, the National Academies, through the National Research Council, has since 1959 annually assembled panels of experts from academia, industry, medicine, and other scientific and engineering environments to assess the quality and effectiveness of the NIST measurements and standards laboratories, of which there are now nine,2 as well as the adequacy of the laboratories’ resources. In 2008, NIST requested that five of its laboratories be assessed: the Building and Fire Research Laboratory, the Manufacturing Engineering Laboratory, the Materials Science and Engineering Laboratory, the NIST Center for Neutron Research, and the Physics Laboratory. Each laboratory was assessed by a separate panel of experts; the findings of the respective panels are summarized in separate reports. This report summarizes the findings of the Panel on Building and Fire Research.

For the FY 2008 assessment, NIST requested that the panel consider the following criteria as part of its assessment:

  1. The technical merit of the current laboratory programs relative to the current state of the art worldwide;

  2. The adequacy of the laboratory facilities, equipment, and human resources, as they affect the quality of the laboratory technical programs; and

  3. The degree to which the laboratory programs in measurement science and standards achieve their stated objectives and desired impact.

The panel adopted the following additional assessment criteria under each broad factor to make them more explicit:

  1. The technical merit of the current laboratory programs relative to the current state of the art worldwide.

    • Relative Technical Caliber: How does the technical quality of the laboratory compare to current state-of-the-art capabilities worldwide?

    • Relevance: Do the projects reflect a broad understanding of comparable work being done elsewhere (at other government laboratories, universities, and industry)?

    • Balance: Does the laboratory adequately balance anticipatory, longer-term research and activities that respond to immediate customer needs?

  1. The adequacy of the laboratory facilities, equipment, and human resources, as they affect the quality of the laboratory technical programs.

    • Available Tools: Is the state of the equipment and facilities adequate to meet project objectives and customer needs?

    • Critical Mass: Are the available scientific and technical competencies adequate to achieve success? Is available funding adequate to achieve

2

The nine NIST laboratories are the Building and Fire Research Laboratory, the Center for Nanoscale Science and Technology, the Chemical Science and Technology Laboratory, the Electronics and Electrical Engineering Laboratory, the Information Technology Laboratory, the Manufacturing Engineering Laboratory, the Materials Science and Engineering Laboratory, the NIST Center for Neutron Research, and the Physics Laboratory.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 5
The Charge to the Panel and the Assessment Process At the request of the National Institute of Standards and Technology, the National Academies, through the National Research Council, has since 1959 annually assembled panels of experts from academia, industry, medicine, and other scientific and engineering environments to assess the quality and effectiveness of the NIST measurements and standards laboratories, of which there are now nine,2 as well as the adequacy of the laboratories’ resources. In 2008, NIST requested that five of its laboratories be assessed: the Building and Fire Research Laboratory, the Manufacturing Engineering Laboratory, the Materials Science and Engineering Laboratory, the NIST Center for Neutron Research, and the Physics Laboratory. Each laboratory was assessed by a separate panel of experts; the findings of the respective panels are summarized in separate reports. This report summarizes the findings of the Panel on Building and Fire Research. For the FY 2008 assessment, NIST requested that the panel consider the following criteria as part of its assessment: 1. The technical merit of the current laboratory programs relative to the current state of the art worldwide; 2. The adequacy of the laboratory facilities, equipment, and human resources, as they affect the quality of the laboratory technical programs; and 3. The degree to which the laboratory programs in measurement science and standards achieve their stated objectives and desired impact. The panel adopted the following additional assessment criteria under each broad factor to make them more explicit: 1. The technical merit of the current laboratory programs relative to the current state of the art worldwide. Relative Technical Caliber: How does the technical quality of the laboratory compare to current state-of-the-art capabilities worldwide? Relevance: Do the projects reflect a broad understanding of comparable work being done elsewhere (at other government laboratories, universities, and industry)? Balance: Does the laboratory adequately balance anticipatory, longer-term research and activities that respond to immediate customer needs? 2. The adequacy of the laboratory facilities, equipment, and human resources, as they affect the quality of the laboratory technical programs. Available Tools: Is the state of the equipment and facilities adequate to meet project objectives and customer needs? Critical Mass: Are the available scientific and technical competencies adequate to achieve success? Is available funding adequate to achieve 2 The nine NIST laboratories are the Building and Fire Research Laboratory, the Center for Nanoscale Science and Technology, the Chemical Science and Technology Laboratory, the Electronics and Electrical Engineering Laboratory, the Information Technology Laboratory, the Manufacturing Engineering Laboratory, the Materials Science and Engineering Laboratory, the NIST Center for Neutron Research, and the Physics Laboratory. 5

OCR for page 5
success? Agility: Is the laboratory sustaining the technical competencies and capacity to respond quickly to critical issues as they arise? 3. The degree to which the laboratory programs in measurement science and standards achieve their stated objectives and desired impact. Technical Planning: Are appropriate milestones identified and do they appear feasible? Dissemination: Is the laboratory regularly implementing sound and effective techniques and practices for delivering products and services to customers? Impact: Will the laboratory products have a consequential, long-term impact? Responsiveness: Are the research projects moving at a pace and in a direction that is well matched to current and emerging customer needs? In addition, because NIST has begun to receive increases in funding through the President’s ACI and the America COMPETES Act of 2007, the Director of NIST requested that the assessment panels specifically examine and review the progress of all of the FY 2007-funded initiatives relevant to their respective laboratories and comment on these program growth areas explicitly in their reports. At the BFRL, that work focuses on disaster-resilient structures and communities, including efforts related to hurricanes, wildland fires, and earthquakes (including community-scale loss estimation). Specifically, the panel was asked to examine and review progress on the following growth areas: the National Earthquake Hazards Reduction Program, wind engineering, multi-hazard failure analysis, and fires at the wildland-urban interface. The context of this technical assessment is the mission of NIST, which is to promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve the quality of life. The NIST laboratories conduct research to anticipate future metrology and standards needs, to enable new scientific and technological advances, and to improve and refine existing measurement methods and services. To accomplish the assessment, the NRC appointed a panel of 18 volunteers whose expertise closely matched that of the work performed by the BFRL staff. On the basis of their own individual expertise areas, the panel members were also assigned to five Strategic Priority Areas that organized the research programs within the BFRL into core competency areas aligned with high-level BFRL goals. The five Strategic Priority Areas are (1) Measurement Science for Building Energy Technologies, (2) Measurement Science for Breakthrough Improvements in Construction Productivity, (3) Measurement Science for Predicting Life Cycle Performance of Nanocomposite Infrastructure Materials, (4) Disaster Resilient Structures and Communities (Hurricanes and Earthquakes), and (5) Disaster Resilient Structures and Communities (Fires). The BFRL Web site provides additional information on BFRL programs.3 Each of the panel’s Strategic Priority Area subgroups separately visited the BFRL staff working in its respective area for 1 day, during which it attended presentations, tours, and demonstrations and had a number of interactive sessions with the staff. These 3 See http://www.bfrl.nist.gov/ for more information on BFRL programs. Accessed August 12, 2008. 6

OCR for page 5
reviews were conducted at the NIST facilities in Gaithersburg, Maryland, on March 18, 2008. Immediately following the subgroup visits, the entire panel assembled for a day- and-a-half meeting at the BFRL facility in Gaithersburg; during that time it attended interactive sessions with the laboratory managers, and the panel also met in a closed session to deliberate its findings and to define the contents of this assessment report. The BFRL provided the panel with background materials, including a recent BFRL annual report; information on the BFRL budget, staffing, awards, and standards activities; a BFRL organizational chart; the NIST programmatic plan for FY 2009-2011; and a list of BFRL publications for FY 2007. BFRL staff also set up a Web site for panel members that provided additional links to further reference materials. The approach of the panel to the assessment relied on the experience, technical knowledge, and expertise of its members, whose backgrounds were carefully matched to the technical areas within which the BFRL activities are conducted. The panel reviewed selected examples of the standards and measurements activities and the technological research presented by the BFRL. It was not possible to review the BFRL programs and projects exhaustively. The goal of the panel was to identify and report representative examples of accomplishments and to highlight opportunities for further improvement with respect to the technical merit of the BFRL work and its perceived impact with respect to achieving its own definition of its objectives, and to highlight specific elements of the resources and infrastructure within BFRL that are intended to support the technical work. These highlighted examples for each BFRL Strategic Priority Area are intended to collectively portray an overall impression of the laboratory, while preserving useful mention of specific suggestions for projects and programs that the panel considered to be of special note within the set of those examined. The assessment is currently scheduled to be repeated biennially. While the panel applied a largely qualitative rather than a quantitative approach to the assessment, it is possible that future assessments will be informed by further consideration of various analytical methods that can be applied. The comments in this report are not intended to address each program within the BFRL exhaustively. Instead, this report identifies key issues and focuses on representative programs and projects relevant to those issues. Given the necessarily nonexhaustive nature of the review process, the omission of any particular BFRL program or project should not be interpreted as a negative reflection on the omitted program or project. 7