1
The Charge to the Panel and the Assessment Process
At the request of the National Institute of Standards and Technology (NIST), the National Research Council (NRC) has, since 1959, annually assembled panels of experts from academia, industry, medicine, and other scientific and engineering environments to assess the quality and effectiveness of the NIST measurements and standards laboratories, of which there are now seven,1 as well as the adequacy of the laboratories’ resources.
At the request of the Director of NIST, in 2013 the NRC formed the Panel on Review of the Engineering Laboratory at the National Institute of Standards and Technology and established the following statement of task for the panel:
The Panel on Review of the Engineering Laboratory at the National Institute of Standards and Technology will assess the scientific and technical work performed by the National Institute of Standards and Technology (NIST) Engineering Laboratory. The panel will attend an orientation session at the NIST facility, will review technical reports and technical program descriptions prepared by NIST staff, and will visit the facilities of the NIST laboratory. Visits will include technical presentations by NIST staff, demonstrations of NIST projects, tours of NIST facilities, and discussions with NIST staff. The panel will deliberate findings in a closed session panel meeting and will prepare a report summarizing its assessment findings.
The Director of NIST requested that the panel focus its assessment on the following factors:
1. Assess the organization’s technical programs.
- How does the quality of the research compare to similar world class research in the technical program areas?
- Is the quality of the technical programs adequate for the organization to reach its stated technical objectives? How could it be improved?
2. Assess the portfolio of scientific expertise within the organization.
- Does the organization have world class scientific expertise in the areas of the organization’s mission and program objectives? If not, what areas should be improved?
- How well does the organization’s scientific expertise support the organization’s technical programs and the organization’s ability to achieve its stated objectives?
3. Assess the adequacy of the organization’s facilities, equipment, and human resources.
_____________
1 The seven NIST laboratories are the Engineering Laboratory, the Physical Measurement Laboratory, the Information Technology Laboratory, the Material Measurement Laboratory, the Communication Technology Laboratory, the Center for Nanoscale Science and Technology, and the NIST Center for Neutron Research.
- How well do the facilities, equipment, and human resources support the organization’s technical programs and its ability to achieve its state objectives? How could they be improved?
4. Assess the effectiveness by which the organization disseminates its program outputs.
- How well are the organization’s research programs driven by stakeholder needs?
- How effective are the technology transfer mechanisms used by the organization? Are these mechanisms sufficiently comprehensive?
- How well is the organization monitoring stakeholder use and impact of program outputs? How could this be improved?
The context of this technical assessment is the mission of NIST, which is to promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve the quality of life. The NIST laboratories conduct research to anticipate future metrology and standards needs, to enable new scientific and technological advances, and to improve and refine existing measurement methods and services.
In order to accomplish the assessment, the NRC assembled a panel of 23 volunteers, whose expertise matches that of the work performed by the Engineering Laboratory (EL) staff.2
In April 2014, most of the panel members attended one of two orientation sessions provided at the NIST facility in Gaithersburg, Maryland, during which they attended interactive presentations by the NIST Director, the Director of the NIST Program Office, and EL management and staff.
On July 22-24, 2014, the panel assembled for two and a half days at the NIST facility during which it received welcoming remarks from the NIST Director’s representative, heard an overview presentation by EL management, and attended an interactive session with EL management. Each panel member was assigned to one of the five division review teams, whose expertise matched that of the work performed in the five divisions of the EL: (1) Energy and Environment, (2) Fire Research, (3) Intelligent Systems, (4) Materials and Structural Systems, and (5) Systems Integration. The division review teams separately attended division-level presentations and visited division laboratories. The panel also met in a closed session to deliberate on its findings and to define the contents of this assessment report.
The panel’s approach to the assessment relied on the experience, technical knowledge, and expertise of its members. The panel reviewed selected examples of the technical research performed at the EL; because of time constraints, it was not possible to review the EL programs and projects exhaustively. The examples reviewed by the panel were selected by the EL. The panel’s goal was to identify and report salient examples of accomplishments, challenges, and opportunities for improvement with respect to the factors suggested above by the Director of NIST. These examples are intended collectively to portray an overall impression of the laboratory, while preserving useful suggestions specific to the projects and programs that the panel examined. The panel applied a largely qualitative rather than a quantitative approach to the assessment, although it is possible that future assessments will be informed by further consideration of various analytical methods that can be applied.
Given the necessarily nonexhaustive nature of the review, the omission of any particular EL program or project should not be interpreted as a negative reflection on the omitted program or project.
_____________
2 See http://www.nist.gov/el/ for information on Engineering Laboratory organization and programs, accessed August 6, 2014.
