1
The Charge to the Panel and the Assessment Process
At the request of the National Institute of Standards and Technology (NIST), the National Academies of Sciences, Engineering, and Medicine has, since 1959, annually assembled panels of experts from academia, industry, medicine, and other scientific and engineering communities to assess the quality and effectiveness of NIST measurements and standards laboratories, of which there are now seven,1 as well as the adequacy of the laboratories’ resources. The context of this technical assessment is the mission of NIST, which is to promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve the quality of life. NIST laboratories conduct research to anticipate future metrology and standards needs, to enable scientific and technological advances, and to improve and refine existing measurement methods and services.
At the request of the acting director of NIST, in 2017 the National Academies formed the Panel on Review of the Engineering Laboratory at the National Institute of Standards and Technology and established the following statement of task for the panel:
The Panel on Review of the Engineering Laboratory at the National Institute of Standards and Technology will assess the scientific and technical work performed by the National Institute of Standards and Technology (NIST) Engineering Laboratory (EL). The panel will review technical reports and technical program descriptions prepared by NIST staff, and will visit the facilities of the NIST laboratory. The visit will include technical presentations by NIST staff, demonstrations of NIST projects, tours of NIST facilities, and discussions with NIST staff. The panel will deliberate findings in a closed session panel meeting and will prepare a report summarizing its assessment findings.
The acting director of NIST requested that in 2017 the panel confine its assessment to the following smart manufacturing programs conducted at the Engineering Laboratory, which conducts activities in other areas as well: Measurement Science for Additive Manufacturing (MSAM), Robotic Systems for Smart Manufacturing (RSSM), Smart Manufacturing Operations Planning and Control (SMOPAC), and Smart Manufacturing Systems Design and Analysis (SMSDA).
The acting director of NIST also suggested that the panel consider during its assessment the following factors:
- Assess the organization’s technical programs.
___________________
1 The seven National Institute of Standards and Technology (NIST) laboratories are the Engineering Laboratory, the Physical Measurement Laboratory, the Information Technology Laboratory, the Material Measurement Laboratory, the Communication Technology Laboratory, the Center for Nanoscale Science and Technology, and the NIST Center for Neutron Research.
-
- How does the quality of the research compare to similar world-class research in the technical program areas?
- Is the quality of the technical programs adequate for the organization to reach its stated technical objectives? How could it be improved?
- Assess the portfolio of scientific expertise within the organization.
- Does the organization have world-class scientific expertise in the areas of the organization’s mission and program objectives? If not, what areas should be improved?
- How well does the organization’s scientific expertise support the organization’s technical programs and the organization’s ability to achieve its stated objectives?
- Assess the adequacy of the organization’s facilities, equipment, and human resources.
- How well do the facilities, equipment, and human resources support the organization’s technical programs and its ability to achieve its stated objectives? How could they be improved?
To accomplish the assessment, the National Academies assembled a panel of 24 volunteers whose expertise matched that of the work performed by the EL staff.2
On March 28-30, 2017, the panel assembled at the NIST facility in Gaithersburg, Maryland, for a two and a half day assessment, during which it received welcoming remarks from the EL director, heard overview presentations by EL management and presentations by researchers at the EL, toured portions of the EL facility, and attended an interactive session with EL management. The panel also met in a closed session to deliberate on its findings and to define the contents of this assessment report.
The panel’s approach to the assessment relied on the experience, technical knowledge, and expertise of its members. The panel did not attempt to report an exhaustive assessment of every project reviewed. Rather, the panel’s goal was to identify and report accomplishments and opportunities for further improvement with respect to the following: the quality of the technical programs at the EL; the portfolio of scientific expertise within the laboratory; and the adequacy of the laboratory’s facilities, equipment, and human resources. The panel illustrated its conclusions with salient examples of programs and projects that are intended collectively to portray an overall impression of the laboratory, while preserving useful suggestions specific to projects and programs.
To accomplish its mission the panel reviewed the material provided by the EL prior to and during the review meeting. The choice of projects to be reviewed was made by the EL. The panel applied a largely qualitative approach to the assessment. Given the nonexhaustive nature of the review, the omission in this report of any particular EL project should not be interpreted as a negative reflection on the omitted project.
___________________
2 See the NIST Engineering Laboratory website, at https://www.nist.gov/el, for information on the Engineering Laboratory organization and programs (accessed April 21, 2017).
