Since 1959 the National Academies of Sciences, Engineering, and Medicine has annually assembled panels of experts—from academia, industry, medicine, and other scientific and engineering communities of practice—to assess the quality and effectiveness of the National Institute of Standards and Technology (NIST) measurements and standards laboratories, of which there are six,1 as well as the adequacy of the laboratories’ resources. These reviews are conducted under contract at the request of the NIST.
In 2020, at the request of the Director of NIST, the National Academies formed the Panel on Review of the Material Measurement Laboratory of the National Institute of Standards and Technology (the “panel”), having earlier established the following statement of work:
The National Academies of Sciences, Engineering, and Medicine Panel on Review of the Material Measurement Laboratory at the National Institute of Standards and Technology will assess the scientific and technical work performed by the National Institute of Standards and Technology (NIST) Material Measurement Laboratory. The panel will review technical reports and technical program descriptions prepared by NIST staff and will visit the facilities of the NIST laboratory. The visit will include technical presentations by NIST staff, demonstrations of NIST projects, tours of NIST facilities, and discussions with NIST staff. The panel will deliberate findings, conclusions, and recommendations in a closed session panel meeting and will prepare a report summarizing its assessment of findings, conclusions, and recommendations.
The assessment shall be responsive to the charge from the NIST Director. The following are the criteria for the assessment:
- The technical merit of the current laboratory program relative to current state-of-the-art programs worldwide;
- The portfolio of scientific expertise as it supports the ability of the organization to achieve its stated objectives;
- The adequacy of the laboratory budget, facilities, equipment, and human resources, as they affect the quality of the laboratory’s technical programs; and
- The effectiveness by which the laboratory disseminates its program outputs.
The panel’s review covered the six divisions and two offices that comprise the MML. The panel conducted its review remotely on September 9, 10, and 11, 2020, working in a synchronous fashion with MML staff and the NIST director, all of whom provided substantive and informative presentations. These presentations were interspersed with discussions between NIST staff and the panel. NIST staff further provided written responses to the panel’s queries.
The panel’s approach to the assessment relied on the experience, technical knowledge, and expertise of its members. Time constraints did not allow the panel to explore all aspects of the MML.
1 The six National Institute of Standards and Technology (NIST) laboratories are the Communications Technology Laboratory, the Engineering Laboratory, the Information Technology Laboratory, the Material Measurement Laboratory, the Physical Measurement Laboratory, and the Center for Neutron Research.
Rather, the panel focused on the research that the leadership of the MML chose to present to it and on a number of issues related to laboratory development that the panel identified as requiring particular attention. The panel’s report includes recommendations that specify “who should do what” to address any determinations as to what might warrant action. The panel underpins the recommendations with salient examples of programs and projects that are intended collectively to portray an overall impression of the laboratory, while preserving useful suggestions specific to projects and programs.
To accomplish its mission, the panel reviewed the material provided by the MML prior to and during the review meeting. The choice of projects to be reviewed was made by the MML. The panel applied a largely qualitative approach to the assessment. Given the nonexhaustive nature of the review, the omission in this report of any particular MML project should not be interpreted as a negative reflection on the omitted project.