1
The Charge to the Panel and the Assessment Process
At the request of the National Institute of Standards and Technology, the National Research Council has, since 1959, annually assembled panels of experts from academia, industry, medicine, and other scientific and engineering environments to assess the quality and effectiveness of the NIST measurements and standards laboratories, of which there are now seven,1 as well as the adequacy of the laboratories’ resources.
At the request of the Director of NIST, in 2013 the National Research Council formed the Panel on Review of the Material Measurement Laboratory at the National Institute of Standards and Technology and established the following statement of task for the panel:
The Panel on Review of the Material Measurement Laboratory at the National Institute of Standards and Technology will assess the scientific and technical work performed by the National Institute of Standards and Technology (NIST) Material Measurement Laboratory. The panel will attend an orientation session at the NIST facility, will review technical reports and technical program descriptions prepared by NIST staff, and will visit the facilities of the NIST laboratory. Visits will include technical presentations by NIST staff, demonstrations of NIST projects, tours of NIST facilities, and discussions with NIST staff. The panel will deliberate findings in a closed session panel meeting and will prepare a report summarizing its assessment findings.
The Director of NIST requested that the panel focus its assessment on the following factors:
1. Assess the organization’s technical programs.
• How does the quality of the research compare to similar world class research in the technical program areas?
• Is the quality of the technical programs adequate for the organization to reach its stated technical objectives? How could it be improved?
2. Assess the portfolio of scientific expertise within the organization.
• Does the organization have world class scientific expertise in the areas of the organization’s mission and program objectives? If not, what areas should it be improved?
• How well does the organization’s scientific expertise support the organization’s technical programs and the organization’s ability to achieve its stated objectives?
3. Assess the adequacy of the organization’s facilities, equipment, and human resources.
• How well do the facilities, equipment, and human resources support the organization’s technical programs and its ability to achieve its state objectives? How could they be improved?
___________________
1 The seven NIST laboratories are the Engineering Laboratory, the Physical Measurement Laboratory, the Information Technology Laboratory, the Material Measurement Laboratory, the Communication Technology Laboratory, the Center for Nanoscale Science and Technology, and the NIST Center for Neutron Research.
4. Assess the effectiveness by which the organization disseminates its program outputs.
• How well are the organization’s research programs driven by stakeholder needs?
• How effective are the technology transfer mechanisms used by the organization? Are these mechanisms sufficiently comprehensive?
• How well is the organization monitoring stakeholder use and impact of program outputs? How could this be improved?
The context of this technical assessment is the mission of NIST, which is to promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve the quality of life. The NIST laboratories conduct research that anticipates future metrology and standards needs, enables new scientific and technological advances, and improves and refines existing measurement methods and services.
In order to accomplish the assessment, the NRC assembled a panel of 19 volunteers whose expertise is in the same areas as the work performed by the MML staff.2
In April 2014, most of the panel members attended one of two orientation sessions provided at the NIST facility in Maryland. They attended interactive presentations by the NIST Director, the Director of the NIST Program Office, and MML management and staff, and were given tours of the NIST facility.
The entire panel assembled for a day and a half at the NIST facilities in Gaithersburg, Maryland, on June 17-19, 2014, during which time it heard welcoming remarks from the NIST Director’s representative and an overview presentation by MML management. Six individual division review teams separately attended division-level presentations and visited the laboratories of the MML divisions: Applied Chemicals and Materials Division, Biomolecular Measurement Division, Biosystems and Biomaterials Division, Chemical Sciences Division, Materials Measurement Science Division, and Materials Science and Engineering Division. Only the Applied Chemicals and Materials team visited the Boulder, Colorado, facility. The panel also met in a closed session to deliberate on its findings and to begin drafting this assessment report.
The approach of the panel to the assessment relied on the experience, technical knowledge, and expertise of its members, whose backgrounds were carefully matched to the technical areas of MML activities. The panel reviewed selected examples of the technical research covered by the MML; because of time constraints, it was not possible to review the MML programs and projects exhaustively. The examples reviewed by the panel were selected by the MML. The panel’s goal was to identify and report salient examples of accomplishments and opportunities for further improvement with respect to the following: the quality of the technical programs at the MML; the portfolio of scientific expertise within the laboratory; the adequacy of the laboratory’s facilities, equipment, and human resources; and the effectiveness with which the MML disseminates its program outputs. The examples were intended collectively to portray an overall impression of the laboratory, while providing useful suggestions specific to projects and programs that the panel examined. The panel applied a largely qualitative rather than a quantitative approach to the assessment, although it is possible that future assessments will be informed by various analytical methods that can be applied.
The comments in this report are not intended to address each program within the MML exhaustively. Instead, the report identifies key issues. Given the necessarily nonexhaustive nature of the review process, the omission of any particular MML program or project should not be interpreted as a negative reflection on the omitted program or project.
___________________
2 See http://www.nist.gov/mml/ for more information on Material Measurement Laboratory programs, accessed July 15, 2014.
