The National Institute of Standards and Technology’s (NIST’s) Material Measurement Laboratory (MML) is the nation’s reference laboratory for measurements in the chemical, biological, and materials sciences and engineering. Staff of the MML develop state-of-the-art measurement techniques and conduct fundamental research related to measuring the composition, structure, and properties of substances. Tools that include reference materials, data, and measurement services are developed to support industries that range from transportation to biotechnology and to address problems such as climate change, environmental sciences, renewable energy, health care, infrastructure, food safety and nutrition, and forensics.
At the request of the Director of NIST, the National Research Council formed the Panel on Review of the Material Measurement Laboratory at the National Institute of Standards and Technology and established the following statement of task for the panel:
The Panel on Review of the Material Measurement Laboratory at the National Institute of Standards and Technology will assess the scientific and technical work performed by the National Institute of Standards and Technology (NIST) Material Measurement Laboratory. The panel will attend an orientation session at the NIST facility, will review technical reports and technical program descriptions prepared by NIST staff, and will visit the facilities of the NIST laboratory. Visits will include technical presentations by NIST staff, demonstrations of NIST projects, tours of NIST facilities, and discussions with NIST staff. The panel will deliberate findings in a closed session panel meeting and will prepare a report summarizing its assessment findings.
The Director of NIST requested that the panel focus its assessment on the following factors:
1. Assess the organization’s technical programs.
• How does the quality of the research compare to similar world class research in the technical program areas?
• Is the quality of the technical programs adequate for the organization to reach its stated technical objectives? How could it be improved?
2. Assess the portfolio of scientific expertise within the organization.
• Does the organization have world class scientific expertise in the areas of the organization’s mission and program objectives? If not, what areas should be improved?
• How well does the organization’s scientific expertise support the organization’s technical programs and the organization’s ability to achieve its stated objectives?
3. Assess the adequacy of the organization’s facilities, equipment, and human resources.
• How well do the facilities, equipment, and human resources support the organization’s technical programs and its ability to achieve its stated objectives? How could they be improved?
4. Assess the effectiveness by which the organization disseminates its program outputs.
• How well are the organization’s research programs driven by stakeholder needs?
• How effective are the technology transfer mechanisms used by the organization? Are these mechanisms sufficiently comprehensive?
• How well is the organization monitoring stakeholder use and impact of program outputs? How could this be improved?
The MML was formed in 2011 by combining the former Chemical Science and Technology Laboratory, the Materials Science and Engineering Laboratory, and the portion of Technology Services responsible for production, packaging, and sales of NIST reference materials and reference data products. A subsequent reorganization within the MML in 2012 produced six technical divisions (down from 10) and two offices. The two offices manage programs related to NIST Standard Reference Materials and NIST Data Products.
The technical divisions engage in research and development of the measurement science, standards, technology, and data required to support the nation’s need to design, develop, manufacture, and use materials. These divisions interact extensively with both industry and public institutions to advance the economy and provide tools for creation of knowledge.
The Materials Science and Engineering Division (MSED) provides the measurement science, standards, technology, and data required to support the nation’s need to design, develop, manufacture, and use materials with the intent of advancing technology and facilitating manufacturing in industrial sectors such as energy, electronics, transportation, and the environment. The Materials Measurement Science Division (MMSD) develops state-of-the-art instrumentation, analytical methods, models, and software to accurately and precisely measure materials properties, structure, and composition over a wide range of length and time scales for applications such as public safety, forensics, advanced materials characterization, homeland security, and nanotechnology. The Biomolecular Measurement Division (BMD) develops measurement science and standards for macromolecules such as proteins, nucleic acids, carbohydrates, lipids, and metabolites with applications to biopharmaceuticals, DNA forensics, biomedical and bioscience research, and health care.
The Biosystems and Biomaterials Division (BBD) quantifies characteristics of complex biological systems, materials, and processes, from the nanoscale to the macroscale, with the intent of fostering innovation in biomedicine and health care, manufacturing, food safety, environmental health, and national security. The Chemical Sciences Division (CSD) focuses on chemical composition and the chemical structure of gases, organic, and inorganic species and in the measurement of a wide variety of chemical properties and processes, including chemical reactivity and mechanisms, and thermochemical properties. The Applied Chemicals and Materials Division (ACMD) examines the thermophysical and mechanical properties of chemicals and materials, analyzes the reliability and performance of materials and structures, and creates information systems for chemical and materials engineering, with the intent of fostering innovation and confidence in the nation’s physical and energy infrastructures, enabling advances in chemical manufacturing and in electronics, and promoting sustainability.
The offices have much more clearly defined functions. The MML Laboratory Office houses the MML’s executive leadership (director, deputy director, and executive officer), scientific advisor detailees from the divisions, technical program directors, administrative professionals, and specialists for the functions of safety, communications, grants and contracts, information technology, and human subject protections. The Laboratory Office leads strategic planning, solicits and funds exploratory research, coordinates cross-division scientific programs and the MML’s response to national initiatives, and directs top-level communications with stakeholders and the public. The Office for Reference Materials (ORM) provides business, administrative, product, and technical support for the NIST standard reference materials (SRM) program. This includes the infrastructure to produce, package, inventory, and market SRM products and then sell them to customers worldwide. The Office of Data and Informatics (ODI), formed in 2014, will provide the capacity to handle large and information-rich data sources and transform such data into products, such as NIST standard reference data, that can be reliably and broadly shared.
KEY FINDINGS AND RECOMMENDATIONS
Several common themes emerged across the assessments of the MML divisions. Traditional endeavors at NIST prior to the formation of the MML focused on research to improve measurements and development of standard reference materials and data. The work was performed by small groups in divisions stovepiped by scientific discipline. This type of traditional endeavor continues, but now MML scientists and engineers are expected to interact across disciplinary boundaries and participate in large-scale initiatives that require interdisciplinary collaboration within NIST and across the nation, including increased collaborations with academic, industrial, and other government organizations. The MML is continually refining methods for expanding, coordinating, and improving the efficiency of these interactions. The research-oriented staff generally find this change exciting and enabling, but staff whose primary function has been supporting the production of standard reference materials are, in many cases, unclear about their value in the new organization. It is important that the MML respond to the perceptions of staff that there is a need for improved communication of strategic plans at the laboratory and division levels and of the metrics by which successful performance is judged, and that staff desire increased empowerment with respect to the formulation of plans.
The general caliber of technical expertise is competitive with the best large research institutions in the country, but the products generated are generally more varied, because standard reference data, materials, and methods are just as important at the MML as seminal publications or patents.
The high-quality technical work at the MML, in both traditional activities (supporting the development of standards, standard reference materials, and standard reference data) and nontraditional activities (performing research), supported by an excellent equipment infrastructure, is enabling the MML to meet its technical goals. However, the MML is in high demand by external stakeholders, its relevant technologies are leading-edge and dynamically changing, and there is competition for individuals with the expertise required of its staff. The MML therefore faces challenges with respect to achieving an effective balance between maintaining its success in ongoing efforts and initiating new ones that represent appropriate investment of MML resources in niches that will produce the best impact; appropriately allocating the resources of its staff, who are stretched thin in several areas; arranging for staff travel to and participation in activities that maintain and enrich their expertise; maintaining a proper ratio of scientists to supporting technicians; devising succession plans in anticipation of staff who will leave; maintaining an equipment infrastructure by applying make/buy/borrow/share strategies that provide state-of-the-art capabilities within cost constraints; and ensuring that the mission priorities of MML scientists and NIST administrative and legal staffs are aligned with respect to purchasing of equipment and other resources.
MML staff participate in many standards organizations and other professional organizations, and research staff publish papers in peer-reviewed journals. The MML disseminates its many products broadly and has increasing opportunities to expand its dissemination and outreach activities through improved Internet communications.
The findings and recommendations below address key elements in common themes that focus primarily on opportunities to increase the productivity of the MML as a highly valuable and targeted national resource with the potential to play an even greater role in U.S. innovation and competitiveness and to provide an even more broadly based inventory of metrics for use by industry and regulatory agencies.
The MML continues outstanding technical performance in areas related to NIST’s traditional endeavors—that is, producing reference materials and supporting the development of international standards. In some cases, the groups in the MML have reached their capacity to support production of reference materials. It is critical for national innovation and industrial competitiveness that leadership in standards development and production of reference materials be supported by first-rate staff and state-of-the-art equipment.
Recommendation 1. The Material Measurement Laboratory should develop a strategy, with significant input from stakeholders, especially industry, to determine which standard reference materials (SRMs) should be terminated, continued, or added. Succession planning for technical staff and project leadership should be carefully addressed to make sure that the capabilities and corporate knowledge pertaining to SRMs do not erode over time. Total manpower and the balance between technicians and Ph.D. staff, financial allocations, pursuit of intellectual property to induce industrial production of SRMs, and other aspects of the process of standards production should be included in the strategic evaluation.
The MML is developing reference data and materials to support less well established industries, including standards for biological materials, nanomaterials, and advanced materials-based products. MML staff are working hard to establish and maintain relationships with relevant industrial and regulatory stakeholders. The identification of which standards or measurement technologies will expedite the critical paths for processing and validation is very difficult. The role of area-specific project coordinators to funnel external queries to the appropriate NIST expert is proving extremely useful. Personnel exchanges with industry would be very useful: MML staff would better understand the development-to-product process and industrial partners could provide input for standards selection. For example, industry fellows could spend time at the MML, and MML staff could spend short sabbaticals learning industrial production processes.
Recommendation 2. The Material Measurement Laboratory (MML) should make much more apparent the focal points for channeling external interactions. The MML should increase the number of project coordinators to funnel external queries to the appropriate NIST expert. The MML should ensure that its portion of the NIST website is updated to market MML capabilities and activities more effectively to potential users of standard methods and materials. The MML should develop additional mechanisms to increase its interactions with industry.
The MML is starting new research initiatives and programs to be responsive to national mandates. These initiatives bring researchers together across MML divisions, across NIST, and externally, with the result that MML staff are very aware of needs and trends for the future. The MML has named technical program directors to manage cross-cutting initiatives across division boundaries. However, at the group level, the near-term objectives and expected deliverables are unevenly defined and the quality of the research is highly variable.
Recommendation 3. The Material Measurement Laboratory should perform technical assessments of research projects at the group, division, and laboratory levels to evaluate the balance between productive existing efforts and new initiatives, to ensure a balance that supports the Material Measurement Laboratory mission, is consistent with available resources, and clearly states the objectives and expected products to all participants.
The Office of Data and Informatics (ODI) has recently been established, and its director has been hired. The ODI will play an important role in support of the Materials Genome Initiative (MGI). However, other MML projects and initiatives are producing highly complex data and need to extract meaningful information from that data. Resources capable of manipulating complex data, data mining to produce knowledge, and disseminating the findings are needed across the MML. The reorganization of the MML has provided opportunities for increased collaborations. One example of leveraging external collaboration is the placement of MML staff at Stanford University to develop biological measurement science, where interactions with faculty and students is a win-win experience for both MML and Stanford.
Recommendation 4. The Material Measurement Laboratory should provide the resources required by the Office of Data and Informatics (ODI) as rapidly as possible. The Material Measurement Laboratory (MML) should integrate the ODI into activities across the MML, especially with any divisions producing information on biological/organic systems. To the extent possible, the MML should leverage outside collaborations while building its own capacity; it should consider using the satellite facility at Stanford as a beachhead to strengthen such connections.
The staff is empowered by the continuity of funding. They are enthusiastic about the opportunity to contribute to national innovation and competitiveness by producing better measurement methods and standards. Staff reported that productivity is clearly hampered by slow responses and multiple levels of administrative oversight external to the MML, especially in the areas of contracting for purchases of supplies and equipment and for legal approvals for industry interactions, work for other government agencies, and use of clinical materials. Achieving the mission of the MML suffers when the controlling priority is compliance with legal and contracting procedural stipulations; both mission accomplishment and compliance should be possible if the priorities are rationally balanced.
Recommendation 5. The mission priorities of Material Measurement Laboratory (MML) scientists and NIST administrative and legal staffs should be aligned. The MML should work to ensure that (a) its staff scientists have simplified training in contracting procedures so they can provide the necessary information at the beginning of a procurement process, and so contracting staff can see how they help the MML to achieve its technical mission; (b) unnecessary legal review and rereview is curtailed by bringing the processes into compliance with standards in other government laboratories and by delegating approval authority; and (c) standard times for review are established to manage MML staff expectations and planning.
The recent reorganization of NIST and the MML has brought the research and standards efforts into much closer contact, in many cases resulting in staff members participating in both endeavors. Therefore, the research is more clearly targeted at producing better measurement systems that will facilitate manufacturing and quality control. Nonetheless, the melding of the standards and research cultures is not without challenges. Some MML staff expressed concerns about the management’s perceptions of relative values of traditional activities versus newer research and development activities. Personnel management is uneven. The demonstration project process for personnel evaluation is generally good but needs to be improved. It is administered in different ways in different divisions, and an unacceptable number of supervisors are perceived by staff as ineffective in motivating staff or fostering career development.
Recommendation 6. The Material Measurement Laboratory (MML) should ensure that at the project level the objectives and interrelationship between research and standards development are clearly stated and that supervisors at all levels make clear that both research and standards products/metrics are considered important elements of personnel evaluation, resource allocation, and intellectual regard. The MML should enhance understanding of the performance expectations by providing to all staff annual tutorials covering expectations and metrics for productivity and impact. The MML should establish clear and uniform approaches for creating individual performance plans and reviews; should ensure that reviews are administered using the same process; and should provide supervisor training for all supervisors who have not already taken such training.