The creation of the Material Measurement Laboratory (MML) has yielded positive results. The discipline-specific projects for producing standards for traditional industries are evolving into flexible, collaborative, cross-disciplinary endeavors that are much more appropriate for addressing the needs of modern industry. Teams are formed within and across MML divisions that are capable of developing technology and standards for the characterization of advanced materials, such as biological therapeutics, nanomaterials, and organic electronics. Standards for such products will not only promote innovation and expedite development of manufacturing processes but will also facilitate evaluation and quality control for commercialization and interaction with regulatory organizations. The knowledge, technology, reference standards, and data developed by MML teams will contribute to quality assurance tools for addressing problems of national importance such as climate change, renewable energy, health care, infrastructure, food safety and nutrition, forensics, and homeland security.
Two factors have had the most impact on the evolution of the MML: (1) the collocation of staff members doing research into measurement systems and staff focused on characterization, production, and dissemination of standard reference materials and data and (2) mandated initiatives that are conducted by staff from many departments. The collocation of research and production staff has led to some confusion about the relative value of the different endeavors. However, the research staff are more focused on potential opportunities for new standards and measurement methods than they may have been earlier, while the individuals focused on production of reference materials are exposed to a broader spectrum of possible uses and opportunities for new standards. Both types of staff members take pride in their work, and senior management recognizes the value of both endeavors. More communication, more strategic staffing, and better personnel management training could facilitate sustainability of these interactions and the respective functions.
The mandated projects are demanded as a result of national initiatives, and MML management and staff want to address these important problems. In some cases, such as forensics, preexisting projects are reoriented to address the mandates to the extent possible. For additive manufacturing or safety of nanomaterials, MML management, staff, and external stakeholders do a very good job of defining a program that spans divisions to take advantage of relevant expertise and facilities and specifying potential objectives that will make a critical difference to the initiative in terms of measurement capabilities, reference materials and data, and technology transfer. Based on reports by MML staff, one area where these cross-cutting programs could improve is communicating the measures for success to staff members doing the work in the various divisions, so that each individual can understand how their part fits into the whole or what balance of their effort is appropriate for different projects. According to comments from the MML staff, most of the technical expertise required to support the cross-cutting initiatives is available within the MML or the broader NIST community, although identifying appropriate participants may be difficult. If the biobased initiatives are expanded, additional staff may be required in relevant areas, because these biologically oriented disciplines have not been pursued to such a significant extent at NIST. The expansion of the informatics operation (at Office of Data and Informatics, ODI) is essential because the MML needs increasingly to evaluate complex data sets and extract meaningful knowledge for assessing the validity and relevance of the measurements and extracting knowledge.
In general, the quality of the research programs is comparable to that at other premier research institutions. While interesting scientific observations are valued, the work is very much targeted at the production of new measurement methods, data, and standards of use in knowledge generation and manufacturing. The staff publishes in refereed journals at a reasonable rate, with occasional papers published in tier 1 journals, but most of the publications are more archival in nature and can be found in specialty journals where they will reach an application-specific audience. The patent process is not well developed at NIST, and this is problematic for the development of intellectual property, which might be useful for expanding the scope of the MML mission by encouraging commercialization of MML-developed measurement technologies and production of standard reference materials.
As the focus of MML activities evolves and the need for new types of standards and reference materials becomes clear, there is a concern that the areas in which the MML has previously excelled may be lost. For example, NIST has an international reputation for characterization of ceramics and for mass spectrometry of chemicals. Such expertise needs to be maintained as the expansion into biomaterials and nanomaterials receives increased attention.
The MML is very well supported in terms of facilities and equipment, but MML staff reported that administrative support at the NIST level for purchasing, contracts, and legal review slow technical progress. The equipment provided to support the technical programs is generally state-of-the-art. However, the new overhead structure decreases funds available for equipment, and the maintenance costs are considerable. The overhead structure was changed to encourage the devotion of more resources to manpower. This is not necessarily a bad policy, but the MML will be less able to purchase one-of-a-kind items like the Brookhaven beamline or the 900 MHz nuclear magnetic resonance (NMR) machine. The allocation of space needs to be reevaluated from the standpoint of laboratory functionality in order to optimize efficiency and reduce clutter from outdated or unused equipment. Purchasing procedures, contracts management, and legal review are not timely, even compared to other government laboratories. Such operations should be reviewed independently and streamlined as much as possible to help accomplish the NIST mission.
The MML works hard to involve its stakeholders at all levels, by establishing programmatic priorities, collaborating on individual projects, and providing feedback on the utility of reference materials. Topic-specific workshops seem to work particularly well. Communication with stakeholders is an ongoing challenge that demands constant attention. The establishment of project leaders who coordinate outside queries and interests with internal experts has been a useful tactic that should be expanded. Creating and maintaining a more informative website would also be helpful. The MML is cognizant that interactions with potential industrial, government, and academic beneficiaries of their programs is critical to making the right choices in project directions and utilizing the technologies and reference materials developed, and MML staff are constantly exploring new approaches to encourage interactions. In addition to the sales of reference materials, information is distributed to the public by other means such as dissemination of web-based data sets, inclusion of data with commercial instruments (e.g., mass spectrometry library), and publications, and the success of these methods is monitored with the goal of ongoing improvement. Licensing technology has not been a significant mechanism for technology transfer; this is an opportunity wasted. More support for modern web-based communications, faster approvals for collaborations and materials exchanges with industry and other government agencies, and more flexible travel policies would all improve the sharing of MML expertise with stakeholders.
Traditional endeavors at NIST prior to the formation of the MML focused on research to improve measurements and development of standard reference materials and data. The work was performed by small groups in divisions stovepiped by academic area. This type of traditional endeavor continues, but now MML scientists and engineers are expected to interact across disciplinary boundaries and participate in large-scale initiatives that require interdisciplinary collaboration within NIST and across the nation. The research-oriented staff generally find this change exciting and enabling, but staff whose primary function has been supporting production of standard reference materials are, in many cases, unclear about their value in the new organization. The general caliber of technical expertise is competitive with the best large research institutions in the country, but the products generated are generally more varied, because
standard reference data, materials, and methods are just as important at the MML as seminal publications or patents.
The findings and recommendations below focus primarily on opportunities to increase the productivity of the MML as a highly valuable and targeted national resource with the potential to play an even greater role in U.S. innovation and competitiveness and to provide an even more broadly based inventory of metrics for use by industry and regulatory agencies.
The MML continues outstanding technical performance in areas related to NIST’s traditional endeavors—that is, producing reference materials and supporting the development of international standards. In some cases, the groups in the MML have reached their capacity to support production of reference materials. It is critical for national innovation and industrial competitiveness that leadership in standards development and production of reference materials be supported by first-rate staff and state-of-the-art equipment.
Recommendation 1. The Material Measurement Laboratory should develop a strategy, with significant input from stakeholders, especially industry, to determine which standard reference materials (SRMs) should be terminated, continued, or added. Succession planning for technical staff and project leadership should be carefully addressed to make sure that the capabilities and corporate knowledge pertaining to SRMs do not erode over time. Total manpower and the balance between technicians and Ph.D. staff, financial allocations, pursuit of intellectual property to induce industrial production of SRMs, and other aspects of the process of standards production should be included in the strategic evaluation.
The MML is developing reference data and materials to support industries that are not as well established, including standards for biological materials, nanomaterials, and advanced materials-based products. MML staff are working hard to establish and maintain relationships with relevant industrial and regulatory stakeholders. The identification of which standards or measurement technologies will expedite the critical paths for processing and validation is very difficult. The role of area-specific project coordinators to funnel external queries to the appropriate NIST expert is proving extremely useful. Personnel exchanges with industry would be very useful: MML staff would better understand the development-to-product process and industrial partners could provide input for standards selection. For example, industry fellows could spend time at the MML, and MML staff could spend short sabbaticals learning industrial production processes.
Recommendation 2. The Material Measurement Laboratory (MML) should make much more apparent the focal points for channeling external interactions. The MML should increase the number of project coordinators to funnel external queries to the appropriate NIST expert. The MML should ensure that its portion of the NIST website is updated to market MML capabilities and activities more effectively to potential users of standard methods and materials. The MML should develop additional mechanisms to increase its interactions with industry.
The MML is starting new research initiatives and programs to be responsive to national mandates. These initiatives bring researchers together across MML divisions, across NIST, and externally, with the result that MML staff are very aware of needs and trends for the future. The MML has named technical program directors to manage cross-cutting initiatives across division boundaries. However, at the group level, the near-term objectives and expected deliverables are unevenly defined and the quality of the research is highly variable.
Recommendation 3. The Material Measurement Laboratory should perform technical assessments of research projects at the group, division, and laboratory levels to evaluate the balance between productive existing efforts and new initiatives, to ensure a balance that
supports the Material Measurement Laboratory mission, is consistent with available resources, and clearly states the objectives and expected products to all participants.
The ODI has recently been established, and its director has been hired. The ODI will play an important role in support of the Materials Genome Initiative (MGI). However, other MML projects and initiatives are producing highly complex data and need to extract meaningful information from that data. Resources capable of manipulating complex data, data mining to produce knowledge, and disseminating the findings are needed across the MML. The reorganization of the MML has provided opportunities for increased collaborations. One example of leveraging external collaboration is the placement of MML staff at Stanford University to develop biological measurement science, where interactions with faculty and students is a win-win experience for both MML and Stanford.
Recommendation 4. The Material Measurement Laboratory (MML) should provide the resources required by the Office of Data and Informatics (ODI) as rapidly as possible. The MML should integrate the ODI into activities across the MML, especially with any divisions producing information on biological/organic systems. To the extent possible, the MML should leverage outside collaborations while building its own capacity; it should consider using the satellite facility at Stanford as a beachhead to strengthen such connections.
The staff are empowered by the continuity of funding. They are enthusiastic about the opportunity to contribute to national innovation and competitiveness by producing better measurement methods and standards. Staff reported that productivity is clearly hampered by slow responses and multiple levels of administrative oversight external to the MML, especially in the areas of contracting for purchases of supplies and equipment and for legal approvals for industry interactions, work for other government agencies, and use of clinical materials. Achieving the mission of the MML suffers when the controlling priority is compliance with legal and contracting procedural stipulations; both mission accomplishment and compliance should be possible if the priorities are rationally balanced.
Recommendation 5. The mission priorities of Material Measurement Laboratory (MML) scientists and NIST administrative and legal staffs should be aligned. The MML should work to ensure that (a) its staff scientists have simplified training in contracting procedures so they can provide the necessary information at the beginning of a procurement process, and so contracting staff can see how they help the MML to achieve its technical mission; (b) unnecessary legal review and rereview is curtailed by bringing the processes into compliance with standards in other government laboratories and by delegating approval authority; and (c) standard times for review are established to manage MML staff expectations and planning.
The recent reorganization of NIST and the MML has brought the research and standards efforts into much closer contact, in many cases resulting in staff members participating in both endeavors. Therefore, the research is more clearly targeted at producing better measurement systems that will facilitate manufacturing and quality control. Nonetheless, the melding of the standards and research cultures is not without challenges. Some MML staff expressed concerns about the management’s perceptions of relative values of traditional activities versus newer research and development activities. Personnel management is uneven. The demonstration project process for personnel evaluation is generally good but needs to be improved. It is administered in different ways in different divisions, and an unacceptable number of supervisors are perceived by staff as ineffective in motivating staff or fostering career development.
Recommendation 6. The Material Measurement Laboratory (MML) should ensure that at the project level the objectives and interrelationship between research and standards
development are clearly stated and that supervisors at all levels make clear that both research and standards products/metrics are considered important elements of personnel evaluation, resource allocation, and intellectual regard. The MML should enhance understanding of the performance expectations by providing to all staff annual tutorials covering expectations and metrics for productivity and impact. The MML should establish clear and uniform approaches for creating individual performance plans and reviews; should ensure that reviews are administered using the same process; and should provide supervisor training for all supervisors who have not already taken such training.