An Assessment of the National Institute of Standards and Technology Material Measurement Laboratory: Fiscal Year 2014 (2015)

2

Applied Chemicals and Materials Division

The Applied Chemicals and Materials Division (ACMD) consists of 105 staff, who are located in Boulder, Colorado. The ACMD was formed in October 2012 as part of the reorganization of the MML. The reorganization merged the Thermophysical Properties Division (TPD) with the Materials Reliability Division (MRD). These former divisions had very different characters and management styles, which have yet to be completely reconciled. ACMD management presented the following description of its foci:

The ACMD is now organized into a Center (Thermodynamics Research Center) and five groups: Experimental Properties of Fluids, Materials for Biological Environments, Nanoscale Reliability, Structural Materials, and Theory and Modeling of Fluids.

ASSESSMENT OF TECHNICAL PROGRAMS

Accomplishments

Maintaining integrity in everything it does is critical for the ACMD, all of whose programs are built on two critical assets: integrity as an honest broker, with scientific results independent of political, regulatory, and legislative influence; and highest quality equipment and measurement techniques, with the personnel to exploit, maintain, and extend precision measurements.

The commitment to quality is clear and pervasive throughout the ACMD. Management and staff are committed to the quality and precision of measurements. The understanding of the fundamental aspects of the most modern tools and facilities is at the deepest level of detail. The work performed in the ACMD can be broadly classified into two broad types, traditional (supporting the establishment of standards) and nontraditional basic research (focused on measurement science).

All of the traditional work reviewed was excellent—the best or among the best of all such work throughout the world. These projects are long-established and form the core of the ACMD’s work. As one example, the Structural Materials Group (SMG) continually provides SRMs, calibration and measurement services, property data, and predictive models to ensure materials reliability for structural applications.

___________________

¹ National Institute of Standards and Technology Material Measurement Laboratory, “2014 National Research Council Assessment of the NIST Material Measurement Laboratory-Read-Ahead Materials,” Gaithersburg, Md., June 2014.

The SMG has been engaged for over 60 years in mechanical testing; it has also managed the Charpy SRM program for many decades. Similar examples of best-in-the-world excellence in traditional ACMD projects include the Theory and Modeling of Fluids Group’s Reference Fluid Thermodynamic and Transport Properties Database (REFPROP) and the Thermodynamics Research Center (TRC). The TRC has produced a set of data that is the standard around the world. Database utilization is embedded in widely used programs (e.g., ChemKin [chemical kinetics] and ASPEN). The output of these programs supports and enables a variety of research scientists and engineers in both academia and industry.

The nontraditional research projects exhibit a widely varied level in quality, from among the top few worldwide to mid-pack at best. The difference among them lies mainly with the level of innovation and likely scientific impact. All of the projects reviewed utilize or intend to develop the best techniques available. They strive to make highly reliable, precise measurements. In nearly all cases, there was significant industrial interest in each project. All the nontraditional projects are squarely within the NIST mission. The research projects whose work is among the top three in the world are the analytical scanning transmission electron microscopy (TEM) work and the resonance atomic force microscopy (AFM) work (e.g., at the Nanoscale Reliability Group), as well as the nascent work on producing top-quality nanoparticles for a variety of applications, ranging from catalysis to water treatment (e.g., at the Materials for Biological Environments Group). An example of a less innovative project with potential impact is the hydrogen and petrochemical pipeline work at the SMG. This is a traditional NIST specialty area based on high-quality but routine mechanical testing. It has been extended significantly in terms of practical importance recently by the novel importation from the Colorado School of Mines of bacteria reflecting pipeline profiles in order to establish the hydrogen activity at relevant conditions. This is the most reliable practical environment ever created for testing in the world, although otherwise the project is based on standard hydrogen embrittlement/fatigue testing. It is invaluable for industry, but it is not particularly innovative in terms of academic research. The strengths and weaknesses of such an approach are entirely appropriate, reflecting the ACMD’s mission.

Two noteworthy projects had strong cross-cutting objectives. The work on the Materials Genome Initiative (MGI) is not yet at a stage where its quality can be adequately assessed. Very important research could be performed in this area by extending to materials science the database methods developed by the MML for chemical systems. The work on laser welding is also not yet at a stage yet where its quality can be adequately assessed, but the connections are being formed in a promising manner. This program is uniquely coupling the NIST laser expertise with that in process monitoring and materials characterization.

The main focus of the SMG’s programs is in mechanical testing, failure investigations, laser welding, and microbially induced corrosion of structural materials. Thermogravimetric analyses (TGA) give attention to details down to the thermal inertia of the sample holding pans, which is considered imperceptibly small for most research groups yet is accounted for in the uncertainty analyses done at the TRC. Another example is the REFPROP database, where the characterizations of fuel component properties are the best in the world and have been used to help design cooling passages for supersonic flight engines, in which the fuel is used as a coolant.

Understanding the endothermic capability of fuel components is paramount in successful design. There is even recognition that backlash and yaw in a positioning table will result in a quantifiable offset in repeatability. The quantifying of that offset resulted in the installation of independent positioning lasers, completely avoiding reliance on the mechanism of the positioning table. Knowing that there is very slight drift in TGA measurements due to the thermal cycling of the instrument during experimentation is another important element of the required understanding. Therefore, ACMD staff routinely perform calibration before and after an experimental campaign to quantify the drift and incorporate it into the statistical uncertainty of the measurement.

Opportunities and Challenges

The SMG has several new opportunities to expand the scope of its work in numerous NIST initiatives, including those focused on the hydrogen economy, alternative energy, additive manufacturing, and laser welding. The MGI, a cross-cutting effort, is an important initiative that facilitates the extension of the chemical database technology to materials science.

Currently the quality at the TRC is adequate. However, there is a risk in succession planning. The people are the most valuable resource in the research groups. They understand the equipment and the data that are provided by that equipment down to the greatest level of detail. This is developed through a series of steps ranging from understanding the theory of operation to the physical design to limitations of the systems. These steps are a process in learning, using and assimilating acquired knowledge either first hand or through discussions with experienced team members. If there is not sufficient overlap of recent hires with experienced senior scientists, the groups risk having significant intellectual capital not being completely transferred and leveraged.

Overall Assessment of Technical Programs

The SMG fulfills the ACMD mission of providing industry and other customers with measurement science, standards, and technology in structural materials. Researchers at the TRC have produced a set of data that is considered the standard around the world. The research in nanoscale materials with the Nanoscale Reliability Group emphasizes understanding of the measurement technique and the incipient errors; this research is of excellent quality and comparable to the best research performed internationally. The SMG’s research programs directly address the main ACMD mission of providing industry and other customers with measurement science, standards, and technology in the area of structural materials.

PORTFOLIO OF SCIENTIFIC EXPERTISE

The people in the ACMD are its most valuable resource. Their determination and creativity have enabled development of some of the translational techniques for which ACMD is recognized. The participation of ACMD personnel on standards committees is noteworthy.

The scientific experience of the SMG can be easily applied to the new opportunities identified above. Some research may be necessary to understand the new challenges such as those encountered in unusual microstructural development in laser welding or the creation of unusual hydrogen activities in microbially induced corrosion of steels. The TRC has expertise comparable to the best scientists in the world. Also, central to the mission of setting standards is the work done by the Nanoscale Reliability Group, which demonstrated the ability to image nanoparticles of diameter 5 nm to 10 nm using a TEM with energies approaching those of a scanning electron microscope (SEM). This has direct application to catalysis and the rational design of catalytically active material.

Sufficient expertise currently exists to support and execute programs. Both junior and senior staff have expressed frustration with the administrative aspects of their work.

The scientific expertise of the ACMD is adequate to perform the required duties. The scientific expertise of the SMG is adequate for the current work as well as for the new opportunities, which are or can be undertaken by the SMG. Recently, the ACMD has built up capabilities in computational materials science to better understand the initiation of fracture.

ADEQUACY OF FACILITIES, EQUIPMENT, AND HUMAN RESOURCES

In general, the experimental facilities in the ACMD are excellent. They include all of the standard characterization and mechanical property equipment. The SMG possesses a variety of equipment for mechanical testing (deformation, fatigue, and fracture), with extensive strain measurement and temperature capability, corrosion testing at high hydrogen pressures coupled with mechanical testing, and Charpy verification. It also has an excellent metallographic laboratory, a microcryocooler test facility, and acoustic emission calibration equipment. This key equipment is adequate to perform the intended work of the SMG. Three technicians maintain and operate this equipment.

The Precision Imaging Facility at NIST Boulder’s Precision Measurement Laboratory is a state-of-the-art facility that includes TEM, SEM, atom probe, helium ion microscope, and focused ion beam (FIB) microscopes. The facility appears to be well-supported and has the appropriate staff to run it efficiently. It has a good balance of support projects and technique development.

There is concern with respect to the equipment needed to develop the capability and technique to measure nanoparticles. Based on the discussions with MML staff, if the Nanoscale Reliability Group had not managed, through great patience and perseverance, to assemble the necessary resources, the group’s breakthrough might never have happened. Furthermore, it appears difficult for the group to get an updated apparatus to take the measurement science to the next level. The large investment of time and energy devoted to equipment acquisition seems in this case to have been a constant uphill battle, and there is risk that staff may view taking the project beyond the proof-of-concept stage as not worth the effort to initiate this new idea.

There is also concern about mid-career engineers and scientists—namely, there are a small number of mid-career personnel and an associated issue with keeping them engaged, encouraging them to become champions of new directions, and visualizing a path to promotion. Problems and frustrations with the administrative staff seem to contribute to mid-level career burnout. If not improved, this could turn into a problem with succession planning. There needs to be more attention to attracting, hiring, and retaining mid-career personnel. Federal employee positions generally require U.S. citizenship. To attract the best talent in the engineering and science fields, limiting the pool to U.S. citizens may not be the best policy.

Based on tours, discussions with staff engineers and scientists, and an examination of the precision of some of the data generated, it appears the facilities and equipment are adequate. The engineers and scientists understand the equipment to the point where they can modify it to obtain new ways to measure physical properties (such as the AFM vibration test for hardness).

Attention is needed to improve the interest of early-career engineers and scientists in the more mature fields. In addition it appears that two projects were terminated because there was not enough funding to build a critical mass of research activity. It is encouraging that projects can be stopped if it is not possible to obtain data on the needed quality. The nanoneurotoxicity project is an example where it was not possible to obtain reliable cell cultures to generate data sets.

Some groups are not working in novel, leading-edge fields. They could face the risk of not attracting the highest caliber of next-generation scientists and engineers if resources, communicating the importance of a field, and active recruiting are not provided.

The MML director and the ACMD division chief are well-liked and well-respected, and many staff would welcome interacting with them more, particularly on technical matters. Both are good communicators, are open and honest, and seem to have an intimate knowledge of their staff and projects. They seem to provide balanced support. However, both individuals need assistants to give them more time to interact with subordinates individually. The director and the chief seem to have been instilling a better sense of security following the merger, a laudable achievement. All staff with whom discussions were held were willing to talk freely and to express both positive and negative comments. That there is no fear of retribution for being honest is a strong compliment for management style.

Although not stated explicitly by the staff, the rubric “respect for the professional” could subsume the various comments and impressions expressed by some staff. This overarching area demands

improvements; those improvements will not require new resources or major changes and can be accomplished over a few years. Examples of such comments (paraphrased) are these:

• There is insufficient transparent communication about management goals and personnel evaluation processes. Performance review standards sometimes change. ACMD management has not been able to achieve adequate cooperation from support staff, including administrators who configure computer equipment and move office equipment without consulting staff, procurement officers who impose processes that significantly delay procurement times, contractors who do not provide desired items, and legal officers who impose counterproductively lengthy reviews.

• Staff are asked to complete numerous surveys that are conceived by a contractor without consulting the staff to determine meaningful questions, and so participation rates are poor, and a common expectation is that nothing will change in response to survey results.

• Staff do not know the strategic plan for the ACMD. Management decisions seem to lack a known strategic context.

• Staff are not involved in hiring, promotion, or compensation decisions.

• Management practices vary from group to group and in some cases from year to year. Group leaders do not receive management training. There are no commonly understood metrics allowing the performance of one individual or group to be compared with that of others.

In addition, new resources may be allocated to a given project without consulting the research staff. This makes maintaining core resources more difficult and necessitates the siphoning off of resources to support core work. There is a struggle between keeping ongoing activities strong while getting resources for new initiatives.

Discussions with managers and postdoctoral researchers suggested a need to speed up the processes supporting procurement and travel.

Overall, the facilities, equipment, and technician resources available to the ACMD are adequate to perform the division’s work in established as well as in emerging areas. ACMD staff reported that the procurement and legal systems lead to extreme frustration of the staff. The ACMD staff indicated that administrative staff do not appear to work well in support of the research staff and technical managers at the ACMD; this may be one factor contributing to turnover for some research staff (five have recently left). Some of them perceive that management does not evince sufficient respect for the professional staff.

One way to improve scientific productivity is to ensure that administrative obstacles are removed for the staff members. This action will send a positive message and keep staff motivated while encouraging others to initiate projects. For example, if the procurement process is streamlined, it will ensure that the staff members are focused on projects instead of burdensome paperwork. The procurement and legal processes could be considered as partnerships between the technical and service organizations, with decisions made according to cost/benefit to NIST.

DISSEMINATION OF OUTPUTS

The ACMD has done an excellent job of disseminating its output in the traditional area. It has developed an extremely strong customer base and reports that these customers appreciate the work done at the ACMD. This overall area was the easiest to recognize and appreciate within the TRC. The team does a very good job of active publication, seminar and conference presentations, and workshop participation. In addition, there is active engagement with regulating organizations such as the Food and Drug Administration (FDA) and the Environmental Protection Agency (EPA) and with standardization entities such as the International Organization for Standardization (ISO), the American National Standards Institute (ANSI), and the American Society for Testing and Materials (ASTM).

The SMG influences codes and standards for testing and designing with structural materials. It has provided eight Charpy SRMs to support the certification of Charpy machines for more than 25 years

for manufacturers and users worldwide; it evaluates more than 1,000 machines per year from around the world for compliance with ASTM and ISO standards. It is the leading facility for cryogenic flow meter calibrations. The SMG conducts independent, expert investigations into failures of structural materials. In contrast, the dissemination of output in the research area is average in comparison with other research facilities, including universities and national laboratories.

Impact factors and h-indexes from the Institute for Scientific Information (ISI) allow assessing the impact of journal publications. The use of a persistent chemical identifier helps to more accurately track ACMD data used in embedded programs (e.g., ASPEN, ANSYS). ACMD staff monitor and identify stakeholder use, enabling direct discussions with and feedback from all who use their products. For example, the TRC has a consortium consisting of ASPEN personnel, petrochemical manufacturers, and refinery owners.

One improvement needed is to track the progress of new initiatives assessing the quality, pace of progress, and potential payoffs of research. One way this could be done is to hold an open house periodically, where the research champions present their ideas and describe current progress via webinars. This could serve a dual purpose: helping the champion identify and communicate the salient aspects of the project and alerting the community to upcoming techniques and processes coming out of the ACMD.

The SMG has the opportunity to publish in scientific journals its research results in new areas such as laser welding and microbially induced corrosion.

Overall, the ACMD does an excellent job of disseminating their output in the traditional area. Dissemination of output by the TRC is excellent. The SMG disseminates its work through its influence on codes and standards for testing and designing with structural materials. It is well recognized for providing Charpy SRMs worldwide.

FINDINGS AND RECOMMENDATIONS

In general, the traditional projects (e.g., SRMs and databases) are of excellent quality. These activities are prized by the customers in each area as evidenced by communications, purchases of SRMs, and adoption of standards. Examples include setting standards, providing SRMs, and creating and maintaining databases of the best data available (e.g., REFPROP, TRC).

In many cases, the ACMD is the only organization performing this important work worldwide. However, the ACMD is a changing entity, recently reorganized and with many personnel changes. It is important that traditional areas closely related to standards be maintained in spite of current or looming threats. These threats include difficulty in hiring top talent in traditional areas, perceptions that programs in research areas are favored over programs in the traditional standards areas, and difficulty in articulating a commonly understood plan for balancing the two types of activity.

The average nontraditional project is solid technically but lacks the innovation commonly associated with top academic research. This is not necessarily a weakness; rather, in many cases it reflects the strong and beneficial connections to the ACMD mission and direct response to the needs of industry. The nontraditional projects need not aspire to world-class quality in the academic sense. However, there are a few projects that would be considered top-notch at any academic institution and that also connect strongly with the traditional work.

The ACMD has the highest-quality equipment and measurement capability, with the personnel to maintain and facilitate this quality. The ACMD should maintain these excellent capabilities, and ACMD management should work with MML management to ensure that the new tax on capital does not interfere with the excellence of these capabilities.

To the extent that it is generalizable, some staff’s perception of poor institutional respect for the professional is a primary threat to continued technical excellence in the ACMD. The following indicators of this perception were suggested by ACMD staff: salaries are low by comparison with salaries of peers employed elsewhere; top-down management and lack of transparency with respect to strategic planning, funding decisions, and personnel performance criteria contribute to a sense of lack of empowerment; lack of training of group leaders results in inconsistency of management styles and practices; and numerous cumbersome administrative, procurement, and legal practices detract from the focus on scientific and technical work and inhibit opportunities for managers to interact with staff.

Major aspects of the management of the ACMD are effective in promoting quality and merit in its technical output. The ACMD chief has instilled an improved sense of security among staff that is a remarkable success in view of past funding uncertainties and the recent reorganization of the ACMD. The chief has an open-door policy but insufficient time to initiate routine informal technical interactions. The chief is widely liked and respected by the ACMD staff as a manager and scientist and is seen as open and honest. These positive results could be reinforced by more visits to group meetings, holding semiannual all-staff meetings at which strategic issues are discussed, and sponsoring informal social events. Such expansion of communication efforts requires time that could be made up for by delegating other responsibilities.