6

Materials Measurement Science

INTRODUCTION

The Materials Measurement Science Division (MMSD) is one of the largest divisions within the Material Measurement Laboratory (MML), with a total of 179 members: 99 National Institute of Standards and Technology (NIST) scientists, 5 technicians, 7 support staff, 3 fellows, and 65 NIST associates. Owing to the NIST and then the MML reorganizations in 2011, the MMSD was formed via the merger of three independent entities: the Ceramics Division from the Materials Science and Engineering Laboratory (MSEL); the Surface and Microanalysis Science Division (SMSD) from the Chemical Science and Technology Laboratory (CSTL); and a section of the Office of Law Enforcement Standards (now the Security Technologies Group). While most of the MMSD staff and facilities are in Gaithersburg, Maryland, the Synchrotron Science Group is located at the new National Synchrotron Light Source II (NSLS-II) facility at Brookhaven National Laboratory in Upton, New York. This group is responsible for the technical development, construction, and maintenance of three NIST beam line instrumental end stations. From a funding perspective, the MMSD is unique within the MML in that the division receives approximately 25 percent of its support from seven U.S. government agencies outside the Department of Commerce (DOC).

The MMSD conducts a mixture of priority-based fundamental research, standards production, and applied science that couples well with metrology efforts to address stakeholder needs. The MMSD maintains a very broad scientific portfolio encompassing five program areas: the Materials Genome Initiative (MGI); advanced materials; nanometrology; safety, security, and forensics; and synchrotron science. The strategic plan is to ensure that project goals and deliverables are clearly tied to NIST, MML, and stakeholder programmatic needs. Such goals are in place in order to provide managers the ability to analyze division and group portfolios to facilitate strategic and informed decision making. They also allow managers and staff to view competencies in a manner that permits the MMSD to become even more agile and effective in responding to NIST and MML priorities.

ASSESSMENT OF TECHNICAL PROGRAMS

Accomplishments

The Microscopy and Microanalysis Research Group performs fundamental research, develops metrological methodologies, and disseminates results toward the compositional and morphological characterization of materials—from the mesoscale to the atomic scale—using electron, ion, and photon interactions with matter. Through detailed measurements, comprehensive analysis and modeling,¹

___________________

¹ NIST, Microscopy and Microanalysis Research Group, https://www.nist.gov/mml/materials-measurement-science-division/microscopy-and-microanalysis-research-group, accessed September 25, 2017.

theoretical methods, and multidisciplinary microscopy, this group advances and promotes microanalysis to address stakeholder needs in diverse arenas of materials science.

The Nano Materials Research Group develops innovative metrology to advance nanomaterial research and applications for the benefit of U.S. commerce. Best-in-the-world measurement science is both developed and used to determine the physical and chemical properties of a wide variety of organic, inorganic, biomolecular, and hybrid systems, where materials of interest fall within the nanoscale regime.² The resulting standards, methods, reference methods (RMs), and measurement data advance a wide range of technologies and stimulate innovation.

The Materials for Energy and Sustainable Development Group conducts research and develops and disseminates measurement science, data, standards, and technology that pertain to energy-related materials, and materials for sustainable development. Using the MGI approach, the group explores and evaluates the efficient use of materials in manufacturing, transportation, infrastructure, and water. Its activities encompass materials efficiency (conservation, use of earth-abundant elements, substitution, reusing, repurposing, recycling, and lightweighting), materials life cycle assessment, critical materials, materials flow analysis, and mitigation of undesirable impacts on the environment and on human health.

The Surface and Trace Chemical Analysis Group develops, improves, and standardizes analytical techniques used for the elemental, organic, isotopic, radiological, and morphological characterization of surfaces, thin films, and particles. It also develops novel methods of chemical analysis based on optical microscopy, mass spectrometry, chromatography, ion mobility spectrometry, spectroscopy, autoradiography, nuclear counting, and nuclear track methods.³ It also conducts research in generation and size calibration of particle standards. Additionally, the group researches fundamental aspects of the trace detection of explosive and narcotic particles (including the use of computational fluid dynamics and advanced flow visualization). It also develops standard test materials by application of advanced inkjet dispensing technologies, and it applies multiple surface, trace chemical, and nuclear analysis methods to problems in forensics, additive manufacturing, materials science, semiconductor technology, bioscience, and homeland security.

The Synchrotron Science Group develops and disseminates measurement science and technology pertaining to the measurement of the structure, including chemical and electronic, of advanced materials by synchrotron methods. The group provides, maintains, and supports synchrotron measurement capabilities as part of the DOE User Facility Program,⁴ which is primarily located at the NSLS-II.

The Materials Structure and Data Group develops and disseminates measurement science, standards, and technology for determination of the structure of advanced materials. The group determines, compiles, evaluates, and disseminates key data and computational tools tools (such as those for the prediction/interpretation of spectroscopy data) needed to establish the relationships between the structure and performance of inorganic and hybrid materials and devices.⁵

The Nanomechanical Properties Group develops and disseminates measurement science, standards, and technology pertaining to the measurement of mechanical properties of advanced materials and structures at the nanoscale. The group determines, compiles, evaluates, and disseminates key data needed to establish the relationships between structure, mechanical properties, and the performance of inorganic and hybrid materials and devices.

___________________

² NIST, Nano Materials Research Group, https://www.nist.gov/mml/materials-measurement-sciencedivision/nano-materials-research-group, accessed September 25, 2017.

³ NIST, Surface and Trace Chemical Analysis Group, https://www.nist.gov/mml/materials-measurement-science-division/surface-and-trace-chemical-analysis-group, accessed September 25, 2017.

⁴ NIST, Synchrotron Science Group, https://www.nist.gov/mml/materials-measurement-sciencedivision/synchrotron-science-group, accessed September 25, 2017.

⁵ NIST, Materials Structure and Data Group, https://www.nist.gov/mml/materials-measurement-sciencedivision/materials-structure-and-data-group, accessed September 25, 2017.

The Securities Technology Group conducts research and develops and disseminates the measurement science to support the development of performance-based standards and to promote the advancement of their technologies. Such technologies are used in trauma-mitigating materials and material systems; the sensing of concealed threats and contraband; advanced composites; high-performance materials; and safety, security, and forensics.

Opportunities and Challenges

The technical programs within the MMSD are excellent and there are many talented staff members and enthusiastic postdoctoral researchers. It is clear there is a strong culture around the NIST mission with a strong commitment to technology and knowledge transfer. The staff clearly understands that it is uniquely positioned to satisfy national needs, and it works effectively with industry. It is clear there is a willingness to engage with customers, as illustrated by its diverse set of partners and collaborators. It is unclear, however, what the core strategy of the division is and how the different groups fit into the division. How are future program trajectories determined, and how does that align with where the division sees itself in 1, 5, or 10 years?

There seems to be uneven awareness of overlapping expertise and equipment across the groups and divisions. Also, being spread over eight buildings creates some barriers to natural interactions. The MMSD leadership will need to break down silos and create opportunities and incentives for the staff to connect. There are current grass-roots efforts to keep connected by having teas, seminars, and picnics, but it is not clear how well this is working.

It is questionable how the MMSD focus on ballistics work fits into the core NIST mission. The MMSD needs to make sure there is a good connection between the Security Technologies Group and the division. It also needs to connect the ballistic body armor work to similar work at Department of Energy (DOE) and the Department of Defense (DOD) laboratories. Similarly, there is work in trace chemical detection and sensors at other DOE laboratories, and it is unclear whether it is leveraging or communicating with those other efforts.

As an organization, the MMSD is recognized and valued, nationally and internationally, for its technical excellence. Building core competencies at this high-performance level is, by nature, often a career-long endeavor. Yet, as the pace of change in scientific discoveries and technological advancements accelerates, the ability to quickly respond to new materials science problems is increasingly important to MMSD’s stakeholders. The rapid response challenge is twofold: the ability to quickly adapt existing measurement competencies to meet new needs, and the timely development or acquisition of new competencies to address novel measurement science problems. Such challenges exist at both the programmatic and business-of-NIST (procurement) levels.

Program leaders within the MMSD are faced with an ever-increasing need to rapidly assess stakeholder needs, prioritize these needs, assemble (and acquire, if necessary) the required expertise, and develop and complete new programs and projects. Developing the necessary agility to deliver to customers and stakeholders without compromising excellence will require NIST to adopt a new operational culture that integrates a focus on rapid response to stakeholder needs, recruiting the required expertise, and developing new programs.

The MMSD needs to ensure that it not only maintains the high levels of expertise expected of a national metrology institute but also can deploy necessary competencies quickly and effectively. Balancing the need for a long-term research program to maintain excellence in core competencies that serve the existing customer base with the ability to be responsive to new customers’ needs is a challenge.

PORTFOLIO OF SCIENTIFIC EXPERTISE

Accomplishments

The MMSD staff consists of 99 NIST scientists, 5 technicians, 7 support staff, 3 fellows, and 65 NIST associates. The staff members are scientific leaders in the areas of X-ray methods, electron microscopy, and surface analysis. Materials measurement science in the MMSD is applied at multiple time and length scales, and spans a diverse range of scientific expertise.

The Synchrotron Science Group at the Brookhaven NSLS-II is noteworthy for its leadership and innovation in synchrotron spectroscopy and imaging methods, particularly in the regime of soft Xrays. The successful execution of its long-range plan to develop cutting-edge and, in some cases, one-of-a-kind end stations is to be commended. The investments and developments there are enabling cutting-edge methods that will provide new capabilities in measurement science.

The expertise in the Materials Structure and Data Group is strong in local structure and subnanoscale measurements, including point defects and complex real-world structures. For complex, real-world structures, the group develops first-principles-based methods for prediction of atomic arrangements and properties and also studies advanced ceramics and inorganic materials, which are critically enabling elements in real-world devices and complex systems across many technology sectors, such as telecommunications and energy. Collaborations with external industry partners appear to be fruitful and are strengthening the impact of the groups. The expertise in crystallographic and phase equilibria databases remains among the best in the world and exemplifies a key role that NIST plays in supporting industry and the larger scientific community.

In the Microcopy and Microanalysis Research Group, the expertise in metrology and compositional and morphological characterization of materials is creating advances with tangible impact to measurement science. Advances in measurement science instrumentation for atom probe and multimodal microscopy are highlights of this group’s expertise.

The Surface and Trace Chemical Analysis Group shows NIST’s role in both measurement science and real world impact. The group has the expertise to push measurement science for trace chemicals and surface analysis with tangible connection to detection needs for security and forensics.

The Materials for Energy and Sustainment Group has expertise in energy-related materials; materials for sustainable development; the efficient use of materials in manufacturing, transportation, infrastructure and water and material efficiency (conservation, use of earth-abundant elements, substitution, reusing, repurposing, recycling, and lightweighting); materials life cycle assessment; and mitigation of undesirable impacts on the environment and human health.

The Nanomechanical Properties Group has developed special expertise in the technology pertaining to the measurement of mechanical properties of advanced materials and structures at the nanoscale. The group has the ability to determine, compile, evaluate, and disseminate key data needed to establish the relationships between structure, mechanical properties, and performance of inorganic and hybrid materials and devices.

The Security Technologies Group has expertise in the development of performance-based standards and the advancement of the technologies used in trauma-mitigating materials (such as those used in helmets to decrease the incidence of traumatic brain injuries) and material systems and the sensing of concealed threats and contraband.

The Nano Materials Research Group has established expertise in nanoparticle separation and analysis. The nanometrology of particles for development of NIST standards is well aligned with NIST’s mission. The applied research in this group is good, and the structure and chemical characterization expertise is excellent. It is encouraging to see the integration of the scientific staff with the broader community in areas that are too big for one group or institution to tackle.

From speaking with the MMSD postdoctoral fellows and early career staff, it is exciting to see the level of diversity both in breadth of expertise as well as with regards to representation from underrepresented groups.

Opportunities and Challenges

While the MMSD work on nanoparticle separation and high-throughput analysis is very good (it includes measurements and standards for scanning probe microscopy, nanoscale strength, and nanoscale stress), it is not evident that the MMSD nanoparticle expertise is equally strong in all areas of this broad and rapidly expanding field. What innovations are being considered in anticipation of the near-future needs is also not evident. The Materials and Structure Data Group might benefit from increased collaboration with the Synchrotron Science Group at Brookhaven.

The through-barrier radar systems (a pulse-Doppler system using a narrow-band swept-frequency source operating between about 500 MHz and 3.2 GHz, which supports measurement science test methods, test artifacts, and data analysis) are becoming available and the work being performed by the MMSD to develop standards for their performance is extremely timely and important.

The competency database is a good idea for identifying scientific expertise; however, there is a risk in relying on individual input to populate it. The MMSD needs to leverage other databases from which key words could be extracted, such as from publications and presentations.

The junior staff is excellent and reflects the success of the National Research Council (NRC) Associateship Program to attract energetic researchers to the division. They appear very happy with the research environment and are well mentored by the senior staff. Most would like to stay in a permanent position, but they have expressed concern about the increasing demands of paperwork and restrictions on travel.

Because of the dearth of technicians, it appears Ph.D. scientists do routine tasks like equipment maintenance, which is a poor use of their skills and resources and limits productivity. There is a strong need for technician support also in the area of information technology (IT), particularly in view of the mandate to make all data public. More technicians would make the staff more efficient. The postdoctoral fellows are not meant to be primary equipment maintainers.

Long-term process improvements and sound decision making will be required. Additionally, the MMSD needs to leverage other laboratories (the DOE Office of Science and the National Nuclear Security Administration [NNSA], in particular) and universities (the University of Michigan, Stanford University, etc.). There needs to be transparency of how to succeed, especially for early career staff. The MMSD needs to leverage NIST’s postdoctoral network with a professional development program in order to improve connectivity across NIST.

While the staff is excellent, it receives few external awards and has little professional society engagement. External awards for NIST staff bring visibility to the caliber of work and the contribution of NIST to the scientific community. While the MMSD personnel win some external awards, they do not win the number they deserve for the quality of the people and the quality of the work that they do. External awards won by the MMSD include the following: the IEC’s 1906 award, the Roon Foundation award, Fellow of the ACS, Fellow of the Royal Society of Chemistry, ASTM Committee F12 Award of Excellence, the International Centre for Diffraction Data (ICDD) Distinguished Fellow Award, American Ceramic Society (ACS) Robert B. Sosman Award and Lecture, and the ACS James I. Mueller Award and Lecture. Of these awards only two—the ASTM Committee F12 Award of Excellence and the ICDD Distinguished Fellow Award—were won in 2017; the rest were won in 2014. With more organized internal effort, more personnel could become fellows and award winners of the professional societies. There needs to be more of a push for external awards, and involvement in professional societies needs to be more encouraged.

ADEQUACY OF FACILITIES, EQUIPMENT, AND HUMAN RESOURCES

Accomplishments

Overall, the quality and quantity of the equipment at the MMSD is excellent, consistent with the MMSD mission to pursue state-of-the-art metrology, microscopy, and spectroscopy. Within the division, there are a number of capabilities to measure and characterize materials that are unique, either to NIST or to the broader international research community.

The Safety, Security, and Forensics program within the MMSD strives to provide the underpinning measurement science needed to advance threat detection, improve the accuracy of forensics measurements, and ensure the reliability of protective technologies and materials in ways that enable homeland security, the safety of public servants, and effective law enforcement. Chemical microscopy and particle characterization techniques have been developed in the MMSD to evaluate the threat signatures associated with trace explosives detection. A database of the attributes and morphologies of particles in fingerprints and other residues associated with the handling of explosive powders and formulations has been assembled, thereby setting realistic criteria for materials used in technology development and testing. The MMSD has developed several new ambient ionization mass spectrometry techniques that allow, for the first time, chemical analysis of both inorganic and organic explosive and narcotic compounds on surfaces. This development expands the analysis of contraband materials to the analytically challenging but critically important area of homemade explosive formulations—which are difficult to detect with currently deployed technology.

The MMSD maintains two through-barrier sensing and imaging capabilities: (1) a narrow-band doppler-radar-based system (≈3.6 GHz) that can sense humans located inside buildings, buried in rubble, or obscured by foliage, thereby targeting applications in law enforcement and emergency response; and (2) a broadband (≈200 MHz to 5 GHz) holographic imaging system, with a large aperture size (≈20 m × 4.5 m) to provide large-area images of objects behind walls. In addition, the MMSD, in collaboration with the Physical Measurement Laboratory (PML), has developed a state-of-the-art video-rate passive submillimeter-wave imager for detecting threats concealed on a person up to 20 miles away. This imager holds great promise for standoff threat detection. Further, a testbed is maintained for evaluating the imaging performance of the portable X-ray systems used by domestic and military bomb squads, the ubiquitous carry-on baggage X-ray inspection systems, and the backscatter and transmission X-ray systems for body screening. These provide the basis for the development of test artifacts, objective and automated image analysis routines, and international documentary performance standards.

The main objective of the Nanometrology program is to develop or create new measurement techniques and applicable standards to meet the needs of various stakeholders that rely on high-quality nanometer-scale material characterization and associated technologies.

A key measurement capability for advanced materials involves the use of contact resonance atomic force microsocopy (AFM) to measure the mechanical properties (elastic modulus) of surfaces at the nanoscale. Recent MMSD efforts have focused on refining models, calibration, and standards—extending the range of applicability to a wider range of harder materials. It has also focused on the development of a new technique—intermittent contact resonance atomic force microscopy—to provide additional depth-sensing capability on nanostructured materials.

The Surface and Trace Chemical Analysis Group has an excellent set of commercially based instruments, including secondary ion imaging and tomographic atom probe analysis coupled with laser excitation of surface atoms.

NIST and the DOE have had a 35-year ongoing partnership at the National Synchrotron Light Source (now NSLS-II) developing advanced synchrotron measurement methods and delivering excellence in material science. NIST is currently constructing an NSLS-II spectroscopy suite of three state-of-the-art high-throughput beam lines (with X-ray diffraction capability); soft and tender X-ray spectroscopy and microscopy (100 eV to 7.5 kEv) and hard X-ray absorption spectroscopy and diffraction (4.5 kEv to 22 kEv). The suite is expected to be completed in 2017. The NIST NSLS-II Spectroscopy Beamline Suite

will be able to measure the electronic, chemical, and structural properties of almost any material, often at the nanoscale. This facility will provide unique opportunities for collaborative research within the MMSD, with MML personnel across divisions (including the Boulder, Colorado, facility), and with the broader user community—including numerous industrial partners.

Opportunities and Challenges

Across the MMSD, the majority of facilities were in excellent condition. In contrast with some other divisions, the MMSD has facilities that largely match the needs of its staff and instruments. One challenge is that facilities are distributed across eight buildings, which can require more effort to make the groups aware of each other’s capabilities. Also, for new capabilities that are at another site (such as at NSLS-II), it will be important to raise awareness across the MML of new capabilities that could be leveraged by other groups.

During the past three years, NIST has made large investments in two important MMSD-led research facilities that were developed in partnership with the DOE. These facilities are the NIST NSLS-II beam lines, which are located at Brookhaven National Laboratory and the Facility for Adsorbent Characterization and Testing (FACT) laboratory, which was developed by funding from the DOE’s Advanced Research Projects Agency-Energy (ARPA-E) and is located at the NIST campus in Gaithersburg, Maryland. NIST’s NSLS-I beam line suite of instruments has been very successful in accelerating the development of new materials into devices and systems with advanced capabilities. The NIST’s NSLS-I user community has averaged about 100 users per year. Building on this success, the goal for the new NSLS-II is to grow that community as new instruments come on line later in FY 2017. DOE infrastructure is in place to support the process of hosting users, and the MML Laboratory Office has provided support to increase internal and external outreach efforts. The MMSD will maximize the use of this facility by NIST staff and associates, and has been reaching out to additional external users. The challenge for both facilities involves growing the user community with existing staff resources (i.e., without devoting additional resources). If successful, this will provide the opportunity to further extend the MMSD outreach into the materials science community.

The scientific instrumentation is generally of extraordinary quality across many of the groups. Some pieces of equipment are unique, with cutting-edge capabilities, and some are among the best in the world, matching the quality of the staff. There is a lot of high-end capital equipment, and there are questions about how to maintain that investment and maintain or upgrade the facilities infrastructure. For example, how does the MMSD decide what to invest in next, what to replace, and what to let go of? One of the challenges here will be continued investment in maintaining and replacing key instruments.

The human resources are highly accomplished, collaborative, and effective in their equipment usage. The MMSD has been very successful with recruiting diverse, early career scientists, but there are some concerns around intentional fostering for retention. A challenge for the MMSD will be in how to invest in staff members to position them for professional growth and to nurture their retention at NIST. Exemplifying careers that bridge fundamental science up to tech transfer is a good idea. Additionally, the MMSD could use workshops to help raise knowledge of all staff. Having a stable workforce is positive, but it also makes it more difficult for staff promotions. Additionally, the MMSD needs to improve succession planning.

DISSEMINATION OF OUTPUTS

Accomplishments

During the last 3 years, the MMSD personnel have published 467 papers in archived journals, 44 conference proceedings, 21 NIST reports, and 18 books and book chapters. The division’s publications

are in a broad range of fields, reflecting its diverse research activities, and over the past year, have included publications in high-impact journals such as Nature Materials, Nano Letters, ACS Nano, Advanced Energy Materials, Advanced Functional Materials, Chemistry of Materials, and Small. Many of the papers are coauthored with researchers outside of NIST—an indicator of successful collaborations.

Patenting is secondary to many other success metrics for almost all MMSD staff researchers, and during the last 3 years there was only 1 patent issued, 1 published, and 3 more filed.

The MMSD works closely with industrial partners. During the last 3 years, it has held 21 workshops, including one workshop on the Qualification of Uncertainties in Material Sciences, another one on Measurement Needs in Adsorption Sciences, and a third workshop on a focused ion beam scanning electron microscope (FIB SEM). There have been one Cooperative Research and Development Agreement (CRADA) and 42 interagency agreements with seven different agencies.

The division has developed three commercial swipe technologies, and three printer systems have been commercialized. It sells approximately 1700 RM units per year, of which there are almost 80 different types. These materials cover a variety of needs, including X-ray diffraction characterization and calibration, particle size (including nanoparticle sizing), glass viscosity, refractive index, surface area, and chemical composition. About 20 percent of the customers are from academia; 20 percent from state, local, and federal governments; and 60 percent from industry.

Examples of new MMSD products include SI-traceable standard reference materials (SRMs) for calibration of varying diffraction measurement methods. This product is highly effective, and there is hardly an X-ray diffractometer in the world that does not use the NIST standard—roughly a third of new instruments sold worldwide include a NIST standards diffraction package.

The MMSD also provides SRDs covering crystallographic and structural information, phase diagrams, surface spectroscopy, ceramic properties, and X-ray and image analysis, all of which are used worldwide. These standards are a major resource for researchers, and the MMSD expends enormous efforts to incorporate image processing and theoretical/statistical methods within materials databases to expedite experimental data standardization and evaluation.

The MMSD representation on standards-setting committees is outstanding and is a significant output of the division. Many MMSD staff members actively participate on committees in 130 international standards organizations (primarily American Society for Testing and Materials [ASTM] and International Organization for Standardization [ISO]). This participation includes standards committee leadership. Noteworthy examples of standards activities include the following:

1. Absolute Intensity Calibration Standard for Small-angle X-ray Scattering (SAXS; SRM 3600). SAXS measurements characterize the microstructure and nanostructure of heterogeneous material systems—specifically, size distributions of microscale and nanoscale features, volume fractions or number concentrations, and surface areas.
2. The Securities and Technology Group led a significant revision of the internationally used documentary performance standard for cabinet X-ray systems used for carry-on baggage at airports and other security checkpoints.
3. The suite of six X-ray diffraction (XRD) SRM’s provide SI-traceability for calibration of all diffraction measurement methods. All of the manufacturers of the XRD instruments worldwide include a NIST diffraction SRM(s) as the calibration standard.
4. The Fachinformationszentrum Karlsruhe (FIZ)/NIST Inorganic Crystal Structure Database (ICSD; SRD 84) is a collection of crystal structure data for nonorganic compounds including inorganics, ceramics, minerals, pure elements, metals, and intermetallics.
5. The MMSD initiated a collaborative effort with the Food and Drug Administration (FDA) and industry members to develop a new standard test method for size measurement using dynamic light scattering.

Finally, the MMSD has generated for other agencies significant outputs that, for security reasons, are not included in its list of publically available publications.

Opportunities and Challenges

Across the MMSD, the internal assessment of output seems satisfactory and recognizes the different expectations placed on different individuals. Communicating tailored expectations of output based on the role and projects of each staff member needs to be continued. Staff members need to feel empowered knowing the output requirements of each activity they undertake—whether it is authoring a publication, developing an SRM, or developing a new instrument or technique. Staff members need to feel secure knowing that those outputs will be valued in merit reviews and decisions, and ongoing efforts to communicate the various paths to success at NIST need to be continued.

The MMSD has a solid publication rate. For future National Academies laboratory reviews, it would be good to see the author list separated by visiting faculty versus staff to see what is NIST-driven and what is external collaborator-driven. While both first authorship and coauthorship of publications appear to be valued, it seemed that the former was regarded as having higher value. In characterization work, it can be that the paper was largely enabled by the contributions of one author, regardless of final author order. And so, discerning value of the author’s contribution to the paper could enable staff members to directly articulate the impact of their work.

One other aspect of output is the rigor of the data collected. The MMSD largely excels in this area, applying statistical analysis to optimize the quality of data and conclusions. One challenge here though is in the consistency of applying experimental design and statistical analysis across all groups. The uniform application of experimental design (when the project is conducive to it) and statistical analysis is needed. Particularly for exploratory work, making the design of experiments more uniform (when the project is conducive to it) would improve efficiency in determining salient variables to explore. Additionally, the MMSD needs to reach out to other parts of NIST to access statistical expertise that already exists.

Many workshops have been held at NIST, and this is highly beneficial for dissemination of information. Connection to the Office of Data andInformatics (ODI) needs to be encouraged to leverage what the MMSD is developing. More consistency is also needed in the design of experiments. Particularly for exploratory work, the uniform application of experimental design (when the project is conducive to it) would improve efficiency in determining salient variables to explore.