An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2000 (2000)

This volume is the product of a process of assessment that began in October 1999 and ended with the finalization of this report. The Board on Assessment of NIST Programs met three times in 1999-2000; the agendas of those meetings are reproduced in Appendix B. The meetings gave the Board an opportunity to receive briefings and have discussions with National Institute of Standards and Technology (NIST) managers and to deliberate and reach findings in executive sessions. In addition to information obtained from these meetings, this report is also based on the reports of the seven major panels operating under the Board. Each panel met with the managers and staff of one of the seven major NIST Measurement and Standards Laboratories (MSL). Prior to those meetings, subgroups of the panels had spent 1 to 2 days reviewing in detail ongoing programs in their areas of expertise.

This chapter represents the Board's judgments regarding the overall state of the NIST MSL. It offers findings, opinions, and recommendations for further increasing the merit and impact of NIST MSL programs. Chapter 2, Chapter 3, Chapter 4, Chapter 5, Chapter 6, Chapter 7 through Chapter 8 offer more in-depth reviews of each of the seven laboratories of the MSL, with findings and recommendations aimed at their specific programmatic areas. See Appendix C and Appendix D for NIST functions and organization, respectively. The acronyms and abbreviations used in this report may be found in Appendix E.

The panels reviewing the seven laboratories of the MSL are again impressed with the high technical quality of their work. In Chapter 2, Chapter 3, Chapter 4, Chapter 5, Chapter 6, Chapter 7 through Chapter 8 of this report, each panel cites specific work that is at or defines the state of the art in its field. A representative example of high-quality work from each of the seven laboratories follows.

In the Electronics and Electrical Engineering Laboratory (EEEL), an effort is under way to use the phenomenon of single electron tunneling (SET) to measure current at previously unachievable levels of accuracy. SET devices are being developed that will allow the reliable and reproducible control of

individual electrons. At present, there is no fundamental representation of current; the standard for current is realized through the representations of voltage and resistance. An independent representation of current could provide significant additional confidence in the coherency of the realization of the SI (International System of Units) electrical units. This project is a good example of how NIST uses new technologies to update and enhance its capabilities in fundamental metrology.

In the Manufacturing Engineering Laboratory (MEL), the establishment of the X-ray Optics Calibration Interferometer (XCALIBIR) system provides NIST with best-in-the-world capability in optical figure metrology. The instrument is housed in a specially constructed, controlled-environment enclosure and will be used, among other purposes, for calibration of the very-high-precision optical components necessary for the Laser Interferometer Gravitational-wave Observatory (LIGO) and the Department of Energy National Ignition Facility (NIF).

The Building and Fire Research Laboratory (BFRL) program on service life prediction for materials used in construction (such as paints, coatings, and roofing materials) focuses on techniques for measuring durability. Recent accomplishments include the construction of a 2-m ultraviolet integrating sphere that can be used to examine the durability of materials exposed to combinations of well-controlled environmental conditions. This is a significant improvement over techniques commonly used in industry today, which can vary widely from measurement to measurement.

Researchers in the Physics Laboratory (PL) are world leaders in the fabrication and analysis of thin magnetic layers and other nanoscale features. This group is now attempting to develop the measurement capability to allow scientists to probe the underlying physics in quantum confined structures on a nanometer scale. To this end, it has recently completed the design and assembly of a unique facility for fabrication and measurement of such structures. Such measurements are valuable as researchers across the nation seek to understand and develop devices and applications based on nanotechnology.

Materials Science and Engineering Laboratory (MSEL) staff are using the laboratory's outstanding measurement capabilities to better understand copper metallization techniques for chip interconnection. Electrodeposited copper is quickly gaining acceptance as a replacement for aluminum in chip interconnection technology. The quality of the copper fill in these processes is dependent on the use of added agents to promote complete fill and the desired final materials properties, but the mechanism of action of these agents is poorly understood. Recent measurements in the MSEL have begun to illuminate these mechanisms so that industrial processes can be more efficiently designed and optimized.

The Chemical Science and Technology Laboratory (CSTL) continues its development of cavity ring-down spectroscopy (CRDS) for use in sensing low-level gaseous contaminants in manufacturing processes. The measurement precision that the laboratory has obtained with this method is already very close to the 2001 target for measuring water vapor in semiconductor fabrication lines given in the International Technology Roadmap for Semiconductors.¹ The laboratory has also miniaturized this technology to create a CRDS optical cavity that can detect and quantify small amounts of materials in contact with or adsorbed on a surface. CSTL researchers are now exploring its potential as a miniature detector for lab-on-a-chip use.

The Information Technology Laboratory (ITL) and the MSEL have jointly developed a general software tool for modeling real material microstructures —the object-oriented finite-element (OOF) analysis system. The result of an effort initiated in support of an MSEL research project, the OOF system is now used by a number of major companies. Further refinement of the software has attracted

external support from the Department of Energy and includes collaborative work with the Massachusetts Institute of Technology. The OOF software was a winner of Industry Week Magazine's 1999 Technology of the Year award.

It is generally difficult to determine comprehensively the impacts of any given piece of research. However, in the combined judgment and experience of the panels and the Board, the majority of programs in the NIST MSL are successfully meeting the needs of NIST customers in U.S. industry. There is significant evidence to support this assertion. On an annual basis, NIST provides about 9000 calibrations for more than 500 different types of measurements, distributes a total of roughly 40,000 units of 1300 different Standard Reference Materials (SRMs), and distributes about 200,000 copies of 70 different databases. Many industries have chosen to access relevant NIST expertise directly through collaborations or Cooperative Research and Development Agreements (CRADAs) with NIST; some of these relationships are described in more detail in the chapters that follow. Programs have garnered from industrial groups recognition such as the Industry Week Magazine award cited above. Retrospective economic studies have demonstrated quantitatively the return on investment in particularly successful MSL programs. In the observation of the Board and panels, programs are reaching target customers through effective outreach to industry. Some of this outreach is coordinated at the level of the individual laboratory, whereas some occurs on the level of individual researchers. Some laboratories are beginning to use prospective economic analysis in their program planning; all should use potential impact as a selection and continuation criterion throughout the life of a project. Several examples of outstanding recent impact follow; more details can be found in subsequent chapters of this report (see, for example, pages 25, 73-74, and 249).

EEEL staff have recently completed development and dissemination of a computer-aided design program for insulated gate bipolar transistors (IGBTs). Several companies have reported that use of this system greatly enhances their ability to optimize devices. An economic impact study of this project estimated that industry received $18 million in direct savings and $40 million in indirect savings, resulting in a benefit-to-cost ratio of 23:1 for the NIST effort.²

ITL staff are developing interactive, Web-based conformance test suites for the extensible markup language (XML). Use of XML, an enabling technology for e-commerce, is growing rapidly. NIST effectively utilized its reputation as a knowledgeable yet neutral third party for this industry—just 6 months after NIST became involved in the effort to define an XML standard, industry was able to agree on a standard after having been stalled for 2 years by disagreements among various companies. By helping industry to reach agreement on a common standard for this language and by providing the test suites to measure the performance of products against that standard, NIST has strengthened the U.S. position in this growing sector of the economy.

Of significant importance is CSTL's work to develop better screening procedures for cloth swipe samples obtained in inspections of uranium enrichment plants by the International Atomic Energy Agency (IAEA). Such measurements are a critical component of the verification called for by current and proposed arms control treaties. CSTL researchers have used their expertise in isotopic analysis by secondary ion mass spectroscopy to develop procedures that improve the detectability and reliability of

screening measurements for enriched uranium. CSTL scientists are now the leading international authorities in methods to characterize IAEA swipe samples. Although this work is not directly related to commerce, it is appropriate that the nation utilize the outstanding expertise resident at NIST for such a purpose.

Overall, each laboratory in the NIST MSL has directed its programs effectively toward alignment with the laboratories' mission. The results of improved strategic planning in each of the seven laboratories are evident in clearer and better-defined program goals, leading to a better ability to judge program results and achievements. Although each laboratory has selection criteria for initiating and terminating projects, not all of the laboratories communicate these criteria effectively to the individual researcher. More effective communication would improve the alignment of programs with laboratory goals and improve the morale of staff, who would better understand the rationale for programmatic decisions. The Board applauds the laboratories in the MSL for performing their first Baldrige self-assessment. Following Baldrige criteria for quality should move the laboratories further along the path of continual improvement to which they are clearly committed.

Resource challenges facing the NIST MSL come in three interrelated areas: budget, facilities and equipment, and human resources. NIST MSL appropriations saw a significant real increase in the period from 1990 to 1995. NIST responded to this increase by taking better control of its own programs rather than expanding its program level. It reduced the amount of research funded by external dollars (e.g., through contracts with other government agencies), to a level of about 20 percent of total funding from earlier levels of 35 percent. This step allowed NIST to have much greater control over the content of its programs; thus, the MSL were able keep their programs tied much more tightly to the NIST mission. The Board applauded this increased mission focus in its earlier reports.

Since 1995, MSL appropriations have seen only slight increases (see Table 1.1 for recent budgetary data), which generally have been more than offset by inflation and mandated salary increases for staff. This effective decrease in appropriated budget, combined in some cases with reductions in external funding as a result of program completion or termination, has left some units in extremely difficult

financial circumstances. Some programs have suffered sharp decreases that place them at or near a critical mass of resources, for example, programs in the Materials Reliability Division of MSEL and in the Structures Division of BFRL (see pp. 162 and 192). The effectiveness of programs under such circumstances is severely compromised. Other programs have compensated by once again increasing their externally funded projects, an approach that can dilute the focus on the NIST mission.

NOTE: NA = not available.

This effective reduction in budget has also had an impact on the adequacy of NIST facilities. The Board and its panels have in the past several years documented numerous inadequacies in the current NIST physical plant, including poor air quality, poor temperature control, and poor humidity control, as well as power fluctuations that interfere with the use of the highly precise measurement equipment necessary for NIST researchers to perform their work. Most egregious is the facility situation at the Boulder campus (see pp. 30-31, 96-97, and 163). Some buildings lack basic temperature control such as air conditioning, which is crucial to running and maintaining high-precision equipment. The Board toured some of the Boulder facilities on a rainy October day and was disturbed by the number of rooms and corridors with significant leaks from the ceiling. Expensive measurement equipment is not designed to tolerate such conditions.

Some significant specific improvements have been made to facilities in the past 3 years, most notably the completion of the new Advanced Chemical Sciences Laboratory. However, although facilities' inadequacies in a few areas have been alleviated, significant inadequacies still exist. In most cases, NIST staff have found work-arounds to these problems. A great deal of ingenuity on the part of individual NIST researchers has been largely responsible for keeping many programs running under less than optimal conditions. For example, a NIST researcher may collect 100 hours' worth of data and average these data to compensate for poor signal quality due to laboratory environmental conditions. The researcher thus obtains the same precision measurement that a colleague in the Physikalisch-Technische Bundesanstalt, one of Germany 's national measurement institutes, could obtain based on several hours of data collection. In Boulder, leaky roofs make a large plastic garbage bin and plastic sheeting the most important equipment for work on a rainy day. In some cases, even this precaution is insufficient —a ceiling collapse in Wing 6 due to roof leaks resulted in loss of equipment for radio-frequency calibration services and a subsequent shutdown in these services. Although that wing has now been reroofed, equipment is still being rebuilt, and a year's backlog of work exists. Such workarounds and disruptions effectively raise the cost of programs and extend the completion dates, requiring inefficient use of resources and potentially delaying results in fast-paced technical areas to the point that U.S. competitiveness is affected.

The current NIST facility plan, with its proposed Advanced Measurement Laboratory (AML) and significant renovations of existing space, appears to address the remaining problems with facilities. State-of-the-art technical analysis has gone into the development of this plan. However, even if the facility plan is implemented according to the current schedule, AML construction will not be complete for another 4 years, and major renovations of existing facilities are scheduled to occur only after that time. The Board and panels are concerned about the interim period, especially after NIST managers presented the Board with figures indicating a backlog of $790 million worth of safety, capacity, maintenance, and major repair work to NIST facilities, work it has not been able to carry out due to lack of funds in this line item. The Board has seen no evidence of an interim plan to meet the programs' requirements for facilities in the time between now and completion of the AML, and it believes that facility inadequacies are having a negative impact on some technical capabilities and outputs.

Capital equipment is state of the art in some areas of research at NIST, while in other areas researchers cobble together and modify antiquated equipment to meet their research needs. Again, a great deal of staff ingenuity makes many programs viable, though often at the cost of efficiency. The

increasing cost of the equipment necessary for some areas of research (e.g., in materials science and for semiconductor design and processing) is a challenge to NIST managers. However, the Board's primary concern regarding capital equipment is the apparent lack of a major capital equipment plan. The Board and its panels saw no evidence of a long-term plan for acquisition of the capital equipment necessary to support NIST program requirements. This is a particularly crucial requirement in this time of planning for the AML. Equipment purchased now and utilized in current facilities will, in general, be too contaminated to move into the highly controlled and clean environments of the AML. A long-term plan for acquisition of major capital equipment is necessary, and this plan must be integrated with the facilities plan.

Staff is NIST's most valuable resource. The Board and panels have been consistently impressed with the quality of staff members, with their knowledge, creativity, and dedication. Many NIST researchers are among the world 's leaders in their fields; the awards and recognition they have received from their peers are too numerous to detail in this report. In their meetings with staff, the panels have observed generally high staff morale, and Board members are impressed by the good retention rate in the MSL. However, the Board sees indications that this situation may change in the near future. The current strong U.S. economic climate is fueling a heavy demand for skilled workers in industry, leading to increased competition in hiring that has resulted in significantly higher compensation packages in industry. In response, NIST starting salaries for entry positions have been increased in an attempt to compete with industrial offers. However, the NIST salary scale is compressed at the top due to government-wide policies and is inadequate to attract experienced technical leaders. The current federal salary scale means that, compared with their counterparts in industry, staff at NIST have a much diminished potential for salary increases. The impact on NIST is particularly apparent in its difficulty in hiring and retaining high-quality midlevel managers but can be expected to affect retention at all levels of staff if the disparity in earnings potential increases. NIST is already experiencing the effects of competition for technical expertise in areas such as information technology for manufacturing systems integration, an area in which it is unable to retain highly qualified staff members, who are increasingly accepting industrial positions with compensation packages valued at more than twice NIST compensation. Subsequent chapters present more details of human resource issues in the MSL (see pp. 54-55 and 225).

Tight budgets have also affected nonmonetary factors that contribute to recruitment and retention. This current year has seen reductions in force (RIFs) in several units of the MSL. Some of these RIFs deprived NIST of senior staff who had served as mentors for recent recruits. Many researchers join NIST for the opportunity to work side by side with a particular leader in their field, so loss of such a staff person through a RIF can have a ripple effect on staffing. Many staff members are also attracted to NIST by the opportunity to pursue a course of research that is quite independent relative to what is possible for their industrial counterparts, without the need to continually pursue grants as academic researchers must do. As budgets tighten, many of these staff are forced to spend increasing amounts of time seeking external funding to cover the costs of their research programs. Both of these factors reduce the attractiveness of the NIST environment and can be expected to affect recruitment and retention.

The Board notes a particularly inefficient personnel situation in the area of information technology (IT) service support for NIST. NIST has a centralized service available for the IT needs of the MSL. However, many of the laboratories in the MSL choose not to rely primarily on this central support, hiring their own in-house IT service support personnel or outsourcing IT support. Any one of these three strategies—centralized support, decentralized in-house support, or outsourced support—can be an efficient and effective means of obtaining necessary services. Pursuing all three simultaneously in an ad

hoc fashion, however, is inefficient and expensive. The NIST MSL need a cohesive strategy for obtaining IT support. The Board endorses the formation of an MSL-wide committee to consider problems of IT support. To be effective, this committee will have to obtain meaningful feedback and ensure significant buy-in from key constituencies (both users and suppliers) of these services.

The current pace of technology change for many industries far outstrips that of even a decade ago. In some industries, a technology or process advancement can be invented, utilized, and eclipsed by new discoveries in as little as 5 years. Advances in measurement technologies, however, are generally based on many years of exploratory research, and the development of standards is a time-consuming and often contentious process. NIST is clearly still learning how to adapt its culture to this new regime. It must be more acceptable for NIST to take greater risks, pursuing multiple technological approaches simultaneously with the knowledge that only one will eventually be adopted by industry. Such strategies are necessary to ensure that the NIST MSL stay ahead of industry in the pursuit of technology and are ready with technology solutions in time to have the greatest impact on industry.

At the risk of sounding trite, the Board will add its voice to the chorus that points out the increasing irrelevance of international boundaries in the global economy. When the very definition of “U.S. industry” can be debated, the entire NIST mission becomes problematic to interpret. One way in which the United States can improve its competitive standing in this environment is through global measurement standardization. NIST researchers are already active in national and international standards-setting activities and in cooperative programs with other national measurement institutes. However, the Board reiterates the need for the NIST MSL to take a proactive leadership role for the United States in such international activities. NIST managers inform the Board that the MSL have a cohesive strategy in this area, and the Board looks forward to hearing about this strategy at its next meeting. However, globalization affects all aspects of NIST's work, and the nature of industry in a global economy must be a strategic consideration for the MSL as a whole.

Much of the current growth of the U.S. economy—and, one can predict, much of its future growth— is linked to new, emerging areas of technology such as biotechnology, optoelectronics, information technology, and nanotechnology. The need for measurement technologies and standards in these areas is great, and the development of enabling technologies early in the growth of these areas can significantly increase the efficiency of industry's investment in these fields. Having early access to such measurements and standards can also give U.S. industry an edge over its competitors in the global marketplace. NIST already has in place the technical talent needed to address problems in these areas, and individual researchers in the MSL are forming collaborations across disciplinary lines on an ad hoc basis to begin to address the challenges that these technologies present. Mounting sufficient efforts to have an impact in these areas will require careful prioritizing between support of these emerging areas and support of established technologies and industries.

The Board and panels have been pleased with the improved performance resulting from meaningful strategic planning in each of the seven NIST laboratories. However, in its review of the MSL, the Board found that although it consistently had the information to judge the performance of individual laboratories it lacked a measure for the MSL as a whole. As a result, the Board could not determine if the MSL collectively are achieving their full potential impact. The Board believes that to progress to the next level of effectiveness and impact, the MSL require strategic planning and leadership at the level of the MSL as a whole.

A clearly articulated strategic plan at the MSL level would allow NIST to take better advantage of opportunities in biotechnology, optoelectronics, information technology, nanotechnology, and other emerging areas that are highly interdisciplinary in their subject matter and, for greatest effectiveness, require an effort coordinated across NIST's current “stovepipe” organizational structure. Currently, individual researchers are engaging in ad hoc interunit collaborations. A clear MSL-level strategy in these areas would allow each laboratory unit to more effectively align efforts within its competence toward addressing common goals, resulting in a whole that is greater than the sum of the parts.

Facilities and major capital equipment choices could also be improved and integrated in light of an MSL-level strategic plan. Having clear goals for the MSL in particular technology areas will allow facilities and equipment planning to be done with more insight because managers will have a clearer picture of where the MSL should be 5 years out. Facilities and equipment plans could then be clearly integrated with program plans for the MSL as a whole. This should result in better decision making and efficiency, as laboratory units identify more areas for consolidation of their current resources and more areas for joint investment in future facilities and equipment purchases.

Because NIST lacks a clearly articulated MSL-wide plan, the Board and panels were left with the impression that some of the staff reductions in the past year occurred for reasons of expediency (i.e., they occurred where funds were lowest or where staff were already leaving) rather than as a result of strategic considerations. Some outstanding staff were let go, and some competencies were completely lost. A clear MSL-level plan could be used to identify those competencies that are crucial to meeting goals and to help in making strategic, coordinated decisions about which competencies to drop and which to bolster in lean times. Such a plan could provide guidance on areas for new hiring at all times.

Flat budgets, the fast pace of technology change, and the emerging global regime combine to require difficult programmatic decisions. The need to begin programs to address new areas of technology and to accelerate existing programs to meet the window of opportunity for impact may mean that programs in mature technological areas suffer cutbacks. Some industry needs may be served through NIST collaborations or agreements with other national measurement institutes. Other programmatic areas will have to be dropped altogether. New ways of carrying out the NIST mission will have to be considered. A clear MSL-level strategy would provide the mechanism to address such challenges in a rational way and help bring about the desired balance between support of emerging industries and support of established areas. If MSL-level goals are communicated clearly to all levels of staff, such decisions will be easier to implement. Clearly communicated goals would be reflected in better alignment of programs at all levels, leading to greater and more efficient achievements for the NIST laboratories.