An Assessment of the National Institute of Standards and Technology Programs: Fiscal Year 1997 (1997)

Kenneth O. MacFadden, Allied Signal, Chair

Lou Ann Heimbrook, Lucent Technologies, Vice Chair

John L. Anderson, Carnegie Mellon University

Ramon M. Barnes, University of Massachusetts

Paulette Clancy, Cornell University

Anthony M. Dean, Exxon Research & Engineering Company

Robert R. Dorsch, DuPont Research & Development Experimental Station

H. Frederick Dylla, Thomas Jefferson National Accelerator Facility

Arlene A. Garrison, University of Tennessee

Steven M. George, University of Colorado

Wayne O. Johnson, Rohm and Haas Company

Helen H. Lee, University of Cambridge

Douglas E. Leng, LENG Associates

Anne L. Testoni, DEC-Digital Semiconductor

Submitted for the panel by its Chair, Kenneth O. MacFadden, this assessment of the fiscal year 1997 activities of the Chemical Science and Technology Laboratory is based on a site visit by the panel on April 8–9, 1997, and on the annual report of the laboratory.

The Chemical Science and Technology Laboratory (CSTL) stated that its mission is to promote the U.S. economy with particular emphasis on trade sectors that depend on chemical measurements, standards, models and data; provide technical leadership for the nation's measurement and standards infrastructure; and assure the availability of essential data and measurement capabilities. The Chemical Science and Technology Laboratory provides these infrastructural capabilities to enhance U.S. industry's productivity and competitiveness, assure equity in trade and improve public health, safety and environmental quality for U.S. industry, government agencies, and the scientific community.

This mission is consistent with the NIST mission of promoting U.S. economic growth and international competitiveness through a strong laboratory effort focused on infrastructure technologies such as measurement standards, evaluated data, and test methods. In fact, as described in this report, the CSTL leads NIST in these areas by providing 76 percent of the Standard Reference Materials (SRMs), 60 percent of the Standard Reference Data (SRD), and 18 percent of the calibrations.

CSTL programs cover technical areas including process measurement, analytical chemistry, surface and microanalysis science, biotechnology, and physical and chemical properties. In the panel's judgment, the laboratory continues to have chemical measurement capabilities of the highest quality and also conducts world-class basic and applied research in diverse technical fields. The projects capably support NIST's mission and are defined and refined to ensure their effectiveness in enhancing American technical measurement capabilities in chemistry, chemical engineering, and biotechnology. As part of this effort, the laboratory also strives to produce and disseminate data, models, and reference standards.

The separate technical divisions in the CSTL continue to devote time and resources to building and using the traditional technical expertise of NIST in chemical measurement, standard reference data, and calibration studies. These programs are balanced by work in emerging research areas such as biotechnology and nanoscale measurement techniques. As these new technical initiatives emerge within the divisions, it is important that the research maintain a strong focus on and a connection to industry.

The laboratory publishes world-class scientific data in leading journals, and various divisions are disseminating information even more extensively through widespread use of the Internet. The databases maintained by the laboratory are appropriately targeted at industrial needs and also address vital national issues in the environmental, chemical, material, and biological sciences. Continued supervision of NIST databases is required to ensure regular evaluation and updating of the data and to provide and maintain clear labeling that indicates the data's source and quality. Disseminating these databases on the Internet will make it easier for more people to access the world-class information gathered at NIST.

The numerous internal and external scientific awards bestowed on laboratory personnel and projects demonstrate the quality of the work done in the Chemical Science and Technology Laboratory. As leaders in their technical fields, staff often present their findings at conferences, workshops, and technical committee meetings. Such talks further disseminate key scientific findings.

The overall quality of the laboratory's technical programs exceeds expectations, and there are many examples of exemplary work. Each division strives to fulfill NIST's mission by continually redefining its role in aiding U.S. industry competitiveness.

The level of the laboratory's impact on industry is impressive, as indicated by the quality and quantity of measurement capabilities developed and of standards and reference data produced. In fiscal year 1996, the staff wrote 281 journal articles, sold over 30,000 SRMs, performed 1,065 calibrations, and sold 472 units of the 15 standard reference databases maintained by the laboratory. In addition, the laboratory has 49 active Cooperative Research and Development Agreements (CRADAs) with industry and two industry fellows. Seven licenses have been granted from patents on the laboratory's work, four of them during 1996.

To meet increasing expectations for quality products in the global marketplace, U.S. industry will require traceable standards and international intercomparability for such standards. Toward this end, the laboratory 's bilateral comparison activities with Germany, Japan, Italy, New Zealand, the Netherlands, and the United Kingdom are at the forefront of global intercomparability efforts. In addition, since the CSTL currently provides 76 percent of NIST SRMs, the importance of the laboratory's work will grow as U.S. companies seek out the reference materials traceable to NIST that are necessary for ISO (International Organization for Standardization) 9000 certification. This increased emphasis on traceability will challenge NIST management to clarify how traceability is defined, established, and maintained. Currently, organizations that ratify only parts of a standard claim traceability, while others assert that their measurements are more accurate than those made by NIST. In this unsettled environment, NIST personnel can play a pivotal role in unifying various approaches to traceability and clarifying the distinction between precision and accuracy.

One current effort to provide traceability resources for industry is the NIST Traceable Reference Materials (NTRM) Program. An NTRM is a reference material with a well-defined traceability linkage to existing NIST standards that is produced and sold by a commercial supplier. An example of the potential impact of work in this field is the effect of the gas measurement NTRMs developed in the CSTL's Analytical Chemistry Division. The gas NTRM program was implemented in 1992 in partnership with the Environmental Protection Agency (EPA) and specialty gas companies to provide end-users with the wide variety of certified gas standards needed to implement provisions of the 1990 Clean Air Act. Since 1993, eight commercial vendors have produced more than 2,900 NTRMs used to generate more than 150,000 commercial gas standards for end-users.

Another enabling measurement technology that has had a direct impact on a specific industry is the laboratory's development of high accuracy SRMs for determining the amount of sulfur in turbine blades down to levels below 1 ppm. Using these data, the aerospace industry

can predict the degree of adherence between thermal barrier coatings and the high temperature alloy used in jet engine turbine blades.

The laboratory is also providing U.S. industry with the data and models needed to identify and implement environmentally acceptable alternatives to the chlorofluorocarbons (CFCs) and hydrochlorofluorocarbons (HCFCs) used in air-conditioning and refrigeration equipment. After the 1987 signing of the Montreal Protocol on substances that deplete the ozone layer, industry faced the formidable challenge of shifting quickly from well-tested products to new refrigerants. Relying extensively on equation-of-state and transport properties models developed by the CSTL Physical and Chemical Properties Division, industry succeeded in producing a new generation of refrigerants for energy-efficient equipment. These new instruments will result in billions of dollars in savings per year and will create a healthier global environment.

One way of measuring the quality of the impact this laboratory has on industry is to assess the demand for services provided. The quality and quantity of calibrations performed and reference data produced both suggest that the laboratory is making a significant contribution. In addition to tracking sales, the laboratory has commissioned three economic impact studies for completed projects and proposed topics for three more studies. Such studies provide a much more quantitative approach and are the preferred method of assessing the laboratory 's impact. Currently, however, the scope of these reports is somewhat limited, because they do not include input from the laboratory's customers. Efforts in this direction are a good beginning, but the laboratory has not yet achieved its ultimate goal: the development of general metrics that can be used to determine the economic impact of laboratory projects.

The Chemical Science and Technology Laboratory receives most of its support from internal NIST sources. These include NIST Scientific and Technical Research and Services (STRS) funding (a direct appropriation from Congress), Competence Funding from the director's office to build expertise in key areas of future industrial needs, and funding from support of other NIST units such as the Advanced Technology Program (ATP). Income is also generated from the measurement services performed by the laboratory, such as calibrations and sales of SRMs and databases. The final source of support is external funding from other federal agencies (OA), nonfederal government (NF) sources, and CRADAs.

Funding sources for the Chemical Science and Technology Laboratory (in millions of dollars):

The staff of the Chemical Science and Technology Laboratory currently includes 281 full-time permanent (FTP) positions, of which 241 are technical professionals. There are also 97 nonpermanent and supplemental personnel, such as postdoctoral fellows and part-time workers. To date, the laboratory has had 235 visiting guest researchers.

The OA funding of $10.8 million has decreased by $4.1 million since fiscal year 1994. The Air Force provides one-third of the current OA funding, some of which has been used to purchase advanced instrumentation for the Surface and Microanalysis Science Division. The overall level of outside support (20 percent) seems appropriate; the laboratory benefits from the additional resources, but the projects funded by OAs do not dominate the laboratory's efforts or interfere with its mission. However, the level of dependence varies widely from division to division, a disparity addressed in the division reports that follow.

In the summer of 1997, Building 1-B at the Center for Advanced Research in Biotechnology (CARB) is scheduled for completion. This addition will provide expanded space for the Structural Biology Group and will house a new high-field (600 MHz) nuclear magnetic resonance (NMR) instrument. The Advanced Chemical Sciences Laboratory (ACSL), currently under construction on the Gaithersburg campus, will open in late 1998, and the Analytical Chemistry and Biotechnology Divisions will gain 82,000 feet of office and laboratory space.

These two construction projects will definitely improve some of the facilities problems faced by the Chemical Science and Technology Laboratory. Currently, the equipment available to laboratory scientists appears adequate and in many instances represents the state of the art in measurement sciences, but the facilities housing the equipment are impeding the advancement of measurement science. The ACSL and CARB 1-B will relieve crowding and provide adequate facilities for the Analytical Chemistry and Biotechnology Divisions. However, the existing facility in Gaithersburg and Building 3 in Boulder are inadequate and limit the capabilities of the remaining divisions. Advanced atomic-scale microscopic analyses are hampered by building vibrations. Retrofitting current facilities with vibration-dampening equipment is unlikely to lower the vibrations to levels adequate to perform analyses at the accuracies necessary for current and future research. Poor power quality and reliability result in damage to electron microscopes and lasers. Dust and dirt in the ventilation systems raise repair and maintenance costs for microscopes and lasers and also limit the laboratory's ability to manufacture the ultra-high-purity SRMs essential to the electronics industry. In general, in the existing facility the Chemical Science and Technology Laboratory cannot perform the cutting-edge research necessary to address current and next-generation measurement needs, especially in microscopic and laser-based kinetic measurements.

How priorities are set at the laboratory is shaped by a series of well-defined criteria: (1) magnitude and immediacy of industrial need; (2) match to NIST mission; (3) NIST's ability to make a difference; (4) nature and size of anticipated impact; (5) NIST's ability to respond in a timely fashion with high-quality output; and (6) opportunities presented by recent advances in science and technology.

Laboratory management has clearly and concisely formulated and disseminated its mission and strategic technical plans to all divisions. The programs ongoing in the Chemical Science and Technology Laboratory are effective in focusing on the NIST mission and the needs of U.S. industry. The laboratory continues to do an excellent job of acting on input from assessment panels and using that input to formulate technical plans and sunset appropriate programs. The formal strategic plans are well focused, and new thrust efforts such as the work on molecular scale materials characterization are of significant interest to industry. A variety of sources including technical workshops, formal and informal industrial interactions, and CRADAs assist the laboratory in focusing its strategic plan on the advancement of technology for U.S. industry.

The criteria for planning and allocating resources such as technical staff and equipment and the management of such resources are consistent with the NIST strategic plan. The system is designed to allow management and staff at various levels to evaluate decisions and contribute to the process. Despite recent budget constraints, the laboratory has maintained a wealth of scientific expertise in a variety of diverse fields. Guest researchers and temporary appointments constantly replenish the scientific talent base. Given the appropriate turnover in the laboratory's array of projects, maintaining a match between the skills of the permanent staff and the needs of specific programs is a continuing concern. All divisions participate in facility planning; however, due to limited capital funding, many current plans may not be implemented.

The CSTL planning process has evolved in response to new and complex technical challenges posed by both NIST and industry. The panel applauds the current trend toward inclusiveness in the planning process. This will ensure that both laboratory staff and American companies are encouraged to help shape a strategy that will meet the future needs of U.S. industry.

In summary, the CSTL is a highly productive organization with strong leadership and open communication. The laboratory's operation is exemplary, and the quality of CSTL efforts is reflected in the larger roles that many senior managers from this laboratory now play at NIST.

The Biotechnology Division stated that its mission is to provide the measurement infrastructure to advance the commercialization of biotechnology. This is accomplished by establishing a scientific and engineering base along with reliable measurement techniques and

data to enable U.S. industry to produce biochemical products economically and quickly with the appropriate quality control.

Despite its relatively recent formation, the Biotechnology Group activities are well integrated with the laboratory and NIST missions. The collaboration with the University of Maryland through the Center for Advanced Research in Biotechnology (CARB) is a nontraditional method of resource-sharing at NIST, but the joint effort brings great strength and value to the division. This cooperative structure may be the only mechanism through which NIST and the CSTL can connect to leading-edge biotechnology.

The DNA Technologies Group is addressing multiple aspects of an important emerging measurement challenge: how to perform reliable, accurate measurements of DNA patterns in a way that is useful for various applications. Measurements for forensic analysis, genetic disease diagnosis, and human identification analysis all demand nearly perfect qualitative results. NIST's traditional expertise in measurement science and implementation is well suited to tackle these new biological measurement needs. Though the industrial impact of this work as measured by growth of gross domestic product may not be large, the potential positive effects on public health, safety, and security justify the division's work in this field. The group's research and outreach efforts have both made strong progress over the past year. Development of mitochondrial DNA SRMs is nearing completion. This group's efforts are layered in such a way that projects applicable in the near future coexist with more basic research that should make longer-term contributions to biotechnology. The work on detecting and quantifying DNA damage and repair is in an area of broad scientific interest but is not clearly connected to the NIST mission. This technology is also applicable to ongoing work in the use of DNA-based diagnostics for food supply safety and clinical issues, where progress might lead to a larger direct commercial impact.

The Structural Biology Group is performing high-quality basic scientific work. The group's status is reflected in its ability to recruit strong, young faculty members in conjunction with the University of Maryland. Recent accomplishments include the development and operation of the Web-based databases on protein structures and crystallization. These efforts addressed difficult problems and resulted in many first-class publications. The combination of x-ray and nuclear magnetic resonance (NMR) techniques for structural analysis with calorimeter and spectroscopic methods to understand structure/function relationships is useful but strongly weighted toward the structural side of the effort. The new NMR capability to be installed at CARB is a timely addition to the NIST facilities, as the current equipment is well past its prime. The expertise of this group in membrane-bound protein work at CARB is not currently linked to the hybrid bilayer membrane work in the Biomolecular Materials Group. A joint effort could attack some of the challenging problems in structure/function relationships for proteins embedded in membranes.

The Bioprocess Engineering Group has a highly diverse and almost divergent range of projects. Its effort in enzyme-catalyzed reaction thermodynamics is unique in its high level of research staff competence and of industrial interest. As biology-based catalysts are more widely adopted, this project will gain wider recognition and its economic impact will become clearer.

Advancing technology in this field is essential to increasing the use of large-scale, biology-based chemical processes. This group 's establishment of the Enzyme Catalysis Thermodynamics Database on the Web is a step toward accelerating commercialization of new methodologies. The group's emerging program targeted on the pathway to shikimate/chorismate is worthwhile effort, as advancements in this field are likely to have commercial impact within the decade. However, the project on complex pathways for electro-optical proteins may not have realistic applications. Applying bioprocesses in the chemical industry faces recognizable economic challenges beyond those technological issues addressed by this project, and achieving industrial use of the results may not be possible. Therefore, the percentage of available resources devoted to this project appears high, while the amount of effort focused on the aromatic pathway to shikimate/chorismate or on more direct approaches to the P450 oxygenase systems seems low.

The Biomolecular Materials Group mainly focuses on new measurement techniques in two major areas: membrane proteins and biomimetic surfaces. Biology research today concentrates on issues related to the behavior of molecules in solutions, and therefore the field of membrane-associated biology is largely unexplored. This group's growing expertise in this area has produced an expanding measurement toolkit that nicely complements the division's strength in structure analysis and function elucidation. A long-term strategy emphasizing the connection between these fields should yield insight that takes advantage of NIST's unique ability to implement the novel measurement techniques required for success. However, the range of studies currently under way in this group is too broad and the number of projects too large. Again, the short-term economic impact of this work is modest, but the division could seed major growth in membrane-based biotechnology. If the most significant challenges can be identified and attacked, future production with great economic impact is plausible.

As biotechnology is a relatively young science, the ultimate economic impact of most of this division's current projects will be felt. Two exceptions are the DNA diagnostic calibration work and the Enzyme Thermodynamics Database. However, there are many other areas in which divisional work has great potential. The structure and function programs will surely contribute useful new data and develop novel ways to measure and model challenging problems. The dependence of biotechnology-based industries on protein structural data and on understanding mechanisms of protein function cannot be overstated. Over the next 2 decades, tens of billions of dollars of value could come from this field of effort, and the work done at NIST/CARB will have made an important scientific contribution to this growth. The quality of the problem selection process will determine how great NIST's industrial impact will be. Similarly, in bioprocess engineering, balancing problems that can be practically addressed with some longer-range programs is a good approach that assures a positive effect within a reasonable time frame. The greatest opportunity for widespread impact is in the membrane and membrane-bound protein programs, although the commercial benefits of this work will not be apparent for some time. In this area, cautious yet imaginative planning is again essential and will improve the probability of success.

As the Chemical Science and Technology Laboratory and NIST develop general performance metrics, the Biotechnology Division will probably require longer-term, less

stringent measures to assess its effectiveness, as the value of its work is more difficult to define than the worth of projects in more mature fields.

Funding sources for the Biotechnology Division (in millions of dollars):

The staff of the Biotechnology Division currently includes 31 full-time permanent positions, of which 29 are for technical professionals. There are also 25 nonpermanent and supplemental personnel, including postdoctoral fellows and part-time workers.

Because the field of biotechnology is rapidly changing, this division 's focus will be constantly shifting. Such adjustment is difficult at NIST for two reasons. Start-up funds for new projects are harder to find in the currently limited resources of the Chemical Science and Technology Laboratory. When new funding is not available, divisions are forced to realign their resources in support of programs with the highest impact. Such shifts are more difficult in this division because of the other impediment to rapid program variation: NIST 's relatively constraining policy on staff turnover. In such a dynamic field, the primary way to add new skills to the programs is to change the personnel—a difficult procedure at NIST.

To plan and guide research programs effectively, division scientists are active in many forums that provide feedback on long- and short-term needs for commercialization of biotechnology. These activities supplement the division's frequent participation in scientific meetings and topical workshops. For example, during fiscal year 1996, the division continued its active role in the International Union of Pure and Applied Chemistry (IUPAC) Commission on Biophysical Chemistry, the American Society of Testing and Materials Committee E-48 on Biotechnology, and the Biotechnology Industry Organization. Personnel also participated in many important workshops held at NIST, such as the Sixth International Meeting on Chemical Sensors (July 1996).

The Biotechnology Division has worked closely with the NIST Advanced Technology Program (ATP) in various ways. Staff members have served as NIST technical representatives in the annual review of a number of active ATP grants. They have also presented results of division

research programs at ATP-sponsored workshops and public meetings. Several generic research projects in this division are directly correlated with ATP focus areas, and some efforts are directly related to specific ATP grants. In addition, the Biotechnology Division continues to play a major role in the planning and coordination of federal biotechnology research through membership in the Biotechnology Research Subcommittee of the National Science and Technology Council.

Division management is to be commended on the ongoing progress in planning in such a rapidly changing field, even as new members are added to the leadership team. The complexity of the technical challenges and opportunities facing the division suggests that great insight in planning will be the largest determining factor in the division 's success. Maintaining the connection with CARB is one example of an area in which the division must carefully manage its options. Other examples include selecting among the great number of disciplines that make up the field of biotechnology and increasing the synergy between the research on membranes and on structures.

The Process Measurements Division stated that its mission is to develop and provide measurement standards and services, measurement techniques, recommended practices, sensing devices, instrumentation, and mathematical models required for analysis, control, and optimization of industrial processes. The division's research seeks fundamental understanding of, and generates critical data pertinent to, chemical process technology. These efforts include the development and validation of data-predictive computational tools and correlations, computer simulations of processing operations, and provision of requisite chemical, physical, and engineering data.

The panel finds this mission statement to be consistent with the laboratory and NIST missions.

The technical merit of the program is demonstrated in many ways. Widespread demand for the division's calibration services in pressure, humidity, vacuum, temperature, and flow speaks highly for its quality of work. NIST is committed to maintaining measurements and standards traceability. In comparison with industrial research institutions, the CSTL has maintained a higher level of broad, forward-looking technical excellence. Industrial laboratories are designed to minimize cost, and their researchers focus on programs that can quickly turn a profit; work at NIST can focus on more long-range goals.

In accordance with the NIST mission, programs in this division make highly appropriate contributions to promoting U.S. economic growth through work with industry. Over the last 20 years, many American companies have become global giants, while foreign investment in the United States has grown rapidly. Today, cooperation in standards activities is an international

issue. The panel sees strong evidence of the Process Measurements Division's creditable work in this area and compliments this effort. The technical merit and appropriateness of selected programs are discussed below, along with comments on the effectiveness and dissemination of the division's work.

In general, the Internet is becoming a most effective way to distribute reference data. Over the past year, the Web site for the Chemical Kinetic Mechanisms (CKMech) Database had 3,767 visits, which included 2,158 for species data and 1,614 for reaction data. These requests have come from all over the world and represent academic institutions, other national laboratories, and private companies. CKMech is a particularly relevant example of the new approaches possible through the Web because in this project, data are distributed to the researcher community before evaluation is completed. Though it is appropriate for the CSTL to supplement fully evaluated databases with information on work in progress, it is also critical that Internet data contain suitable indicators of the status of the material posted, such as the date of acquisition and the level of certification or evaluation performed.

The Fluid Flow Group has made excellent use of computational fluid dynamics (CFD) to analyze various problems such as flow irregularities produced by pipe fittings, including elbows, bends, and diameter changes. The recently completed exhaust meter calibration facility is designed to simulate car exhaust gas flows. Labview software controls the various flow strengths and environments available in this impressive facility.

The High Temperature Processes Group described two main project areas: spray combustion and hydrothermal processing. The panel toured the related facilities, which appeared cramped and makeshift. The spray combustion program has many good features with appeal for industry: atomizer design, spray/emissions correlation, modeling, process control, and a new laser-driven thermal reactor. Therefore, this project is a good candidate for industrial support to supplement internal funding and improve the quality of its facilities.

This group's program on the hydrothermal processing of industrial wastes currently includes a major effort to determine supercritical heat transfer rates. Previous work in reactor studies indicates that results depend on reactor design; for example, long slender tubes provide more complete conversion than larger, shorter, and wider vessels. The difference is believed to be due to widely varying residence time distributions and possibly to surface effects. Other potential issues include salt deposition, effects of different oxidants, corrosion, and process sensing. The nature of these problems and of other issues related to cost and plugging implies that general commercial use of this technology is a long shot. However, industrial interest in supercritical extraction is widespread, as evidenced by the four well-attended sessions addressing this topic at the 1996 American Institute of Chemical Engineers national meeting. Industrial concern centers on separation or extraction and crystallization processes. A major detriment to furthering useful applications for supercritical processing is the current inability to predict mixture properties. The panel believes that using current technology to advance other areas of supercritical processing will benefit industry.

The Reacting Flows Group works on chemical reacting flows, nanoparticle synthesis, application of molecular modeling to process applications, kinetic modeling of flame and materials chemistries, in situ diagnostics in plasma and thermal reactors, advanced materials at high temperatures, and contaminant-free semiconductor manufacturing. CFD two-phase modeling has been used to predict interesting phenomena such as concentration profiles near a spinning disc; researchers await experimental data from a new rotating disc reactor to compare

with the computational results. The materials chemistry project on destruction of chlorofluorocarbons (CFCs) looks encouraging. The focus is on an approach in which halogens in the CFCs are mineralized into a salt coating using a sodium flame. The costs associated with this process appear reasonable.

The CKMech Database contains information such as thermochemical data for a set of species or rate constants for a set of chemical reactions for high-temperature fluorocarbon chemistry. Consumers can download this material from the Internet, and the number of visitors to the Web site demonstrates this dissemination method's popularity. This on-line database is an excellent vehicle for technology transfer.

Although the Reacting Flows Group shares plasma facilities with the Process Sensing Group, the focus of the research each group conducted using these capabilities is different. The Reacting Flows Group is interested in particle dynamics rather than in sensing. The inadequacy of space assigned to the Reacting Flows Group raised some concerns. Some of this group's work is closely coupled with work on to plasmas and lasers, and the researchers were not near enough to others in these areas.

The Process Sensing Group is making good progress on the application of self-aligning monolayers to DNA probes and has begun a project on the characterization of surfaces using Surface Enhanced Raman Spectroscopy. The multisensor array work takes advantage of major improvements in substrates. The ultimate goal of this commendable effort is qualitative and quantitative analyses of gas species. For example, new detection models can distinguish between ethanol and methanol.

A noteworthy achievement of this group in collaboration with the Pressure and Vacuum Group is the application of the optical measurement technique cavity ring-down spectroscopy (CRDS) to humidity standards. This new method greatly increases the range of measurable concentrations. It represents a primary standard and will ultimately replace other methods with more narrow applications.

Recent focus on the realization of the International Temperature Scale of 1990 (ITS-90) has led to numerous publications by the Thermometry Group. Innovative use of triple points and development of improved cells (including transportable cells) have contributed to this excellent program. There are still some regions of the scale where different methods of measurement overlap and the varying results produce nonunique temperatures; this issue remains unsolved. Staff limitations have been addressed by automation of the thermometry labs. New thermocouples under development have attracted significant attention from silicon wafer manufacturing operations, where temperature variations have resulted in low product yield and suitable thermocouple technology has not been available. Excellent management and innovative programs have resulted in this group's continued provision of SRMs and calibrations in a cost-effective, timely manner. Substantial technology transfer has arisen from such industrial interactions and from workshops.

The extensive international activities of the Thermometry Group have required a major financial commitment from the division. The panel believes that this intercomparison project is highly significant to U.S. industry and is gratified to see that continuation of this work is supported by reprogramming at the laboratory level. The Pressure and Vacuum Group continues to strengthen its vacuum standards over a wide range using existing primary and transfer standards. The highest priority work focuses on the two types of instruments most widely used in the commercial and research and development communities: partial pressure analyzers and

spinning rotor gauges. This focus is an appropriate use of the limited resources allocated to this group.

The group's development of new high-vacuum instrumentation based on optical measurements is proceeding well. A key demonstration of the use of CRDS has been accomplished with oxygen, and there are credible plans to apply the technique to water vapor and other active species of practical interest. In addition, state-of-the-art laser diode equipment has been identified that will allow substantial miniaturization of the equipment.

The group continues to use its expertise in flow measurements to develop high-accuracy primary standards for low gas flows (10⁻⁷–10⁻³moles per sec); this project complements other divisional activities in high-flow regimes. The semiconductor industry often makes flow measurements using thermal conductivity. Current group activities aim to further understanding of the characteristics of the commercially available thermal mass flow meters. This project includes on-site measurements at the equipment manufacturing plants and addresses significant calibration and standardization issues. The panel was surprised to learn that the Semiconductor Manufacturing Technology (SEMATECH) consortium no longer helps fund this effort.

In response to general concerns expressed by previous panels, the Pressure and Vacuum Group has increased efforts and visibility in international comparisons. There are two comparisons in vacuum gauging and four in pressure gauging currently under way.

This group continues to maintain its baseline calibration services based on mercury manometers and piston gauges as primary pressure standards. Added research and development efforts in pressure measurements complement and strengthen this calibration work. Two projects recently initiated in this area are the design of very low pressure gas mixtures (for the Geological Survey) and the use of computational stress and fluid analysis to improve piston gauge performance.

The Process Measurements Division's research results are used, referred to, and implemented effectively. The quality of work is excellent and the results are disseminated efficiently through publications, the Internet, customer visits, and planning sessions. However, methods to encourage customer feedback are insufficient.

Various groups in the Process Measurements Division participate in intercomparability activities among national and regional standards laboratories. For example, the Fluid Flow Group has participated in the international comparisons for gas flow organized by NIST and Japan's National Research Laboratory of Metrology as well as in interlaboratory tests on liquid flow, including hydrocarbon flow measurements. Under the auspices of the Consultative Committee on Thermometry (CCT) of the International Committee of Weights and Measures, NIST served as chair of an 11-nation intercomparison on humidity. CCT is also organizing international comparisons of the realizations of the ITS-90 among its member countries. The Thermometry Group will coordinate the intercomparison in the range from 83.8058 K to 933.473 K and participate in the intercomparisons of other temperature ranges. The Pressure and Vacuum Group is also active in six other international or regional comparisons of standards.

The panel found many indications of the positive impact divisional programs have on industry. Examples such as publications, work with consortia, industrial visits, hits on the Web sites, and extensive use of calibration services from NIST are all evidence that this division reaches out to industry and that companies are aware of and use NIST products. However, data on such activities do not quantitatively measure the value of divisional projects to industry. Though the panel applauds the initial efforts to assess the economic impact of the NIST thermocouple calibration program, such efforts are incomplete, because they include input from instrument manufacturers but not from end-users. A useful evaluation system includes industry input and support, and NIST needs quantitative impact data to help define which programs have the highest value.

Funding sources for the Process Measurements Division (in millions of dollars):

Several years ago, funding from other agencies, service fees, and industry grants supported three-quarters of the activities of certain groups within this division. Today, internal support (NIST-STRS) accounts for approximately 76 percent of the division's budget. Current funding policy has the advantage of enabling programs to proceed efficiently with technical activities, as opposed to devoting effort to obtaining funding. The downside is that the organization is highly dependent on congressional appropriations. The panel believes that efforts to lower the proportion of funding supplied by other agencies and industry have succeeded in increasing the emphasis on core programs but may have gone too far. The current range of financial support for divisional projects is not diverse enough; opportunities for industrial interactions and customer feedback do not occur as naturally when most of the programs are dependent only on internal funding. For example, the current sources of support for the Reacting Flows Group are inadequate. However, some of the particle formation technology this group studies is potentially valuable to industrial scientists involved in gas phase polymerizations, but connections to and support from the appropriate companies have not been exploited.

The staff of the Process Measurements Division currently includes 65 full-time permanent positions, of which 59 are for technical professionals. There are also 16 nonpermanent and supplemental personnel, including postdoctoral fellows and part-time workers. Flat budgets coupled with increased costs have made staffing levels an issue in the

Process Measurements Division. For some projects, the allotted manpower is below the level necessary to sustain quality work. Though the division is making excellent use of temporary staff such as postdoctoral fellows and guest researchers, projects often suffer when temporary help leaves. To management's credit, the division has avoided across-the-board budget cuts by terminating weaker programs. The panel applauds this strategy.

The general-purpose laboratory equipment available to Process Measurement Division researchers is comparable to that in industrial laboratories, but the specialized equipment is significantly better. The array of tools is adequate for current programmatic needs. The one exception is the instrumentation for the mid-range gas flow rate calibrations done by the Fluid Flow Group. Foreign institutions have established flow accuracies two to three times better than the NIST value of ± 0.22 percent. The Chemical Science and Technology Laboratory cannot match foreign standards without upgrading the relevant facilities in this field.

As discussed in many other sections of this report, the current physical plant is a major impediment to research efforts and is the most critical issue facing the division. The physical facilities are cramped, outdated, and inadequate in many ways. The biggest problems are availability of space, inappropriate climate control, and large amounts of airborne particulates. Excessive vibration is an additional problem for some activities. Electrical power outages have seriously affected productivity.

The predominant management concern pervading most of the groups is the need to address the renovation of existing facilities and construction of new facilities to accommodate the work load. The range of facilities issues is so broad that problems cannot be addressed piecemeal.

The division's organizational structure has not changed in the past year. This stability (along with impressive leadership) contributes to excellent progress, high morale, new developments, and good technology transfer to customers. Rigorous annual planning allows the division to reallocate about 5 percent of its resources each year, while laboratory policy mandates the shift of an additional 5 percent. A variety of short-and long-term review processes continue throughout the year and scrutinize each project. The cycle includes the NRC laboratory panel review, NIST program reviews, staff proposal reviews, preliminary assessment of new proposals, off-site program reviews, reprogramming, and identification of areas to emphasize and de-emphasize. The laboratory appears committed to making this an annual process, and the panel applauds this effort. Longer-term strategic planning is undertaken at all levels of management, from group leader to laboratory director. This range of contributions is a key part of a generally commendable planning process.

The subpanel was not provided with a division mission statement. The division chief indicated that a new mission statement is being developed. The overview statement provided in the 1996 Chemical Science and Technology Laboratory Summary of Technical Activities is as follows:

The Surface and Microanalysis Science Division conducts research to: (1) determine the chemistry and physics of surfaces, interfaces, particles, and bulk materials, and their interactions with a broad spectrum of species including electrons, photons, ions atoms, and molecules; (2) determine the chemical and isotopic composition, morphology, crystallography, and electronic structure at scales ranging from millimeters to nanometers; (3) determine the energetics, kinetics mechanisms, and effects of processes occurring on solid surfaces as well as within materials and devices; (4) study the total chemical measurement process as well as source apportionment in atmospheric chemistry using chemometrics; and (5) develop and certify Standard Reference Materials and Standard Reference Data. Emphasis is placed on performing fundamental and applied measurement research, providing data and standards (including software), and developing theories that are needed for accurately measuring chemical composition and dynamic processes that occur on surfaces and in microstructures. This information establishes the relationship between chemical composition at high spatial resolution and the effects of manufacturing processes, device operations or material properties. The information also provides the bases for advances in various technologies, such as chemical catalysis, advanced electronics, and materials science.

The technical content of this overview integrates well into the NIST and laboratory mission statements. However, this statement lacks the appropriate emphasis on promoting U.S. economic growth through collaborations with industry, both through direct contact with individual companies (as through CRADAs) and indirect interactions (as in NIST 's work with industrial consortia).

Overall, the panel believes that the division performs appropriate, high-quality research into the development and application of new measurement technologies in surface and microanalysis science.

The Atmospheric Chemistry Group's work in chemical metrology and isotope analysis for atmospheric science is internationally recognized. In particular, this group is the leader in isotopic analysis of atmospheric species such as ¹⁴C and is establishing global standards for isotope analysis. The Standard Reference Photometer for ozone instrument calibration provides a valuable international service. Another important continuing project uses ¹⁴C to distinguish between pollution caused by fossil fuel and by biomass fuel.

Newly initiated field work has also expanded the group's scope. A specific example is the use of isotope analysis to study long-range transport of biomass-burning aerosol. Another new effort evaluates the history of carbon aerosol trapped in polar ice. This work could

potentially reveal the “history of fire” as recorded in the ice core. In addition, the group continues to examine carbonaceous aerosol in urban areas, such as Denver, to determine the exact source of the pollution.

The Surface Dynamical Processes Group investigates energy transfer and kinetic processes at surfaces and interfaces. The recently concluded series of surface vibrational dynamics studies addressed a major challenge in the field. The group also has strong expertise in surface spectroscopy and is continuing promising studies of radical/surface interactions. The results of this work should be directly applicable in the semiconductor and advanced materials industries. The group has also initiated a new effort in near-field scanning optical microscopy. This work has great potential for advancing optical measurement techniques for very small structures and should build many new ties with industry, especially in catalysis and microelectronics.

The Microanalysis Research Group performs state-of-the-art electron and x-ray beam microanalysis. This work concentrates on elemental and structural analysis of nanometer-scale particles and structures. The group's excellent research has produced leading-edge measurement techniques and applications. Recent accomplishments include developing methods for automated scanning electron microscopy (SEM) particle analysis and evaluating the structure and composition of TiO₂ nanoparticles using transmission electron microscopy (TEM). Developing automated SEM and x-ray spectroscopic analyses of particles will benefit many industries, including catalysis, advanced materials manufacturing, and microelectronics.

One example of the many collaborations between this group and industry is the recent compositional analysis of ZrO₂ thermal barrier coatings, in cooperation with the NIST Materials Science and Engineering Laboratory, Sandia National Laboratories, and Pratt & Whitney. The group has also continued to develop atmospheric pressure (environmental) SEM capabilities that will have important applications and be useful in future collaborations. Work continues on the desktop spectrum analyzer, although widespread adoption of this package has been limited by the software's incompatibility with various hardware platforms. Use in industry is limited because the group has not developed a version than can run under Windows NT. Finally, this group's support of the development of high energy resolution, superconducting microcalorimeter-based x-ray detectors in the Electronics and Electrical Engineering Laboratory at NIST-Boulder has been essential in this research's rapid progress toward commercialization. If successful, these detectors will revolutionize microanalysis work in industrial, academic, and government laboratories.

In surface analysis, Auger electron spectroscopy (AES) and x-ray photoelectron spectroscopy (XPS) are the two techniques most widely used in industry. However, making practical, quantitative measurements using these methods is extremely difficult; as a result, they are often inappropriately (or incompletely) used. Therefore, industry would benefit if this group stepped up its efforts to develop measurement methods for and new applications of AES and XPS. In addition, the existing databases and standard test data could be more widely promoted to industrial users to expand the direct impact of work in this area.

The group is well equipped and staffed. Instrumentation is appropriate and often identical to that available in industrial laboratories. This overlap plays a key role in the smooth transfer of new measurement methods to industrial use. The recent addition of a field emission TEM provides state-of-the art, high-resolution image capabilities.

The Analytical Microscopy Group performs state-of-the-art ion and photon microprobe analysis research. Its main focus is on developing new measurement methods and standards for secondary ion-mass spectrometry (SIMS) microprobe material analysis. This group also conducts photon microprobe analysis of materials. Research results are excellent, and the group is extensively involved with industry. One recent accomplishment is the development of SIMS techniques to evaluate uranium isotopes in particles. This work is important for national security and can be used to verify compliance with international agreements. Another key effort is the study of CoSi₂ formation, which has yielded information applicable to future semiconductor process technology.

Development of three-dimensional (3D) SIMS analysis continues. A good example is the development of methods and standards for boron and arsenic implanted in silicon, techniques especially valuable to the semiconductor industry. Complementing this 3D SIMS analysis is the development of SIMS methods using focused ion beams to allow spatial resolution at the 200-nm level. This technique is on the leading edge of instrumentation and analysis methods. In addition, this group is using laser microprobe mass spectrometry in a promising study of soot formation.

The Surface and Microanalysis Science Division performs research that affects a wide variety of industries. As companies continue to fabricate devices with smaller feature sizes, probes with higher spatial resolution are needed. Product evaluation also requires elemental and molecular composition mapping at these higher resolutions. In addition, detecting extremely low concentrations with high sensitivity is necessary to maintain precision in analysis. The Surface and Microanalysis Science Division is well positioned to make contributions in these and other areas.

Industrial collaborations are particularly evident in the Microanalysis Research and Analytical Microscopy Groups. These two groups have an especially effective set of instruments to carry out the research funded by the Air Force. This collection of tools is an effective arsenal to answer many industrial needs. In addition, these instruments often match the tools used in industry; as a result, companies can benefit from NIST expertise on and calibration of the instruments.

Numerous industries benefit from the research of these two groups. One highlight is the semiconductor industry's use of SIMS SRMs for boron and arsenic in silicon. Surface and microanalysis techniques developed at NIST address the needs of a broad spectrum of industries. Other x-ray spectroscopy and infrared research has investigated ZrO ₂ coatings that serve as thermal barriers on gas turbine blades. In addition, researchers in this division have used TEM techniques to study the formation and structure of the TiO₂ particles that play a critical role in photocatalysis.

Research in the Atmospheric Chemistry Group is more relevant to environmental industrial problems. This work has established the origin of carbonaceous aerosols in urban areas such as Denver. Environmental regulations that result from these studies have an indirect effect on industry. The vast isotopic analysis capabilities of this group are also being used to

trace the origin of other substances. This work should continue to have great environmental and industrial relevance.

The Surface Dynamical Processes Group is developing a near-field scanning optical microscope that will operate at a variety of optical wavelengths and is expected to have novel industrial applications. Other research activities in this group provide high-quality scientific data pertinent to a general understanding of important surface processes. One example is the ongoing study of radical surface interactions that affect semiconductor fabrication. This group has an excellent staff of scientists who are working to increase the industrial relevance of their work. The group leader's recent 6-month tenure at W.R. Grace & Company through the NIST Industrial Fellows Program should also enhance the overlap between divisional project goals and industrial concerns. The panel commends this group for these efforts and the progress made over the past 2 years.

Currently the division does not perform any formal assessment of its impact on industry. However, such evaluations are needed to provide the management team and research staff with direct feedback on the division's importance to U.S. industry and allow them to direct their research into appropriate new areas. The absence of metrics to measure industrial impact makes it harder for the division to focus its research efforts on industrial needs.

Funding sources for the Surface and Microanalysis Science Division (in millions of dollars):

Approximately 40 percent of the OA funding in this division is targeted toward capital equipment purchases and maintenance, which provide long-term benefits. Therefore, only about 30 percent of the division 's yearly operating expenses are covered by outside funds, and the panel believes that this level is appropriate. However, the panel considers overdependence on OA funding for augmenting and updating the division's capital equipment base to be dangerous, as state-of-the-art instrumentation must be maintained even when external support is unavailable.

The staff of the Surface and Microanalysis Science Division currently includes 37 full-time permanent positions, of which 34 are for technical professionals. There are also 13 nonpermanent and supplemental personnel, including postdoctoral fellows and part-time workers.

The personnel include three NIST fellows, and the entire research staff is excellent. Morale is high throughout the division. An examination of program staffing levels shows the distribution of human resources is appropriate. In general, there are no personnel problems, and important projects do not lack key people. Leadership is strong in three of the four groups. However, the absence of an official group leader for the Atmospheric Chemistry Group is a pressing concern, especially because management may be unable to fill this position from the current group of investigators and may need to look outside NIST.

The instrumental resources in the division are excellent. In the Microanalysis Research and Analytical Microscopy Groups, much of the equipment was obtained through Air Force funding. This instrument set is superb, but should Air Force support cease, the cost of replacing this equipment would become a concern. The instrumentation used by the Surface Dynamical Processes and Atmospheric Chemistry Groups is also high quality. New state-of-the-art instruments are being constructed on site, whereas other new equipment orders are pending. The one piece of equipment missing in the division is a commercial, state-of-the-art, high spatial resolution Auger electron spectrometer —a significant omission.

An important problem faced by the division is the quality of experimental facilities. The division's research requires much better space with finer temperature control, air purity, and vibrational stability. (Some examples of current and future difficulties resulting from problems with facilities are described below.) The construction of the Advanced Materials Laboratory would alleviate these issues in the long term. But the division lacks a short-term facility improvement plan as well as a contingency plan, in case the building is not constructed.

The Surface Dynamical Processes Group's development of the new femtosecond laser for future infrared surface studies is promising. However, for these new efforts to be successful, the experiments and lasers need better laboratory space with lower floor vibrations, more stable temperature and humidity, more reliable electrical power, and cleaner air to preserve optics.

The Microanalysis Research Group is pushing the limits of spatial resolution in microanalysis. However, much of the research is currently hampered by the poor laboratory conditions. For example, particles in the air contaminate samples and impede the nanoparticle characterization effort. The laboratories currently have about 10⁶ particles per cubic foot, but the experiments require air cleanliness several orders of magnitude better. Problems with vibration, air temperature, and humidity cause decreases in analytical capability and staff productivity. More stable electrical power is also needed; last year there were more than 50 power outages of greater than 1 sec. The panel was aware of several other institutions with similar power problems where solutions were found locally through installing uninterruptible power supplies or generally by negotiating with local power suppliers to improve power quality sitewide.

The Analytical Microscopy Group provides a world-class facility and national resource for particle analysis. The decision to purchase a new time-of-flight SIMS for the group will complete their suite of analytical microscopy tools. As in the Microanalysis Research Group, the division has gathered excellent scientists who form a high-quality team. However, the ability of these researchers to maintain their leadership role and to continue to define the state of the art in analytical microscopy is being impeded by the state of the facilities.

The Surface and Microanalysis Division's planning process is effective and complete. In particular, focusing the research on major programs that cut across the four technical groups effectively ensures complete reviews of industry research needs and encourages collaboration among division personnel. The research staff feels engaged in the planning process.

The Physical and Chemical Properties Division stated that its mission is to be the nation's reference laboratory for measurements, standards, data, and models for the thermophysical and thermochemical properties of gases, liquids, and solids—both pure materials and mixtures; the rates and mechanisms of chemical reactions in the gas and liquid phases; and fluid-based physical processes and systems, including separations, low-temperature refrigeration, and low-temperature heat transfer and flow. The division develops and maintains state-of-the-art experimental, theoretical, and modeling capabilities and provides Standard Reference Data, Standard Reference Materials, calibrations, and technical research and services to promote U.S. economic growth and to assist U.S. industry, other government agencies, and academia in solving problems of national importance.

The programs of this division remain firmly rooted in the central NIST mission to work closely with industry. Every group in the division has ties to appropriate industries to some degree. Yet this division maintains a balance between research directed toward industrial concerns and work motivated by a more general view of the nation's science and technology needs.

The skills of the staff members in the Physical and Chemical Properties Division are complementary, unique, and highly valued. NIST personnel form virtually the world's only remaining teams actively involved in making fundamental measurements of thermodynamic properties. This division includes the expertise necessary for critical evaluation of existing literature and construction of databases containing useable information. The value of the work done by the division is high, and the research is definitely relevant to industrial needs. An important feature in this division is the strong synergism between experiment and theory, as evidenced by the interactions within and among the various groups.

In the most recent reorganization, the Pressure and Vacuum Group was moved to the Process Measurements Division, and two new groups —the Chemical Reference Data and Modeling Group and the Experimental Kinetics and Thermodynamics Group—were formed within the Physical and Chemical Properties Division. The panel views these changes as part of an ongoing, sensible restructuring and consolidation of laboratory activities. In fact, the strength

of the Physical and Chemical Properties Division has been enhanced by broadening the scope of the research to include work on reactive systems.

The Chemical Reference Data and Modeling Group focuses mainly on modeling and data collection and evaluation. Work centers on providing kinetic and thermodynamic information for a large number of compounds. The group is also responsible for the Mass Spectrometry (MS) Database used by all major vendors of MS machines. The NIST Chemistry WebBook generated by this group is a searchable compilation of chemical data designed for Internet access. This product is available free of charge to the general public and represents positive outreach to both the industrial and university communities. Another ongoing activity of this group is the compilation of kinetics data for chemical reactions in the gas phase. The resulting database is published by NIST's Standard Reference Data Program and was last updated in November 1994. The group is currently working on the next edition, and the panel strongly supports completing and rapidly releasing the update.

Adding a computational chemistry emphasis to the Chemical Reference Data and Modeling Group broadens the group's impact. One goal is critical evaluation of the differences between computationally derived molecular properties and experimental results. Another exciting new venture guides nonexperts in choosing methods and assessing reliability and resource requirements for different types of computations of molecular properties. The vision of this subgroup is ambitious but consistent with the needs of the industrial community as well as with NIST's traditional mission to provide an infrastructure for measurements and standards.

The Experimental Kinetics and Thermodynamics Group uses photolysis, high-pressure mass spectrometry, shock-tubes, and calorimetry to obtain kinetic and thermochemical data. The project on using a cavity ring-down laser system to measure gas-phase reaction rates is an example of how NIST is developing new data-taking techniques to meet future industrial needs. The fluorine bomb calorimetry work could have a significant impact on thermodynamic property evaluation methods for semiconductor materials. The combination of fluorine- and oxygen-based calorimetric techniques allows the division to study a wide range of inorganic and organic materials of industrial interest.

The Fluid Science Group develops state-of-the-art techniques for measuring the thermodynamic and transport properties of pure fluids and mixtures, including alternative refrigerants and semiconductor processing gases. Another research focus is on next-generation primary standards in temperature, pressure, and low-flow rates. The project on transport coefficients such as viscosity near critical points is an example of important research on the development of advanced measurement techniques.

The Theory and Modeling of Fluids Group develops databases, correlations, and predictive models of fluid properties, studies the microstructure of complex (colloidal) fluids, and works toward advancing theories of phase transitions and dilute solution properties. A university researcher on sabbatical at NIST assisted in building interfaces for the computerized databases, making them more flexible and user friendly. The projects in complex fluids are solid basic science and use neutron and light scattering as well as fluid rheology to characterize multiphase media; systems vary from supramolecular (wormlike micelles) to colloidal (clays). The group has appropriately de-emphasized aspects of its computational fluid dynamics program that did not complement other work in the division and refocused existing staff resources on the dissemination of thermophysical properties data.

About three-quarters of the effort by the Cryogenic Technologies Group is in advanced refrigeration systems. The remaining work is on cryogenic waste grinding, two-phase heat transfer in liquid hydrogen, and maintenance and upgrades of the liquid nitrogen flow facility. About 60 percent of the projects in this group are externally funded. The spectrum of process scale is huge: from the nitrogen flow facility to cryogenic catheters for medical applications. The latter project is a good example of intergroup cooperation—the refrigeration gas for the catheter was developed using expertise from the Theory and Modeling of Fluids Group.

Current projects in the Process Separations Group include supercritical fluid vapor-liquid-solid equilibrium, natural gas separation/characterization, membrane separations/properties, adsorptive separations, azeotropic vapor-liquid equilibrium distillation with aqueous electrolytes, alternative refrigerant analysis, and analytical methods development. The effort in membranes research focuses on obtaining property data for polymer/solute pairs and developing models relating structure to property. A novel project is the use of the spectra of aromatic compounds to determine the polarizability of supercritical fluids, including alternative refrigerants.

The Experimental Properties of Fluids Group focuses on precision measurements of vapor-liquid equilibrium, surface tension, equations of state, and thermal properties of alternative refrigerants, ammonia/water mixtures, and other industrially important fluids. Improving measurement techniques is also emphasized. Three examples are twin-cell calorimetry, transient hot-wire thermal conductivity cells, and the vibrating wire viscometer, a device that functions over a broad range of temperatures. The emphasis of the work is on obtaining data over a broad range of temperatures and pressures while maintaining high accuracy.

All the division's groups are making a strong, well-directed effort to convey their results to the scientific and engineering communities. The division is widely recognized for providing U.S. industry with the data and models needed to identify and implement environmentally acceptable alternatives to chlorofluorocarbons (CFCs) and hydrochlorofluorocarbons (HCFCs) in air conditioning and refrigeration equipment. The value of this work is discussed in the laboratory-level assessment.

The impact of the Chemistry WebBook on the chemical industries and on university researchers will likely be considerable. In the 31-day period after the February 1997 release, about 8,000 distinct hosts accessed the WebBook, downloading more than 200,000 pages of information. Many small businesses that cannot afford expensive data resources of their own will benefit from the availability of inexpensive databases over the Web.

In the area of chemical identification, the division has continued to expand and improve its Mass Spectrometry Database. In fact, 60 percent of the 3,000 mass spectrometers sold in the United States annually include the NIST database and software, and more than 25,000 copies of the database are currently installed on mass spectrometers.

The division is working with several companies on cryogenic technologies. It is collaborating with Cryenco, Inc., on the design of a 500-liter/day liquefier for natural gas based on pulse-tube refrigeration technology. This project will make using liquefied natural gas a more cost-effective fuel for fleet vehicles and will open new markets for these smaller-size liquefiers.

Another joint effort is the work with CryoGen, Inc., on development of cryogenic catheters that can destroy damaged or diseased tissue. This technology will create alternative treatments to expensive, intrusive, and all-too-common procedures such as open heart surgery.

NIST has commissioned economic impact assessments for six programs in the Chemical Science and Technology Laboratory. Two of these are from the Physical and Chemical Properties Division: Alternative Refrigerants to Replace Ozone-Depleting CFCs and Advanced Refrigeration Systems for Cryogenic Applications. Both studies are currently under way. In addition, the Standard Reference Data Program commissioned a study to assess the impact of the division's research on patents in the area of alternative refrigerants, which indicated a strong correlation between important patents and use of the division's research results. The panel endorses using such studies to effectively assess impact.

During the past few years, the division has made an effort to gather feedback from its industrial customers in several areas, including alternative refrigerants and liquid fuels. This division has produced three projects recognized by NIST management as successful examples of industrial impact.

The division's model of working closely with industry to identify needs and develop programs assures that the division's work will have impact. Significant evidence of the value of divisional programs is also seen in industrial funding of a number of projects. This support can be direct (through corporate grants) or indirect (through the purchase of data products).

Funding sources for the Physical and Chemical Properties Division (in millions of dollars):

The division spends approximately 7 percent of its annual budget on capital equipment, defined as costing more than $2,500. This spending level, and the resulting instrumentation available to division researchers, appears adequate.

The level of Other Agency (OA) funding is a problem throughout the division. OA money is becoming more difficult to acquire, and the funding agencies and companies are requiring unreasonable levels of cost-sharing. Although most current externally supported projects are consistent with the division's mission, this situation cannot be expected to continue indefinitely. The present level of OA funding —approximately 35 percent—is high, and division management would prefer this figure to be closer to the laboratory average of around 20 percent. Nonetheless, all groups in the division think that maintaining some external support is important,

and the panel concurs that fighting for competitive funds is a healthy process for any research team.

Data collection and evaluation as well as database development and dissemination are important to the general scientific community over the long term. Although such projects represent a core element of the NIST mission, such activities are hard to support using OA resources. The major problem with outside funding is the danger that overdependence on such external sources of income could turn NIST into the research branch of other organizations. Such subordination is clearly undesirable. NIST walks a fine line between universities and industry and acts as a unique bridge between the often short-term interests of industry and the longer-term, more fundamentals-based approach of universities. A lack of outside funding for connective projects would not be a problem if sufficient support were available from NIST's direct congressional appropriations. However, NIST currently does not set aside funding specifically for longer-term data projects that fill the gap between industry and academia.

The staff of the Physical and Chemical Properties Division currently includes 65 full-time permanent positions, of which 54 are for technical professionals. There are also 17 nonpermanent and supplemental personnel, including postdoctoral fellows and part-time workers. The NIST staff is split approximately equally between Gaithersburg and Boulder.

The slow decline in CSTL staff over the past 5 years (from 300 full-time-equivalents to 275) makes the need to use contractors and guest researchers more acute. The Physical and Chemical Properties Division has more guest researchers and contractors (approximately 60 full-time equivalents) than most divisions of the laboratory. The range of expertise these visitors bring to NIST is welcomed by division researchers. Given the wide scope of this division's projects, the panel believes that the current number of guest workers and contractors is appropriate. In addition, all the groups are eager to have a National Research Council postdoctoral fellow, but the supply does not appear to be able to meet the demand.

A NIST Fellow in this division is effectively assuming the responsibilities of managing a group. However, NIST policy precludes his official acceptance of this role, which has led to some uncertainty within the group.

Physical and Chemical Properties Division staff in Gaithersburg are generally satisfied with the physical facilities, although some problems were noted. In certain laboratories, the mechanisms to ensure air cleanliness, dust control, and air filtration are insufficient; the quality, capacity, and reliability of the power supply are problematic; and the exhaust and ventilation systems are inadequate. Future laser-based optical studies will require cleaner rooms and better vibration control. However, the current quality of the laboratories is comparable to that of facilities at research-oriented universities.

In Boulder, the division's laboratory space in Building 2 is generally adequate. The two most significant infrastructure problems have recently been addressed by installing a new heating, ventilation, and air conditioning system for this building and increasing power capacity and quality. However, Building 2 is currently severely overcrowded because laboratory space in Building 3 is totally inadequate. Problems associated with this facility include extremely minimal temperature control (no cooling, only ventilation); extremely poor air cleanliness, dust control, and air filtration; unreliable, low-quality, low-capacity power; poor lighting; and a leaky exterior shell. These problems have a strong negative effect on the division's work.

The Physical and Chemical Properties Division reviews and prioritizes its programs at least annually at both the division and laboratory levels. Reviews include identifying promising new opportunities as well as determining areas in which the level of impact is no longer commensurate with the amount of NIST funds invested. Mechanisms used to identify new opportunities include industrial workshops, attendance at scientific and engineering conferences, participation on committees of industrial trade associations and professional societies and in industrial road-mapping activities, direct collaborations with individual firms and groups of firms, site visits to companies, external evaluation panels, and National Research Council studies. The division uses similar means to identify the needs of other agencies. The criteria used to set priorities are the same as those used at the laboratory level and are listed in the Laboratory Planning section of this report.

This process keeps the division well abreast of new developments in the areas covered by the division mission. NIST personnel are visible at major meetings and conferences in their fields. Overall, the panel finds this planning process to be well organized and appropriate. Management has done an exceptional job of coordinating various activities across group boundaries, connecting the Gaithersburg operation to Boulder, and developing strategic plans for future research. Division leadership also has done a fine job with personnel development and utilization.

The Analytical Chemistry Division stated that its mission is to conduct research concerning the qualitative and quantitative determination of chemical composition; develop and maintain state-of-the-art chemical analysis capabilities; provide measurement quality assurance through reference materials, data, and services, and provide a basis for U.S. chemical traceability and international comparability.

The Analytical Chemistry Division is the fundamental chemical metrology component of the Chemical Science and Technology Laboratory and NIST. The division mission is fully and effectively integrated into the overall mission and goals of both the laboratory and NIST. Divisional programs provide chemical measurement standards, accurate and reliable chemical compositional data, and research in chemical measurement science in support of NIST and laboratory missions.

The division consists of five groups: Spectrochemical Methods, Organic Analytical Methods, Gas Metrology and Classical Methods, Chemical Sensing and Automation Technology, and Nuclear Methods. This division and its organizational structure is the result of the 1994 merger of the Inorganic and Organic Analytical Research Divisions. Research activities

related to chemical measurement sciences are carried out in analytical sensing technologies, classical analytical methods, gas metrology, laboratory automation technology, nuclear analytical methods, organic analytical methods, and spectrochemical measurement methods.

Research in the Spectrochemical Methods Group focuses on the development, critical evaluation, and application of methods for the identification and measurement of inorganic chemical species using x-ray, optical, and mass spectrometry. This division has a unique capability in thermal ionization mass spectrometry (TIMS), and this group has used new sulfur isotope ratio and isotope dilution techniques to solve various important problems. For example, a TIMS technique was developed to measure the natural variation in the ³⁴S/³²S isotopic ratio in atmospheric source and receptor samples. This information allows researchers to distinguish between anthropogenic and natural sources of atmospheric sulfur, and possibly to assess the impact anthropogenic sulfate emissions have on the global distribution of sulfur. Suspended sulfur-bearing particles emitted from stationary sources were analyzed, and the results were used to differentiate among components in source apportionment models. Many of the samples contained less than 100 µg of sulfur; only the TIMS technique developed at NIST was capable of providing the sensitivity and precision required.

NIST has one of only three ultraviolet/visible Fourier transform spectrometers in the world. This group continues to use this instrument to develop techniques for high-accuracy wavelength measurements. The uncertainties in these measurements have been evaluated and resolved using a two-color laser wavelength calibration technique. Work will begin in fiscal year 1997 on the measurement of new ¹⁹⁸Hg wavelength standards.

The Nuclear Analytical Methods Group continues research on the use of neutron beams as analytical probes with both prompt gamma activation analysis (PGAA) and neutron depth profiling (NDP). During an extended reactor shutdown, the cold-neutron PGAA and NDP spectrometers were rebuilt to take advantage of the newly installed cold neutron source at the NIST reactor. New analytical instrument designs complement the new cold source and have improved detection limits of both instruments by a factor of about 4. Efforts continue to increase elemental sensitivities and spatial resolution by developing techniques and methodology for focusing cold neutron beams with a polycapillary fiber and a monolithic lens for analytical applications.

The Organic Analytical Methods Group is developing separation methods and standards to identify and quantify organic and organometallic species. Group researchers are examining the use of supercritical fluid extraction of hair samples as a possible means of detecting illegal drug use patterns. Supercritical fluid chromatography methods under development may be more effective than other techniques in separating optical isomers of pharmaceuticals and agricultural chemicals. Studies are under way to assess and eliminate laboratory-to-laboratory variances in DNA fingerprinting. Liquid chromatography (LC) and mass spectrometry (MS) methods and standards will be able to identify and quantify protein content in baby foods, infant formula, and human serum albumin. Capillary electrophoretic methods are being developed to characterize the structure and effectiveness of liposomes (phospholipid micelles) in selectively delivering bioactive agents across cell membranes to active enzyme sites. Recent breakthroughs in organometallic speciation have produced the first certified values for methyl mercury in mussel tissue. Work is under way to assess the bias between Karl Fischer volumetric and coulometric determination of trace amounts of water in oils used as insulators in electrical transformers.

Adoption of these method modifications is being discussed with American Society for Testing and Materials Committee D-27.

The Gas Metrology and Classical Methods Group is focused on gas metrology, classical wet chemical methods (such as gravimetry and titrimetry), coulometry, ion chromatography, optical spectroscopy, and maintenance of the theoretical infrastructure for pH and conductivity measurements. Currently researchers are developing methods to monitor trace levels of nitrogen dioxide in nitric oxide (NO). NO therapy is highly effective in reversing persistent pulmonary hypertension in newborns (“blue baby” syndrome, characterized by low oxygen uptake), but oxidation can produce trace amounts of nitrogen dioxide in the mixture that result in pulmonary edema. An infrastructure has been implemented to support production of standards, and NIST Traceable Reference Materials (NTRMs) have been developed for use in medical labs. Fourier transform infrared measurement capabilities are being developed with low sensitivity to carbon dioxide and water. These methods, when perfected, will provide standard spectra and absorption coefficients for 100 of the 189 Environmental Protection Agency (EPA) hazardous air pollutant molecules listed in the EPA's Clean Air Act. These techniques will then allow for quantitative fence-line stack emissions. Work is progressing to qualify a continuous wave, solid-state, tunable laser source for on-line determination of oxygenated hydrocarbon emissions from vehicles. This group is actively involved with international intercomparisons with a European collaboration on measurement standards in such areas as primary gases for the automotive industry, conductivity for water quality, and pH standards. This will allow European 12/22/00companies to use local standards in making regulatory and trade agreements with the United States. The group is also actively involved in developing gas, metal, spectrometric, and environmental traceability for U.S.-produced secondary reference materials. This activity is important because the demand for SRMs traceable to primary standards has grown rapidly. Such SRM stocks are difficult and costly to maintain.

The Chemical Sensing and Automation Group develops and applies new technologies, techniques, and standards for chemical sensing, sample preparation, and laboratory automation for chemical analysis. For more than 25 years, NIST has provided optical filters to calibrate the transmittance and wavelength scales of visible and ultraviolet spectrophotometers. Recently this group built a database that contains all available information about optical filter SRMs. New data created during filter certifications and recertifications are automatically entered into the database. A candidate filter material for a glass near infrared (NIR) wavelength standard (700 to 1700 nm) was developed, and test filters were evaluated for wavelength reproducibility, stability over a limited temperature range, and acceptable transmission characteristics using a Fourier transform NIR spectrophotometer. A round-robin activity is being scheduled to allow potential customers to evaluate and comment on the new material. A new Fourier transform infrared (FTIR) and Raman spectroscopy instrument was developed to be used in noninvasive determination of oxygenates in SRM gasoline ampuoles. A workshop with the clinical chemistry and instrument manufacturing communities was held to determine and prioritize needs for low-level and single-photon-counting light intensity standards. It was decided that the most important areas were work on low-level artifact standards that permit traceability to the SI base units and development of standard chemical reactions that produce a known number of photons per mole. Parametric evaluations of candidate technologies for the artifact standards are in progress.

The panel is impressed with the quality and impact of the research programs under way in the Analytical Chemistry Division. The projects described have effectively support the division's mission and the laboratory's strategic plan. Personnel are highly competent, and the reference laboratories of all five groups are world class. The programs have significant impact on a wide range of industrial sectors, and the excellent dissemination mechanisms include reference materials and reference datasets, calibration services, workshops, CRADAs, consortia, and round robins. Division scientists remain active, as evidenced by their abundant number of publications, presentations, national and international committee memberships, and other service contributions. The project on international comparability of chemical measurements is a world leader in the development of SRMs, NTRMs, proficiency testing programs, and international intercomparisons. The number of CRADAs (12), calibrations or recertifications (264), and SRMs/RMs (141) has remained constant over the past 3 fiscal years, and the number of NIST Traceable Reference Materials (NTRMs) has increased. The division continues to add new skills by hiring high-caliber postdoctoral fellows for 1- to 2-year terms.

One of the panel's concerns is that remarkably few patents have been obtained (only one in fiscal year 1996). The panel noted that this low level may be because the division does not encourage staff to use patent disclosures as an alternative or complement to professional publication.

The Analytical Chemistry Division and each of its groups maintain working information pages on the NIST Chemical Science and Technology Laboratory Web site. These pages help disseminate information about the activities and recent technical highlights of the various groups. Certified SRM values are currently available on the NIST Web site, but the pages do not include detailed background information such as particle size distribution.

Another concern the panel raised is that the division's current efforts on thin films for the semiconductor industry appear limited. No CSTL reference materials currently exist to benchmark measurement accuracy of monitors on composition, area density, and uniformity of thin films on various substrates during fabrication operations. Glow discharge source research (including radio frequency operation and mass spectrometry detection) would respond to the Semiconductor Manufacturing Technology (SEMATECH) consortium's request for the development of thin-film SRMs. Focus on use in thin-film quantification and possible thin-film SRM evaluation would complement the X-ray fluorescence characterization of these films.

As is clear from the programs discussed here, the division's contributions to industry are significant. New analytical methods and procedures as well as important SRMs and NTRMs meet industrial needs in chemicals, electronics, automotive research, petroleum refining, instrumentation, biotechnology, environmental technologies, health care, and aerospace. The division's work is the foundation for the certification of chemical composition in more than 850 SRMs important to U.S. industry, government agencies, and educational institutions. Measurement methods derived from division research activities are also used to establish and maintain chemical measurement traceability links for producers of commercial reference materials (such as gases, optical filters, and spectrometric solutions) and measurement comparability links with chemical metrology laboratories worldwide.

The capabilities gained from these activities establish and maintain the national infrastructure that provides U.S. industry with the tools necessary to achieve international comparability of chemical measurements. Techniques used include high priority SRMs, NTRMs from commercial sources, interlaboratory proficiency testing programs in critical areas, international intercomparisons of primary measurement methods and standards with other national metrology laboratories, specialty analyses, and critical analytical databases, such as those on inductively coupled plasma (ICP) wavelengths, FTIR remote sensing, and Henry's law constants. These standards and methods are the basis for international trade equity and for establishing and enforcing safety, health, and environmental standards.

For example, definitive isotope dilution MS methods have been applied to calcium, magnesium, lithium, and potassium for the certification of SRM 956a (Electrolytes in Frozen Human Serum). This SRM is intended for use as a calibration standard in clinical laboratories, thus establishing NIST traceability for important health care diagnostic markers. Work is under way to establish reference methods for other important biomarkers. Development of accurate methods for two of these analytes is an important component of the ongoing collaboration with the College of American Pathologists to establish NIST traceability for important health markers. Several other new SRMs are being developed to support measurement activities in the environmental, health care, food, and nutritional labeling arenas.

The division has contributed to successful implementation of the NTRM Program, as described in the laboratory-level assessment. Plans to extend the NTRM model to other sectors of the commercial reference materials community will initially emphasize spectrometric solutions.

The division actively participates in intercomparability programs among national and regional standards laboratories to facilitate international trade. Structured intercomparison programs with other national metrology laboratories remain the basis for formal establishment of equivalence among primary methods and standards important for global commerce. During fiscal year 1996, the “Declaration of Equivalence ” for primary gas standards between NIST and the Netherlands Measurements Institute was expanded to include seven primary mixture suites. In addition, NIST, the Danish Institute of Fundamental Metrology, and the Hungarian National Office of Measures have signed a new memorandum of cooperation to intercompare primary standards in conductivity.

An area in which the division is not currently active is the development of computer validation protocols and methodologies to ensure compliance with EPA and Food and Drug Administration regulations. The industrial benefits from such work might justify a NIST effort in this area.

To assess its impact on industry, the division continuously solicits input from U.S. companies on their technological needs. These interactions help shape divisional planning and programs. Extensive ties to industry and participation in national and international committees and professional organizations permit direct community assessment of NIST programs. The division plans to evaluate the mechanisms used to assess impact in several areas, such as health and food standards and gas NTRMs.

Funding sources for the Analytical Chemistry Division (in millions of dollars):

There is substantial need to upgrade or replace aging instrumentation in this division. A number of inorganic mass spectrometers for TIMS are unique, but they are old and need to be replaced. The division does not have a high-resolution ICP mass spectrometer. Some optical emission spectrometers and a wavelength dispersive x-ray fluorescence spectrometer are also aging and need to be upgraded or replaced. Augmentation or upgrades of the benchtop gas chromatography/mass spectrometry (GC/MS) and GC/atomic emission detector systems are necessary, because these systems are used heavily. A new high-resolution organic magnetic sector mass spectrometer for the organic definitive method program was acquired, but there is still no triple quadrupole MS nor a benchtop LC/MS in this division. The magnetic sector MS for gas metrology is 35 years old, and it is time to replace it. Finally, the Chemical Sensing and Automation Group lacks a fluorescence microscope and imaging system and semiconductor processing equipment.

The division had already planned to replace selected capital equipment to meet long-term needs. However, cutbacks in funding for equipment have delayed this expenditure. The expected occupation of the Advanced Chemical Sciences Laboratory (ACSL) in January 1999 is an opportunity to confront instrumental issues. Relocating precision equipment is costly, so the most practical approach is to replace dated or tired instrumentation when it comes time to move to the new building.

Construction of the ACSL, a state-of-the-art building to support CSTL research programs, is under way. When it is completed in late 1998 or early 1999, this building will house most Analytical Chemistry Division activities. The division will then have sufficient space and modern physical plant facilities to maintain its research and standards efforts.

The staff of the Analytical Chemistry Division currently includes 68 full-time permanent positions, of which 62 are for technical professionals. There are also 21 nonpermanent and supplemental personnel, including postdoctoral fellows and part-time workers. As the number of permanent staffers has decreased approximately 5 percent over the past 3 years, non-NIST researchers have come to play a major role in divisional activities. Nonetheless, the division still has too few scientists. Although all methods are documented and published, a certain amount of art is still inherent in practicing scientific methodology. The panel was concerned because the division does not cross-train enough personnel to ensure that there is a backup staff able to perform each function; nor does the division have enough succession planning for group leaders.

All division research and service projects are reviewed on an annual basis to assess the quality of the work and whether it remains in keeping with the mission and with customer needs. Projects that are not progressing or that have successfully achieved their goals are concluded so that new programs may be funded. The general focus of the research and measurement service activities within the division will be stable over the next year. The one exception is in the Chemical Sensing and Automation Program, where the attention to measurement and standards needs in process analytical chemistry will increase. This increased emphasis on real-time measurement and standards issues is reflected in the new projects that will be initiated over the next 2 years.

During the past 3 years, CSTL planning has been formalized through off-site meetings with staff and management. A strategic plan is now in place to encourage staff and management to use greater initiative and assume greater responsibility and authority in achieving their goals.

In the summer of 1994, the former Inorganic and Organic Analytical Divisions merged. The new Analytical Chemistry Division then developed a strategic plan for research and service activities. The goals of this plan included changing the funding profile for the division and reengineering delivery of SRM services. Over the past 3 years, this plan has been carefully implemented as the division has successfully adjusted its structure and personnel in response to budgetary pressures. For example, some reductions in personnel will be offset by the shift from SRM production to NTRM commercial products in spectrometric solutions and optical filters. The panel strongly supports and endorses the division's strategic plan. Some directions that are particularly appropriate include the efforts to reduce SRM backlog, increase the ratio of new to renewal SRM effort, evaluate modes for SRM value assignment, and establish infrastructure for traceability and comparability for chemical measurements though intercomparison of primary standards among national and international laboratories. The NTRM Program has been successful with gases. However, the metrological community also requires NTRMs from other SRM types, such as spectrometric solution, optical filters, and organic NTRMs. In related work, the panel is pleased that this division's participation in international intercomparison programs is expected to increase.

The division has effectively sought desirable research and service support and has distributed available funding to achieve strategic objectives. Management determined that OA support should be about 20 percent of total funding and has adjusted the budget and the programmatic distribution to reach this goal. Physical facility planning efforts have contributed to the custom design of space in the ACSL to meet the division's specific needs. Although the number of full-time NIST personnel in this division has decreased, management has maintained an adequate staffing level for divisional programs through reassignment of current staff and use of non-NIST personnel. Graduate students, guest scientists, and collaborators from other agencies have augmented the permanent NIST staff and introduced new technology into divisional research and development activities. One concern is that all of the groups are not equally effective at recruiting outside personnel.

• The Chemical Science and Technology Laboratory publishes world-class scientific data and produces SRMs that appropriately target national needs. In addition, the laboratory is deeply involved in national and international measurement intercomparisons that support U.S. companies in the global market.

• The overall quality of the technical programs meets or exceeds the panel's expectations.

• The laboratory has made a good start on the effort to determine the economic impact of its activities. So far, economic impact studies have been used to calculate the value of specific projects. However, the laboratory does not yet have a general set of metrics that can be used to evaluate the economic impact of future activities.

• The panel was impressed by the laboratory's active presence on the Internet and by the quantity of NIST data available on the Web. However, not all of the data included the indication of its age and level of evaluation necessary to ensure appropriate use. In addition, the NIST Web site did not contain all of the SRM certification data or include detailed background information on the SRMs.

• Promotion of the mass spectral databases to instrument manufacturers appears to be an effective means of dissemination. It was not clear to the panel why the surface analysis data are not aggressively promoted in a similar fashion.

• The panel was pleased to observe the successful upgrade of Building 2 in Boulder and to hear about the new facilities (NIST's ACSL and CARB's Building 1-B) soon to be completed in Maryland. Unfortunately, Building 3 in Boulder, like the existing facility in Gaithersburg, is in urgent need of upgrading or replacement. The inadequacies of these two facilities impede the laboratory's goal of performing cutting-edge research that addresses current and next-generation needs.

• The poor quality of the power supplied to the site has a negative impact on various pieces of equipment and experiments.