An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2002 (2002)

Marvin F. DeVries, University of Wisconsin-Madison, Chair

Richard A. Curless, Cincinnati Machine, a UNOVA Company, Vice Chair

Hadi A. Akeel, FANUC Robotics NA, Inc. (retired)

Christopher P. Ausschnitt, IBM Microelectronics Division

Robert Bridges, SMX Corporation

Richard J. Furness, Ford Motor Company

Marion B. Grant, Jr., Cummins Technical Center

David E. Hardt, Massachusetts Institute of Technology

Mark C. Malburg, Digital Metrology Solutions, Inc.

Newman M. Marsilius III, PMT Group

Eugene S. Meieran, Intel Corporation

Carmen Pancerella, Sandia National Laboratories

Jay Ramanathan, Concentus Technology Corporation

Wolfgang H. Sachse, Cornell University

Arthur C. Sanderson, Rensselaer Polytechnic Institute

Masayoshi Tomizuka, University of California, Berkeley

Peter M. Will, Information Sciences Institute/University of Southern California

David H. Youden, Eastman Kodak Company

Submitted for the panel by its Chair, Marvin F. DeVries, and its Vice Chair, Richard A. Curless, this assessment of the fiscal year 2002 activities of the Manufacturing Engineering Laboratory is based on site visits by individual panel members, a formal meeting of the panel on March 19-20, 2002, in Gaithersburg, Md., and the documents provided by the laboratory.¹

According to laboratory documentation, the mission of the Manufacturing Engineering Laboratory (MEL) is to satisfy the measurements and standards needs of U.S. manufacturers in mechanical and dimensional metrology and in advanced manufacturing technology by conducting research and development, providing services, and participating in standards activities. This mission statement has one small but important change from previous years: it no longer specifies that MEL meets the needs of “discrete-part manufacturers,” but now specifies simply “manufacturers.” The panel applauds this change, because it indicates recognition of the changing nature of manufacturing throughout the world, and particularly in the United States, where outsourcing, including outsourcing to overseas locations, has become a major element of the emerging “lean manufacturing” philosophy. In keeping with these new realities of manufacturing, the panel suggests that MEL review and consider the needs of all manufacturers in the supply chain, from large, original-equipment manufacturers down through procuring subcontractors as well as small, discrete-parts manufacturers.

The new mission statement lacks specific mention of information technology or knowledge software, which have become important technology within manufacturing today, and this may merit reconsideration by the laboratory. MEL could also consider changing the verb “to satisfy” in its mission statement to something stronger, denoting the leadership position that it is striving to achieve. Finally, MEL managers at all levels should ensure that the missions of each division dovetail with the MEL and NIST mission statements to form a coherent picture of MEL’s role in manufacturing technology.

The quality of research in the laboratory is high overall. In some areas, MEL work is state of the art relative to work being performed worldwide; in others it is not. While recognizing that for some activities it may not be necessary to be world-class, the panel notes that where MEL is not achieving world-class status, it is generally because resources are spread too thinly. Considering current budget and operating constraints and taking into account that achieving world-class status is not possible for every activity, MEL emphasis on collaboration is to be complimented and encouraged. In general, the staff is highly competent and motivated to have a positive impact on U.S. competitiveness and is capable of the technical leadership that MEL should be striving for.

The Manufacturing Engineering Laboratory is organized in five divisions: Precision Engineering, Manufacturing Metrology, Intelligent Systems, Manufacturing Systems Integration, and Fabrication Technology (see Figure 3.1). The first four divisions are reviewed in turn under “Divisional Reviews” below in this chapter; the Fabrication Technology Division, which houses the NIST machine shop, is not reviewed separately, but its “Shop Floor as National Measurement Institute” program is discussed in the reviews of its collaborating divisions.

MEL has a unique role to play in U.S. manufacturing through its expertise in measurements and standards of all sorts. The panel was impressed with the number of MEL researchers who had received awards and recognition from external organizations, including international industrial and standards organizations such as IEEE, ASTM, and the International Organization for Standardization (ISO). These awards are evidence of the value that their colleagues throughout the world place on their achievements and contributions to the manufacturing community.

FIGURE 3.1 Organizational structure of the Manufacturing Engineering Laboratory. Listed under each division are the division’s groups.

The panel was also pleased to learn of the prestigious Department of Commerce Gold Medal award given to the team of NIST researchers, including MEL staff, for their contributions to the highly visible Charters of Freedom project involving the design and construction of new display and preservation cases for the Declaration of Independence, the Bill of Rights, and the Constitution. This award, along with a Bronze Medal for work on the M48 Coordinate Measuring Machine, provides further evidence of the quality of the research done in MEL.

By refining its own focus and improving its customer focus, the laboratory could achieve even more significant impact from its expertise. The panel has seen increased efforts among MEL staff to identify customers and their needs as a means of directing programmatic efforts. This is responsive to earlier panel feedback on the need to better identify customers.²

It is critical that MEL identify its customers correctly, and to accomplish this, the laboratory needs to further refine its idea of the customer. In seeking its customers and their needs, MEL has not yet captured the viewpoint of firms deep in the supply chain. MEL must extend its customer focus in that direction in order to have the level of impact that it seeks. Also, it needs to focus on the needs of users in addition to those of vendors of advanced manufacturing technology. Customer contact should be sought at all levels of an organization, from bench-level scientists through top management. Better definition of the customer will only be accomplished through a combination of top-down and bottom-up approaches, in which strategic management is utilized to determine target industries, and working-level knowledge of those industries is used to target appropriate organizations, firms, and individuals.

MEL has made considerable progress over the past several years in program planning and in alignment of the MEL plan with NIST objectives. MEL commitment to long-term program base funding is recognized and applauded. Strategic planning is still being carried out to differing degrees throughout MEL. Planning is not uniform among the divisions, and what planning exists has not yet been fully integrated into programmatic decisions. Many ongoing programs appear to have been started for opportunistic reasons, but that cannot be the major factor driving program prioritization if MEL is to achieve its full potential for impact. As the NIST strategic plan is finalized, the panel anticipates that MEL will revisit its own plan for purposes of its relevance and appropriateness and adjust the MEL plan as necessary. The panel looks forward to reviewing the results of MEL planning processes at its next assessment.

MEL could benefit from a more systematic self-evaluation of its own effectiveness, which would provide it with information needed to optimize the use of resources, especially staff talents. The panel suggests that MEL consider adopting regular use of the Baldrige quality criteria to measure the effectiveness of its staff and the effectiveness and timeliness of its programs and outcomes.

MEL presented the panel with its responses to the newly defined, NIST-wide Strategic Focus Areas. The panel has the following reaction to these presentations.

In the area of homeland security, critical infrastructure protection must include the U.S. manufacturing infrastructure. MEL’s unique expertise and experience can be used to help define the issues and strategy in this area. Knowledgeable risk analysis of distributed, global, information-based manufacturing systems will help define the research agenda to ensure secure interoperable systems. MEL expertise in hierarchical information architectures and reliable data exchange could support such efforts. MEL should proactively seek participation in NIST-level, interagency, and industrial planning for infrastructure protection. MEL is in a favorable position to coordinate action across the federal/industrial boundary and in related technical aspects of this issue.

While the issue of software vulnerability and secure interoperable systems is one important facet of securing the manufacturing infrastructure, other broader issues exist that MEL might address or at least raise. Examples of areas that need consideration include identification of critical manufacturing technologies and practices that must be protected; the development of robust supply networks, especially those that have overseas components, and understanding of the economic impact of destabilization of these supplier chains; and identification of environmental impact in case of catastrophic events in manufacturing plants dealing with hazardous materials.

With its advanced measurement capabilities, NIST has an important role to play in establishing nanoscale measurement methods and standards. To date, NIST’s role has not been widely recognized, nor has NIST received a significant amount of the substantial increases in federal investment in this area. The completion of the Advanced Measurement Laboratory at NIST Gaithersburg will give NIST even greater capabilities on this scale, and this event should be capitalized upon by MEL, which has a clear role to play in nanotechnology. This role is a natural extension of MEL’s existing competence in metrology, measurement methods, and standards for length. As these are all enabling technologies for the growth of nanoscale industrial processes, the panel hopes to see MEL aggressively pursue its measurements and standards role in nanotechnology research.

The MEL role in health care is not entirely clear to the panel. Nonetheless, the size and importance of this sector and the rapid acceleration of technology adoption in medical practice make it worthwhile for MEL to consider what impact it might have in health care. The panel suggests that MEL identify key customers in this area to find out if they have needs that MEL can address and determine the areas in which MEL’s contribution could be unique, implementable, and appreciated. MEL should look particularly for those areas in which its capabilities would be complementary to its ongoing efforts, areas such as manufacturing systems interoperability, measurement techniques and algorithms, and standards development. On the basis of these considerations, MEL should decide whether it will try to have an impact in this sector.

The panel is enthusiastic about the choice of information and knowledge management as a NIST Strategic Focus Area (SFA). This SFA plays to many of MEL’s existing technical strengths, and the panel expects that each division of MEL will contribute strongly to this SFA’s objectives. The SFA seeks to enable the conversion of data into usable knowledge, and it can draw heavily on MEL expertise in the architecture of sensing and actuation and the development of related standards. It is not yet clear to the panel how the choice of this area as an SFA will affect NIST’s strategy and funding distribution. The panel looks forward to more information and evidence of progress in this area in the FY 2003 assessment.

Funding sources for the Manufacturing Engineering Laboratory are shown in Table 3.1. As of January 2002, staffing for the laboratory included 200 full-time permanent positions, of which 136 were for technical professionals. There were also 27 nonpermanent or supplemental personnel, such as postdoctoral research associates and temporary or part-time workers.

MEL’s budget has failed to keep up with inflation for several years. This failure, combined with mandatory salary increases for staff, has meant that the full-time permanent staff of the laboratory has shrunk significantly in the past several years, presenting considerable problems for MEL management as it seeks to address laboratory goals, objectives, and priorities. These problems also appear to be having a negative impact on staff morale, as staff experience the frustration of declining budgets, uncertainty about program futures, and having to spend increasing amounts of their time chasing external funds for their research.

While recognizing the challenge of managing under such difficult resource constraints, the panel believes that MEL could improve the use of its resources through more specific resource planning. MEL progress in strategic planning has not been matched by resource planning. There is need for a resource plan that encompasses human resources, equipment, and facilities and that is integrated with the MEL strategic plan to assure that resources are available for and directed toward the laboratory’s highest-priority programs.

In the area of human resources, the panel suggests that MEL adopt a 5-year personnel plan to predict the mix of skills it will need to achieve major objectives and that it chart how to maintain or obtain these skills. The panel recognizes that no manager can perfectly predict retirements, separations, or available new hires, but anticipating these events to the extent feasible and developing a strategy to ensure that the

necessary skill mix is available for the future will help increase the effectiveness of MEL’s use of resources and of its programs overall.

In developing a personnel plan, MEL should give particular consideration to the balance between administrative staff and technical staff and also to the balance between managers and bench-level staff. The panel members found the ratio of administrative staff to technical staff to be quite high compared with that of their organizations. Some divisions have already begun reprogramming administrative positions to technical positions as the former are vacated. As administrative positions open up, the panel believes that managers should consider carefully whether those positions’ duties can be effectively reassigned to remaining administrative staff and the resources used to broaden or deepen the technical skill base. The panel also noted a large ratio of manager to bench-level staff with the laboratory’s current matrix management approach. If the matrix managers are still functioning mainly as scientists, the ratio may not be inappropriate.

The panel is in agreement with the matrix management approach MEL is taking to meeting its programmatic objectives, but it notes the difficulties that accompany implementation of this approach. Frustration has arisen among program managers who are not also group leaders, as well as among the technical staff who report to these program managers. It is not reasonable to expect an employee to work on a project for which he or she is not evaluated. In many cases, the objectives of a group leader,

who does have reporting responsibility, are at odds with those of a program manager who does not have line reporting responsibility. In such situations, the technical staff member is given the uncomfortable choice of either supporting his or her group leader and getting a good evaluation or supporting the program leader who has responsibility for achieving specific program objectives.

Such tensions, which are not unusual when implementing a matrix approach, can be significantly ameliorated by appropriate employee performance evaluation. Because some projects cross division boundaries, the MEL director, the division chiefs, and the program managers should all be involved in the employee evaluation process. Program managers as well as group leaders should submit evaluations of all technical staff members working on their projects. Evaluations of group leaders and division chiefs should consider their support of cross-program projects. Such an approach can relieve much of the tension experienced by personnel in a matrix-managed situation. MEL management should consider talking to colleagues at other government laboratories that utilize matrix management (for example, Lawrence Livermore National Laboratory) to see how they implement the matrix approach.

The panel was pleased to see the many instances of internal recognition of staff accomplishments. Such recognition by top MEL management helps build staff morale and can give greater emphasis to laboratory priorities. The panel encourages increased use of internal awards and staff recognition.

Existing equipment within MEL is generally acceptable. However, as the panel noted last year, no detailed plan exists for equipping the Advanced Measurement Laboratory (AML). As this laboratory comes closer to completion, the need for a specific equipment plan becomes more urgent. This plan must detail both the equipment that is to be purchased and the existing equipment that is to be moved and refurbished. The AML offers MEL and NIST the capability to do world-class work in a number of important areas, but this potential will be realized only if the facility is properly equipped. As lead time for the purchase of major instruments can be several years, planning for equipment in this new facility is overdue.

MEL presented the panel with information on NIST’s new Organizational Focus Areas (OFAs)— personnel, customer focus, and information technology (IT) infrastructure. The panel applauds these choices; having discussed customer focus above, the panel comments below on personnel and IT infrastructure.

MEL’s plan in the personnel OFA needs to address the work environment. In particular, to be successful in its matrix-management approach, MEL needs to pay closer attention to the nurturing of staff skills and development. A long-range plan is needed for staff career development. Attention is needed to long-term guest researchers, who are generally overlooked in plans for mentoring, rewards, and advancement but who nonetheless make significant contributions to MEL’s research agenda.

This OFA complements the SFA in information and knowledge management. Knowledge management can be used to create a collaborative environment across programs at NIST. By creating, collecting, storing, and sharing data and knowledge, staff can use knowledge from across NIST to achieve results and can create virtual “centers of expertise.” The OFA must consider rewards for sharing data and collaborating.

MEL efforts in personnel should include tracking of results to enable continuous improvement in personnel practices. MEL presented the panel with its own internal survey of staff, but the sample size presented and the response received were too small to be meaningful. Additional means of obtaining meaningful data—such as results of the biennial NIST-wide employee survey—should be used to track employee satisfaction in MEL.

An efficient and effective IT infrastructure is vital in any modern organization. This OFA concentrates on reducing IT costs by using a combination of centralized and decentralized approaches to service. The panel did not hear any discussion of the use of collaborative software for managing knowledge and business practices; use of such software could contribute significantly to the OFA goals. Since information- and knowledge-based manufacturing is a major component of MEL’s work, MEL is a significant stakeholder in this effort and should be a major contributor to planning in this area.

In its previous report, the panel’s primary recommendations with respect to MEL concerned the declining level of staffing, the need for better strategic and program planning, and the need for an equipment plan to ensure future capabilities, especially in the AML. The new MEL director provided responses to all three of these concerns. On the issues of staffing and planning, it is clear that the laboratory is making efforts in the right direction. The remaining issues in both of these areas are matters requiring longer-term attention; the panel will look forward to increased progress in these areas over time. The panel would also like to see metrics that capture progress over longer periods of time, while recognizing that recent changes in management structure make the continuity of such metrics difficult. As noted above, the panel was disappointed to see that no action had been taken on the need for a plan to equip the AML. The time horizon for occupying the AML is growing shorter, and the time at which this planning should have been initiated is already past. The panel urges MEL and NIST as a whole to develop the equipment plan quickly but carefully and to begin its implementation, since the process of refurbishing and procuring the necessary equipment can be expected to take several years. The capabilities of the AML offer NIST extraordinary opportunities to advance measurement science, but these opportunities will be wasted if the facility is not properly equipped.

The panel presents the following major observations:

• The panel concurs with the broadening of the Maufacturing Engineering Laboratory mission statement to recognize manufacturing beyond that of discrete parts. MEL should consider whether its mission should state its role in information technology more explicitly and whether the mission statement should be posed in more proactive terms.

• MEL has made progress in its strategic and program planning efforts. More remains to be done to achieve an integrated plan for MEL efforts at all levels. In particular, the laboratory needs a resource plan that can be integrated with the strategic plan to ensure that MEL will have the skills, equipment, and facilities it needs to meet its intermediate-term goals and objectives.

• MEL has improved its customer focus but needs to continue to work to define its customers better. In particular, to have the impact on manufacturing that it seeks, MEL must broaden its customer focus by looking deeper into the supply chain. It should also consider customers at all levels of the companies and organizations with which it interacts, not just at the level of scientific and engineering peers.

• The panel agrees with MEL’s matrix management approach as a means to best utilize staff skills to accomplish laboratory objectives. Changes in the employee evaluation process may be necessary to better align evaluation with the program management structure.

• The panel is concerned about the decline in the number of MEL technical staff and its impact on the laboratory’s ability to meet its goals and objectives. The laboratory lacks a human resource plan that anticipates skills needed to meet goals, takes staff retirements and separations into account, and lays out a strategy to ensure that MEL has or can obtain the skills necessary to meet its highest-priority objectives. Careful consideration should also be given to the ratio of administrative support staff to technical staff and to the ratio of managers to technical staff.

The mission of the Precision Engineering Division is to provide the foundation of dimensional measurement that meets the needs of the U.S. industrial and scientific communities—by conducting research in dimensional measurements, developing measurement methods, providing measurement services, and disseminating the resulting technology and length-based standards. Within the division is a diverse set of programs, organized in four groups: Nanoscale Metrology, Surface and Microform Metrology, Engineering Metrology, and Large-Scale Coordinate Metrology. The division also contributes to the Shop Floor as NMI Program.

The panel is very impressed with the quality of the work and the array of capabilities represented within the Precision Engineering Division. The division performs length measurements over 12 orders of magnitude, all at state-of-the-art precision for national measurement institutes (NMIs). This range is divided into four overlapping segments, corresponding to the groups listed above. Each group has successfully achieved many well-defined goals since the last review in 2001 and is making significant contributions to the overall success of the division. The division’s staff continues to perform outstanding work that is recorded in archival publications. Individual staff members have garnered many technical awards and have played leadership roles in major conferences and industry consortia. The technical quality of the measurement work is also quantitatively benchmarked by round-robin measurement activities with other NMIs around the world, with results that reflect well on NIST.

Nanoscale Metrology Group. The Nanoscale Metrology Group extends dimensional metrology to the submicron scale, providing standards, measurement capability, and measurement uncertainty guidelines for the semiconductor and nanotechnology industries. Its stated goal is to provide to the U.S. microelectronics industry the reference measurements, reference standards, and metrology necessary to realize the production goal of 100-nm devices by 2005. This encompasses Standard Reference Materials (SRMs) and metrology methods for photomask and wafer critical dimension, pattern placement and overlay metrology, as well as the development of three-dimensional structures of controlled geometry whose dimensions can be traced directly to the intrinsic crystal lattice. The program has been strongly guided by the International Technology Roadmap for Semiconductors (ITRS). Members of the Nanoscale Metrology Group continue to work in close collaboration with the industry consortium International SEMATECH.

Notable achievements of this group within the past year include the development of a process by which the ultrahigh-vacuum scanning tunneling microscope (UHV STM) can write features as small as 10 nm and the completion of studies leading to the construction of a next-generation Line Scale Interferometer. The group also developed a maskless lithography system, completed calibration and

delivery of photomask standard artifacts for SRM 2800, fabricated a second-generation overlay test reticle set, fabricated a new scanning electron microscope (SEM) test pattern, and issued SEM Sharpness Standard SRM 8091.

As predicted in last year’s report, the division’s ability to support semiconductor mask suppliers with meaningful standards is slipping because of the lack of state-of-the-art equipment. In the push to produce 100-nm and smaller wafer feature sizes, the semiconductor industry is relying increasingly on enhancements to mask technology, such as phase-shift masks, proximity correction, and subresolution assist features. Consequently, the dimensional metrology issues associated with masks are expanding rapidly, especially given the role of the so-called mask enhancement factor (MEF) in increasing the sensitivity of wafer dimensions to mask dimension variation. Mask dimensional metrology is an area in which the accuracy issues central to the NIST mission play a critical role. Nonetheless, according to panel members’ informal survey of colleagues in industry, U.S. mask suppliers are not utilizing NIST standards as widely as should be expected. The two-dimensional placement standard that they require is not yet available from NIST, and the German standard is used instead. The NIST standard is only slated for release in the fall of 2002. One reason given by the division for the delay is NIST’s reliance on the availability of two-dimensional grid equipment at Photronics; hence the need for internal equipment. The uncertainty associated with current optical critical dimension standards from NIST is greater than the 10-nm specification that critical levels require. To its credit, NIST has pioneered work in reducing line edge roughness, a significant contributor to measurement uncertainty. Mask makers are increasingly turning to SEM metrology to control and validate their manufacturing. In the absence of state-of-the-art SEM capability, NIST is not in a position to support their needs.

Surface and Microform Metrology Group. The Surface and Microform Metrology Group works primarily in the measurement of nanometer- to micron-scale surface features, utilizing microscopy or stylus-based instrumentation. Several areas of the division’s work support areas of industrial interest, such as microelectronics, optics, other manufacturing industries, and homeland security.

Most projects target a specific need and are focused. However, the group seems to lack an identity and has been driven to simply “go where the money is,” rather than to remain focused on its core competencies and charter. Specific projects have led the group to work more in the nanoscale than in the traditional micrometer-scale work that remains important to industries such as the automotive and aerospace industries. This tendency may also be driven by the equipment and staffing of the group, which are better suited for nanometer work. In micrometer surface metrology areas, particularly stylus-based measures, primary customers are metal-cutting industries. These more traditional industries, because of years of optimization and cost reductions, do not often have the budgets for research. However, in terms of the manufacturing economy, they still require ongoing support.

Staff members in the Surface and Microform Metrology Group are highly regarded in the technical community, and their work is world-class. In some cases, however, the NIST instrumentation is lagging behind what is currently available in industrially based laboratories. As a result, the group is sometimes forced to choose projects primarily because of the instrumentation available rather than for strategic reasons.

Engineering Metrology Group. The Engineering Metrology Group works to manage and reduce the contribution to uncertainty of the traceability of length, location, and spacing measurements as well as other traditional geometric, dimensioning, and tolerancing dimensional controls (roundness, cylindricity, perpendicularity, angle, and so on). The group characterizes, evaluates, and improves instruments that measure length and coordinates, and it develops new measurement techniques for measurements from

1 micrometer to about 1 meter. This group also supports the availability of alternate routes to establish traceability, such as National Voluntary Laboratory Accreditation Program (NVLAP) laboratories (for example, the Oak Ridge Y12 Metrology Laboratory and the Starrett-Webber Gage Block Laboratory) and other ISO-17025-compliant laboratories, as well as collaboration with other recognized NMIs such as the National Physical Laboratory of the United Kingdom and Physikalisch Technische Bundesanstalt of Germany. The group’s primary customers are the U.S.-based discrete-parts manufacturing industry and the measurement equipment makers that serve them (such as those affiliated with the American Measuring Tools Manufacturing Association).

Because of collaborations with measurement equipment makers that have provided necessary instruments to update several key measures, the group’s equipment is up to date relative to that used by its customers. The group has achieved world-leading status in length traceability and evaluation of two-dimensional coordinate measurement machine (CMM) artifacts through the use of an M48 Moore Special Tool CMM with Brown & Sharpe control and analysis software and systems. This capability is world leading at 1 micrometer error or less anywhere in the artifact volume. The advanced automated master angle calibration system (AAMACS) polygon calibration system is also of world-leading status in technology and capability. The group’s gage block calibration capability is world-class, and ongoing research into the effects of deformation and surface finish are maintaining this traceability at that level.

A reasonable amount of research is being done to support the discrete-parts manufacturing and measurement equipment makers’ current and future measurement needs. The projects include these:

• Research in collaboration with NIST Boulder to decrease the uncertainty of laser interferometry measurements for greater capability in length and frequency metrology,

• Research in dilatometry to measure thermal expansion with lower uncertainty and carry the capability to two-dimensional grid expansion,

• Cylindricity capability research in collaboration with industry,

• New laser micrometer methods for measurement of sperical and cylindrical diameters, and

• New probing methods for submillimeter feature measurement and for use in new calibration methodologies.

The Engineering Metrology Group also is continuing to support the research and standardization of uncertainty management. It supports the Shop Floor as NMI Program, which seeks to provide documentary standards and to educate industrial users about how to manage measurement uncertainty. In all projects the group endeavors to include studies in levels of confidence in its measurement developments and services. Collaboration with international NMIs is ongoing to compare group results with those of its international colleagues.

The group’s services will be shut down around 2004 for several months in order to move into the new AML. During this time, the group plans to rely more heavily on partner laboratories such as Y12 for continued customer support.

Large-Scale Coordinate Metrology Group and Shop Floor as NMI. The Large-Scale Coordinate Metrology Group works to characterize, evaluate, and improve instruments that measure lengths and coordinates greater than 1 m. Originally the group considered mostly fixed CMMs, but today it looks increasingly to a wide variety of portable CMMs. Some of the group’s projects are partly funded by outside grants and contracts. These include the development of a laser-rail calibration system for laser trackers and a total-station measurement system for shipbuilding. One of the group’s important goals, is the development of measurements and simulations that help in the calculation of measurement

uncertainty. In addition, Chebyshev referenced algorithms and rigid-surface fitting algorithms are being created for industry’s self-assessment of measurement and will be placed on the Web site. The members of the technical staff hold many leadership positions in professional organizations and standard-setting groups.

The Shop Floor as NMI Program aims to help U.S. parts manufacturers support their claims of measurement traceability, even without a direct NMI dimensional calibration, by providing documentary standards, guidelines, and reports. Several of these documents were completed this year; one was recognized as the best paper of the 2001 International Meeting of the National Conference of Standards Laboratories. The program is also undertaking projects to help instrument users characterize uncertainties in shop-floor measurements. Projects include the development of a smart artifact (a calibrated artifact that serves as a surrogate workpiece, containing advanced GD&T features); a ball-bar calibrator; a method for fitting complex surfaces; and a method for estimating thermal distortion. In response to last year’s report, the program has been reduced in size, and the new projects are focused on goals having greater applicability and impact.

Technical professionals and lower-level managers in the Precision Engineering Division have good contacts in industry and academia. At the project level, strong interactions with customers and contributing professionals outside of NIST contribute to the relevance and effectiveness of the individual projects. The involvement of high-level division and MEL management with customers and industry leaders is not apparent, however, and individual programs seem to be left on their own to define and carry out their goals. While some benefits accrue to a bottom-up approach, it also leads to an inability to formulate a broader strategy that might provide the basis for better program direction and a stronger funding outlook.

The division’s current attempts at quantifying goals in terms of time, money, or savings are erratic. For example, last year the Surface and Microform Metrology Group articulated the goal of saving industry $80 million over the next 5 years. This year, no mention was made of that goal or of progress toward it. The panel does not mean to suggest that all programs should be driven by potential savings to industry, particularly since such numbers are often hard to validate. Where meaningful, however, the quantification of benefits is essential to tracking progress and to establishing priority among many possible tasks. It is also a powerful tool for marketing the division—both to improve the visibility of its programs to industry and to solicit industry funding for critical expenditures.

The division’s setting of goals and reporting of progress would be improved by adopting methods more consistent with those used in industry. Progress reports too often consist of a recitation of technical details without any perspective on performance trends over time. Reporting would be more effective if the division used conventional methods that enable rapid assimilation and comparison of results. For example, SRM introduction could be tracked using time-to-introduction Gant charts, and measurement services could be tracked by year-to-year volume, revenue, and cost.

The division is effective in communicating with customers via publications and conferences, and its Web site contains a wealth of information. In many cases, programs respond to specific customer requests, but the division does not appear to track customer usage of its results. An informal survey of colleagues conducted by panel members revealed significant doubts and misconceptions on the part of potential customers regarding the relevance of NIST results to the semiconductor industry.

The Engineering Metrology Group and its resources are heavily used by its customers, with many customers relying on the group’s services. Among all of the MEL groups, Engineering Metrology has

the highest level of income from services ($800,000 in calendar year 2001), making most of its calibration services self-supporting. The group has also received equipment from industry to use to push the state of the art in several areas, which demonstrates strong trust, collaboration, and rapport with this customer segment. Thus, the group seems to have a clear picture of its area of responsibility and its duties to satisfy customer needs. However, the group is struggling to reduce calibration costs, which are due in part to rising overhead rates. It is difficult for the group to provide any calibration for less than $5,000, and this level of cost pushes customers to use alternate routes such as National Voluntary Laboratory Accreditation Program (NVLAP) and ISO 17025 laboratories.

The Engineering Metrology Group has good customer relations, but it could establish stronger ties to discrete-parts manufacturing customers by communicating its capabilities, desired research directions, and ongoing developments. The group might be able to bring in more industry money and backing to support such work. Participation in consortia, standards committees, and trade shows could be used more widely to inform, receive feedback, and solicit support.

Funding sources for the Precision Engineering Division are shown in Table 3.2. As of January 2002, staffing for the division included 35 full-time permanent positions, of which 32 were for technical professionals. There were also 5 nonpermanent or supplemental personnel, such as postdoctoral research associates and temporary or part-time workers.

Funding levels and staffing appear to have stabilized over the past year. As a result, staff morale has improved. Retention has also improved, no doubt in part due to the decrease in attractive high-tech opportunities elsewhere.

Underlying the division’s lack of capital planning is its lack of strategic planning, also raised in the recommendations of the past 2 years.³,⁴ Significant capital purchases cannot be justified without a comprehensive plan showing that they are necessary to meet critical program objectives. This strategic plan must be based on close engagement between NIST upper management and the customer community.

Despite the high technical capability and progress demonstrated, the Nanoscale Metrology Group’s programs suffer from lack of state-of-the-art metrology equipment to provide meaningful industry support in the nanometer regime. This deficiency was noted in last year’s report. Both NIST researchers and industry customers have identified two critical equipment needs: (1) a current-generation critical dimension-SEM (CD-SEM) for dimensional metrology for both masks and wafers and (2) a nanometer-scale two-demensional grid metrology system. NIST acquisition of a combined SEM/ focused-ion-beam tool for the in situ cross-sectioning of masks and wafers would support this rapidly growing area of metrology activity in the semiconductor industry. All of these are “big-ticket” items, each costing upwards of $2 million to $3 million. Existing funding strategies do not allow such capital purchases. In fact, MEL management recently rejected a proposal that would have had SEMATECH fund much of the cost of a new CD-SEM. The division’s lack of advanced metrology equipment is already hampering its efforts to support industry. Given that nanotechnology has been identified as a national initiative and a NIST Strategic Focus Area, NIST management must find ways to enable the acquisition of these essential metrology tools. Providing to industry metrology services that are enabled by new equipment, perhaps modeled after the services provided by the Engineering Metrology Group, could at least partially fund the equipment.

As stated above, the equipment budget allocated for the Advanced Measurement Laboratory appears inadequate. Furthermore, division researchers have apparently not been actively engaged in planning equipment purchases for the AML facility, although they plan to migrate important programs there. If AML occupation is going forward in 2004, equipment planning is already overdue. The panel is concerned that the division may not be able to make full use of the AML’s capabilities for lack of appropriate equipment.

Funding continues to be an ongoing battle for the Surface and Microform Metrology Group. It is very difficult for the group to initiate a new project or program with its own funding, and few customers for this area are willing to provide significant funding. This lack is also evident in terms of the age and capabilities of NIST instrumentation compared with industry capabilities. Many of the group’s instruments, while they are well understood and well utilized by the staff, are not up to date; SEM calibration is one example—NIST cannot produce an artifact that is needed by industry because of the lack of instrumentation necessary to calibrate it.

According to division documentation, the mission of the Manufacturing Metrology Division is to fulfill the measurements and standards needs of the United States in mechanical metrology and advanced manufacturing technology by the following:

• Conducting research and development in realizing and disseminating SI mechanical units;

• Developing methods, models, sensors, and data to improve metrology, machines, and processes;

• Providing services in mechanical metrology, machine metrology, process metrology, and sensor integration; and

• Leading in the development of national and international standards.

The division is organized in four groups: Mass and Force, Machine Tool Metrology, Manufacturing Process Metrology, and Sensor Development and Application.

The name of the division—Manufacturing Metrology—implies a broad scope of activities related to manufacturing. The mission also can be interpreted as being quite expansive. Recognizing the limitations of budget and number of personnel, the division’s management has made a strategic decision to limit its scope to four major programs: Advanced Optics Metrology, Mechanical Metrology, Smart Machine Tools, and Predictive Process Engineering. The division also plays a supporting role in other programs—Nanomanufacturing, Shop Floor as NMI, and Nanoscale Metrology. The panel commends the division for focusing its scope to improve the potential impact of each program. These programs and the technical projects within them are closely tied to the current core competencies of the division.

Overall, the Advanced Optics Metrology, Mechanical Metrology, and Smart Machine Tools Programs are well focused on areas of significant need and are of high technical quality. The panel was particularly impressed with the Advanced Optics Metrology Program, which addresses an area of significant need in semiconductor manufacturing and appears to be well connected with users through International SEMATECH. The panel has some concern regarding current program leadership in Advanced Optics Metrology. As a result of staff turnover, the division director has assumed the role of interim manager of this program. The panel hopes that the division will use this opportunity to recruit someone with strong technical as well as managerial skills to lead this important program.

In response to the previous panel review, the division significantly altered its Smart Machine Tools Program. In fact, this program has changed titles, and the scope of the program is now very well defined. The critical technical issues have been identified, and the team has established a preliminary road map for projects appropriate for NIST. The panel believes that the road map for this program should continue to evolve, as it is not yet fully developed.

The panel believes that the goals of the Predictive Process Engineering Program, which is only partially supported by the division, are admirable, but they are not new and likely not realizable. The panel commends the program leaders for responding to last year’s recommendation to narrow its scope. As a result of that action, the integration of projects within the program is better defined, and the program focuses on just two metal-cutting operations. However, despite this restricted scope, the panel remains concerned about the complexity of the total integration of all the subprojects and believes that the program’s focus is still too broad and that its stated goals are unattainable. The panel also believes that the program’s efforts on process models are misplaced and should be focused instead on manufacturing software interoperability and standards, which would be more in line with NIST’s mission and expertise. The panel recommends that in the coming year, the group select a demonstration part that each project in the program can utilize, integrate the available simulation models, and thus focus and link all the important technical project deliverables essential for fulfilling the program vision. Doing so will highlight the technical capabilities of the concept, and any unforeseen technical challenges will be exposed. The panel thinks that the demonstration part will be critical for tracking the ongoing progress of the projects and the overall program. The panel recommends that after the demonstration part is completed, the division seriously reevaluate its available technical resources for this program, and, if necessary, secure the appropriate expertise to meet revised program goals.

The panel continues to be impressed with the division’s work on microforce measurements. This team has made significant progress in the past year, and the panel believes that the effort exemplifies the type of work in which NIST should be engaged.

The continuing role of the division as a leader in the development of national and international standards ensures that U.S. interests are protected. For this reason, the division must continue to provide adequate resources in this area. The division devotes significant resources to the key comparisons and other requirements called for in the Mutual Recognition Agreement⁵ signed in 1999 by the NMIs of the 38 member states of the Metre Convention. The division’s continued active role in this area is imperative.

The Manufacturing Metrology Division serves two primary roles. First, the division is the nation’s reference laboratory for the units of mass, force, vibration, and sound pressure. In this role, the division serves the nation by providing calibration services, developing advanced methods for mechanical metrology, developing national and international standards, and leading efforts with international standards organizations. This role is critical for the nation’s manufacturing industry, and it is also critical for distributed international manufacturing and commerce. The division retains world-class capabilities and has state-of-the-art facilities for a number of metrology services. Given limited technical resources, the division has decided to phase out its ultrasonic standards work. While this may alleviate the shortage of resources in the near term, the long-term implications of the reduction of measurement and standardization capability cannot be predicted, particularly in view of NIST’s stated intent to play a role in health care.

The second major role of the division is to develop manufacturing and mechanical metrology technology. The customers of this effort include both industrial and governmental communities. The division fulfills this role through its own internal projects and by acting as a catalyst or facilitator for collaborative efforts between government, industry, and academia.

The division’s work is disseminated through workshops, consortia, and standards committees. Workshops are also used to identify customer needs. Division staff are frequently invited speakers at conferences and seminars. These presentations, the issuance of standards, and the division’s technical publications all provide evidence that the division is serving its customers, although these indicators cannot be used to measure the impact of these results quantitatively. The impact of calibration services is more readily determined by considering the number of paying customers. A user survey by the division indicates that the customers for these services, while concerned about costs, are satisfied. A clearer indication is achieved by the Mass and Force Group, which serves individual paying customers directly and is working with an industrial partner in implementing new, higher-resolution mass calibrations. The continued request for division involvement in standards activities (e.g., IEEE 1451, ANSI/ ASME [American National Standards Institute/American Society of Mechanical Engineers] standards, and international standards) also indicates the relevance of the division’s work.

Toward the beginning of calendar year 2001, the division introduced a matrix-management structure utilizing group leaders, program managers, and project leaders. Though matrix management has been very successful in large organizations with several distinct cross-functional responsibilities, the

panel wonders whether the same benefits are being realized in this division because of its smaller organization with rather homogeneous responsibilities. The panel believes that the transition to the matrix-management structure has created some unnecessary and costly duplication. Specifically, the panel wonders whether the roles of the Mass and Force Group leader and the Mechanical Metrology Program leader might be filled by just one individual, thus freeing up one person for other assignments. From a discussion with bench-level researchers, the panel found that the new management structure has led to considerable frustration as researchers attempt to “serve two masters” (group leaders as well as program managers). As a result, the researchers are experiencing work insecurity and inefficiency. Also, the perception exists among the researchers that employee reviews are not adequately coordinated between group leaders and program managers, although the panel heard that the management team tries to collaborate on these reviews. Group leaders appear to find the new management structure awkward at times but are trying to make it work. They find it a challenge at times to match employee skills with research needs. The panel believes that while matrix management may be beneficial for the division, its implementation should be continually monitored and modified as necessary to ensure that the hoped-for results are realized at all levels. Supervisory training is especially important during the transition to matrix management, as is the need for increased communication at all levels of the organization.

The panel heard little discussion of project selection criteria in this review, as was also observed in last year’s report. This should be an area of continual attention in order to ensure relevance in project selection. The panel recommends that each project be clearly connected to both MEL’s strategic plan and the division’s mission statement.

Funding sources for the Manufacturing Metrology Division are shown in Table 3.3. As of January 2002, staffing for the division included 40 full-time permanent positions, of which 36 were for technical professionals. There were also 5 nonpermanent or supplemental personnel, such as postdoctoral research associates and temporary or part-time workers.

The panel is deeply concerned about the division’s ability to retain talented technical professionals and to recruit equally talented new employees. Several technically strong, key personnel left the division in the past year. The success of the division’s programs relies heavily on the technical competence of its staff. The division now has a small number of researchers covering many technical areas, which can negatively affect program results. Strong fiscal constraints appear to preclude new hires.

The panel commends the division for its renewed efforts to augment the permanent staff with postdoctoral fellows but is concerned that if the number of topflight researchers who might serve as mentors declines, the goal of increasing the number of postdoctoral researchers will become difficult to realize. In a meeting with the panel, division staff voiced concerns over budget, time allocation, the matrix-management system, and scarce resources. All these issues are of concern and affect staff morale.

The division’s technical resources have been flat or declining over the past several years. The panel recommends that the division explore cost-effective ways of expanding its technical capabilities, such as giving critical consideration to a significant reduction of its nontechnical staff. This may represent the only real opportunity for the division to develop its technical capabilities without adding to the total number of staff members.

According to division documentation, the mission of the Intelligent Systems Division is to develop the measurements and standards infrastructure needed for the application of intelligent systems by manufacturing industries and government agencies. The division is organized in five groups: Perception Systems, Knowledge Systems, Control Systems, Machine Systems, and Systems Integration. The division manages Intelligent Open Architecture Control of Manufacturing Systems, Intelligent Control of Mobility Systems, and Critical Infrastructure Protection Programs. It also supports Predictive Process Engineering, Shop Floor as NMI, and Nanomanufacturing Programs. The division manages a Competence Development project, which extends basic research that was formerly part of the Research and Engineering of Intelligent Systems Program.

The Intelligent Open Architecture Control of Manufacturing Systems Program continues important work related to easing the problem of interoperability of manufacturing hardware systems. It has a goal of developing a suite of standards in this area by 2005. This is clearly an area in which the division can have significant impact on industry through the promulgation of standards for intelligent systems. The program comprises work on a number of standards and on testing and validation of standards for several different industries and equipment types: for example, standards for machine tools, motion control in general, robotic welding, automated inspection, factory robots, CMMs, and so on. The program has performed a number of demonstrations on tasks ranging from integrated metrology systems to welding control systems. It is currently developing a metrology testbed, in part to test standards for automated inspection and inspection data handling. The program has also made progress on the use of open systems; the panel was particularly interested in the promising work on the real-time use of Linux in control systems. One of the most mature projects involves the Standard Exchange of Product data standard for computer-aided design and manufacturing (CAD/CAM applied to numerical control [STEP-NC]). While STEP-NC has been demonstrated to allow sharing of data among systems, differing interpretations of the standard can result in machined parts that differ from their intended design. To prevent this, the division is developing conformance tests for CAM system and machine tool controller

adherence to STEP-NC. This effort involves both virtual and solid model comparisons, development of measurement artifacts, and physical measurements.

The goal of the Intelligent Control of Mobility Systems Program is to satisfy the measurement and standards needs of U.S. industry and government agencies in developing and applying intelligent control technology to mobility systems in order reduce cost, improve safety, and save lives. The program consists of three projects: Industrial Material Handling, DOD Unmanned Ground Vehicles (UGV), and Performance Measures for Mobile Robots. A significant portion of the 2001 expenditure on this program ($3.5 million out of a total of $4.0 million) was obtained from other agencies. The division participated in the Army Research Laboratory’s Demo IIIC, providing the control systems architecture, advanced sensor systems, and standards to achieve autonomous mobility for unmanned vehicles. The demonstration was very successful, and a significant level of further funding is expected—which evidences the leadership status of the division in this area. It is among the world leaders in this technology.

The division responded to the need of the U.S. search-and-rescue community to develop and disseminate reference test arenas to enable measurement and understanding of robot capabilities. The test course designed by the division was adopted by the American Association for Artificial Intelligence conference in 2000. NIST will chair the RoboCupRescue competition in 2002 in Japan. The division has used funding from other agencies (OA) in advanced robotics projects to remain at the forefront of research in the autonomous control of complex systems, and it has invested direct appropriations to transfer advanced robotics technology to manufacturing applications. For example, a project on the vision guidance of robots that handle industrial materials attempts to demonstrate and transfer the DOD UGV technology to industrial applications. It utilizes a conventional vision camera to track double lane markers along its desired path, a configuration typical of aerospace facilities. The panel is pleased with the division’s technology transfer effort. At the same time, the panel notes the need to objectively assess the difference, if any, between military problems and manufacturing problems. The division’s focus should be on adapting its technology, that is, ladar guidance in an unstructured environment, to provide advantages in the industrial environment, rather than on applying its resources to solve a specific user need with conventional technology.

The goal of the Critical Infrastructure Protection Program is to increase the security of computer systems that control production and distribution in critical infrastructure industries—including utilities, processing industries such as oil and gas, chemicals, pharmaceuticals, metals and mining, and pulp and paper, as well as consumer products and discrete-parts manufacturing industries. The program has the objectives of defining and applying standard information-security requirements, developing best practices for information security, and developing laboratory and field-test methods for information-security products applied to the process control sector. The division has rationalized the criticality of this program by identifying typical sources of vulnerability in process control systems, historically isolated but now increasingly connected to outside information and communications networks. Furthermore, the changing controls and customer support technologies often enable outsiders to gain access to process control systems through wired and wireless communications systems, which appreciably increases an enterprise’s vulnerability to hackers and destructive interference. The criticality of the program is further underscored by the potential consequences of attack in terms of loss of life and production, endangerment of public health and safety, social disruption, and environmental damage.

The division has followed a pragmatic approach to managing the Critical Infrastructure Protection Program by forming the Process Control Security Requirements Forum, with representation from related industries, concerned government and private agencies, and other relevant NIST divisions. The forum’s goal is to increase the security of industrial control systems through the definition and application of a common set of information-security requirements for these systems. This goal is well aligned

with the division’s goal. The testbed identified for testing of proposed protection products is simple and adequate, and it appears to be expandable so that it can accommodate new elements as they are recognized. A task list has been developed for FY 2002, with specific tasks related to the core goal. The tasks appear to be chosen as the basis of a structured approach to attain the goal, though no overall plan is obvious. A high-level plan would be valuable, with a road map leading from the current state of vulnerability to the desired state of protection. This road map should include definite dated milestones associated with available funding and resources.

The panel agrees with the approach utilized to support the Critical Infrastructure Protection Program. NIST, through the Intelligent Systems Division, is a natural lead entity for such a critical program. Because the program requires diversified resources not necessarily present at or appropriate to NIST and since it requires the active participation of many proficient and highly specialized resources from private industry and other government agencies, the panel believes that the division should assume leadership in the capacity of a program manager. The division may develop an overall plan and a program strategy, align and assign contracts, manage the budget, test and evaluate the results, and develop the standards necessary for identified security measures. Of particular importance is leveraging existing expertise in system vulnerability identification (white-hat hacking) to identify weaknesses and recommend remedies. The panel recommends that the division acquire such expertise for the execution of this program.

The Competence Development project encompasses the core research themes that were formerly included in the Research and Engineering of Intelligent Systems Program. The goal of the project is to provide the fundamental research underlying the intelligent systems that support the evaluation, specification, and integration of systems applied to manufacturing and other applications.

This project continues the division’s long-standing commitment to the development and application of a hierarchical intelligent-systems architecture. This framework and its history are presented in a recently published, comprehensive overview by Albus and Meystel.⁶ This book is an important milestone for the project, as it assembles the conceptual basis of this architecture of intelligent control in a form that will support new applications and encourage the comparative analysis and integrated implementation of these concepts.

The NIST real-time control system (RCS) hierarchical architecture forms the basis for applications programs in the Intelligent Systems Division, including manufacturing, open architecture, interoperability, and critical infrastructure. The architecture provides a framework with explicit model hierarchies that integrate sensing. The resulting applications may serve many customers, ranging from the academic and industrial research community to manufacturing and defense industries. The division strategy in this area, which is sound and working well, bases diverse applications on a common architecture. The division must view the architecture as an umbrella for integration rather than as a rigid template. It has the opportunity to further increase its partners and collaborators in this development. Flexibility that enables adopting new concepts within the RCS framework will be important, since new methods for adaptivity and behavioral systems may become modular components of the more general hierarchical system.

A principal strength of the Competence Development and Infrastructure Program is the division’s capability to implement and evaluate RCSs in several different applications domains. This experimental

prototyping approach is unique and important in research programs in robotics and automation, and it should continue to be an area of emphasis in the structure of the research programs. Current projects stimulate fundamental research and analysis through the evaluation of performance in real systems, and they also directly influence and disseminate practical solutions to industrial and government users. The division should continue this role of implementation and evaluation and not be drawn too strongly into pure analysis and algorithm development. Partnerships with academia and other basic researchers can provide the fundamental research support, while the division programs retain a broad capability to integrate, explore, and evaluate new approaches. This program is essential to the pursuit of division goals. The program goals are appropriate and well conceived, but implications of the program may actually be broader than stated, with applications in industries beyond manufacturing. The principal program themes should be examined to emphasize those in which key progress and milestones will be achieved. Alternative themes that align with key domain areas (such as software development and security issues) could be considered, while more abstract themes (such as learning, knowledge engineering, and hybrid control) may be contracted to research partners. The fundamental strength of the division in implementing and evaluating real systems through prototypes and testbeds should be emphasized.

The Intelligent Open Architecture Control of Manufacturing Systems Program has had numerous ties with industry, most notably with automobile manufacturers and their suppliers. However, the division’s key industrial partners or customers are currently industry groups such as Open Modular Architecture Control, the Robotics Industry Association, the Automotive Industry Action Group, the Metrology Automation Association, the American Welding Society, and the Consortium for Advanced Manufacturing International. These may not be the best sources of information, as they often lack a strategic cross-industry view of the problem. The division has for several years quoted figures of the cost to industry from the lack of interoperability—for example $200 million to $400 million lost per vehicle program, 100 staff-years for each $10 million in capital expenditure, and even costs as high as $1 billion per year for the automotive industry supply chain. In the panel’s judgement, these figures are highly credible, which makes the program especially relevant, with great potential for realizing economic and competitive advantages for U.S. industry. However, the panel thinks that if the need is this great, the division’s efforts should be larger and more unified than they now appear. Also, a push for this technology from the highest level of industry is lacking, despite its strong cost drivers, suggesting that more outreach to high-level corporate officials is needed. While the goals for the program are reasonable and useful, they will result in a number of rather specific niche standards. More general schemes that can be applied to a broader set of equipment and industry segments would be more useful and more appropriate. The new reality of major industries such as the automotive and aerospace industries is that they are quickly becoming a highly dispersed but even more tightly integrated supply chain that needs to share ever-more-detailed information. The importance of interoperability will be growing even stronger in this atmosphere. It is imperative that this program stay ahead of this trend, and it is recommended that an effort to work with higher-level industry representatives concerned with the longer range be considered.

The primary customers of the Intelligent Control of Mobility Systems Program are the Army Research Laboratory, the U.S. Department of Transportation (DOT), and DARPA. Those customers provide funding for the continued support of this program. The DOD UGV has been a demonstration testbed for the NIST RCS architecture and real-time sensing and measurement. The enormous potential saving of both cost and human life, and the potential safety improvements justify research on UGV for

military applications. The division also has a project on side sensing for vehicles, sponsored by DOT. Intelligent transportation systems (ITSs) remain one of the most important technologies, and the project is relevant not only to DOT but also to the automotive industry. The panel recognizes the division’s effort to transfer the UGV technology for military applications to manufacturing problems, though industrial customers are yet to be identified.

The Critical Infrastructure Protection Program is a timely and critical assignment for the Intelligent Systems Division. The division is the natural leader for program management and for funding and mobilizing the expertise of proficient and specialized private resources and concerned government agencies. The division has made excellent progress in identifying the problems to be addressed and involving potential funding agencies.

The Competence Development project focuses on four principal themes: performance metrics, knowledge engineering, architectures and tools, and learning. The division has coordinated the development of metrics through an annual conference, Performance Metrics for Intelligent System (PerMIS), that attracted significant participation from the research community in 2000. PerMIS can also serve as an important forum for further discussion of related architectures and concepts that address the challenges of intelligent-systems development. Work in the area of architectures and tools has also progressed well, as demonstrated by the dissemination of RCS architecture for applications to both manufacturing and mobile robotic systems. Collaborations and subcontracts have engaged additional systems users. Strong participation in the DARPA mobile autonomous robot software program has resulted in impressive demonstrations of RCS architecture in mobile autonomous vehicle systems. Advances in knowledge engineering and learning are less convincing, considering the work presented. There are several themes of conceptual work, but the panel heard little discussion of new and unique contributions or applications.

Funding sources for the Intelligent Systems Division are shown in Table 3.4. As of January 2002, staffing for the division included 33 full-time permanent positions, of which 31 were for technical

professionals. There were also 11 nonpermanent or supplemental personnel, such as postdoctoral research associates and temporary or part-time workers.

Division personnel are highly qualified, and they express motivation and confidence in their management. The division is privileged to have the only NIST fellow in MEL, benefiting from his contribution to the division’s technical objectives.

The panel supports the division’s decision not to constrain its research to technologies using computing power available at low cost, and to assume that, when these technologies mature, the computing cost for targeted practical applications will likely be economical.

Because of the large effort and highly specialized expertise necessary for the Critical Infrastructure Protection Program, the panel advises that the division leverage its expertise with outside contracts and focus on program management, funding, evaluation and testing and on developing evolving standards. It is advised that the division add to the program team a resident expert in the identification of system vulnerability. Otherwise the division appears to be properly staffed and funded to support its programs.

The stated mission of the Manufacturing Systems Integration Division is to promote economic growth by working with industry to develop and apply measurements and standards that advance the use of information-based manufacturing technology. As noted in last year’s assessment report, this mission is very encompassing and is not likely to be met without significant internal and external cooperation. But, based on the site visit, the panel believes the wording of the division’s mission may be interpreted to mean, “promote economic growth by working with industry to define, refine, and drive interoperability standards for software used in all aspects of manufacturing.” If this is an appropriate restatement, division focus on the interoperability of both product and process models becomes an achievable goal. The interoperability focus becomes immediately important (i.e., in the 5-year horizon) and will energize the division in the right direction. The panel suggests developing a divisional strategy to facilitate internal communications, external communications, and the assessment of improvements.

The division is organized in four groups: Design and Process, Enterprise Systems, Manufacturing Simulation and Modeling, and Manufacturing Standards Metrology. The work of the division supports five programs:

• Product Engineering, which includes projects in parametrics exchange, design-analysis integration, assembly and tolerance representation, heterogeneous material representation, and knowledge representation for next generation CAD;

• Predictive Process Engineering, which includes projects in process metrology, representation, modeling, and application;

• Manufacturing Simulation and Visualization, which includes projects on distributed manufacturing simulation environments, and manufacturing simulation transactions and simulation templates and model formats;

• Nanomanufacturing, which includes projects in atomic-scale measurement, manipulation, and manufacturing, and molecular-scale measurement, manipulation and assembly; and

• Manufacturing Enterprise Integration, which includes projects on a B2B testbed, and self-integrating system.

Other projects are grouped under the title Special Activities.

The division has recently undergone a major realignment of programs, leading to significantly improved performance. Division management has faced a monumental task in reorienting division activities. The division has much experience and expertise in mechanical engineering and manufacturing, but the industry needs assistance in the areas of formal specifications, semantic representation, knowledge-based systems, ontologies, and software conformance. These emerging areas are still a challenge even to computer scientists, and additional complexities arise when these areas are applied to manufacturing and engineering. The mechanical engineering and manufacturing knowledge and experience of the NIST scientists in this division are highly essential for success in systems integration. The key question is how to train existing personnel and attract new people who can bring the latest, newest technology into the group, especially in this time of flat budgets.

The division is engaged in work at several levels of abstraction in systems integration capabilities, ranging from standards and measurements, to process representation, through integration and modeling capabilities, to the use of software to enhance manufacturing performance. Within this framework, the panel identified two fairly specific technical areas as the foundations for the division’s efforts:

• Interoperability—Semantic models and knowledge representation methodologies for manufacturing interoperability, for product and process modeling and simulation, and for enterprise integration.

• Metrology—Measurements and testing with (1) algorithm testing and evaluation (e.g., looking at variability in physical measurements due to the nature of the software used in the measurement capability), and (2) optical measurement systems to be used for manufacturing at the nanoscale.

Most programs, except Nanomanufacturing, fit this categorization. The programs and projects that focus on interoperability and on metrology and standards are certainly of high quality, have state-of-the-art technical content, and are staffed by highly competent scientists and engineers. In addition, there appears to be a potential for a significant impact on commerce and the economy. Activities reflect both the immediate industrial testbed needs and the longer-term research needs.

The objectives of the Nanomanufacturing Program are not yet clearly articulated. Even so, the panel encourages this multidivisional activity, since the program is in a new and emerging area of technology, and it is wise to consider manufacturing early in any engineering activity. The panel suggests only that, as the division proceeds to work in this domain, it continue to focus resources on the conformance and interoperability issues rather than on instrumentation or basic physics.

For the most part, the division’s technical work is of high quality and is carried out with zeal and enthusiasm. However, most programs still have too many activities, and the participating workforce appears to be spread too thin. Unless the division gets more funding, the number of activities will have to be reduced. It seems unreasonable to expect that a staff of relatively constant size can be expected to cover an exponentially increasing domain of needs without loss of quality.

The panel found work on evaluating and certifying software manufacturing coordinate metrology to be an extremely interesting and novel approach to the problems associated with metrology and software certification. The panel strongly encourages continuation and expansion of this unique concept and program.

The panel applauds the care taken by the Manufacturing Systems Integration Division to ensure technology transfer. In this regard, the division’s leadership in STEP over many years is to be commended. Indeed, without NIST support, STEP would likely not exist.

The Manufacturing Simulation and Visualization Program is doing very commendable work. However, the panel believes that more attention should be paid to “best in class” work outside NIST when determining what should be done within NIST. The panel also suggests that team members solicit the active participation of constituencies in the prioritization of new activities in order to ensure their relevance. This may be done by workshops, site visits, collaborations, seminars, or other similar mechanisms.

Testbed infrastructure development has been recognized as the most immediate, fundamental mechanism for serving constituencies. It would be appropriate to identify performance measures up front for the short-term impact of testbed activities and for the longer-term impact of the NIST semantic testbed.

The panel thinks that the division could increase its dissemination of results through the Manufacturing Extension Program. This program provides the division with a unique opportunity to reach important portions of the supply chain across the country.

In order to maximize its influence, the division needs to issue a clear statement of objectives against 1-year, 2-year, and 5-year time lines. Each program needs to articulate succinct, meaningful, measurable, and significant deliverables within this framework.

Enterprisewide interoperability is a key need, but it is not clear to the panel which constituency will be satisfied by the current research activities in the Manufacturing Enterprise Integration Program. Even after the complete reorganization of this program during the past year, its objectives and scope are not clear to the panel. Because enterprise integration is such a large area, the program should focus on some specific customer’s needs—small manufacturing enterprises, for example—rather than on trying to solve the entire spectrum of needs.

Funding sources for the Manufacturing Systems Integration Division are shown in Table 3.5. As of January 2002, staffing for the division included 30 full-time permanent positions, of which 23 were for technical professionals. There were also 5 nonpermanent or supplemental personnel, such as postdoctoral research associates and temporary or part-time workers.

Division funding is flat, and total funding is projected to be lower in FY 2002 than in FY 2001. Permanent staff has decreased significantly during a time of unprecedented expansion of the manufacturing domain. The majority of division researchers are nonpermanent staff. Both staff members and this panel are concerned that little or no attempt is being made to replace key personnel because of budget pressures. The flat budget and the decrease of the permanent staff are the two biggest concerns of the panel.

While the panel commends the division for leveraging resources by using guest researchers to maximize efficiency and to create an interesting mix of permanent and flux employees, the panel is concerned with maintaining institutional memory and core competencies. Furthermore, the permanent staff is typically the source of future management, and if this group continues to shrink, it will have a long-term negative impact on the division and its institutional memory. The division chief has been extremely adept at using guest scientists, postdoctoral researchers, and other assignees to supplement the permanent staff. Unfortunately, this approach must be considered only an interim step toward establishing a larger, more permanent core team. The most obvious way to accomplish a change in technology focus is to bring in people with the requisite knowledge and skills to complement the existing staff.

Even though guest researchers are indispensable to division success and play key roles in the

division, they unfortunately are not treated as first-class citizens. For example, no mechanism exists for including guest researchers in the bonus program, and they cannot be trained for emerging technologies at NIST expense even though this education furthers NIST’s needs. Since it is anticipated that such nonpermanent but often long-term employees will continue to make up a majority of the workforce, such issues need to be considered by NIST.

Staff is the division’s most important resource; there is clear and evident pride on the part of the scientists, engineers, and administrative people working for NIST. Working for NIST with its high-caliber staff in the objective and open NIST environment contributed to positive staff morale. Division staff members interact with their counterparts in other divisions and enjoy the open exchange of ideas through technical seminars. Knowledge sharing is part of the culture of this division, and there seems to be ample opportunity for staff learning and education.

The panel believes that the division might further enhance staff morale through its recognition of technical accomplishments. Financial rewards might be difficult, but certainly public recognition might further enhance the already-positive morale. Clearly, the technical accomplishments of the group are significant, and being perceived as such by peers and customers would be received positively.

Computer resources seem to be appropriate. Both hardware and software appear adequate and are highly leveraged by division staff.

Division programs are matrix-managed, with a large number of projects and tasks in each program. Because the staff is thinly spread across these projects and tasks, it is highly important that programs be aimed at common and mutually supportive objectives and that the scientists and engineers know their roles in the master plan of the division. In addition, each person must be able to articulate his or her place and contribution to the major goals. These intradivisional interactions need attention. The scientific and support staff have strong faith in the management and will support the leadership. As stated, the division still has too many tasks for the size of its staff, and although the number of projects has been pruned since last year’s assessment, there are still too many. The panel would be happier with a lower ratio of tasks to people to ensure quality and timeliness of results.