An Assessment of the National Institute of Standards and Technology Programs Fiscal Year 1994 (1994)

Stuart G. Miller, General Electric Corporate Research and Development (retired), Chair

Jan D. Achenbach, Northwestern University

Robert Bate, Texas Instruments, Inc. (retired)

Jean M. Bennett, Naval Weapons Division

D. Jeffrey Bostock, Martin Marietta Energy Systems, Inc.

Lee Boswell, Hewlett-Packard Company

John F. Cassidy, United Technologies Research Center

Melvin I. Cohen, AT&T Bell Laboratories

Johnson A. Edosomwan, Johnson & Johnson Associates, Inc.

Albert R. George, Cornell University

Frederick Hayes-Roth, Cimflex Teknowledge Corporation

E. Ray McClure, Moore Special Tool Co., Inc.

Roger N. Nagel, Lehigh University

Richard P. Paul, University of Pennsylvania

Invited Participant

Sushil Birla, General Motors Corporation

Submitted for the panel by its Chair, Stuart G. Miller, this assessment of the fiscal year 1994 activities of the Manufacturing Engineering Laboratory is based on a site visit of the panel on March 15-17, 1994, and on the annual report of the laboratory.

The mission of the Manufacturing Engineering Laboratory (MEL) as presented to the panel by its new director is to improve the competitiveness of U.S. manufacturing by working with industry to develop and apply technology, measurements, and standards.

The Manufacturing Engineering Laboratory is in a state of transition as new management repositions and realigns the organization with new industrial customers and their expectations. This represents a profound cultural change and an opportunity for MEL, made possible by increased NIST funding. This opportunity for MEL to play a critical role in U.S. competitiveness necessitates a well-orchestrated strategic plan that matches MEL's capabilities to industrial needs. The process of developing such a plan is endorsed by the laboratory director and was ongoing at the time of this assessment.

MEL's fiscal year 1993 funding included $11.8 million from NIST Scientific and Technical Research and Services (STRS) funding, $17.7 million from other (federal) agency (OA) funding sources, and $7.9 million from other funds such as the Advanced Technology Program, calibration services, standard reference materials, reimbursable services, and NIST overhead support. Fiscal year 1994 funding at the time of the assessment was estimated at $19.3 million from STRS, $14.2 million from OA, and $8.0 million from other sources. The laboratory has 259 full-time permanent employees, with 145 technical professionals and 80 other technical staff.

The current laboratory-level strategic plan is too broad and global in its statements. A more specific plan is necessary, with program justification and charting of milestones and deliverables. The laboratory director recognizes the need for a revised strategy, and the panel recognizes the difficulty of achieving a rapid cultural change.

The laboratory's strengths are intelligent machines, control augmented processing, information integration, and precision machining. The laboratory also has the complementary skills necessary to develop precision instruments, integrate equipment with software systems (either information systems or real-time control systems), and achieve these goals in an environment mindful of standards ramifications. This combination of skills and knowledge is unique. The organization, however, has been constrained in pursuing targeted developments because of overdependence on external funding. In fiscal year 1994, external funding represents 76 percent of the MEL budget. Although collaborations with other government agencies and

select industry partners can provide insight into customer need, these externally funded projects have driven MEL's direction, rather than an MEL mission, leaving MEL as executor but not owner of its projects.

Each division shows some appreciation of and sensitivity to its customers. The Precision Engineering Division shows the most appropriate customer-driven responsiveness. The panel applauds the division's efforts to give its customers “working standards.” Avoiding the implied danger of being tainted by “impure” standards practices or compromised foundations seems to require little more than ordinary prudence.

Both the Robot Systems Division and the Factory Automation Systems Division have impressive teams and high-quality visions. Nevertheless, they both seem to suffer from inadequate focusing of resources on programs that are likely to produce incremental, compelling wins. On the other hand, these two divisions' conceptual foundations and their grasp of the problems being faced are most impressive. They should plan “guaranteed wins” and then supplement these efforts with higher-risk, longer-range activities. They must bring their visions to a usable level of maturity earlier to produce value-adding implementations and put these into the hands of many potential users and producers. If something is worth doing really well in 5 years, it is worth doing partially and effectively in 12 to 18 months.

Resources are spread thinly across the current number of activities. The panel supports the leveraging of MEL resources by using non-NIST capabilities in the fulfillment of NIST's mission, an example of which is the cooperative program with the Department of Energy (DOE) Gear Metrology Center at the Y-12 Plant in Oak Ridge, Tennessee. In the changing environment, there is a growing need for formal, visible project management with attendant milestones and project control. Several major projects are either at or nearing completion, creating a sense of urgency in planning the laboratory's new direction. The most important of these projects are the Automated Manufacturing Research Facility and the Office of Industrial Relations. In planning follow-on efforts for these projects, as in all project planning, MEL must ascertain industrial need for the projects before proceeding and carry out the projects in a way that facilitates industrial adoption of results.

Detailed assessments of the Manufacturing Engineering Laboratory's technical programs are found in the divisional assessments below, but special attention is called here to the molecular measuring machine development. This measurement capability is of significant importance to the semiconductor industry, but the machine has been in development since 1988, with new problems emerging in each development step. The panel recommends a mid-year 1994 design review to affirm the appropriateness of the development process and preempt future development problems so as to bring the project to rapid, successful completion.

The following are the panel's recommendations for MEL as a whole.

• In planning its programs, MEL must seek to match industry needs with the laboratory's capabilities, keeping in mind the magnitude and immediacy of those needs. MEL must also assure that the solutions it proposes to such needs match industry's ability to utilize them.

• MEL should better identify and couple with its customer base to determine the proper balance between research and development and response to immediate industrial needs.

• To be successful in the long run, the laboratory needs to understand how its own capability ranks against similar capabilities found in other government laboratories, in universities, within industry, or internationally. This can be achieved through benchmarking and applying total quality management to the activities of the laboratory.

• To enhance the effectiveness and impact of this annual assessment, the panel recommends that MEL respond formally to the panel's fiscal year 1994 recommendations at the time of the fiscal year 1995 assessment.

The mission of the Precision Engineering Division is to help meet the needs of U.S. manufacturing in the area of length-based dimensional metrology by working with industry to develop and apply technology, measurements, and standards.

The Precision Engineering Division's general strategy is to develop the length-based dimensional metrology needed by U.S. manufacturing industries to support near-term (1 to 5 years) and long-range (>5 years) needs. A new strategy is being developed to leverage NIST resources by using NIST personnel in collaboration with non-NIST facilities and personnel to achieve NIST-traceable standards. Three different projects have been implemented. The new NIST-DOE Gear Metrology Center will provide NIST-traceable and NIST-calibrated gear elements, master gears, and first articles of types and accuracies specified by industry. A second effort involves use of a Y-12 M60 coordinate measuring machine (CMM) by NIST personnel to calibrate high-accuracy CMM step gauges. This service will be superior to that of the best foreign laboratory, Germany's national laboratory, the Physicalische Technische Bundesanshalt. A third project involves joint industry-NIST certification of reference artifacts for advanced microelectronics manufacturing processes.

The Precision Enginering Division's 1993 funding totaled $3.6 million from STRS, $3.5 million from OA, and $1.5 million from other sources. Fiscal year 1994 funding is estimated at the time of the assessment to be $4.3 million from STRS, $3.0 million from OA, and $2.1 million from other sources. The division has 57 total staff, of whom 30 are technical professionals and 17 are other technical personnel.

The leveraged strategy approach is an innovative solution to increasing the responsiveness of NIST to its customers. The Precision Engineering Division has done an excellent job of identifying its customers and its customers' needs. The strategy currently being implemented to accomplish the division's mission is on target.

The Precision Engineering Division had many noteworthy accomplishments during fiscal year 1993; only highlights are given here.

During fiscal year 1993, problems of excessive vibration surfaced in the molecular measuring machine. These have been identified and corrected. All systems were checked in air and parts disassembled and cleaned for ultrahigh-vacuum operation. Unfortunately, some of the optical components came apart during cleaning and are now being reassembled and realigned. Also, the fine motion carriages were found to distort out of the vertical range of the instrument pickup. This problem has been corrected, and at the time of the assessment the instrument was expected to be fully operational in ultrahigh vacuum by summer 1994.

The project to produce a calibration atomic force microscope is well under way. Commercially available components are being incorporated into a novel nanometrology system. Subsystem performance in the 1-to 2-nm range has been achieved. This instrument addresses the long-range need for calibration standards accurate to subatomic dimensions.

A new, inexpensive method has been developed for stabilizing the green line of a heliumneon laser, and a prototype interferometer has been built that can use any visible laser line for fringe counting. These accomplishments will make high-accuracy length calibrations available to many more customers than previously.

Studies of the nonideal wavefronts of commercial phase-measuring interferometers have resulted in a model to correct for systematic errors. This work will directly benefit the many optical companies that routinely use these instruments.

Ductile-regime grinding has been demonstrated in aluminum-titanium carbide using resinbonded wheels and in silicon nitride using metal-bonded wheels. The NIST-developed electrically assisted grinding process to erode the iron bond and expose fresh diamonds while grinding was also demonstrated. A theory was developed to explain for the first time why certain materials can or cannot be successfully machined by single-point diamond turning. These accomplishments

extend the types of materials that can be shaped to final form using precision grinding or single-point diamond turning.

An image analysis software package has been developed for interlaboratory scanning electron microscopy (SEM) studies and calibrations of x-ray masks. Also, work on new, low-accelerating voltage SEM magnification calibration standards has been completed. Linewidth work addresses the needs of the microelectronics industry for high-accuracy linewidth measurement for future x-ray masks. A new technique has been proposed for measuring features on photomasks that circumvents the uncertainty in modeling geometrical line shapes and is more closely related to customer needs.

A calibration procedure has been developed to measure the shape of Rockwell microhardness indenters to less than one-tenth the measurement uncertainty required by present American Society for Testing and Materials and International Organization for Standardization standards. The shape accuracy achieved at NIST is superior to that of any other international calibration laboratory.

In spite of the excellent productivity of the Microelectronics Dimensional Metrology Group during the past year, the group is greatly below critical mass with only three scientists and one part-time guest worker. Since the work of this group is vital for the development of linewidth standards for the microelectronics industry, every effort should be made to ease this staffing shortfall so that important projects currently on hold can proceed expeditiously.

The following are the panel's recommendations for the Precision Engineering Division.

• NIST-leveraged activities should proceed as planned; however, contingency plans should be developed in case some of the industry partners are unable to participate in the program.

• The status of the molecular measuring machine should be assessed to determine how the machine optimization, calibration, and measurement projects should proceed.

• Since the work of the Microelectronics Dimensional Metrology Group is vital for the development of linewidth standards for the microelectronics industry, every effort should be made to ease the staffing shortfall so that important projects currently on hold can proceed expeditiously.

The Automated Production Technology Division (APTD) develops and maintains competence in the integration of machine tools and robots; develops the interfaces and networks necessary to combine robots and machines into workstations and/or manufacturing cells; develops and maintains computer-assisted techniques required for the generation of the computer codes necessary for the integration of machine tools and robots; performs research and integration tests

necessary for in-process monitoring and gauging; and maintains competence in engineering measurements and sensors required by the manufacturing industries, including mass, vibration, acoustics, force, and ultrasonics.

The APTD has in recent years proceeded according to what is essentially a two-pronged plan. It has developed a basic, generic technology based on its mission to provide and support standards for mass, force, vibration, ultrasonics, and acoustics. On the platform of these basic technologies, a special capability for the development and use of sensors involving both sensing and signal processing is continuously evolved. The other prong consists of applications projects based on sensor technology with specific product or process goals organized around a theme that, until recently, was labeled quality in automation (QIA). QIA is a logical organization of means to suppress the effects of flaws on the performance of a system by separately treating repeatable and nonrepeatable flaws and detecting and compensating for overall systems performance defects. The QIA system works by means of a network of sensors and actuators integrated into a computer control system for which process models and associated software are developed as needed.

Fiscal year 1993 funding for APTD included $2.1 million from STRS, $1.8 million from OA, and $1.4 million from other sources. Fiscal year 1994 funding is estimated at the time of the assessment to be $3.0 million from STRS, $1.8 million from OA, and $1.3 million from other sources. The division has 49 staff members, of whom 30 are technical professionals and 11 are other technical workers.

The panel again observes, as in previous assessments, that the APTD 's general strategy is not clearly represented by a written plan that incorporates specific action items with associated schedules and implementation methodology and is subsequently used to guide project execution.

The division management has been able to acquire the industrial involvement that is dictated by recent shifts in government priorities and that is generally necessary to take technology to the marketplace. Cooperative Research and Development Agreements and consortia of commercial organizations have been successfully initiated, indicating promise for more joint NIST-industry efforts in the future.

The panel found the turnaround time for applications projects too long. In particular, the 14 months required to implement the Giddings and Lewis project for precision turning of pistons was not timely.

Cross-institutional coupling has shifted APTD's work from purely speculative development efforts to more applications efforts with greater concern for cost-effectiveness. The panel approves this shift. The enhanced machine tool controller is a good example. This project emphasizes integration of off-the-shelf products to enable upgrading of older, inflexible controllers so that they become candidates for QIA strategies, thereby facilitating improvements in the overall performance of older equipment. To enhance its visibility and increase its effectiveness, the division should be certain to claim appropriate credit for its contributions to projects such as this and the high-speed actuator project.

The panel is concerned that, although the expressed intent of the division's work is to improve “manufacturing,” the improvements are being realized in the narrower field of “machining.” Expansion of applications of the basic, generic technologies developed under QIA has not been undertaken.

The following are the panel's recommendations for the Automated Production Technology Division.

• The Automated Production Technology Division's work should be planned and executed according to commonly used project management methodologies.

• The division should network with a broader customer base than the machining and machining technology industry in order to identify other applications for the generic technologies developed under QIA and its follow-on efforts.

• The division should take the initiative in diffusing applications of the technologies in which it leads. An example is the use of digital computers for compensation of repeatable errors of length, displacement, and geometry in mechanisms; the division should host a workshop to establish a more uniform utilization of error correction in machine design by U.S. industry.

The Robot Systems Division (RSD) develops and maintains competence in intelligent machines and systems, robotics, real-time sensory-interactive control, reference model architectures, software development techniques, and methods for intelligent system design and testing; conducts research in new concepts and techniques for planning and control, world modeling, sensory processing, and interactive graphics for operator interfaces; develops methods

for software development and testing; applies artificial intelligence techniques to real-time sensory-interactive control for computer-integrated manufacturing systems and other applications with high national priority, such as construction, military, and environmental cleanup; develops measures of performance for robots and intelligent systems; and performs research and technical activities relating to standards for intelligent machine systems.

The Robot Systems Division seeks to carry out its mission to meet the goal of improved competitiveness of U.S. industry by working with industry, academia, and other agencies to develop and apply intelligent systems technologies. Its approach to meeting this goal is to identify customers, study customer needs, establish strategic alliances, develop project plans, perform research and development, and establish consortia of commercial suppliers.

The division presented the panel with projects plans that include long-term goals, approach, background, and yearly goals.

Fiscal year 1993 funding for the Robot Systems Division included $3.0 million from STRS, $2.6 million from OA, and $0.2 million from other sources. Fiscal year 1994 funding is estimated at the time of the assessment to be $4.4 million from STRS, $3.2 million from OA, and $0.03 million from other sources. The division has a total of 52 staff members, of whom 38 are technical professionals and 3 are other technical staff.

As is the case throughout MEL, RSD's planning document is too broad and general. To be effective, a strategic plan must be more specific; for example, yearly goals do not focus effort as do quarterly milestones and deliverables.

The RSD is well organized for its competence in intelligent machines and systems, robotics, real-time sensory-interactive control, and reference model architectures. The research application projects currently under way, however, are too numerous and diverse (spanning such areas as unmanned vehicles and large-scale robots), spreading RSD's resources too thinly. Research application projects should focus more on manufacturing. As OA-funded projects terminate and internal funding opportunities become available, the division should move toward manufacturing applications.

The panel endorses RSD's plan to use NIST funding to bring in student research associates. This plan would both alleviate shortfalls in personnel and increase the interaction between NIST and academia, contributing to student training and perhaps transferring NIST technology to industry as former students take industrial positions.

RSD's work on an open architecture for a real-time controller is appropriate. The various controller-related projects such as the shop floor controller, the Next Generation Inspection System (NGIS) controller, and the laboratory development controller should be coordinated in order to evolve a common, platform-independent open architecture with a well-defined migration path. A strong chief architect should be appointed. These projects should also be coordinated more closely with various open architecture projects worldwide. The staff should participate in and organize workshops to evolve and define a unified open architecture controller standard.

The following are the panel's recommendations for the Robot Systems Division.

• The Robot Systems Division should coordinate the various controller-related projects such as the shop floor controller, the NGIS controller, and the laboratory development controller to evolve a common, open, platform-independent architecture with a well-defined migration path. The division should appoint a strong chief architect.

• The division should coordinate more closely with various open architecture projects worldwide and participate in and organize workshops to evolve and define a unified open architecture controller standard.

• The division should focus on manufacturing processing applications, including physical transformation processes and their associated inspection and material handling.

• The division should pursue collaborations with universities by inviting student research associates as funding becomes available and consider inviting faculty and industry researchers. NIST staff should have opportunities for industrial sabbaticals as additional means of encouraging technology exchange and collaboration.

The Factory Automation Systems Division (FASD) provides a focus for research and development of standards and technologies related to systems integration for manufacturing; emphasizes the use of information technology for manufacturing applications, including the implementation of a proposed research “testbed” required for measuring the ability of engineering systems to perform in conformance with product and process interface standards; develops and maintains competence in product data exchange, flexible computer-integrated manufacturing, enterprise integration, manufacturing communications, and manufacturing database systems; and develops methods for the software development and testing of manufacturing systems and data interfaces.

The FASD is driven by a vision of the future manufacturing environment as a “virtual enterprise” in which independent enterprises are integrated by an information network into an effective system. Within each of these enterprises, the various product-related functions and product life-cycle stages are integrated through the sharing of product and process data, although each stage maintains its own view of the product. This vision is built on the concept of concurrent engineering and the conclusions of the National Critical Technologies Panel, the Agile Manufacturing Forum, and the Advanced Manufacturing Initiative of the former Federal Coordinating Committee for Science, Engineering, and Technology.

The goal of the division is to assist industry in producing the technology and standards required for U.S. manufacturing in the twenty-first century. The division will make use of a proposed facility known as the Advanced Manufacturing Systems and Networking Testbed. The purposes of the facility are the following: perform research and development in the areas of advanced manufacturing systems and networking; test and develop computer software, manufacturing systems, and systems integration necessary for successful operation within the U.S. manufacturing community; assist industry in implementing voluntary consensus standards, including standards for networking, electronic data interchange, and digital product data sharing; make high-performance computing and networking technologies an integral part of design and production of products; conduct research to identify and overcome technical barriers to the successful and cost-effective operation of advanced manufacturing systems and networks; facilitate industry efforts to develop and test new applications of advanced manufacturing systems and networks; involve companies that both develop manufacturing systems and use those systems; provide training for industry on the effective use of these new technologies; work with private industry to develop standards for the use of advanced computer-based training systems, including multimedia and interactive learning technologies, that will accelerate the efficient use of advanced manufacturing systems; and provide mechanisms for the exchange of information about advanced manufacturing systems and networking.

The division presented the panel with a strategic plan document with yearly goals.

Fiscal year 1993 funding for FASD totaled $2.2 million from STRS, $3.9 million from OA, and $0.9 million from other sources. Fiscal year 1994 funding is estimated at the time of the assessment to be $5.4 million from STRS, $3.8 million from OA, and $0.8 million from other sources. The division has 62 staff members, of whom 45 are technical professionals and 3 other technical staff.

As in other MEL planning documents, the Factory Automation Systems Division's strategic plan is too broad and general, discussing goals appropriate for an entire industry but not for a 62-person division. The strategic plan should consist of specific programs that have been carefully selected for focus, with specific quarterly milestones and deliverables for each project. There is little evidence that the current document guides day-to-day division operations; a more specific plan could be an effective project management tool, giving the staff focus and allowing it to make daily decisions based on specific, mission-oriented goals.

The Factory Automation Systems Division is doing a good job of developing a vision of the virtual enterprise and other developments needing its technology. The vision and strategy should show the FASD strategic impact in the larger picture of the changing nature of enterprises, not just in manufacturing.

Project ideas evolving from the recent NIST Integration Workshop should receive serious consideration. Future industry input on system integration, however, should be more focused, originate from higher-level industry executives, and result in a mutual set of short- and long-term objectives.

The division's current efforts to reorganize and change its name are appropriate, given MEL's reorganization. The name and structure chosen should enhance the division's visibility and apparent value to industry.

Acquiring responsibility for the systems integration for manufacturing applications (SIMA) portion of the federal High Performance Computing and Communication initiative is a major accomplishment that FASD should be proud of. A well-defined and documented understanding of the problem for which SIMA is the solution would be an important step in making the significance and challenge of SIMA clear to potential industry beneficiaries, partners, and implementers. A single point of responsibility for the architecture of SIMA would significantly increase the probability of success and aid in collaboration and cooperation in meeting the SIMA challenge. The role of FASD as coordinator of SIMA is not clear. How does industry participate in goal setting, realistic testing, and so on? What criteria will be used to measure Factory Automation Systems Division success in SIMA inside MEL, within NIST, in other parts of government, and in industry? A clear goal with measurable objectives is needed.

The division's projects reflect NIST's significant growth and have the growing respect of customers; however, they seem to be more theoretical than practical. Information and technology are used by industry to impact the bottom line, and the division needs a greater awareness of this. Project planning should document specific industry applications for FASD's generic solutions as proof of project relevance and of the significance of the underlying concepts. In particular, FASD staff members need to be more cognizant of current industry problems, focusing significant energy to accelerate the definition, development, implementation, and acceptance of innovations.

The following are the panel's recommendations for the Factory Automation Systems Division.

• The Factory Automation Systems Division must clearly define its role and demonstrate its potential impact within the context of the changing nature of manufacturing, e.g., the globalization of business, integration of processes, and so on.

• The division should consider forming an industry-led group similar to that participating in the NIST Integration Workshop. Measurable objectives with specifically identified impacts on industrial competitiveness should result from such an interaction.

• Decisions on a new division name or structure should not be made too quickly but should be part of the overall MEL reorganization and strategy selection.

• A chief architect should be appointed for SIMA. This chief architect must be certain that the problem for which SIMA is the solution is well defined and documented so that the significance and challenge of SIMA is clear to potential industry beneficiaries, partners, and implementers.

• The Factory Automation Systems Division should give more attention to how its information technology impacts industry's bottom line.

The Fabrication Technology Division is a basic service division, providing machine shop services to MEL and other NIST laboratories. The true value of this organization is the one-on-one coupling between in-house craftsmen and the research scientist. The value of this relationship both now and in the future is a NIST management issue. The panel points out, however, that precision drives all NIST activities, including fabrication of experimental apparatus and precision instruments. With a perpetual need for quality craftsmen, the issue then is the competitiveness of in-house shops with external contractors and the level of skills that should be possessed by personnel retained on the NIST payroll.

The Office of Industrial Relations was established as a temporary office to organize the National Initiative for Product Data Exchange. It is now nearing the end of its charter, and NIST needs to plan an orderly transition of this work to a new, permanent organization.

The function of the Office of Manufacturing Programs is to manage cross-division OA research. Its major funding, through the Navy Automated Manufacturing Research Facility Program, is now winding down after some 12 years of sponsorship. The panel views the program as a success, with some 33 commercial products, 18 patents, and 39 standards as key accomplishments. The panel believes that it is now time to redefine and clarify the mission of the office and to determine through industry consensus whether a follow-on effort is needed and whether it should take the form proposed in MEL's Advanced Manufacturing Systems and Networking Testbed.