3

Robotic Systems for Smart Manufacturing

INTRODUCTION

The mission of the Robotic Systems for Smart Manufacturing (RSSM) program is to

This is achieved through a variety of technical activities, including the following: discussions with stakeholders to assess needs; identifying gaps in standards; researching in relevant hardware to define useful metrics; developing technological best practices; defining test methods and artifacts; disseminating results that contribute to standards development; forming working groups of relevant stakeholders; and leading in standards development. This work is conducted primarily by personnel from the EL Intelligent Systems Division. The scientific staff is organized into five groups: (1) Cognition and Collaboration Systems; (2) Manipulation and Mobility Systems; (3) Networked Control Systems; (4) Production Systems; and (5) Sensing and Perception Systems. The program has a budget of $7 million. There are 16 full-time equivalent (FTE) NIST employees in this area, along with 3 guest researchers and one contractor.

ASSESSMENT OF TECHNICAL PROGRAMS

Accomplishments

The technical activities of the RSSM program seek to address several current challenges experienced by SMEs in their use of robotics. In particular, the vision is for SMEs to have access to robots that are easy to install and integrate into their industrial enterprise; that are reconfigurable, adaptable, and agile; and that can partner with humans to amplify productivity and quality. Technical work is organized around five projects that address these open challenges: (1) Performance Assessment Framework for Robotic Systems; (2) Performance of Collaborative Robotic Systems; (3) Tools for Collaborative Robots within SME Workcells; (4) Agility Performance of Robotic Systems; and (5) Robotic Systems Interoperability and Integration. The RSSM program has acquired state-of-the-art equipment and developed appropriate testbeds for the respective projects. The RSSM vision and projects are well aligned with the current state

___________________

¹ National Institute of Standards and Technology, “Robotic Systems for Smart Manufacturing Program,” updated July 20, 2017, https://www.nist.gov/programs-projects/robotic-systems-smart-manufacturing-program.

of the field, and adeptly address the key challenges faced by SMEs that currently limit the use of robotics in industry.

The RSSM staff has demonstrated good alignment with NIST’s mission through leadership and active participation in standards and metrics developments. The researchers have gathered input and identified needs from stakeholders to formulate project objectives and directions. To engage the broader community and ensure impact, the staff has participated in leadership roles in standards committees, professional organizations, and trade organizations. Results have been disseminated through publications, workshops with professional societies, and inputs to consortia. The staff needs to be particularly commended for using competitions at major robotics conferences (e.g., Intelligent Robots and Systems [IROS] and International Conference on Robotics and Automation [ICRA]) to engage the community and generate inputs and requirements. The five technical projects within the RSSM program have all made notable technical contributions.

Significant accomplishments of the RSSM program involve substantial contributions to important new standards. The Agility Performance of Robotics Systems project has led to the development of a new standard for knowledge representation for robot systems, Institute for Electrical and Electronics Engineering (IEEE) 1872-2015 Standard Ontologies for Robotics and Automation, which has been adopted and used by other organizations.

The Performance of Collaborative Robot Systems project has provided critical contributions to the development of multiple standards for safe operations of collaborative robots, including the first international technical specification for safe operation of collaborative industrial robot systems, ISO Technical Specification 15066 Robot and Robotic Devices—Collaborative Robots.

The RSSM Performance Assessment Framework for Robotic Systems project team founded the Committee on Performance Standards for Industrial Vehicles, and developed the ANSI/ITSDG B56.5 Safety Standard for Driverless, Automatic Guided Industrial Vehicles and Automated Functions of Manned Industrial Vehicles. It also led the standards development through ASTM E57 3D Imaging Systems for image-based test systems, including test methods for six-dimensional pose measurement.

The RSSM staff has also served in a leadership role in several activities that will drive future standards. These include the development of evaluation metrics for robotics, led by the Performance Assessment Framework for Robotic Systems project, which resulted in the formation of the IEEE Grasp Metrics Working Group for grasping metrics. The Agility Performance of Robotic Systems project and the Robotic Systems Interoperability and Integration project created a canonical robot command language and tools that have been adopted by other organizations. The agility project also conceived, developed, and conducted competitions in conjunction with the IEEE Conference on Automation Science and Engineering (CASE) conference to drive future standards on robot agility with participation within the United States and internationally, including both industry and academia. These projects also developed test methods to identify inadequacies in existing tools, such as Robot Operating System (ROS) Industrial, that impede interoperability and agility.

The RSSM staff have also successfully built a set of relevant testbeds for generation of technical input to standards, including the mobile manipulation, collaborative robotics, multifinger grasp, and robot agility testbeds. These testbeds offer the opportunity to explore and evaluate interoperability and integration. These testbeds are critically important for the RSSM research that leads toward standards development.

All of the RSSM projects have actively disseminated their results through publications, data sharing, and invited talks. In particular, the RSSM staff have established a healthy and strong publication record in leading robotics journals and conferences such as IEEE Transactions on Automation Science and Engineering; IEEE Transactions on Systems, Man, and Cybernetics; and others. The Performance Assessment Framework for Robotic Systems project has shared some of its test data with the research community, including its grasp test data for commercial multifinger hands. They also regularly speak at relevant technical venues to discuss their activities in standards development.

The RSSM staff also actively engage with the user community through workshops and other meetings. For example, the team conducted the 2015 Collaborative Robotics Workshop and reached out

to Manufacturing Extension Partnership (MEP) centers to develop a deeper understanding of the key obstacles to robotics adoption, as well as the technology needs of SMEs. They have also actively participated in various consortiums, including the ROS-Industrial Consortium, Robotic Industries Association, and the Advanced Robotics for Manufacturing (ARM) Institute to discuss standards development. They regularly collaborate with other organizations, including universities, companies, and consortiums, to conduct joint tests and leverage results.

Opportunities and Challenges

The robotics technical area is rapidly changing, especially in software and data-driven methods, such as deep learning, and this creates a number of technical challenges in keeping up with the state of the field and the practical impact of the approaches being developed in the RSSM program. However, the RSSM program also has unique opportunities to leverage NIST’s traditional leadership in standards and technology, its in-house expertise, and its broad array of coalitions to forge consensus, break new paths, and deliver lasting impact. One example of an opportunity would be for EL to broaden its industrial impact through stronger interactions with MEP centers and through other industrial consortia. EL can develop a set of use cases and obtain more in-depth feedback on standards development through these deepened interactions. EL can also ensure that the selected projects achieve the broadest impact on industry.

Furthermore, the RSSM staff could lead in the development of a more systematic methodology for generating test and use cases. This methodology could include gap identification, research, developing metrics, developing test methods and artifacts, standard development, evaluation, and dissemination.

The RSSM program can further ensure relevance by working more closely with industry (including newer companies) in testing and evaluation. The RSSM staff could also aim more of their dissemination activities to industry by publishing more articles in trade magazines and by holding workshops in industry tradeshows (e.g., Automate).

The RSSM program has a lack of crosscutting activities that take advantage of synergistic objectives of the individual projects. Integrating the testing and evaluation work across the overall program would provide more synergy across the projects, as well as an opportunity to achieve broader impact of the program work as a whole, rather than only at the project level. The RSSM staff also has potential opportunities to leverage other fast-developing robotics domains, such as driverless cars, for relevant knowledge of standards development in smart manufacturing and industrial automation.

While vision sensors are used in several testbeds, the RSSM program has an unexplored opportunity to investigate how machine vision, and associated algorithms such as deep learning, can be used to enhance the flexibility, user-friendliness, and safety of robots for SMEs. The exploration of how to take advantage of machine vision to simplify robot programming, develop self-teaching techniques, ensure safe human-robot interaction, and adapt to the varied application environments at SME facilities conforms well to the EL objectives for the RSSM program. There is also an increasing utilization of cloud and edge resources for robotics and industrial automation. The Industry 4.0, Microsoft Azure, and Amazon Web Services (AWS) efforts are trying to integrate resources in a consistent and efficient framework, and the RSSM program has the opportunity to contribute to and take a leadership role in this emerging field.

The RSSM has developed many test methods, artifacts, and metrics to guide in the design and use of grippers for dexterous grasping. They could leverage this research to create community-wide test sets that can assist the research community in developing new approaches to dexterous grasping. They could make a contribution to the research community similar to that of the Yale-CMU-Berkeley (YCB) Object and Model Set data set, which helps advance robotic manipulation.

PORTFOLIO OF SCIENTIFIC EXPERTISE

Accomplishments

NIST has a long history in robotics research. The RSSM program has assembled a team of researchers with expertise in computer vision and perception, human-machine interaction, collaborative robots, machining learning, mathematics and algorithms, performance metrics, and testing. Their educational backgrounds include computer science, computer engineering, electrical engineering, and mechanical engineering, as well as engineering sciences and mathematics. Across the team, 52 percent of the research staff hold Ph.D.s, 36 percent hold master’s degrees, and the remainder have B.S. degrees.

The RSSM program’s scientific staff is strong and intellectually diverse, with broad expertise and educational preparation in the many technical areas that are necessary for achieving the program goals. This excellent mix of research interests and professional preparation enables the staff to accomplish the research necessary to support the development, testing, and evaluation of performance standards for robotics in manufacturing.

Several RSSM staff members are well established in their fields, as evidenced by their high number of top-quality peer-reviewed publications; their leadership roles in standards development, professional societies, and conferences; as well as their many invited presentations. They have achieved a high level of productivity comparable to highly performing faculty in top research universities. RSSM researchers are also recipients of several external awards, including the IEEE Standards Association Emerging Technology Award and the ASTM International Robert J. Painter Memorial Award for meritorious service.

Because the RSSM researchers are experts in their respective fields of research and have many collective years of experience in the profession, they understand the state of the art in robotics for manufacturing. They are also active in technical meetings and have established strong connections with relevant professional organizations and collaborators. Members of the staff serve in leadership roles in technical committees (TCs), including the IEEE Robotics and Automation Society (RAS) TC on Performance Evaluation and Benchmarking, and the IEEE RAS TC on Robotic Hands Grasping and Manipulation. The RSSM staff are also active as organizers of professional conferences, workshops, competitions, and summer schools. The RSSM program has established collaborations with a number of universities and companies. Four guest researchers currently on the RSSM staff are from the collaborative partner universities. All of these engagement activities are important for enabling the RSSM staff to understand the needs of the industrial user community and draw on the collective manufacturing robotics expertise across the technical community.

Opportunities and Challenges

Given the strong records of performance of the RSSM researchers, greater recognition for the program (and for NIST) could be achieved through external merit-based robotics awards, early-career investigator awards, elevation in professional societies, as well as invitations to deliver opening and keynote presentations at conferences. NIST researchers are eligible for many prestigious awards, depending on their current career stage. Examples of relevant awards include fellows of professional societies, the Robotics Industry Association (RIA) Engelberger Robotics Award, invitations to Frontiers of Engineering meetings, the Society of Manufacturing Engineers Outstanding Young Manufacturing Engineer Award, the International Symposium for Flexible Automation Young Investigator Award, and the IEEE RAS Early Career Award. This type of external recognition can serve not only to award deserving individuals but also to raise the visibility of EL in the field.

The RSSM program could more systematically leverage (and seek out) expertise from other organizations, including university and industry, and through an extended use of the guest researcher programs. A consortium of industrial companies, both large and small, as well as universities that meet

with the RSSM staff on an annual or semi-annual basis, could serve as additional sources of expertise for needs assessment, gap identification, and the testing and validation of standards. The RSSM program could also leverage regional programs and facilities affiliated with MEPs. In the interoperability area, the RSSM program could coordinate with broader efforts in industrial automation, including Industrial Internet, Industry 4.0, ROS 2.0, and commercial activities such as Microsoft Azure and Amazon AWS.

RSSM researchers publish their work in highly technical journals and professional conferences. While such publications are encouraged, the RSSM could also expand its scientific network and the impact of its work by enhancing translation of its technical work through publication of critical use cases in trade magazines. They could also conduct a road show to discuss important standards-relevant challenges, coordinating with ARM and other organizations such as ROS-Industrial in the roll-out of standards and performance testing and evaluation methods.

Human-machine collaboration is an important challenge in robotics for manufacturing. The RSSM program appears to have an expertise gap in human robot interaction due to lack of permanent staff in this area. It is important to actively recruit researchers in human-robot interaction into the EL permanent staff in order to establish this topic as an active research area.

ADEQUACY OF FACILITIES, EQUIPMENT, AND HUMAN RESOURCES

Accomplishments

The RSSM researchers have several well-equipped laboratories for conducting their research projects. These laboratories include testbeds and facilities for safety validation systems, multirobot collaboration, human-robot interaction, dexterous grasping performance measurement, task-oriented performance measurement, mobile industrial vehicle testing, mobile manipulator performance, agility testing, and sensor/robot calibration and registration. These facilities are well equipped with many state-of-the-art industrial and collaborative robots from top robotics vendors (e.g., Kuka, Rethink Robotics, Universal Robots, ABB Robotics, Fanuc, and Motoman); state-of-the-art versatile grippers and high-degree-of-freedom hands with touch sensors (e.g., Schunk, Robotiq, Allegro Robotics LDT, SoftRobotics, and Empire Robotics); mobile industrial vehicles with manipulation (e.g., America in Motion Automatic Guided Vehicles [AGVs], Adept Lynx); many relevant sensor systems (e.g., monocular and stereo vision and structured light); devices for human-robot interaction (e.g., consumer wearables, and devices for virtual and augmented reality); and high-accuracy ground truth position tracking measurement systems. These facilities are comparable to the best-equipped robotics laboratories in the world, and so provide an excellent environment in which to conduct standards development research in the RSSM.

In summary, the RSSM program has adequate facilities and equipment supportive of the type and requirements of their projects. The laboratory facilities are well equipped with machines, robots, instruments, equipment, and related materials that are necessary to conduct high-quality research. For the type and number of projects pursued, the human resources are also qualified and adequate. These resources appear to be appropriate for the capacity of the assigned space, without observable waste or redundancy.

Opportunities and Challenges

The RSSM program has two main opportunities to better accomplish its objectives through enhanced facilities, equipment, and human resources. The first is to improve the visibility of the program’s activities and output by increasing human resources for media outreach and information dissemination. The ability of RSSM staff to share videos, images, and other information through social media is hindered by the lengthy approval process at NIST and the lack of staff with expertise in social media outreach. As a result, the RSSM has very little visibility on social media, which is a missed opportunity to improve the

impact of the program’s activities. The RSSM needs to consider adding staff who can improve dissemination of technical accomplishments through social media, trade publications, and/or popular press. While the RSSM program does have some presence in scientific and trade associations, such as the IEEE and the American Society of Mechanical Engineers (ASME), it could increase its visibility in trade publications and popular press that are read by industry practitioners. For example, increasing media coverage of the limitations of current standards for robotic safety and agility and the work the EL has done to address these limitations would help disseminate the work of the EL to more practitioners.

The second area of opportunity is to arrange the equipment and facilities to allow for closer integration of activities across the program. While research facilities are adequate, they are distributed across the NIST campus and are physically disconnected across multiple buildings on the campus. The lack of crosscutting activities across all the projects of the RSSM program is not helped by the distance between laboratory spaces. Given that the RSSM research projects share many common technologies, science, and resource needs, and can benefit from knowledge exchange, these separations between the project teams might contribute to less cohesiveness and overall impact than might be possible through co-located facilities.

Several of the testbeds developed for individual projects have overlapping capabilities, but cannot currently be integrated because they do not have interoperable software. There is an opportunity to develop software infrastructure resources to facilitate the interoperability of testbeds developed in individual projects.