National Academies Press: OpenBook

Intelligent Human-Machine Collaboration: Summary of a Workshop (2012)

Chapter: 4 Common Challenges and Breakthroughs

« Previous: 3 Human-Machine Teamwork Panels
Suggested Citation:"4 Common Challenges and Breakthroughs." National Research Council. 2012. Intelligent Human-Machine Collaboration: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/13479.
×

4

Common Challenges and Breakthroughs

After the scenario and panel discussions, workshop participants discussed common challenges in IH-MC, as well as breakthroughs to transform the ways in which humans and machines will collaborate in the future. Some of these breakthroughs, discussed below, were also touched upon during the scenario exercises.

Rethinking Roles for Humans and Machines

For some participants, achieving the desired breakthroughs begins with a reevaluation of the role humans play in IH-MC. According to Kruijff, this begins with recognizing that humans have a central role. Rather than focusing on the human as a part of the problem or only as a partial solution, he proposed that collaboration should be designed with and around humans and take into consideration the broader sociotechnological context.

From the machine side, Sidner observed that current robots have limited sensing, manipulation, and communication capabilities. However, as machine capabilities improve, machines will be able to play a larger and more complex role in IH-MC. For some observers, this could result not only in new roles for machines in human-machine collaboration, but also a shift in humanmachine team dynamics. Beyond humans leading teams of machines, Hoffman hypothesized that one day computational devices could serve as mentors or trainers for human (and/or human-robot) teams. If that were to happen, he asked, how might humans be trained to work with machine partners or even machine mentors? Understanding these questions, according to Liz Sonenberg, will benefit from a better understanding of what makes a human a good member/leader of a human-machine or all-machine team.

Jeff Bradshaw suggested that intelligent human-machine systems may also have a role in virtual and real-world training activities. For example, by designing experiments that require individuals to adapt in a changing environment, both on the field and in the laboratory, researchers could study individual and team dynamics. There was a debate over how closely human-machine training could replicate human team training (given the elaborate physical, cognitive, and communications skills that humans bring to bear). The efficiency of proficiency

Suggested Citation:"4 Common Challenges and Breakthroughs." National Research Council. 2012. Intelligent Human-Machine Collaboration: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/13479.
×

scaling through these types of training was seen by some as likely to be domain dependent.

During these discussions, participants highlighted several common research areas necessary to advance IH-MC, such as communication, flexibility and resilience, human-machine models, user experience and system design, testbeds, and data overload.

Communication

For many participants, effective communication represents a significant barrier to advances in human-machine collaboration. According to Kruijff, it would be useful if robots could better explain to humans what they can or cannot do and what they actually do. An inability to effectively communicate this, he added, makes it difficult for humans and machines to gain common ground. In addition to verbal communication, Sidner noted the challenges posed by nonverbal behavior—for example, how might a robot “notice” what a human notices?

Other participants commented on the benefit of further research in how humans and machines communicate and understand intent. Bradshaw emphasized that by improving machine observability or “apparency,” humans will be better able to understand a device’s intent. In contrast, Veloso observed that cases may exist in which a human does not necessarily need to monitor or understand what the robot is doing, as long as he or she trusts the robot to proactively ask for help when necessary. Oron-Gilad cautioned that effectively conveying intent between two human operators, let alone between humans and robots, is still a challenge. For example, if a software agent incorrectly “guesses” a human’s intent, it might unnecessarily automate a task—thus leading to dangerous and unintended consequences.

Padgham proposed the development of a “teaming compact” whereby humans and machines mutually communicate their capabilities, goals, and intentions. Perhaps what is required, she said, is one common and simple language that can be used by any system. Also necessary, Matthias Scheutz added, are feedback mechanisms and intelligent and tangible interfaces between humans and agents. Other participants commented that this feedback should be dynamic so that human-machine collaboration can change over time—for example, as a result of training or changes in familiarity or trust.

This prompted a discussion on whether new ways for robots to communicate with one another could reduce the number of humans in human-robot teams. In response, one participant suggested that challenges of effective robot-to-robot communication would be made simpler by removing the cultural baggage of human communication.

Suggested Citation:"4 Common Challenges and Breakthroughs." National Research Council. 2012. Intelligent Human-Machine Collaboration: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/13479.
×

Flexibility and Resilience

Some participants commented that improved human-machine collaboration will require improved flexibility and resilience. For example, Hoffman observed that human-machine interfaces would be improved by engineering for resilience, that is, designed for unanticipated tasks. Such flexibility is particularly important, Jean Scholtz added, as the tasks people do today will not be identical to those being done tomorrow or in five or ten years. Humans can rapidly adapt and apply their capabilities to new situations, so how can this flexibility and learning be applied to robots without significant programming?

Frank Dignum suggested that lessons may be learned from human adaptability—for example, the adaptation of human language to widespread adoption of text messaging. Rather than wait for convergence in, for example, natural language between humans and robots, he proposed a deeper examination into situations in which humans but not robots are able to adapt.

This resilience, Neerincx noted, will require breakthroughs in context-driven adaptive autonomy. Both Hoffman and Sidner commented that such high levels of complex autonomy would first depend on significant breakthroughs in commonsense knowledge and practical manipulation tasks.

Modeling

Another common challenge discussed was the potential benefit of improved human, machine, and shared human-machine models. Goodrich spoke to the difficulties of developing such shared models by describing human and machine dynamic asymmetries in experience, understanding, goals, and capabilities.

Although some participants emphasized the need to provide robots with better models of humans, Scheutz noted the challenges of building correct models of robots for humans. Human models of robots, he said, need to be compatible with the ways humans will interact with them. For example, if a robot does not have good natural language or good visual sensing capabilities, perhaps anthropomorphized robot mouths or eyes will mislead humans to overestimate the robot’s capabilities. While highly realistic Geminoid robots exist, Holly Yanco added that the “uncanny valley” factor should also be taken into consideration.

According to Dignum, new social reality models may allow machines to do things “with” humans, and not just “for” humans in a limited role. Many of the workshop participants remarked that it would help for humans to develop social understanding and acceptance of such sophisticated machine capabilities in order for this level of collaboration to occur.

Suggested Citation:"4 Common Challenges and Breakthroughs." National Research Council. 2012. Intelligent Human-Machine Collaboration: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/13479.
×

Testbeds and Fielded Systems

Several participants suggested that more and improved large-scale dynamic testbeds (as well as their ongoing evaluation) would benefit many of the previously discussed research issues. Going beyond testbeds, Kruijff observed that some of the challenges the group discussed would best be studied using deployed or fielded systems. For example, some challenges, such as philosophical linguistics issues, are more likely to arise in the field as opposed to the laboratory. True collaboration under stressful circumstances, he noted, cannot be replicated in the laboratory.

“Big Data”

Challenges posed by “data overload” were also highlighted, in the context of improving both human-machine interaction and teamwork, as well as the value of using “big data” to solve large-scale problems.

Some suggested that new ways for teams to share concrete and dynamic information about their environment could provide novel perspectives that lead to interesting and new solutions. For Satoshi Tadokoro, this is particularly relevant in the context of supporting teams composed of one human and multiple robots. This type of coordination, he observed, requires significant amounts of and access to data. In the rescue domain, this means data about human-robot coordination and interaction, as well as the environment.

Morison proposed that significant opportunities exist for breakthroughs in the ways that large-scale robot/sensor data are used to expand the ways humans perceive the world. Systems could be designed, he said, for effective exploration so that relevant information can be quickly extracted.

Shared Resources for Shared Problems

Using IH-MC to solve highly complex problems, Hoffman noted, requires big research budgets, often in harsh economic climates. One path forward, he proposed, might be to choose a single problem large enough to require international funding efforts.

Padgham acknowledged that many fields associated with humanmachine collaboration have not been as successful as others in disciplining themselves to combine resources in pursuit of solving larger-scale challenges. In

Suggested Citation:"4 Common Challenges and Breakthroughs." National Research Council. 2012. Intelligent Human-Machine Collaboration: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/13479.
×

part, this is because long-term funding to support such initiatives has not been as available in this field as it has been in others. As an example of a successful initiative, Sidner posited that the success of the physics research communities in effectively combining resources has, to some degree, been a result of 400 years of maturation within a set of unified fields. Perhaps, Padgham suggested, IHMC efforts to manage a small disaster would be appropriate for international funding and combined large-scale research efforts.

Suggested Citation:"4 Common Challenges and Breakthroughs." National Research Council. 2012. Intelligent Human-Machine Collaboration: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/13479.
×

This page intentionally left blank.

Suggested Citation:"4 Common Challenges and Breakthroughs." National Research Council. 2012. Intelligent Human-Machine Collaboration: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/13479.
×
Page 17
Suggested Citation:"4 Common Challenges and Breakthroughs." National Research Council. 2012. Intelligent Human-Machine Collaboration: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/13479.
×
Page 18
Suggested Citation:"4 Common Challenges and Breakthroughs." National Research Council. 2012. Intelligent Human-Machine Collaboration: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/13479.
×
Page 19
Suggested Citation:"4 Common Challenges and Breakthroughs." National Research Council. 2012. Intelligent Human-Machine Collaboration: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/13479.
×
Page 20
Suggested Citation:"4 Common Challenges and Breakthroughs." National Research Council. 2012. Intelligent Human-Machine Collaboration: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/13479.
×
Page 21
Suggested Citation:"4 Common Challenges and Breakthroughs." National Research Council. 2012. Intelligent Human-Machine Collaboration: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/13479.
×
Page 22
Next: 5 Global and Transnational Issues »
Intelligent Human-Machine Collaboration: Summary of a Workshop Get This Book
×
Buy Paperback | $29.00 Buy Ebook | $23.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

On June 12-14, 2012, the Board on Global Science and Technology held an international, multidisciplinary workshop in Washington, D.C., to explore the challenges and advances in intelligent human-machine collaboration (IH-MC), particularly as it applies to unstructured environments. This workshop convened researchers from a range of science and engineering disciplines, including robotics, human-robot and human-machine interaction, software agents and multi-agentsystems, cognitive sciences, and human-machine teamwork. Participants were drawn from research organizations in Australia, China, Germany, Israel, Italy, Japan, the Netherlands, the United Arab Emirates, the United Kingdom, and the United States.

The first day of the workshop participants worked to determine how advances in IH-MC over the next two to three years could be applied solving a variety of different real-world scenarios in dynamic unstructured environments, ranging from managing a natural disaster to improving small-lot agile manufacturing. On the second day of the workshop, participants organized into small groups for a deeper exploration of research topics that had arisen, discussion of common challenges, hoped-for breakthroughs, and the national, transnational, and global context in which this research occurs. Day three of the workshop consisted of small groups focusing on longer term research deliverables, as well as identifying challenges and opportunities from different disciplinary and cultural perspectives. In addition, ten participants gave presentations on their research, ranging from human-robot communication, to disaster response robots, to human-in-the-loop control of robot systems.

Intelligent Human-Machine Collaboration: Summary of a Workshop describes in detail the discussions and happenings of the three day workshop.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!