National Academies Press: OpenBook

Intelligent Human-Machine Collaboration: Summary of a Workshop (2012)

Chapter: 6 Revisiting the Scenarios

« Previous: 5 Global and Transnational Issues
Suggested Citation:"6 Revisiting the Scenarios." National Research Council. 2012. Intelligent Human-Machine Collaboration: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/13479.
×

6

Revisiting the Scenarios

On day three of the workshop, participants were placed into small groups and given the opportunity to revisit previous scenarios for a second round of analysis. Loosely following the DARPA Grand Challenge competition, each group developed a 10-year research proposal on a topic of their own choosing, using two of the earlier scenario discussions as a starting point: Hospital Service Robotics and Preparing For and Managing a Major Disaster. In addition to receiving an unlimited research budget, each group was obligated to rely on the actual expertise of its members. For example, if a group did not possess a natural language expert, its delivered system could not employ sophisticated or innovative natural language.

Group 1: Disaster Management System for a Collapsed Urban Hotel
Moderator: Alex Morison
Group Members: Paul Maglio, Alex Morison, Don Mottaz, Gopal Ramchurn

The moderator, Alex Morison, spoke on behalf of the group. Based on the large-scale volcanic eruption scenario, he discussed the group’s development of a Disaster Management System for search and rescue efforts following the collapse of an urban hotel. As a result of the collapse, people are believed to be trapped in the rubble within contained cavities that are not navigable by humans or dogs. The group’s system would make effective use of robots to map cavities within the rubble (for size, location, interconnectedness) and coordinate the exploration. Key technological challenges include: mobility, structural stability, communications, environmental awareness, multi-robot coordination, and “big data” sense making. The system’s design considerations would very likely include both staged rescue scenarios and real-world rescue efforts with actual rescue personnel.

To provide the mobility necessary for such a system, the group proposed the design of a crawling robot composed of multiple modular sensor units. This “slug-like” robot would consist of a series of sensor arrays; for example, one module might be an antenna system to improve communication capabilities.

Suggested Citation:"6 Revisiting the Scenarios." National Research Council. 2012. Intelligent Human-Machine Collaboration: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/13479.
×

In addition to navigating confined and unstable spaces, multiple robots—each with different perspectives—would provide improved spatial awareness; new reasoning functions could provide 3-D mapping capabilities.

Morison noted that the system’s success will depend on how well multiple robots can work together as a team. In fact, the group identified multi-robot teamwork as the group’s most significant challenge, citing the current lack of breakthroughs in communications protocols and multi-agent coordination. Effective multi-robot coordination becomes especially critical in post-disaster environments that are often resource limited and unpredictable. Under some circumstances, humans would assume a larger or primary role in coordination efforts—for example, under system failure or when human expertise is required. In cases where humans and robots share responsibilities, automated reasoning would be combined with human reasoning.

Lastly, the group observed that as robots develop increased autonomous capabilities, there may be a push for increased autonomous decision making. The group questioned what if anything might limit such autonomy. For example, what ethical considerations exist for human robot rescue teams (with varying various levels of autonomous capabilities) that triage lost or injured individuals?

Group 2: Team Clean

Moderator: Michael Beetz

Group Member: Michael Beetz, Andreas Hofmann, Mark Neerincx, Liz Sonenberg

Michael Beetz, the moderator, provided a summary of the group’s discussions. Beetz indicated that the group focused its efforts on designing a home robotic cleaning team, “Team Clean,” composed of multiple machines (e.g., humanoid robot, vacuum cleaner, small UAV to “map” the environment), and potentially a human director. The team would be capable of accomplishing a number of tasks with varying degrees of difficulty, from cleaning bathrooms to washing dishes, vacuuming, and doing the laundry.

To do this, a number of research challenges would be addressed, including: practical task manipulation (e.g., picking up fragile objects), smooth locomotion and navigation in a dynamic environment (e.g., going up stairs and opening doors), safety (e.g., not getting in the way of residents or pets), human-robot communication, and social robotics. In addition, machines would have to be able to learn and recover from mistakes and possess sufficient knowledge intensiveness (e.g., to go from an abstract task “to clean up” to understanding how clean is “clean enough”).

Some tasks, Beetz acknowledged, would require varying degrees of interaction between machines and residents. In some cases, a robot may request feedback from the resident. For example, a robot might ask whether a dirty glass

Suggested Citation:"6 Revisiting the Scenarios." National Research Council. 2012. Intelligent Human-Machine Collaboration: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/13479.
×

situated near the resident is currently being used or in need of washing. In other cases, the system should be adaptable if it is re-tasked by the resident. This could occur if the resident’s cleaning expectations differ from those of the robot. The robot may also need to resolve conflicting resident demands—for example, balancing a parental request to “pick things up off the floor” and a teenager’s request to “leave the bedroom as it is.” To deal with this situation, the group proposed a system with one “chief,” as well as an organizational structure to deal with conflicting goals.

Lastly, the group identified performance evaluation as a significant element of the system. For example, how many tasks were accomplished and in what time frame? How well did the team function? As the team evolved, the system would be scalable to accomplish a wider range of tasks.

Group 3: Biped Hospital Companion Robot
Moderator: Candy Sidner
Group Members: Robert Hoffman, Lakmal Seneviratne, Candy Sidner, Rong Xiong

The moderator, Candy Sidner, provided a description of the group’s proposal for creating a biped hospital companion robot (based on an earlier discussion of hospital service robotics). This robot would undertake personal care activities (e.g., dressing and bathing patients and picking up laundry) and provide mobility/balance support by preventing mobility-related accidents and catching patients who are falling. In addition to physical manipulation requirements, some basis for human-robot communication is required and humans need to be comfortable receiving robotic assistance. For this reason, human-robot trust is an important systems requirement.

To accomplish these tasks, the proposed biped robot would be designed with articulated, touch-sensitive hands and somewhat soft bodies with suitable, nonaversive “skins.” In addition, visual recognition would be integrated with touch and task-manipulation capabilities. Communication between the robot and patient would be computer-controlled and employ simple dialogue—for example, questions that can be answered with a “yes” or “no” or with a very short statement.

Sidner added that algorithms, such as those used to predict the movements of rapidly traveling Ping-Pong balls, would be tuned and applied to predict when a human is falling and to respond appropriately. This would require the robot to distinguish not only between types of falling (e.g., falling while conscious or unconscious), but also between similar actions (e.g., falling versus bending over to pick something up). For some frail individuals, the group noted, the line between falling and bending over is thin. By merging robot companion and robot assistant technologies, the robot could also act as an instructor or coach for patients. For example, a robot that has learned to balance itself could

Suggested Citation:"6 Revisiting the Scenarios." National Research Council. 2012. Intelligent Human-Machine Collaboration: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/13479.
×

not only help feeble patients cross hospital floors but could also act as a physical therapy coach.

As a part of its design, the group would also develop a number of testbeds to assess both communication and trust issues, as well as appropriate and safe interactions between robots and patients. Questions addressed would include: How does the nature of human-robot communication and interaction change when robots are working with patients who may be sick, feeble, or physically or cognitively impaired? How anthropomorphic should a robot companion be, and should it be more or less anthropomorphic if it is engaging in a conversation or dressing/undressing a patient? How might robot companions work with other robot companions in this environment?

Group 4: The Robotic Patient Advocate
Moderator: Michael Freed
Group Members: Michael Freed, Yukie Nagai, Jean Scholtz, Satoshi Tadokoro, Manuela Veloso

The moderator, Michael Freed, spoke on behalf of the group. Using the medical service robots as a starting point, Freed described the group’s proposal for creating a robotic patient advocate that would work either as an intermediary between the patient and hospital staff or directly with patients. The robotic patient advocate would keep nurses up-to-date (e.g.., monitor and report changes in patient physical or emotional states), provide continuity when nurses change shifts or when patients are assigned new doctors, and communicate with nurses when patients are asleep or unable to effectively communicate. In addition, the advocate would directly provide information to confused or forgetful patients (e.g., asking “Why am I being wheeled to Room 108?” or “Have I taken my medication already?”). The advocate would also support medical staff when the patient required encouragement. Lastly, the advocate would run interference with visitors and people who stay too long or get in the way of medical staff.

As Freed explained, the advocate would leverage the group’s collective experience in autonomous systems, communications and dialogue, human emotion, machine-human interaction, and performance evaluation of both robots and humans. Although some of these capabilities were possible using conventional technologies, six key breakthroughs would be required. (1) Dialog: The advocate should be capable of high-level discussions with people possessing different knowledge, motives, and cultures. (2) Multimodal Sensing: The advocate should be able to tap into—via many and complex sensors—a hospital’s data-rich environments to access a patient’s medical records, real-time physiological conditions, test results, and schedules. (3) Strategic Planning: The advocate should take action by balancing a patient’s immediate goals and requests with longterm patient support that considers legal and safety issues. (4) Safe Navigation: The advocate should navigate a complex environment of constantly changing

Suggested Citation:"6 Revisiting the Scenarios." National Research Council. 2012. Intelligent Human-Machine Collaboration: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/13479.
×

people, carts, beds, and equipment. (5) Social Understanding: This might require the advocate to know when and how to “shoo” away visitors who are unwanted or have stayed too long. (6) Multi-Persona Negotiation: The advocate should be a middleman or “middlerobot” among a floating team consisting of patients, family members, doctors, nurses, and others.

The group acknowledged that evaluation of the system was critical; thus, the advocate would be developed first with limited capabilities that would be expanded on the basis of experience and learning.

Group 5: Providing Post-Disaster Basic Services
Moderator: Lin Padgham
Group Members: Tal Oron-Gilad, Lin Padgham, Dirk Schulz, Holly Yanco

Lin Padgham, the moderator, provided the description of the group’s proposal. As one component of the volcanic eruption disaster-management scenario, Padgham described the group’s design of an information management system to provide basic services, such as communications, food, power, and water, in the first week following a major disaster. The system would not provide total coordination across the entire disaster management value chain, but rather would provide on-the-ground individuals with decision support.

Such a decision-support system would require data inputs from numerous sources, including cell phones, sensors, weather reports, and UAVs. The system would take in data in a variety of formats and then organize and share those data with a range of specialized users. For example, data inputs from UAVs that show downed power lines could be used to coordinate prompt robot deliveries of electrical and other power sources to neighborhoods lacking electricity. Effectively distributing and acting on this information will require simple yet specialized human-machine interfaces.

Ongoing access to massive amounts of parallel data would allow management officials to better prioritize their attention and efforts—for example, whether to immediately evacuate a neighborhood or to first restore basic infrastructure. Padgham added that the system could be used as a simulation tool in advance of a disaster to improve emergency management response. By assessing the efficacy of different communications protocols and of evacuation routes under different environmental and social circumstances, authorities can identify where critical post-disaster response failures are likely to occur.

The system would also make “individualized” information available to both specialized users (e.g., UAV operators with specific data needs to survey for downed power lines) and to untrained users who are stranded in their homes with limited food and water. Although acknowledging that such a system would provide complex decision support, the group noted that human judgment will always remain key.

Suggested Citation:"6 Revisiting the Scenarios." National Research Council. 2012. Intelligent Human-Machine Collaboration: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/13479.
×

This page intentionally left blank.

Suggested Citation:"6 Revisiting the Scenarios." National Research Council. 2012. Intelligent Human-Machine Collaboration: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/13479.
×
Page 25
Suggested Citation:"6 Revisiting the Scenarios." National Research Council. 2012. Intelligent Human-Machine Collaboration: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/13479.
×
Page 26
Suggested Citation:"6 Revisiting the Scenarios." National Research Council. 2012. Intelligent Human-Machine Collaboration: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/13479.
×
Page 27
Suggested Citation:"6 Revisiting the Scenarios." National Research Council. 2012. Intelligent Human-Machine Collaboration: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/13479.
×
Page 28
Suggested Citation:"6 Revisiting the Scenarios." National Research Council. 2012. Intelligent Human-Machine Collaboration: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/13479.
×
Page 29
Suggested Citation:"6 Revisiting the Scenarios." National Research Council. 2012. Intelligent Human-Machine Collaboration: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/13479.
×
Page 30
Next: Appendix A: Workshop Participants »
Intelligent Human-Machine Collaboration: Summary of a Workshop Get This Book
×
Buy Paperback | $29.00 Buy Ebook | $23.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

On June 12-14, 2012, the Board on Global Science and Technology held an international, multidisciplinary workshop in Washington, D.C., to explore the challenges and advances in intelligent human-machine collaboration (IH-MC), particularly as it applies to unstructured environments. This workshop convened researchers from a range of science and engineering disciplines, including robotics, human-robot and human-machine interaction, software agents and multi-agentsystems, cognitive sciences, and human-machine teamwork. Participants were drawn from research organizations in Australia, China, Germany, Israel, Italy, Japan, the Netherlands, the United Arab Emirates, the United Kingdom, and the United States.

The first day of the workshop participants worked to determine how advances in IH-MC over the next two to three years could be applied solving a variety of different real-world scenarios in dynamic unstructured environments, ranging from managing a natural disaster to improving small-lot agile manufacturing. On the second day of the workshop, participants organized into small groups for a deeper exploration of research topics that had arisen, discussion of common challenges, hoped-for breakthroughs, and the national, transnational, and global context in which this research occurs. Day three of the workshop consisted of small groups focusing on longer term research deliverables, as well as identifying challenges and opportunities from different disciplinary and cultural perspectives. In addition, ten participants gave presentations on their research, ranging from human-robot communication, to disaster response robots, to human-in-the-loop control of robot systems.

Intelligent Human-Machine Collaboration: Summary of a Workshop describes in detail the discussions and happenings of the three day workshop.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!