National Academies Press: OpenBook
« Previous: Front Matter
Suggested Citation:"Summary ." National Research Council. 2014. Complex Operational Decision Making in Networked Systems of Humans and Machines: A Multidisciplinary Approach. Washington, DC: The National Academies Press. doi: 10.17226/18844.
×

SUMMARY

Over the past 2 decades, computing and communications networks have made it possible for decision makers to access enormous amounts of information. Entire libraries are available for searching, multiple streams of sensed data may be tapped, and collections of other network users can in many cases be accessed readily. Machine learning and natural language processing have advanced to the point where they can often provide basic pre-processing of diverse types of data. Computational support in the form of large-scale data collection and analysis, visualization, etc. has been readily incorporated into some human decision making processes. For example, computation is in the control processes for all manner of processing plants (chemical processing, nuclear power generation and petroleum refining), infrastructure (electric grid and telecommunications), manufacturing (chip fabrication and large scale baking plants), assembly (electronics and automotive robotic assembly), transactions (credit card and banking) and the military (management of theater operations).

However, the ease with which humans have already integrated computational systems into decision making ranging from ordinary to critical, from simple to complex, belies a deeper truth: this area of inquiry is still in its infancy relative to where multi-disciplinary research could take it over the next generation. This state of affairs has generated an environment that is ripe for a rethinking of human-computer collaboration in the context of complex decision making. The vast amount of information that can be brought to bear does not guarantee better decisions or a more straightforward or reliable decision-making process. How to take advantage of these capabilities is the subject of this report of the Committee on Integrating Humans, Machines and Networks: A Global Review of Data-to-Decision Technologies.

The multidisciplinary committee that was formed at the request of the National Ground Intelligence Center of the U.S. Army (NGIC) included experts in autonomous agents, cognitive science, decision analysis, machine learning, neuroscience, statistics, and other areas. The sponsor wanted to better understand how enabling technologies are being integrated to inform and improve computer-assisted decision making; what some of the impediments are to their integration; and obtain a sense of the research that is occurring in university, government, and industrial labs inside and outside of the United States.

Early on in its deliberations, the committee perceived the unbounded nature of this broad topic—that is, the more they learned as a group about the varied aspects of human-machine collaboration for decision making, the more there was to study. Thus the committee concluded that a useful contribution to this topic at this time would be a preliminary exploration of the issues that could provide a roadmap for future multidisciplinary research. The report’s structure takes a linear path: from human decision making; to relevant new computing capabilities; to emerging explorations of human-computer team decision making; to several research challenges that need to be overcome in order to realize the next steps in human-machine collaboration for complex decision making.

Following are the committee’s findings. They are listed in the order that they appear in the text:

Suggested Citation:"Summary ." National Research Council. 2014. Complex Operational Decision Making in Networked Systems of Humans and Machines: A Multidisciplinary Approach. Washington, DC: The National Academies Press. doi: 10.17226/18844.
×

Finding 1: A common representation of the decision-making process, used to train fighter pilots in rapid decision making for air combat, calls for sequential steps to observe, update beliefs, choose an action, and take the action (the so-called OODA loop). While those steps are inherent to any careful decision making, for complex decisions the OODA loop framework does not readily reflect feedback loops between the steps and branching to consider multiple le choices of action, both of which are common. The study of decision making in complex situations, and the design of automated decision support systems, requires an understanding of those complexities. Thus the OODA-loop framework may not be sufficient in those contexts.

Finding 2: Increasingly the data used to support computer-assisted decisions are drawn from heterogeneous sources (e.g. unstructured text, images, simulation outputs). Current techniques for filtering and aggregating these disparate data types into a well-characterized input for decision making are limited, which therefore limits the quality of the decisions.

Finding 3: While improved information availability can improve the quality of decision making, more information alone is not sufficient. This is particularly evident in complex scenarios where the goals of different team members are not completely aligned and delays make it difficult to attribute effects to actions.

Finding 4: Computer assists to human decision making will “come of age” when some of the computational elements are not simply assistive, but perform at a level that they are trusted as “near-peer” teammates in an integrated human-computer system. One of the key challenges of this integration will be the development of new techniques for test and evaluation that build trust between the human partner and the computational elements.

Finding 5: Humans and computation have different strengths in what they accomplish and there are several aspects of human decision making that can benefit from computer-aided systems, such as cognition, recognition of errors in judgment and task allocation. Similarly, there are several aspects of computer processing that can benefit from human guidance, such as prioritization, dealing with unusual or unexpected situations, understanding social and cultural context, and taking environmental and contextual information into account. The committee finds that the computational assists to human decision making are best when the human is thought of as a partner in solving problems and executing decision processes, where the strengths and benefits of machine and humans are treated as complementary co-systems.

In addition to these findings, the committee identified a number of promising research directions to improve the scientific basis for strong human-computer decision making:

  • Data-to-decisions is an umbrella term that is not clearly defined. We need a better understanding of how cognitive functions can be supported over time and in context and an overall framework for thinking about how to design human-computer decision systems;
  • The ubiquitous capability to capture, store, reproduce, move, and reuse data has led to decisions increasingly being made by networks composed of humans and machines. Yet, the exploitation of that data is often ad hoc. Research is needed to frame and systematize how we exploit that data;
Suggested Citation:"Summary ." National Research Council. 2014. Complex Operational Decision Making in Networked Systems of Humans and Machines: A Multidisciplinary Approach. Washington, DC: The National Academies Press. doi: 10.17226/18844.
×
  • At any moment, whether a particular datum will be relevant or irrelevant into the future is task- and context-dependent, so there is an incentive to retain more, rather than less. Thus, a key challenge is to build task and context models that enable data to be filtered and processed into “useful information”;
  • Another challenge is developing systems that allow both humans and computers to work together in a harmonious team, rather than one supervising the other. This requires research to help individual and team exploration of (partial and incomplete) hypotheses, to enable continuous learning by the system (e.g., so the system can learn how to predict an analyst’s needs and preferences); to guide continuous ingesting of data and its metadata and fusing it into the existing data; to cue decision makers to relevant, unexplored data or behavior; and to facilitate the sharing of hypotheses and derived knowledge among team members (such as by developing languages that make it easy for decision makers to state what they want the data to tell them). Creating harmonious human-computer teams would also be helped by research in comparing the different roles of humans and computers in mixed teams;
  • Complex decision making often takes place in a complex environment, with multiple activities occurring simultaneously. This leads to frequent interruptions and the need to switch tasks and revise priorities. Current human-computer systems do not handle interruptions well and they need to provide more support for the resumption of interrupted activities. More research is needed on computational-interruption management techniques and algorithms, rooted in an understanding of people’s cognitive and attentional capabilities; and
  • More work is needed to develop a methodology for evaluating and assigning metrics for each individual piece of the collaboration and for the quality of the decisions made by the overall human-machine collaborative system.

The committee anticipates that human-computer decision-making systems will continue to advance, but this outcome is not certain. A concerted and thoughtfully guided effort will improve the chances of success.

Suggested Citation:"Summary ." National Research Council. 2014. Complex Operational Decision Making in Networked Systems of Humans and Machines: A Multidisciplinary Approach. Washington, DC: The National Academies Press. doi: 10.17226/18844.
×

This page intentionally left blank.

Suggested Citation:"Summary ." National Research Council. 2014. Complex Operational Decision Making in Networked Systems of Humans and Machines: A Multidisciplinary Approach. Washington, DC: The National Academies Press. doi: 10.17226/18844.
×
Page 1
Suggested Citation:"Summary ." National Research Council. 2014. Complex Operational Decision Making in Networked Systems of Humans and Machines: A Multidisciplinary Approach. Washington, DC: The National Academies Press. doi: 10.17226/18844.
×
Page 2
Suggested Citation:"Summary ." National Research Council. 2014. Complex Operational Decision Making in Networked Systems of Humans and Machines: A Multidisciplinary Approach. Washington, DC: The National Academies Press. doi: 10.17226/18844.
×
Page 3
Suggested Citation:"Summary ." National Research Council. 2014. Complex Operational Decision Making in Networked Systems of Humans and Machines: A Multidisciplinary Approach. Washington, DC: The National Academies Press. doi: 10.17226/18844.
×
Page 4
Next: Chapter 1 Introduction »
Complex Operational Decision Making in Networked Systems of Humans and Machines: A Multidisciplinary Approach Get This Book
×
Buy Paperback | $44.00 Buy Ebook | $35.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Over the last two decades, computers have become omnipresent in daily life. Their increased power and accessibility have enabled the accumulation, organization, and analysis of massive amounts of data. These data, in turn, have been transformed into practical knowledge that can be applied to simple and complex decision making alike. In many of today's activities, decision making is no longer an exclusively human endeavor. In both virtual and real ways, technology has vastly extended people's range of movement, speed and access to massive amounts of data. Consequently, the scope of complex decisions that human beings are capable of making has greatly expanded. At the same time, some of these technologies have also complicated the decision making process. The potential for changes to complex decision making is particularly significant now, as advances in software, memory storage and access to large amounts of multimodal data have dramatically increased. Increasingly, our decision making process integrates input from human judgment, computing results and assistance, and networks. Human beings do not have the ability to analyze the vast quantities of computer-generated or -mediated data that are now available. How might humans and computers team up to turn data into reliable (and when necessary, speedy) decisions?

Complex Operational Decision Making in Networked Systems of Humans and Machines explores the possibilities for better decision making through collaboration between humans and computers. This study is situated around the essence of decision making; the vast amounts of data that have become available as the basis for complex decision making; and the nature of collaboration that is possible between humans and machines in the process of making complex decisions. This report discusses the research goals and relevant milestones in several enabling subfields as they relate to enhanced human-machine collaboration for complex decision making; the relevant impediments and systems-integration challenges that are preventing technological breakthroughs in these subfields; and a sense of the research that is occurring in university, government and industrial labs outside of the United States, and the implications of this research for U.S. policy. The development of human-machine collaboration for complex decision making is still in its infancy relative to where cross-disciplinary research could take it over the next generation. Complex Operational Decision Making explores challenges to progress, impediments to achieving technological breakthroughs, opportunities, and key research goals.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!