Finding 1: A common representation of the decision-making process, used to train fighter pilots in rapid decision making for air combat, calls for sequential steps to observe, update beliefs, choose an action, and take the action (the so-called OODA loop). While those steps are inherent to any careful decision making, for complex decisions the OODA loop framework does not readily reflect feedback loops between the steps and branching to consider multiple le choices of action, both of which are common. The study of decision making in complex situations, and the design of automated decision support systems, requires an understanding of those complexities. Thus the OODA-loop framework may not be sufficient in those contexts.

Finding 2: Increasingly the data used to support computer-assisted decisions are drawn from heterogeneous sources (e.g. unstructured text, images, simulation outputs). Current techniques for filtering and aggregating these disparate data types into a well-characterized input for decision making are limited, which therefore limits the quality of the decisions.

Finding 3: While improved information availability can improve the quality of decision making, more information alone is not sufficient. This is particularly evident in complex scenarios where the goals of different team members are not completely aligned and delays make it difficult to attribute effects to actions.

Finding 4: Computer assists to human decision making will “come of age” when some of the computational elements are not simply assistive, but perform at a level that they are trusted as “near-peer” teammates in an integrated human-computer system. One of the key challenges of this integration will be the development of new techniques for test and evaluation that build trust between the human partner and the computational elements.

Finding 5: Humans and computation have different strengths in what they accomplish and there are several aspects of human decision making that can benefit from computer-aided systems, such as cognition, recognition of errors in judgment and task allocation. Similarly, there are several aspects of computer processing that can benefit from human guidance, such as prioritization, dealing with unusual or unexpected situations, understanding social and cultural context, and taking environmental and contextual information into account. The committee finds that the computational assists to human decision making are best when the human is thought of as a partner in solving problems and executing decision processes, where the strengths and benefits of machine and humans are treated as complementary co-systems.

In addition to these findings, the committee identified a number of promising research directions to improve the scientific basis for strong human-computer decision making:

  • Data-to-decisions is an umbrella term that is not clearly defined. We need a better understanding of how cognitive functions can be supported over time and in context and an overall framework for thinking about how to design human-computer decision systems;
  • The ubiquitous capability to capture, store, reproduce, move, and reuse data has led to decisions increasingly being made by networks composed of humans and machines. Yet, the exploitation of that data is often ad hoc. Research is needed to frame and systematize how we exploit that data;


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement