Attention and Multitasking
Divided attention and multitasking—doing several things at once—are ubiquitous in combat operations. An infantryman may have to decide on a general course of action, plan his path of movement, run, and fire his weapon simultaneously. When engaging multiple targets, a tank crew must continuously navigate and control the vehicle, search for targets, aim and fire the gun, and assess battle damage. A pilot must simultaneously control his aircraft, plan maneuvers, navigate, communicate with his wingman, control sensors, aim and fire weapons, and monitor and manage other aircraft systems. A commander responsible for several units must divide his attention among the units as he attempts to accomplish multiple, concurrent, perhaps conflicting goals. In each of these settings, several decisions and actions may have to be evaluated and then executed in overlapping time frames.
In most situations lasting more than a few seconds, the individual or the team ideally should or actually does review current goals to consider and prioritize them; assess progress made toward accomplishing each goal; and then allocate immediate attention to tasks in accordance with scheduling priorities, importance, urgency, probabilities, training, and anticipated ability to accomplish certain tasks or processes in parallel, with specified loss due to sharing. This management-like activity should occur continuously and generally represents an attempt to allocate cognitive resources efficiently. In some cases, the decision maker may choose to deal with competing tasks by devoting attention to each in turn. In most cases, however, a realistic representation of the performance of competing tasks or processes will require some degree of overlap or sharing. In
this chapter, models of such situations are termed multitasking models. Both theories and models of attention and multitasking behavior are reviewed. Conclusions and goals emerging from this review are presented in the final section. First, however, some essential details related to attention and multitasking are added to the vignette presented in Chapter 2, and some key concepts and terms are defined.
Hasty Defense Vignette: Additional Details
To frame the discussion and provide examples of attention and multitasking concepts, it is necessary to add some detail to the hasty defense vignette described in Chapter 2. These details include specific tasks the platoon leader is responsible for performing.
Suppose that after the initial engagement, the tank platoon has moved to battle position 1 (BP1). All tanks have moved into initial hide positions, and all tank commanders have identified fire and alternative hide positions. All the tank commanders and gunners (including the platoon leader and his gunner) are scanning for additional enemy forces. The scenario unfolds according to the event sequence found in Exhibit 4.1. At this point in the scenario, the platoon leader is attempting to perform the following tasks:
Maintain general situation awareness, and initiate appropriate tasks
Report enemy contact to A Company commander
Assess battle damage (to first T-80)
Monitor movement to alternate position
Monitor fire on second T-80—interrupted by third T-80
Assess damage to own tank
Direct turret slew toward target (third T-80)
Communicate with platoon
Monitor firing (on T-80)
Clearly, the platoon leader cannot perform all these tasks simultaneously. Furthermore—and significant to the theme of this chapter—the way he allocates his attention to these tasks will have a significant effect on the outcome of the battle.
Key Concepts and Terms
Relation to Learning
The relationship between learning and attention and multitasking has intrigued researchers and theorists from the earliest days of experimental psychology. For example, Bryan and Harter (1899) studied improvements in the sending and receiving of telegraphy. They proposed that naive performers needed to
Exhibit 4.1 Tank Platoon Scenario Event Sequence
allocate scarce attentional resources to the tasks involved, but that training allowed automatization of processes, or automatism (discussed further below), freeing attention for increasingly higher-level cognitive activities. Perhaps the most easily observable and largest effects in the fields of performance and cognitive behavior are those that occur during the often quite extended periods of deliberate practice, known as the development of expertise (and skill) (see Ericsson and Smith, 1991). Part of this gain in skill is known to depend on the storage in memory of a vast amount of relevant knowledge and behavioral procedures that can be accessed and executed with relatively low demands on attention (e.g., see Chase and Simon, 1973).
Other researchers have studied the degree to which training allows performers to accomplish two largely unrelated simultaneous tasks. For example, Downey and Anderson (1915) showed that extensive training would allow performers to ''read chapters while writing memorized verses" with little cost in terms of a performance decrement or errors in the written passages. Schneider
and Shiffrin (1977) and Shiffrin and Schneider (1977) carried these ideas further, tested them empirically in a number of visual and memory search studies, and proposed a general theory of attentive and automatic processing. The idea was that certain processes that are trained consistently may be learned as automatic units, reducing demands for attentional resources. Logan (1988) also explored such issues by showing how recourse to learned procedures allowed performers to bypass the need to accomplish tasks by algorithmic means, that is, by a slower, sequential series of smaller steps. In both of these instances, the authors identified specific processes that were learned and thereby reduced processing demands, but these processes were specific to the tasks under study. A valid criticism would note that the general theory simply describes the shift from procedures requiring scarce resources to ones that bypass such demands, without providing a method for predicting what sort of learning might take place in other situations.
Models of learning are not the subject of this chapter. It is assumed that the participant is at a given level of skill development, one at which the need for multitasking is critical to carrying out the task. At a fixed stage of learning, or level of skill, the issue is not what changes with training, but what set of available cognitive resources is allocated to accomplish the tasks at hand. The general term used to describe such allocation is selective attention. Both some brief historical notes on selective attention and a discussion of current models are presented later in the chapter.
Relation to Working Memory
Although the allocation of limited attentional resources is often described as selective attention, these processes are difficult to separate from the general operations that control the cognitive processing system. Such control processes are usually thought to reside in a temporarily active memory system, and are also referred to as working memory (see Chapter 5). It would probably not be theoretically possible to draw lines between working memory, selective attention, and multitasking. However, it has been traditional to talk about selective attention with respect to tasks that involve perception and motor performance and are usually fairly simple; to talk about multitasking with respect to several tasks or processes that are complex, relatively independent, and distinct; and to talk about working memory with respect to tasks involving coding, learning, and retrieval from memory. Although these terms are used in this chapter (and in Chapter 5), no theoretically important distinction is implied by the choice of a particular term.
Tasks and Processes
Sometimes an individual (or group) must try to accomplish two or more externally defined tasks that are largely unrelated and independent. An example
involving the platoon leader in the above vignette would be directing the slew of the turret while trying to reset the radio. The demands on the performer in such cases are perhaps most aptly described as multitasking. In other cases, multiple internal processes must be used to accomplish a single externally defined task. For example, a driver may need to reach a designated location, nominally a single task, but to do so must scan for obstacles and enemies, plan a route, manipulate the controls, listen to and act on orders from the tank commander, and so forth. The language of selective attention is more often used in such cases. There is really no hard and fast line between uses of the terms, and multitasking is used in this chapter to refer to both situations; that is, tasks are taken to refer to both external and internal activities.
Automatism offers a means of accomplishing multitasking with less sharing and less sequential application of attentional resources. It is usually developed through extended training. For example, driving a tank requires that multiple concurrent tasks be accomplished; a novice must usually focus on just one task at a time, such as steering. After extended training, a skilled tank crew member may be carrying out 10 or more tasks in a generally concurrent fashion, sharing attention among them, and even have enough attention left over to carry on a simultaneous conversation on an unrelated topic. Automatism is closely related to the development of skill and expertise. For a few experimental paradigms, the processes of automatism have been worked out. Examples include chunking in memory search (Schneider and Shiffrin, 1977), attention attraction in visual search (Shiffrin and Schneider, 1977), memorization in alphabetic arithmetic (Logan and Klapp, 1991), learning of responses in Stroop situations (MacLeod and Dunbar, 1988), and unitization in the perception of novel characters (Shiffrin and Lightfoot, 1997). The processes by which automatism develops generally are not yet well understood, although some existing models, such as adaptive control of thought (ACT) and Soar, incorporate automatization (see Chapter 3 for further discussion). Thus simulation modeling for a given task would require specific implementation of the components of automatism appropriate for that task.
Informally, attention may be thought of as the focus of conscious thought, though this is an inadequate definition. Somewhat more formally, attention may be thought of as the means by which scarce or limited processing resources are allocated to accomplish multitasking. There are several broad reviews of attention as it relates potentially to human behavior modeling (e.g., Parasuraman and Davies, 1984; Shiffrin, 1988). The following brief summary is based heavily on Wickens (1992:74-115).
Selective attention is a process through which the human selectively allocates processing resources to some things over others. The things involved could include internal processes of all kinds, but the term was used more restrictively in the early years of the field to refer to perceptual processing of sensory stimuli. Thus the term originally referred to decisions to attend to some stimuli, or to some aspects or attributes of stimuli, in preference to others (Kahneman, 1973:3). Examples of selective attention drawn from our vignette would include visual sampling (in which the platoon leader, early in the vignette, selectively attends to his integrated display), visual target search (in which he scans his vision blocks or the independent thermal viewer for enemy armor), and auditory selective attention (in which he monitors his radio for transmissions related to him and his platoon).
Selective attention is by definition limited. A human can attend substantially or fully to a relatively small number of stimuli and/or stimulus attributes at one time. Not only is attention limited, but the selection process may be inaccurate and inappropriate for the tasks at hand. For example, the platoon leader could become momentarily preoccupied with the radio problem when the more immediately important task is to slew the turret to the third T-80 threat.
Focused attention is a process in which the human rejects some processes in favor of others; in perceptual domains the term usually denotes the rejection of irrelevant stimuli (Schneider et al., 1984:9). For example, the platoon leader uses focused attention to ignore vegetation and cultural features as he searches for enemy forces. He also uses it to filter out radio transmissions that are not relevant to him and his platoon.
The inability to reject irrelevant stimuli and/or information or, more generally, irrelevant processing, marks a failure of focused attention. Focusing sometimes fails because attention is attracted by a singular or intrusive event in the environment (e.g., an irrelevant stimulus that attracts attention, as in Shiffrin and Schneider , Yantis , and Theeuwes , or a target that is noticed and causes a subsequent target to be missed, as in Shapiro and Raymond ). For example, the platoon leader may be distracted by his gunner's complaints about heat and humidity inside the tank and miss an important radio transmission.
Divided attention describes a situation in which the human attempts to carry on many processes simultaneously, distributing resources among them. In the perceptual domain, such situations usually involve processing more than one
stimulus at a time. For example, the platoon leader may have to try to scan his independent thermal viewer for enemy forces and watch his integrated display simultaneously. He may have to listen to a status report from his gunner while listening to a potentially relevant radio transmission from the A Company commander to the platoon on his right flank. Humans are clearly limited in their ability to divide attention in this manner. For example, the platoon leader may miss an enemy tank while watching his integrated display, or miss relevant information from the radio while listening to the gunner.
Automatization of component processes is the typical means by which people increase their ability to carry out simultaneous tasks. Relatively little is known about interactions among tasks, but when tasks differ substantially, there is often a cost associated with switching between them, even after some practice (e.g., Allport et al., 1994).
Theories and Models of Selective Attention
Theories and models of selective attention are still in an early formative stage (as are the models of working memory of which they are a subset). Historically, Broadbent (e.g., 1957) introduced his filter theory for application to the processing of sensory information. According to Broadbent, primitive sensory features are processed in parallel, preattentively, without capacity limitations. Slightly later in the processing stream, a filter or blockage is reached, and further processing requires selective allocation of attention to some feature or dimension of the input information (such as one of the two ears); information having some other feature or arriving on some other dimension, termed a channel, is blocked from further processing (e.g., the information arriving on the other ear will not be processed further). Subsequent research demonstrated that certain information on unattended channels does get processed deeply enough for its meaning to have an impact (e.g., one's own name presented to an unattended ear). This finding led Treisman (e.g., 1969) to modify Broadbent's theory and propose that the processing of information on unattended channels is attenuated rather than blocked. An alternative theoretical approach was suggested by Deutsch and Deutsch (1963). They posited that all incoming information is processed to deep levels, but that the attentional capacity limitations are those of memory (e.g., selective forgetting of processed information from short-term memory). A more fully developed version of this concept, the theory of automatic and attentive processing (then called automatic and controlled processing) was presented by Shiffrin and Schneider (1977) and summarized and updated by Shiffrin (1988).
All the above approaches share the assumption that the difficulty for the processing system is limited processing capacity. Early theories explicitly or implicitly assumed that capacity represents a common pool of allocatable resources. Later researchers proposed that capacity is better conceived as a group of overlapping pools, so that increasing the difficulty of a task may require
sharing of resources and attention across similar domains (such as between the two ears), but not across dissimilar domains (such as between the ears and eyes). Examples of this view include Wickens (1984) and Navon and Gopher (1979).
It must be emphasized that these issues continue to undergo intense empirical testing and theoretical development today, and a general or simple resolution has not been achieved. For certain tasks, the capacity limitations are almost certainly quite central and related to deep processes such as decision making and/or forgetting in short-term memory (e.g., Shiffrin and Gardner, 1972; Palmer, 1994); for others, however, the blockage may be at a more peripheral locus.
Because the field of attention is so complex and relatively new and comprises largely empirical studies, theory development is still at an early age. Some models are little more than metaphors; an example is the "spotlight" theory, in which attention is spatially compact and moves continuously across the visual field. Detailed and well-developed computer simulation and mathematical models have been devised for particular tasks with some success, but they are based on extensive collection of data within those tasks and tailored to those domains (e.g., Schneider and Shiffrin, 1977; Wolfe, 1994; Sperling and Weichselgartner, 1995; Meyer and Kieras, 1997a, 1997b). Theory development has not yet proceeded to the point where the theories can generalize well across tasks or allow extrapolation to new domains in which extensive empirical research has not occurred. In this regard, the work of Meyer and Kieras (1997a, 1997b) on executive-process interactive control (EPIC) offers a promising approach. (EPIC is discussed in detail in Chapter 3.)
If one broadens the notion of attending to stimuli to encompass attending to processes and tasks, theories and models of attention can be expanded from a focus on perceptual processing to become theories and models of multitasking—the topic of the next section.
It is difficult to imagine a situation in which the modeling of multitasking in the general sense would not be needed for military simulations. Whether in the guise of a model of working memory or of selective attention or multitasking, this modeling will have similar conceptual underpinnings. Applications are sometimes needed when there are two or more externally defined and somewhat independent tasks to be accomplished, and sometimes when one (complex) task requires a variety of internal processes that need more resources than are available concurrently. Of course, some military simulations do incorporate limited multitasking, if only to permit the interruption of one task for another, but the extent to which such capability is based on psychological theory is not always clear. When overload occurs, there are several potential outcomes including (1) continuing to do everything but less well, (2) reducing the number of things being attended to, (3) putting tasks in a queue, and (4) dropping everything and walking away. It is
worth nothing that there may be situations in which one would not want to model a real human with real limitations; rather, one might find it useful to assume that the human has unlimited parallel processing capabilities (e.g., if a model of a "superopponent" were desired).
Theories and Models of Multitasking
There are a number of excellent reviews of multitasking and related topics, including several by Wickens (1984, 1989, 1992) and Adams et al. (1991). The discussion below covers engineering and psychological theories and models of multitasking. Then, to summarize that information in a form more likely to be usable by military simulation modelers, a composite theory of multitasking is presented. Note that the models described here are generally not computational models and are therefore not directly applicable to military simulations. But these models and the theories they interpret could serve as a valuable base on which to construct computational models. Relevant computational models of multitasking are described in more detail in the context of integrative models, which are discussed in Chapter 3.
Engineering Theories and Models of Multitasking
Engineering theories of human behavior are generally concerned with describing gross human behaviors, not the cognitive or psychomotor mechanisms that underlie them. This is particularly true of engineering theories and models of multitasking. Pattipati and Kleinman (1991) present a summary of such models; the summary here is based on but also expands on their account.
As mentioned above, multitasking theories and models can be viewed as an extension of theories and models of attention. An example of this point is multitasking theories and models based on queuing theory . Queuing theory is a branch of operations research that addresses systems capable of being described in terms of one or more servers and a population of customers queuing (lining up) for service. Queuing theory was first applied to the domain of human operator modeling by Carbonell and colleagues, who used it to model the visual scanning behavior of a pilot or other operator obtaining information from several different displays (Carbonell, 1966; Carbonell et al., 1968). Their theory was that the operator's visual attention could be described as a server and the instruments to be read as customers queuing for service. They were then able to use queuing theory formulas to generate estimates of sampling frequencies and other parameters.
The notion of attending to multiple displays was expanded to the broader issue of attending to multiple tasks by a number of researchers (e.g., Walden and Rouse, 1978; Chu and Rouse, 1979; Greenstein and Rouse, 1982). Their general approach was to describe human high-level attention as a server with a given
service time probability distribution having specified parameters. They described tasks competing for the operator's attention as customers queuing up for service according to one or more arrival time distributions with specified parameters. This approach allowed them to model the operator's attention allocation policy in terms of queuing discipline, such as first-in-first-out or last-in-first-out, possibly involving balking (the decision of a customer to leave the queue if waiting time becomes too great). To describe multitasking as a queuing process enabled these researchers to use queuing theory formulas to develop general information about the multitasking behavior of the operator. For example, the ability to compute mean waiting time gives insight into the time required before a task must be attended to and helps in deriving mean task execution time.
The value of such estimates of overall multitasking behavior in the present context may be limited to constructive military simulations, in which the moment-to-moment activity of the modeled human is relatively unimportant to the user. However, these queuing-based theories and models of multitasking behavior provide one basis for the higher-resolution, discrete-event computational models described briefly below and in more detail in Chapter 3 of this report. For a more thorough review of queuing theory models, see Liu (1996).
Engineering research has also contributed multitasking theories and models based on control and estimation theory. These have their roots in optimal control theory and optimal control models of the human operator (e.g., Kleinman et al., 1970, 1971). The application of optimal control theory to human behavior is based on the assumption that the performance of an experienced human operator in controlling a continuous-state system (such as flying an aircraft or controlling a chemical plant) approaches that of a nonhuman optimal control system. Optimal control models of human performance have been shown to predict accurately the performance of real humans who are well practiced at a control task.
In an attempt to extend this success to the modeling of human multitasking behavior, a number of researchers have applied optimal control theory to that domain (e.g., Tulga and Sheridan, 1980; Pattipati and Kleinman, 1991). An optimal control theory of human multitasking behavior has the following elements. A task is represented as a dynamic subsystem of the controlled system (the plant, in optimal control terminology). Thus the plant represents not just an airplane or a tank, but an airplane or a tank augmented by the tasks the operator is trying to perform. The plant is acted upon by disturbances beyond the operator's control. The task state is the state of the plant (possibly including its environment) with respect to the task. The decision state is the time required for each task, the time available for the task, and so on. A display element delays and adds noise to true task states. A human limitations and monitoring element yields perceived task states. A Kalman filter/predictor yields estimates of true task states. The decision state submodel calculates decision variables. The attractiveness measures submodel yields attractiveness measures for each task, and the stochastic choice model computes probabilities of working on each task, which in
turn affect the plant. These elements are synthesized through optimal control theory in relation with a carefully specified performance index selected to minimize the system dynamic error.
Like queuing theory models of multitasking, control and estimation theory models can yield estimates of overall human performance, especially for well-trained individuals or groups. Since they are applicable to hardware and software controllers that must operate in real time (or more rapidly), they are also potentially capable of generating representations of moment-to-moment human behavior. As a result, such models may be useful in both constructive and virtual military simulations. One must keep in mind, however, that models assuming optimum control are almost certainly best applied to humans who have already developed considerable expertise and levels of skill.
Psychological Theories and Models of Multitasking
In general, psychological theories and models of multitasking are distinguished from engineering theories and models in that the former are more concerned with understanding and representing the mechanisms underlying behaviors.
Resource Theories and Models Theories of perceptual attention treat the visual or auditory system as a limited resource to be allocated among two or more competing stimuli or information channels. This view has been naturally extended to the concept of multitasking, in which more complex resources must be allocated to tasks (e.g., Navon and Gopher, 1979). Resource theories are typically based on data obtained in dual-task experiments, in which subjects perform two concurrent tasks (such as tracking a target on a cathode ray tube screen while doing mental arithmetic) while performance on each task is measured.
In single resource theory (Wickens, 1992:366–374), cognitive mechanisms, including those used for memory and decision making, are viewed as a single, undifferentiated resource pool. Task performance is dependent on the amount of resources allocated to the task, and is sometimes defined more formally by means of a performance resource function, which gives the performance of a task as a function of the amount of resources allocated to that task. The performance resource function can be used to characterize a task with respect to whether it is resource limited (i.e., not enough resources to perform it perfectly) or data limited (i.e., limited by the quantity and/or quality of information available to perform it).
When two tasks compete concurrently for such resources and there are insufficient resources available to perform both tasks perfectly, a tradeoff occurs: one task is performed better at the expense of poorer performance on the other. This relationship is sometimes specified more formally by means of a performance operating characteristic. The performance operating characteristic is a function that describes the performance on two concurrent tasks as a function of resource allocation policy (the amount of resources allocated to each task). Here, too,
resource-limited vs. data-limited performance is characterized by the shape of the curve.
The single resource theory is a significant step toward understanding multitasking, but has limitations. For example, single resource theory cannot easily account for dual-task data indicating that interference between two tasks could not be predicted from their difficulty, only from their structure. Another failure of the single resource theory is its inability to explain why, in some cases, two demanding tasks can be time-shared perfectly.
Such limitations led to the development of two related alternative theories. One of these is the theory of automatic and controlled processing, discussed earlier (e.g., Shiffrin and Schneider, 1977; Shiffrin, 1988). In this theory, differential ability to carry on two or more simultaneous tasks is due to differential development of automatic processes and procedures that allow attentional limitations to be bypassed. The other approach is that of multiple resource theory (Wickens, 1992:375–382). In multiple resource theory, resources are differentiated according to information processing stages (encoding and central processing or responding), perceptual modality (auditory or visual), and processing codes (spatial or verbal). Different tasks require different amounts of different resources. For example, for the platoon leader to detect and interpret a symbol on his commander's integrated display would require more encoding resources (processing stage category), while for him to acknowledge a radio transmission from the A Company commander would require more responding resources.
With this refinement, the concept of resources can be used to explain some of the dual-task data beyond the capabilities of the single resource theory. For example, near-perfect time sharing of the two tasks described in the previous paragraph can be explained by their need for different resources. Presumably, the platoon leader can allocate sufficient encoding resources to display interpretation and at the same time allocate adequate responding resources to perform the acknowledgment task.
The multiple resource theory has been used in an attempt to formalize the notion of mental workload (Wickens, 1992:389–402). Here mental workload is defined as the resource demand. Thus poor performance in situations deemed to impose "high workload" is explained in terms of excess demands for specific resources. The multiple resource theory and derivative theories of mental workload have been at least partially validated in realistically complex domains and are already in use in applications designed to evaluate human-machine interfaces and operator procedures (e.g., W/INDEX, North and Riley, 1989).
It is important to realize that both the theory of automatic and controlled processing and multiple resource theory are really general frameworks and for the most part do not provide specific models for new tasks and task environments. However, they can be the basis for such models. One example is a queuing model that integrates aspects of single-channel queuing with multiple-resource-based parallel processing (Liu, 1997).
Strategic Workload Management Theories and Models The multiple resource theory and workload theories and models generally do not address explicitly the issue of how the human allocates resources. Recognizing this, Hart (1989) observed that pilots and operators of other complex systems seem to schedule task performance and augment and reduce task load so as to maintain a ''comfortable" level of workload. Raby and Wickens (1994) validated this theory in a study examining how pilots manage activities while flying simulated landing approaches. Moray et al. (1991) found that while Hart's theory may be true, humans are suboptimal in scheduling tasks, especially when time pressure is great. A recent theory relying heavily on scheduling priorities and timing to explain capacity limitations and multitasking is that of Meyer and Kieras (1997a, 1997b; see the discussion of EPIC in Chapter 3).
Theories and Models of Task Interruptions An issue closely related to managing activities is interruptions. In a simulator study of airline pilot multitasking behavior, Latorella (1996a, 1996b) found that task modality, level of goals in the mission goal hierarchy, task interrelationships, and level of environmental stress affect the way humans handle interrupting tasks and the ongoing tasks that are interrupted. Damos (forthcoming) is currently trying to identify how airline pilots prioritize tasks.
Theories and Models of Task Management There have been a number of efforts to define the process by which operators of complex systems (especially pilots of modern aircraft) manage tasks. For example, Funk and colleagues developed a preliminary normative theory of cockpit task management (Funk, 1991; Chou et al., 1996; Funk and McCoy, 1996). According to this theory, managing a set of cockpit tasks involves the following activities:
Assessing the current situation
Activating new tasks in response to recent events
Assessing task status to determine whether each task is being performed satisfactorily
Terminating tasks with achieved or unachievable goals
Assessing task resource requirements (both human and machine)
Prioritizing active tasks
Allocating resources to tasks in order of priority (initiating, interrupting, and resuming them, as necessary)
Updating the task set
Rogers (1996) used structured interviews to refine and expand the concept of task management, and Schutte and Trujillo (1996) studied task management in non-normal flight situations. The conclusions to be drawn from these studies are that task management is ubiquitous and significant and that it plays an important role in aviation safety. These conclusions almost certainly generalize to other
complex systems. Detailed models of these complex activities are still under development.
Connectionist and Neurally Based Models Neurally motivated computer simulations yield connectionist theories and models of multitasking . For example, Detweiler and Schneider (1991) describe a connectionist model in which separate, radiating columns or modules of nodes and connections represent separate channels or operator "resources" (visual, auditory, motor, and speech systems). In their model, all columns are connected to a single inner loop so that pathways can be established between modules. Multiple pathways imply the capacity for parallel performance. The nature and extent of the connections dictate the nature and extent of multiple task performance. In Detweiler and Schneider's model, the development of connections can be used to model the acquisition of multitasking skill.
Prospects for a Composite Theory of Multitasking
A composite, comprehensive account of multitasking is essentially equivalent to a comprehensive model of human cognition. That is, almost any task of reasonable complexity, especially one likely to be incorporated in real-world military simulations, will involve resource allocation, motor performance, strategic use of working memory, scheduling, retrieval from long-term memory, decision making, and all other components of a general model of cognition and performance. No one would pretend that anyone has yet come close to producing such a model. The closest approximations available are applications of varying degrees of specificity that are tailored to particular task environments (e.g., detailed process models of sequential choice reaction time in the laboratory or less detailed models of pilot performance). A number of such approaches have been described in this section. Although continued progress can be expected in the development of large-scale models, it is unlikely that anything like a comprehensive model will be available within a time horizon of immediate interest to the armed forces. Thus for the near future, applications with the greatest utility will be based on models developed for and tailored to specific task environments.
INTEGRATING CONCEPTUAL FRAMEWORKS
Theories and models of attention and multitasking cover a wide and complex range of human behavior, and there have been several attempts to integrate them into a single, coherent framework. One example is Adams et al. (1991). Their objective was to summarize what is known about attention and multitasking, including task management. Their approach was to review existing psychological literature, to extend and extrapolate research findings to realistically complex domains, and to present a framework for understanding multitasking and task
management. The results of their efforts are summarized in the following paragraphs.
Task management involves task prioritization. Task prioritization depends on situation awareness, which in turn depends on perception. Perception is schema based; that is, input information is interpreted in the context of structured expectations about situations and events. This implies that information used to update situation models must be anticipated and prepared for. Long-term memory is a connectionist, associative structure, and the attentional focus corresponds to areas of long-term memory activation. Multitasking in turn depends on attentional shifts, which are cognitively difficult and take measurable time. Human behavior is goal driven, and goals help determine how and where attention will be shifted. A goal hierarchy comprising goals and subgoals is the basis for task ordering or prioritization when simultaneous performance of all tasks is impossible. Tasks correspond to knowledge structures in long-term memory (one structure per task, though certain structural elements are shared across tasks). Since information processing is resource limited, the human can allocate conscious mental effort to only one task while queuing others. This is the motivation for task prioritization.
There is a tendency to process only that incoming information which is relevant to the task currently being attended to. If incoming information is not relevant to that task, the human must interrupt it to determine which queued task (if any) the information concerns. Such information tends to be "elusive" and subject to neglect, since there is no schema-based expectation or preparation for it. However, noticing and processing a stimulus or event implies interrupting the ongoing task. Humans resist interruptions and can even become irritable when they occur. Tasks associated with lower-level (more specific) goals are more resistant to interruption. But interruptions do occur; fortunately, memory for interrupted tasks is highly persistent.
Task management further involves task scheduling. The ability to schedule depends on the individual's understanding of temporal constraints on goals and tasks. Subjects in task-scheduling studies are capable of responding appropriately to task priority, but scheduling performance may break down under time pressure and other stressors. Task management itself is an information processing function, and it is most crucial when information processing load is at its highest, for example, when there are more tasks to manage. Therefore, task management is a significant element of human behavior.
The conceptual framework of Adams et al. (1991) formed part of the basis for the operator model architecture (OMAR) model of human performance (see Chapter 3). Another framework for attention and multitasking that is beginning to yield interesting results from validation study is EPIC (Meyer and Kieras, 1997a, 1997b) (also described in Chapter 3).
This review of theories and models of attention and multitasking has focused on the engineering and psychological literature, which often proposes explicit mechanisms of attention allocation and task management. There is also some
promising computer science work in which attention and multitasking are emergent phenomena that result not necessarily from explicit representations, but from properties of information processing mechanisms. Selfridge's pandemonium theory (Selfridge, 1959) is an early example.
CONCLUSIONS AND GOALS
Modeling of multitasking is clearly relevant to military simulations and to human performance generally. The field has reached a point at which there is real potential to produce useful simulations of multitasking behaviors. However, doing so will not be easy or quick. Currently, the relevant theories and models are not well developed or validated, and the computational models are somewhat ad hoc. But current theories and models do provide a starting point from which acceptable computational models can be built. We offer the following goals for further development.
Conduct studies to identify the factors that influence the allocation of attention to tasks (e.g., importance, urgency, salience of stimuli), using domain experts as subjects.
When possible, use existing military models that support multitasking. Otherwise, augment existing serial models to support the execution of concurrent tasks.
Delineate carefully the different approaches of sharing tasks versus switching between tasks.
Because most research has focused on attentional effects on perception, devote resources to incorporating in the models the effects of attentional allocation on memory management, decision making, and translation of cognitive activities into action and motor control.
Develop models of attentional capacity, expanding on the concepts of sharing and switching.
Identify the factors that influence the allocation of attention to tasks (e.g., importance, urgency, salience of stimuli).
Validate the behavior of the models by comparing the tasks they accomplish by the model and their performance with similar data from human domain experts.
Begin the process of collecting data on cognitive resource sharing in military domain situations.
Explore alternative model representations, including rule-based, exemplar-based, and neural net representations.
Investigate the effects of various factors (e.g., salience and uniqueness of stimuli) on attention allocation strategies.
Develop models of multitasking behavior in realistically complex military domains, incorporating a wide range of cognitive and motor processes that are affected by attention and resource allocation.
Validate such models against real-world data, especially from the military domain.
Expand the applicability of models of attentional allocation to other cognitive modules mentioned in this report, from memory and learning (Chapter 5) through planning and decision making (Chapters 8 and 6, respectively), to group behavior (Chapter 10) and information warfare (Chapter 11).