Appendix D
Research on Managing Information Overload in Soldiers Under Stress

This appendix provides background information on research in augmented cognition (AugCog) conducted by the Defense Advanced Research Projects Agency (DARPA) and the U.S. Army and discusses several of the neuroscience research and engineering challenges.

THE DARPA AUGCOG PROGRAM

DARPA began its research program in the area of augmented cognition in 2001 with a focus on challenges and opportunities presented by the real-time monitoring of cognitive state with physiological sensors (Augmented Cognition International Society, 2008a). The program, now known as the Improving Warfighter Information Intake Under Stress Program, was initially called the Augmented Cognition program, or AugCog. At first it had two stated purposes: (1) to gain battlefield information superiority and (2) neurology-related clinical applications. Linked to the idea of network-centric warfare, the stated rationale for AugCog was that “order of magnitude increases in available, net thinking power resulting from linked human-machine dyads will provide such clear information superiority that few rational individuals or organizations would challenge under the consequences of mortality” (Schmorrow and McBride, 2004). Another early AugCog objective was stated as the enhancement of a single operator’s capabilities such that he or she could carry out the functions of three or more individuals (McDowell, 2002).

It was proposed that information superiority could be achieved by enhancing an operator’s cognitive abilities and by measuring neurological and physiologic variables. One aspect of the enhancement would involve tracking markers of cognitive state, such as respiration, heart rate, or eye movement, using devices for electroencephalography (EEG) or functional optical imaging. The operator would wear a headset that contained such devices and might be connected to other devices as well—perhaps galvanic skin response sensors or pressure sensors in a seat—interacting with them by means of a traditional computer interface device such as a mouse or a joystick. Another aspect of enhancing cognitive ability would be dynamic control of the rate as well as the source of information by a “communications scheduler.” The former DARPA program manager envisioned that technologies developed by the program would be operational within 10 years. He predicted that within 20 years, the technology would be woven into the fabric of our daily lives (Augmented Cognition International Society, 2008b).

Proof-of-concept work for AugCog occurred in two phases. In Phase 1, researchers attempted to detect changes in cognitive activity in near real time in an operationally relevant setting. One relatively large demonstration, called the Technical Integration Experiment, was conducted with mixed results (St. John et al., 2003). Its objective was to determine which psychophysiological measurements could consistently detect changes in cognitive activity during a supervisory control task. Using 20 gauges of cognitive state (CSGs) such as EEG, functional near infrared (fNIR), and body posture, as well as input device measures (mouse pressure and mouse clicks), eight participants completed a series of four simplified aircraft monitoring and threat response tasks in 1 hour. Eleven of the CSGs, including fNIR and EEG measures, were reported to be significant and reliable for at least one independent variable, but no CSG was significant and reliable across all of three independent variables (St. John et al., 2004). Two CSGs, mouse clicks and mouse pressure, demonstrated statistically significant results across two of the independent variables, but these were probably already highly correlated.

There were significant problems with technical integration, several of which were acknowledged by St. John and his colleagues in Technical Integration Experiment (2004). These included questions about the study design constructs and their external validity, the statistical methods applied to the data, important data missing from the report of results, and the familiar problem in psychophysiological research of noisy data (substantial variability across test participants, test



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 117
appendix d research on managing information overload in soldiers under stress This appendix provides background information on by means of a traditional computer interface device such as research in augmented cognition (AugCog) conducted by a mouse or a joystick. Another aspect of enhancing cognitive the Defense Advanced Research Projects Agency (DARPA) ability would be dynamic control of the rate as well as the and the U.S. Army and discusses several of the neuroscience source of information by a “communications scheduler.” The research and engineering challenges. former DARPA program manager envisioned that technolo- gies developed by the program would be operational within 10 years. He predicted that within 20 years, the technology The darPa auGcoG ProGram would be woven into the fabric of our daily lives (Augmented DARPA began its research program in the area of Cognition International Society, 2008b). augmented cognition in 2001 with a focus on challenges Proof-of-concept work for AugCog occurred in two and opportunities presented by the real-time monitoring phases. In Phase 1, researchers attempted to detect changes of cognitive state with physiological sensors (Augmented in cognitive activity in near real time in an operationally Cognition International Society, 2008a). The program, now relevant setting. One relatively large demonstration, called known as the Improving Warfighter Information Intake the Technical Integration Experiment, was conducted with Under Stress Program, was initially called the Augmented mixed results (St. John et al., 2003). Its objective was to Cognition program, or AugCog. At first it had two stated determine which psychophysiological measurements could purposes: (1) to gain battlefield information superiority consistently detect changes in cognitive activity during a and (2) neurology-related clinical applications. Linked to supervisory control task. Using 20 gauges of cognitive state the idea of network-centric warfare, the stated rationale for (CSGs) such as EEG, functional near infrared (fNIR), and AugCog was that “order of magnitude increases in available, body posture, as well as input device measures (mouse pres- net thinking power resulting from linked human-machine sure and mouse clicks), eight participants completed a series dyads will provide such clear information superiority that of four simplified aircraft monitoring and threat response few rational individuals or organizations would challenge tasks in 1 hour. Eleven of the CSGs, including fNIR and under the consequences of mortality” (Schmorrow and EEG measures, were reported to be significant and reliable McBride, 2004). Another early AugCog objective was stated for at least one independent variable, but no CSG was sig- as the enhancement of a single operator’s capabilities such nificant and reliable across all of three independent variables that he or she could carry out the functions of three or more (St. John et al., 2004). Two CSGs, mouse clicks and mouse individuals (McDowell, 2002). pressure, demonstrated statistically significant results across It was proposed that information superiority could be two of the independent variables, but these were probably achieved by enhancing an operator’s cognitive abilities and already highly correlated. by measuring neurological and physiologic variables. One There were significant problems with technical integra- aspect of the enhancement would involve tracking markers tion, several of which were acknowledged by St. John and of cognitive state, such as respiration, heart rate, or eye his colleagues in Technical Integration experiment (2004). movement, using devices for electroencephalography (EEG) These included questions about the study design constructs or functional optical imaging. The operator would wear a and their external validity, the statistical methods applied to headset that contained such devices and might be connected the data, important data missing from the report of results, to other devices as well—perhaps galvanic skin response and the familiar problem in psychophysiological research of sensors or pressure sensors in a seat—interacting with them noisy data (substantial variability across test participants, test 

OCR for page 117
8 OPPORTUNITIeS IN NeUROSCIeNCe fOR fUTURe ARMY APPLICATIONS runs, etc.). Given the number and severity of confounding such a system and thus about the time frame for an initial factors, the results of the study are preliminary at best and operational capability. Unfortunately, no follow-on studies in no way constitute unequivocal scientific evidence that the have reported how the successful CSGs could or would CSGs identified as statistically significant can effectively be combined in an operational system. The engineering detect change in cognitive activity in a complex human super- obstacles to combining EEG, fNIR, and eye-tracking devices visory control task. are substantial. Unless dramatic leaps are made soon in the A second set of four experiments was conducted in miniaturization of these technologies and in improved signal- Phase 2 of AugCog. The stated objective of Phase 2 was to processing algorithms, the realization of a single headset that manipulate an operator’s cognitive state as a result of near- can combine all—or even a subset—of these technologies is real-time psychophysiological measurements (Dorneich et at least a decade away. al., 2005). The experiments used a video game environment Other engineering problems, such as how to measure to simulate military operations in urban terrain (MOUT) EEG signals in a dynamic, noisy environment, have not been in either a desktop setting or a motion-capture labora - addressed, at least in the open literature. Basic sensor system tory. In addition to the primary task, navigating through engineering problems like these will be critical to any opera- the MOUT, participants had to distinguish friends from tional deployment of these technologies. A similar engineer- foes while monitoring and responding to communications. ing problem underlay the use of the eye-tracking devices A communications scheduler, part of the Honeywell Joint assumed for AugCog applications. These devices currently Human–Automation Augmented Cognition System,1 deter- require a sophisticated head-tracking device in addition to the mined operator workload via a cognitive state profile (CSP) eye-tracking device, and encapsulating this technology into and prioritized incoming messages accordingly. The CSP an unobtrusive device that can be worn in the field appears was an amalgam of signals from cardiac interbeat interval, also to be at least 10 years in the future. heart rate, pupil diameter, EEG P300, cardiac quasi-random In addition to hardware limitations on the use of neural signal (QRS), and EEG power at the frontal (FCZ) and cen- and physiological technologies in an operational field setting, tral midline (CPZ) sites (Dorneich et al., 2005). the software/hardware suite required to interpret cognitive As in the Phase 1 experiment, there were only a few state reliably in real time is beyond current capabilities, participants (16 or fewer) in each of the four Phase 2 experi- particularly in the highly dynamic, stochastic settings typi- ments. Construct validity and statistical models were ques- cal of command-and-control environments. The experiments tionable, with significant experimental confounds. There is for the AugCog program were conducted under controlled no open account of how the neurological and physiological laboratory conditions. While this is to be expected for pre- variables were combined to form the CSP, making indepen- liminary, proof-of-concept studies, such a limitation con- dent peer-researched replication of these experiments diffi- strains the extrapolation of the reported results. For example, cult. In light of these concerns, claims such as a 100 percent the communications scheduler in the Phase 2 experiments improvement in message comprehension, a 125 percent made changes in information presentation based on gross improvement in situation awareness, a 150 percent increase differences in perceived cognitive state. In actual battlefield in working memory, and a more than 350 percent improve- conditions, the amount of task-relevant information and ment in survivability should be considered tentative. In addi- the degrees of freedom in cognitive state will require more tion, the authors claim anecdotal evidence that their CSGs precision and reliability in ascertaining an operator’s condi- can indicate operator inability to comprehend a message tion and making situation-appropriate adjustments rather (Dorneich et al., 2005). than limiting access, perhaps inappropriately, to information The focus in these experiments appears to have been that may be critical for a real-time decision. Not only must on generating measurable outcomes on a very tight time the sensors and signal-processing algorithms improve sub- schedule. Most of the technical data on the performance of stantially; significant advances are also needed in decision- the actual sensors and of the signal processing and combina- theoretic modeling. In particular, these models will have to tion algorithms were not published. This information would accommodate a significant range of individual variability. have been useful for further scientific evaluation and confir- Overall, the AugCog goal of enhancing operator perfor- mation of the reported results. mance through psychophysiological sensing and automation- The problem with AugCog as a development lies less based reasoning is desirable but faces major challenges as an in the intrinsic concept of managing cognitive workload active information filter. Suppose a system is implemented through neural and physiological feedback to a smart infor- that can change information streams and decrease the volume mation system than in the assumptions that were made about by filtering incoming information presented to a user. How the maturity of the technologies required to implement is the system to know that its filtering in a specific situa- tion is both helpful to this user and passes along the correct information for the current situation? The system software 1Honeywell was the prime contractor for the DARPA AugCog program must correctly determine an optimal cognitive load for an and remains the prime contractor for the follow-on Army program in aug - individual in a dynamic, highly uncertain context and decide mented cognition.

OCR for page 117
9 APPeNDIX D which information to emphasize and which information to viduals for a portion of the experiment. In addition, the minimize or even filter out altogether. The problem is that, approach to classifying cognitive state was extremely limited in command-and-control settings, there are no general prin - in state estimation (i.e., costs of actions were not considered), ciples for what information is truly optimal. Before deploy- and it depended on relatively short temporal gaps between ing such an active-filter system, which controls inputs to training and testing (Dorneich et al., 2007). This latter con- a military operator, rigorous testing and validation would straint in an operational setting means that soldiers would have to demonstrate that at the least, it does no worse than require extended training to “condition” the system before an unaided soldier. each mission. Since actual combat never follows a carefully planned script, an issue yet to be addressed is how a priori classification training can ever be based on events that are army FolloW-oN Work real enough to give reliable results. The ultimate problem is When the DARPA AugCog program formally ended, the not that the information filter might fail to accurately gauge Army continued working with portions of the concept at the the cognitive state of the user, but that it might act in a way U.S. Army Natick Soldier Research, Development and Engi- that results in a bad decision. neering Center. The original goal of the Army effort was to incorporate AugCog technology into the Army Future Force reFereNces Warrior Program by 2007 (U.S. Army, 2005). The primary Augmented Cognition International Society. 2008a. History: Emergence focus of the effort has since changed from operational to of augmented cognition. Available at http://www.augmentedcognition. training applications; moreover, the technology focus has com/history.htm. Last accessed August 17, 2008. narrowed to the use of EEG and electrocardiography sen- Augmented Cognition International Society. 2008b. Frequently Asked sors instead of the array of neurological and physiological Q uestions. Available at http://www.augmentedcognition.org/faq. sensors envisioned for the DARPA AugCog scheme (Boland, htm#q2. Last accessed August 17, 2008. Boland, R. 2008. Army uses advanced systems to understand what soldiers 2008). know. Signal. Fairfax, Va.: Armed Forces Communications and Elec- An experiment was conducted to extend the previ- tronics Association. ous DARPA effort, directed by the same Honeywell team Dorneich, M.C., P.M. Ververs, M. Santosh, and S.D. Whitlow. 2005. A joint that performed the set of experiments described above. human–automation cognitive system to support rapid decision-making This Army-sponsored experiment focused on developing in hostile environments. Pp. 2390- 2395 in IEEE International Confer- ence on Systems, Man and Cybernetics, Vol. 3. Los Alamitos, Calif.: an experimental test bed for mobile cognitive-state classi- IEEE Publications Office. fication and testing it in a dismounted-soldier field setting Dorneich, M.C., S.D. Whitlow, S. Mathan, P.M. Ververs, D. Erodogmus, A. using the previously discussed communications scheduler Adami, M. Pavel, and T. Lan. 2007. Supporting real-time cognitive state (Dorneich et al., 2007). The authors developed an EEG head- classification on a mobile system. Journal of Cognitive Engineering and set connected to a laptop computer worn in a backpack by Decision Making 1(3): 240-270. McDowell, P. 2002. The MOVES Institute’s Context Machine Project. the test subject. This laptop supported the signal processing Available online at http://www.movesinstitute.org/~mcdowell/augCog/. algorithms, the communications scheduler, and other experi- Last accessed May 12, 2008. mental testing elements. For the experiment, eight subjects Schmorrow, D., and D. McBride. 2004. Introduction: Special issue on aug - with no military experience completed a 1-hour navigation mented cognition. International Journal of Human-Computer Interaction and communication task with a handheld radio, a personal 17(2): 127-130. St. John, M., D.A. Kobus, and J.G. Morrison. 2003. DARPA Augmented digital assistant, and a 35-lb backpack. The authors reported Cognition Technical Integration Experiment (TIED). Technical Report that, with the communications scheduler prioritizing mes- 1905, December. San Diego, Calif.: U.S. Navy Space and Naval Warfare sages based on whether the subjects were in a low-task-load Systems Center. or high-task-load condition, mission performance metrics St. John, M., D.A. Kobus, J.G. Morrison, and D. Schmorrow. 2004. improved from 68 percent to 96 percent with cognitive-state Overview of the DARPA Augmented Cognition technical integration experiment. International Journal of Human–Computer Interaction mitigation. 17(2), 131-149. As with the earlier experiments, significant confounds U.S. Army. 2005. Mission: Augmented cognition. The Warrior. Natick, limited the validity of the results. Problems were reported Mass.: Public Affairs Office, U.S. Army Natick Soldier Research, Devel- with movement-induced signal noise, as well as significant opment and Engineering Center. loss of data that reduced the subject pool to just four indi-

OCR for page 117