Opportunities in Neuroscience for Future Army Applications (2009)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Appendix D Research on Managing Information Overload in Soldiers Under Stress This appendix provides background information on by means of a traditional computer interface device such as research in augmented cognition (AugCog) conducted by a mouse or a joystick. Another aspect of enhancing cognitive the Defense Advanced Research Projects Agency (DARPA) ability would be dynamic control of the rate as well as the and the U.S. Army and discusses several of the neuroscience source of information by a “communications scheduler.” The research and engineering challenges. former DARPA program manager envisioned that technolo- gies developed by the program would be operational within 10 years. He predicted that within 20 years, the technology The DARPA AugCog Program would be woven into the fabric of our daily lives (Augmented DARPA began its research program in the area of Cognition International Society, 2008b). augmented cognition in 2001 with a focus on challenges Proof-of-concept work for AugCog occurred in two and opportunities presented by the real-time monitoring phases. In Phase 1, researchers attempted to detect changes of cognitive state with physiological sensors (Augmented in cognitive activity in near real time in an operationally Cognition International Society, 2008a). The program, now relevant setting. One relatively large demonstration, called known as the Improving Warfighter Information Intake the Technical Integration Experiment, was conducted with Under Stress Program, was initially called the Augmented mixed results (St. John et al., 2003). Its objective was to Cognition program, or AugCog. At first it had two stated determine which psychophysiological measurements could purposes: (1) to gain battlefield information superiority consistently detect changes in cognitive activity during a and (2) neurology-related clinical applications. Linked to supervisory control task. Using 20 gauges of cognitive state the idea of network-centric warfare, the stated rationale for (CSGs) such as EEG, functional near infrared (fNIR), and AugCog was that “order of magnitude increases in available, body posture, as well as input device measures (mouse pres- net thinking power resulting from linked human-machine sure and mouse clicks), eight participants completed a series d ­ yads will provide such clear information superiority that of four simplified aircraft monitoring and threat response few rational individuals or organizations would challenge tasks in 1 hour. Eleven of the CSGs, including fNIR and under the consequences of mortality” (Schmorrow and EEG measures, were reported to be significant and reliable M ­ cBride, 2004). Another early AugCog objective was stated for at least one independent variable, but no CSG was sig- as the enhancement of a single operator’s capabilities such nificant and reliable across all of three independent variables that he or she could carry out the functions of three or more (St. John et al., 2004). Two CSGs, mouse clicks and mouse individuals (McDowell, 2002). pressure, demonstrated statistically significant results across It was proposed that information superiority could be two of the independent variables, but these were probably achieved by enhancing an operator’s cognitive abilities and already highly correlated. by measuring neurological and physiologic variables. One There were significant problems with technical integra- aspect of the enhancement would involve tracking ­markers tion, several of which were acknowledged by St. John and of cognitive state, such as respiration, heart rate, or eye his colleagues in Technical Integration Experiment (2004). movement, using devices for electroencephalography (EEG) These included questions about the study design constructs or functional optical imaging. The operator would wear a and their external validity, the statistical methods applied to headset that contained such devices and might be connected the data, important data missing from the report of results, to other devices as well—perhaps galvanic skin response and the familiar problem in psychophysiological research of sensors or pressure sensors in a seat—interacting with them noisy data (substantial variability across test participants, test 117 

118 OPPORTUNITIES IN NEUROSCIENCE FOR FUTURE ARMY APPLICATIONS runs, etc.). Given the number and severity of confounding such a system and thus about the time frame for an initial factors, the results of the study are preliminary at best and operational capability. Unfortunately, no follow-on studies in no way constitute unequivocal scientific evidence that the have reported how the successful CSGs could or would CSGs identified as statistically significant can effectively be combined in an operational system. The engineering detect change in cognitive activity in a complex human super­ o ­ bstacles to combining EEG, fNIR, and eye-tracking devices visory control task. are substantial. Unless dramatic leaps are made soon in the A second set of four experiments was conducted in miniaturization of these technologies and in improved signal- Phase 2 of AugCog. The stated objective of Phase 2 was to processing algorithms, the realization of a single headset that manipulate an operator’s cognitive state as a result of near- can combine all—or even a subset—of these technologies is real-time psychophysiological measurements (Dorneich et at least a decade away. al., 2005). The experiments used a video game environment Other engineering problems, such as how to measure to simulate military operations in urban terrain (MOUT) EEG signals in a dynamic, noisy environment, have not been in either a desktop setting or a motion-capture labora- addressed, at least in the open literature. Basic sensor system tory. In addition to the primary task, navigating through engineering problems like these will be critical to any opera- the MOUT, participants had to distinguish friends from tional deployment of these technologies. A similar engineer- foes while monitoring and responding to communications. ing problem underlay the use of the eye-tracking devices A communications scheduler, part of the Honeywell Joint assumed for AugCog applications. These devices currently Human–­Automation Augmented Cognition System, deter- require a sophisticated head-tracking device in addition to the mined operator workload via a cognitive state profile (CSP) eye-tracking device, and encapsulating this technology into and prioritized incoming messages accordingly. The CSP an unobtrusive device that can be worn in the field appears was an amalgam of signals from cardiac interbeat interval, also to be at least 10 years in the future. heart rate, pupil diameter, EEG P300, cardiac quasi-random In addition to hardware limitations on the use of neural signal (QRS), and EEG power at the frontal (FCZ) and cen- and physiological technologies in an operational field setting, tral midline (CPZ) sites (Dorneich et al., 2005). the software/hardware suite required to interpret cognitive As in the Phase 1 experiment, there were only a few state reliably in real time is beyond current capabilities, participants (16 or fewer) in each of the four Phase 2 experi­ particularly in the highly dynamic, stochastic settings typi- ments. Construct validity and statistical models were ques- cal of command-and-control environments. The experiments tionable, with significant experimental confounds. There is for the AugCog program were conducted under controlled no open account of how the neurological and physiological laboratory conditions. While this is to be expected for pre- variables were combined to form the CSP, making indepen- liminary, proof-of-concept studies, such a limitation con- dent peer-researched replication of these experiments diffi- strains the extrapolation of the reported results. For example, cult. In light of these concerns, claims such as a 100 percent the communications scheduler in the Phase 2 experiments improvement in message comprehension, a 125 percent made changes in information presentation based on gross improvement in situation awareness, a 150 percent increase differences in perceived cognitive state. In actual battlefield in working memory, and a more than 350 percent improve- conditions, the amount of task-relevant information and ment in survivability should be considered tentative. In addi­ the degrees of freedom in cognitive state will require more tion, the authors claim anecdotal evidence that their CSGs precision and reliability in ascertaining an operator’s condi- can indicate operator inability to comprehend a message tion and making situation-appropriate adjustments rather (Dorneich et al., 2005). than limiting access, perhaps inappropriately, to information The focus in these experiments appears to have been that may be critical for a real-time decision. Not only must on generating measurable outcomes on a very tight time the sensors and signal-processing algorithms improve sub- s ­ chedule. Most of the technical data on the performance of stantially; significant advances are also needed in decision- the actual sensors and of the signal processing and combina- theoretic modeling. In particular, these models will have to tion algorithms were not published. This information would accommodate a significant range of individual variability. have been useful for further scientific evaluation and confir- Overall, the AugCog goal of enhancing operator perfor- mation of the reported results. mance through psychophysiological sensing and automation- The problem with AugCog as a development lies less based reasoning is desirable but faces major challenges as an in the intrinsic concept of managing cognitive workload active information filter. Suppose a system is implemented through neural and physiological feedback to a smart infor­ that can change information streams and decrease the volume mation system than in the assumptions that were made about by filtering incoming information presented to a user. How the maturity of the technologies required to implement is the system to know that its filtering in a specific situa- tion is both helpful to this user and passes along the correct information for the current situation? The system software Honeywell was the prime contractor for the DARPA AugCog program must correctly determine an optimal cognitive load for an and remains the prime contractor for the follow-on Army program in aug- mented cognition. individual in a dynamic, highly uncertain context and decide 

APPENDIX D 119 which information to emphasize and which information to viduals for a portion of the experiment. In addition, the minimize or even filter out altogether. The problem is that, a ­ pproach to classifying cognitive state was extremely limited in command-and-control settings, there are no general prin- in state estimation (i.e., costs of actions were not considered), ciples for what information is truly optimal. Before deploy­ and it depended on relatively short temporal gaps between ing such an active-filter system, which controls inputs to training and testing (Dorneich et al., 2007). This latter con- a military operator, rigorous testing and validation would straint in an operational setting means that soldiers would have to demonstrate that at the least, it does no worse than require extended training to “condition” the system before an unaided soldier. each mission. Since actual combat never follows a carefully planned script, an issue yet to be addressed is how a priori classification training can ever be based on events that are Army Follow-on Work real enough to give reliable results. The ultimate problem is When the DARPA AugCog program formally ended, the not that the information filter might fail to accurately gauge Army continued working with portions of the concept at the the cognitive state of the user, but that it might act in a way U.S. Army Natick Soldier Research, Development and Engi- that results in a bad decision. neering Center. The original goal of the Army effort was to incorporate AugCog technology into the Army Future Force References Warrior Program by 2007 (U.S. Army, 2005). The primary focus of the effort has since changed from operational to Augmented Cognition International Society. 2008a. History: Emergence of augmented cognition. Available at http://www.augmentedcognition. training applications; moreover, the technology focus has com/history.htm. Last accessed August 17, 2008. narrowed to the use of EEG and electrocardiography sen- Augmented Cognition International Society. 2008b. Frequently Asked sors instead of the array of neurological and physiological Questions. Available at http://www.augmentedcognition.org/faq. sensors envisioned for the DARPA AugCog scheme (Boland, htm#q2. Last accessed August 17, 2008. 2008). Boland, R. 2008. Army uses advanced systems to understand what soldiers know. Signal. Fairfax, Va.: Armed Forces Communications and Elec- An experiment was conducted to extend the previ- tronics Association. ous DARPA effort, directed by the same Honeywell team Dorneich, M.C., P.M. Ververs, M. Santosh, and S.D. Whitlow. 2005. A joint that performed the set of experiments described above. human–automation cognitive system to support rapid decision-making This Army-sponsored experiment focused on developing in hostile environments. Pp. 2390- 2395 in IEEE International Confer- an experi­mental test bed for mobile cognitive-state classi­ ence on Systems, Man and Cybernetics, Vol. 3. Los Alamitos, Calif.: IEEE Publications Office. fication and testing it in a dismounted-soldier field setting Dorneich, M.C., S.D. Whitlow, S. Mathan, P.M. Ververs, D. Erodogmus, A. using the previously discussed communications scheduler Adami, M. Pavel, and T. Lan. 2007. Supporting real-time cognitive state ( ­ Dorneich et al., 2007). The authors developed an EEG head- classification on a mobile system. Journal of Cognitive Engineering and set connected to a laptop computer worn in a backpack by Decision Making 1(3): 240-270. the test subject. This laptop supported the signal processing McDowell, P. 2002. The MOVES Institute’s Context Machine Project. Available online at http://www.movesinstitute.org/~mcdowell/augCog/. algorithms, the communications scheduler, and other experi- Last accessed May 12, 2008. mental testing elements. For the experiment, eight subjects Schmorrow, D., and D. McBride. 2004. Introduction: Special issue on aug- with no military experience completed a 1-hour navigation mented cognition. International Journal of Human-Computer Interaction and communication task with a handheld radio, a personal 17(2): 127-130. digital assistant, and a 35-lb backpack. The authors reported St. John, M., D.A. Kobus, and J.G. Morrison. 2003. DARPA Augmented Cognition Technical Integration Experiment (TIED). Technical Report that, with the communications scheduler prioritizing mes- 1905, December. San Diego, Calif.: U.S. Navy Space and Naval Warfare sages based on whether the subjects were in a low-task-load Systems Center. or high-task-load condition, mission performance metrics St. John, M., D.A. Kobus, J.G. Morrison, and D. Schmorrow. 2004. improved from 68 percent to 96 percent with cognitive-state Overview of the DARPA Augmented Cognition technical integration mitigation. experiment. International Journal of Human–Computer Interaction 17(2), 131-149. As with the earlier experiments, significant confounds U.S. Army. 2005. Mission: Augmented cognition. The Warrior. Natick, limited the validity of the results. Problems were reported Mass.: Public Affairs Office, U.S. Army Natick Soldier Research, Devel­ with movement-induced signal noise, as well as significant opment and Engineering Center. loss of data that reduced the subject pool to just four indi­