Measuring Human Capabilities: An Agenda for Basic Research on the Assessment of Individual and Group Performance Potential for Military Accession (2015)

10

Assessment of Individual Differences Through Neuroscience Measures

Committee Conclusion: A wide variety of measures fall within the domain of neuroscience (e.g., direct neuroscience measures such as electroencephalography [EEG], positron emission tomography, magnetic resonance imagery [MRI], or functional MRI [fMRI] and indirect biomarkers of neural activity such as heart rate or eye blink). These measures may take multiple roles in the Army accession process including (a) monitoring test takers for constructs such as anxiety, attention, and motivation during other assessments; (b) use in research settings as criteria for evaluating other potential assessments; and (c) use as direct selection and classification assessments. Although the third role may be well in the future in terms of technically feasible and cost-effective assessment, the first two uses have near-term promise. The committee concludes that the neuroscience domain merits inclusion in a program of basic research with the long-term goal of improving the Army’s enlisted accession system.

INTRODUCTION

The breadth, scope, and history of neuroscience are well summarized in the context of Army applications in the 2009 National Research Council report, Opportunities in Neuroscience for Future Army Applications. In that report, the Army was encouraged to engage in research and development “to take best advantage of variations in the neural bases of behavior that contribute to performance” (p. 4).

The current study committee was charged with recommending a research agenda based upon the biological basis of individual differences

in behavior, including performance potential, and the determination of biomarkers indicating the state of biological systems that affect capability to perform. (See the complete Statement of Task in Chapter 1, Box 1-1; also see Appendix C for a more complete description of biomarkers.) Neuroscience methods may not be optimal for carrying out the job of testing recruits and classifying their performance capability—at least not at present. However, these methods do provide insight into how best to design test environments that allow recruits to perform up to their capacity, and they do have the potential to predict and even improve the learning of people in advance of test performance.

In this chapter, the committee uses the following measures, which it considers to incorporate aspects of neuroscience in the sense that they measure bodily functions that have been linked to psychological state or behavior. These measures include the following:

1. psychophysiological measures, such as heart rate and cardio rhythms, eye position, galvanic skin response, and pupil size;
2. neuroimaging measures, such as fMRI, electrical recording, and optical imaging with near infrared spectroscopy; and
3. biochemical measures, such as level of neuromodulators or hormones related to stress.

Appendix D provides a brief tutorial describing neural signals and their relationship to neuroimaging measures.

Most of the measures described above are not appropriate for routine use during large scale testing of recruits. It is often very difficult to interpret the results on an individual basis, and they can be influenced by many environmental factors. Moreover, their use with individuals may be seen as an invasion of privacy or could lead to even higher levels of stress than the testing itself. However, the committee believes there currently are less obtrusive methods, such as salivary measures of cortisol for monitoring stress during testing, that could benefit the Army’s selection research and practice, and future neuroscience methodologies may become less intrusive and more effective in application. More generally, we suggest that neuroscience methods be used in research and field testing in which the measures would be used with volunteer samples of recruits to help determine the optimal methods of behavior testing given logistical and technical constraints. These neuroscience measures could currently be used to shape and improve the behavioral testing environment, thus reducing stress and increasing attention during testing. In the future they may allow better prediction of which individual will be best able to learn the material to be tested, and they may help in the design of tests or interventions that reduce impediments due to

test anxiety, poor concentration, or other learning problems. We cover these possibilities below.

MONITORING TEST PERFORMANCE

For certain individuals, poor performance in a testing situation (especially timed high-stakes testing situations) can attribute to anxiety for reasons unrelated to the task, and several decades of theory and data support the idea that removal of off-task distraction (e.g., anxiety) through training can reduce interference with the primary task of test performance (Wine, 1971). More recent research has focused on performance failures at critical periods for lower-ability examinees (DeCaro et al., 2011). Although certain aspects of test anxiety can be appropriate due to the examinee’s accurate expectation of not performing well on a test in a given domain (e.g., anxiety in a test or performance for which the examinee has insufficient skill or knowledge), the committee views inappropriate levels of test anxiety during assessments as a potential validity threat to the Armed Services Vocational Aptitude Battery (ASVAB) and the Tailored Adaptive Personality Assessment System (TAPAS; see Chapter 1 for further discussion of these tests).

As discussed in Chapter 6, the ability to perform under stress is an important attribute in many situations that require effective performance under situational pressures such as time, evaluation, or danger. To be successful in an Army career, performance in a wide range of stress and anxiety circumstances is necessary (Hancock and Szalma, 2008). The concept of a physiological reaction to stress—called strain—dates back to the idea of self-regulation producing homeostasis; that is, changes to the external environment—called stressors—result in physiological changes that maintain a body’s internal environment within nominal operating parameters (Cannon, 1932). In this report, the committee focuses on measuring strain, the psychological consequences of stress. Psychological strain can be objectively defined as any action or situation that causes acute release of stress hormones, predominantly catecholamines and glucocorticoids.¹

An important difference exists between stress response and emotional response. Stress is the general application of an external change, and the stress response is the resulting release of hormones to cause some internal compensating change. Emotional responses of fear and anxiety are strains related to the fight or flight response (see also Chapter 6) with an increased state of arousal in the autonomic nervous system—for example, cardiac, respiratory, and perspiration changes. Stress can induce such emotions, and

_________________

¹ Catecholamines in the human body include epinephrine, norepinephrine, and dopamine. The most well-known glucocorticoid is cortisol, which is necessary for cardiovascular, metabolic, immunologic, and homeostatic functions.

some emotions in turn can cause stress. In the research literature, four basic characteristics of situations are emerging as common triggers for stress response in humans: novelty, unpredictability, threats to survival or ego, and low sense of control.

Anxiety, for example, is an emotional response to a stress that is often longer lasting than the duration of the stress that triggers it. The trigger stress can sometimes be difficult to identify or predict within aspects of a given situation (simple or complex; they are highly subjective), and therefore it can be difficult for people to self-manage anxiety or for situations to be modified to manage anxiety. Increases in anxiety tend to result in changes to the autonomic nervous system and to brain activity (van Stegeren, 2009). Stress induces changes to brain activity patterns, commonly in the anterior cingulate cortex and orbitofrontal regions (Cannon, 1932; Dedovic et al., 2009; Lupien, 2009; van Stegeren, 2009). Stress and anxiety are normal parts of human homeostatic reaction that is usefully incorporated into a model of adaptability (see Figure 10-1) historically in-

FIGURE 10-1 Physiological adaptive capability (blue line) and psychological adaptive capability (green line) can be equated with attentional resource capacity, plotted qualitatively here as a function of stress level. An individual’s performance will degrade at either end of the curve. Homeostatic pressure directs mental and physiological processes to move toward the center of the stress axis. There are also a central normative zone and a slightly more flexible comfort zone.
SOURCE: Hancock, P.A., and J.S. Warm. (1989). A dynamic model of stress and sustained attention. Human Factors, (31)5:528. Copyright SAGE Publications. ©Reproduced by permission of SAGE Publications.

spired by Yerkes-Dodson and proposed by Hancock (Hancock and Warm, 1989). In this model, ultralow or ultrahigh levels of anxiety or stress are outside the window of an individual’s ability to adapt (see also Chapter 7 for a discussion of adaptability).

It is possible to monitor levels of stress through the analysis of saliva for cortisol hormone levels (Hucklebridge et al., 1998). Testing situations can pose a cognitive challenge to some individuals that then lead to elevated hormonal levels within a few minutes of testing. By taking saliva samples before and after the test, one can examine the effect of stress level on test performance. Moreover, as described below, there are effective means for combatting the stress produced by cognitive challenges (Fan et al., 2014). These could be used with recruits who demonstrate an unusually strong response to the test situation, perhaps focusing on those whose performance appears out of line with predictions or other knowledge about the person.

DESIGNING BETTER TESTS AND TESTING ENVIRONMENTS

Comparing Environments

A testing situation is an inherently evaluative experience and therefore often a stressful one. However, both the environment in which the test is conducted and the format of the test itself can contribute to increased stress and anxiety. As a part of a quasi-experimental research effort, neuroscience measures such as salivary cortisol could be used to assess examinees in test environments that vary naturally. For example, environments that are hot or noisy, have poor ventilation, or require longer testing periods may be more stressful than others. Environments that appear similar may still lead to different stress responses because of the context in which stress level is measured, such as the time of day, the test taker’s preceding activities, or the presence of sleep deprivation.

In addition to the stress response, sympathetic and parasympathetic activity can be measured by use of heart rate variability measures (Axelrod et al., 1981). Scalp electrodes can be used to assess the presence of signs of attentive processing during the session (Posner, 2008).

Comparing Tests

How does cognitive activity vary as a function of different types of test content? One can measure the time to process information in the test by examining successive eye movements and dwell time on relevant information (Rayner and Pollatsek, 1989). To get a finer picture of the speed and location of brain activity in processing the test material, one can combine

EEG and neuroimaging methods to obtain the location and timing of brain activity, as has been done in the study of word and sentence processing (Posner et al., 1999). This information might enable improvements in the presentation and content of the test material used for recruits.

PREDICTING HUMAN PERFORMANCE

This section summarizes research supporting the possibility of predicting how recruits may perform in testing situations through neuroscience methods and also of modifying (mitigating) their stress reactions to testing. Although the committee cites evidence for each of these points, they remain areas of ongoing research that will require additional effort to develop the fundamental understanding necessary for long term potential application in assessment processes. However, the preliminary evidence offers great promise for the future application of neuroscience methods in improving predictability and performance by military personnel.

Brain Changes in Learning

Learning modifies the connectivity and activity of the brain. If, for example, one is presented with a word (e.g., “hammer”) and then is asked to develop a use for it (e.g., pounding) a set of neural areas become active. These include areas in the left frontal and temporal lobe, the anterior cingulate and portions of the right cerebellum. However, if one practices a few minutes with the same association, that pathway changes: the frontal, cingulate, and cerebellar activations diminish and the visual and motor brain areas that resemble those used in reading aloud remain (Raichle et al., 1994).

This finding is a good example of what happens when carrying out any task. There are a set of brain areas that are active, and they are orchestrated over a short period to produce behavior (Posner and Raichle, 1994). Practice changes the areas of activation, concentrates them, and improves the connections between them (Zatorre et al., 2012). These changes are reflected behaviorally in terms of a more predictable, proceduralized, and faster response.

Predicting Behavior from Brain Activity

Recently several forms of fMRI have been employed to predict performance in cognitive tasks. For example, in one study, resting-state fMRI was used to assess the functional connectivity between brain areas that were related to a visual discrimination task to be performed subsequently

by human participants (Baldassarre et al., 2012). The study found that resting-state connectivity predicted both initial performance and speed of learning in the subsequent task. A similar study in the motor system measured connectivity with high-density EEG and was able to predict the rate of subsequent motor learning (Wu et al., 2014). So far, only rather simple cognitive tasks have been predicted from the connectivity pattern. However, connectivity in brain areas related to executive attention and working memory (see Chapter 2) may have a robust influence on many tasks that are critically important in military situations, such as map reading, navigation, and decision making. Research into prediction of such high-level skills from patterns of connection derived from resting state MRI and other neuroscience methods would be of high priority.

Many studies (See Zatorre et al., 2012, for a summary) show that the act of learning a skill improves white matter connectivity between brain areas related to the task. It seems reasonable that assessment of the preexisting connectivity should predict the rate of improvement.

Although the studies above use tasks where the areas of activation and patterns of connectivity are known in advance, new statistical methods suggest it will be possible in the future to make predictions even in situations where there is little advanced knowledge of the brain areas and connectivity involved (Friston et al., 2003; Norman et al., 2006).

Another possible route to prediction is via measures of the neural systems and genetics related to individual differences in personality (Canli et al., 2001; DeYoung and Gray, 2009). It has been common to view differences in personality as enduring aspects that describe traits that can influence behavior (Goldberg and Rosolack, 1994). Efforts have been made to describe the neural systems involved and to understand the genetic basis of some of these traits (Canli and Lesch, 2007). This is a kind of construct validation process. One of the most important personality traits is conscientiousness, which is related to effort control in the research literature addressing the temperament of children (Rothbart, 2011). Measures of effortful control assessed in childhood have proven predictive of a wide range of behaviors and outcomes over the life span (Moffitt et al., 2011). Neural systems related to effortful control involve a network including the anterior cingulate, anterior insula, and underlying striatum (Posner et al., 2007). Some dopaminergic genes that influence the efficiency of this network have been studied. Research into effortful control suggests that higher effortful control relates to more functional adult behaviors, including the ability to learn new things (Posner and Rothbart, 2007) and emotion regulation (see Chapter 6). Thus, studies of the neural systems related to effortful control would be of great value in the evaluation of recruits.

Training

In addition to predicting which recruits will do better in subsequent tests, neuroscience measures have been important in evaluating two methods that might reduce stress and thereby potentially improve the ability to learn new material.

Training working memory is one method that has led to improvements in activation of lateral areas of the frontal and parietal lobes (Olesen et al., 2004). In initial studies, it was reported to transfer to general intelligence and other cognitive tasks (Jaeggi et al., 2008), although transfer to intelligence and other measures remote from the training remains a highly disputed issue, given subsequent data failing to find such empirical support (Redick et al., 2013).

Another method with potential for wide-ranging consequences for stress reduction as well as cognitive performance involves mindfulness meditation training (Tang and Posner, 2014). In one study, 1- to-4-week mindfulness training reduced stress to a cognitive challenge as measured by salivary cortisol secretion in a dose-dependent fashion (Fan et al., 2014). In other studies, 1 to 4 weeks of mindfulness meditation in comparison to a control group given relaxation training improved attention and mood and changed white matter connectivity between the anterior cingulate and other areas as measured by diffusion tensor imaging (Tang et al., 2012). As mentioned above, such white matter changes can lead to improvements in learning.

Given these two key methods and the promise for others to arise, the committee suggests further research designed to use neuroscience methods to improve the testing experience and training of recruits.

RESEARCH RECOMMENDATION

1. Research seeking refinement of current and future Army assessments (e.g., the Armed Services Vocational Aptitude Battery and Tailored Adaptive Personality Assessment System) through a deeper understanding of the constructs measured in selection and classification testing. In this role, biomarkers may reveal underlying neurophysiological correlates of constructs of interest (e.g., cortisol as a biomarker for anxiety). Deeper understanding of physiology has the potential to differentiate complex constructs or alternatively to reveal the relative strength of measures.
2. Independent use of biomarkers as direct selection instruments or their use in combination with traditional assessments. Research should identify biomarker correlates (e.g., consistent gaze, pupilometry, reaction time in a vigilance test) of abilities and outcomes. Test stimuli or conditions for eliciting biological responses from test takers (e.g., simulated rifle drill or other novel muscle coordination task to assess parietal-dominant brain) should be developed.

1. Monitoring candidates for attributes such as anxiety during assessment and offering training on mitigation strategies for applicants not selected on the basis of their test scores. Such attributes can contribute to bias in test scores, and success in controlling for the effects of these attributes can result in more valid assessment. The committee expects challenges in determining whether applicants’ observed performance reflects their true ability (e.g., whether applicants are experiencing normal performance stress or an interfering level of anxiety). Additionally, we expect challenges in designing a simple and effective mitigation program.
2. Basic research to apply modern neurophysiological tools to model test-taker response data (e.g., response time distributions, answer patterns that may suggest unmotivated responding or intentional distortion).

REFERENCES

Axelrod, S., D. Gordon, F.A. Ubel, D.C. Shannon, A.C. Barger, and R.J. Cohen. (1981). Power spectrum analysis of heart rate fluctuation: A quantitative probe of beat-to-beat cardiovascular control. Science, 213:220–222.

Baldassarre, A., C.M. Lewis, G. Committeri, A.Z. Snyder, G.L. Romani, and M. Corbetta. (2012). Individual variability in functional connectivity predicts performance of a perceptual task. Proceedings of the National Academy of Sciences of the United States of America, 109(9):3,516–3,521.

Canli, T., and K.P. Lesch. (2007). Long story short: The serotonin transporter in emotion regulation and social cognition. Nature Neuroscience, 10:1,103–1,109.

Canli, T., Z. Zhao, J.E. Desmond, E. Kang, J. Gross, and J.D.E. Gabrieli. (2001). An fMRI study of personality influences on brain reactivity to emotional stimuli. Behavioral Neuroscience, 115:33–42.

Cannon, W.B. (1932). The Wisdom of the Body. New York: W.W. Norton.

DeCaro, M.S., R.D. Thomas, N.B. Albert, and S.L. Beilock. (2011). Choking under pressure: Multiple routes to skill failure. Journal of Experimental Psychology General, 140(3):390–406.

Dedovic, K., A. Duchesne, J. Andrews, V. Engert, and J.C. Pruessner. (2009). The brain and the stress axis: The neural correlates of cortisol regulation in response to stress. Neuroimage, 47(3):864–871.

DeYoung, C.G., and J.R. Gray. (2009). Personality neuroscience: Explaining individual differences in affect, behavior, and cognition. In P.J. Corr and G. Matthews, Eds., Cambridge Handbook of Personality Psychology (pp. 323–346). New York: Cambridge University Press.

Fan, Y.X., Y.Y. Tang, and M.I. Posner. (2014). Cortisol level modulated by integrative meditation in a dose dependent fashion. Stress and Health, 30(1):65–70.

Friston, K.J., L. Harrison, and W. Penny. (2003) Dynamic causal modelling. Neuroimage, 19(4):1,273–1,302.

Goldberg, L., and T.K. Rosolack. (1994). The Big Five factor structure as an integrative framework. In C.F. Haverson, G.A. Kohnstamm, and R. Martin, Eds., The Developing Structure of Temperament and Personality from Infancy to Adulthood (pp. 7–35). Hillsdale, NJ: Lawrence Erlbaum Associates.

Hancock, P.A., and J.L. Szalma. (2008). Performance Under Stress. Aldershot, England; Burlington, VT: Ashgate.

Hancock, P.A., and J.S. Warm. (1989). A dynamic model of stress and sustained attention. Human Factors, 31(5):519–537.

Hucklebridge, F., A. Clow, and P. Evans. (1998). The relationship between salivary secretory immunoglobulin A and cortisol: Neuroendocrine response to awakening and the diurnal cycle. International Journal of Psychophysiology, 31:69–76.

Jaeggi, S.M., M. Buschkuehl, J. Jonides, and W.J. Perrig. (2008). Improving fluid intelligence with training on working memory. Proceedings of the National Academy of Sciences of the United States of America, 105(19):6,829–6,833.

Lupien, S.J. (2009). Brains under stress. Canadian Journal of Psychiatry, 54(1):4–5.

Moffitt, T.E., L. Arseneault, D. Belsky, N. Dickson, R.J. Hancox, H.L. Harrington, R. Houts, R. Poulton, B.W. Roberts, S. Ross, M.R. Sears, W.M. Thomson, and A. Caspi. (2011). A gradient of childhood self-control predicts health, wealth and public safety. Proceedings of the National Academy of Sciences of the United States of America, 108(7):2,693–2,698.

National Research Council. (2009). Opportunities in Neuroscience for Future Army Applications. Committee on Opportunities in Neuroscience for Future Army Applications. Board on Army Science and Technology, Division on Engineering and Physical Sciences. Washington, DC: The National Academies Press.

Norman, K.A., S.M. Polyn, G.J. Detre, and J.V. Haxby. (2006). Beyond mind-reading: Multivoxel pattern analysis of fMRI data. Trends in Cognitive Science, 10(9):424–430.

Olesen, P.J., H. Westerberg, and T. Klingberg. (2004). Increased prefrontal and parietal activity after training of working memory. Nature Neuroscience, 7:75–79.

Posner, M.I. (2008). Measuring alertness. In D.W Pfaff and B.L. Kieffer, Eds., Molecular and Biophysical Mechanisms of Arousal, Alertness and Attention (pp. 193–199). Boston, MA: Wiley Blackwell.

Posner, M.I., and M.E. Raichle. (1994). Images of Mind. Scientific American Library. New York: W.H. Freeman.

Posner, M.I., and M.K. Rothbart. (2007). Research on attention networks as a model for the integration of psychological science. Annual Review of Psychology, 58:1–23.

Posner, M.I., Y. Abdullaev, B.D. McCandliss, and S.E. Sereno. (1999). Anatomy, circuitry, and plasticity of word reading. In J. Everatt, Ed., Reading and Dyslexia: Visual and Attentional Processes (pp. 137–162). London: Routledge.

Posner, M.I., M.K. Rothbart, and B.E. Sheese. (2007). Attention genes. Developmental Science, 10:24–29.

Raichle, M.E., J.A. Fiez, T.O. Videen, A.M.K. McLeod, J.V. Pardo, P.T. Fox, and S.E. Petersen. (1994). Practice related changes in the human brain: Functional anatomy during non-motor learning. Cerebral Cortex, 4:8–26.

Rayner, K., and A. Pollatsek. (1989). Psychology of Reading. Englewood Cliffs, NJ: Prentice-Hall.

Redick, T.S., Z. Shipstead, T.L. Harrison, K.L. Hicks, D.E. Fried, D.Z. Hambrick, M.J. Kane, and R.W. Engle. (2013). No evidence of intelligence improvement after working memory training: A randomized, placebo-controlled study. Journal of Experimental Psychology General, 142:359–379.

Rothbart, M.K. (2011). Becoming Who We Are: Temperament, Personality, and Development. New York: Guilford Press.

Tang, Y.Y., and M.I. Posner. (2014). Training brain networks and states. Trends in Cognitive Sciences, 18(7):345–350.

Tang, Y.Y., Q. Lu, M. Fan, Y. Yang, and M.I. Posner. (2012). Mechanisms of white matter changes induced by meditation. Proceedings of the National Academy of Sciences of the United States of America, 109(26):10,570–10,574.

van Stegeren, A.H. (2009). Imaging stress effects on memory: A review of neuroimaging studies. Canadian Journal of Psychiatry, 54(1):16–27.

Wine, J. (1971). Test anxiety and direction of attention. Psychological Bulletin, 76(2):92–104.

Wu, J., R. Srinivasan, A. Kaur, and S.C. Cramer. (2014). Resting-state cortical connectivity predicts motor skill acquisition. Neuroimage, 91:84–90.

Zatorre, R.J., R.D. Fields, and H. Johansen-Berg. (2012). Plasticity in gray and white; Neuroimaging changes in brain structure during learning. Nature Neuroscience, 15:528–536.

This page intentionally left blank.