Measuring Human Capabilities: An Agenda for Basic Research on the Assessment of Individual and Group Performance Potential for Military Accession (2015)

2

Fluid Intelligence, Working Memory Capacity, Executive Attention, and Inhibitory Control

Committee Conclusion: The constructs of fluid intelligence (novel reasoning), working memory capacity, executive attention, and inhibitory control are important to a wide range of situations relevant to the military, from initial selection, selection for a particular job, and training regimes to issues having to do with emotional, behavioral, and impulse control in individuals after accession. These constructs reflect a range of cognitive, personality, and physiological dimensions that are largely unused in current assessment regimes. The committee concludes that these topics merit inclusion in a program of basic research with the long-term goal of improving the Army’s enlisted accession system.

The committee considers the areas of fluid intelligence, working memory capacity, executive attention, and inhibitory control as offering new constructs for the Army’s consideration, even though some aspects of these ideas have been studied for decades. The newer research brings these several heretofore separate topics together and extends the relevance of the constructs beyond performance on specific tasks to broader issues of cognitive and emotional control. These topics are presented in a single chapter because there is considerable evidence that they overlap in terms of their theoretical motivations and definitions, their measurement, their variance, and their patterns of prediction. These topics are also brought together because, at the same time these overlaps are evident, future research must determine whether these various constructs reflect a single common mechanism or highly related but separate psychological mechanisms that might play different roles in the regulation of behavior, thought, and emotion.

If the latter hypothesis is supported, then a second issue is whether much more specific assessment of those separate mechanisms can add predictive validity for performance in the jobs for which potential military recruits are assessed.

Each section of the chapter begins with a brief history about one or more of the constructs listed in the title, focusing on how research on these constructs has converged and diverged over time. It then presents findings from various researchers who have studied these issues most recently, describes the evidence for the validity of the constructs in predicting performance of real-world tasks, and discusses the transition of what has been fairly basic research agenda on these topics to a more testing-oriented agenda. The sections end with a discussion of questions that should be addressed in future projects.

FLUID INTELLIGENCE

The idea that intelligence could be thought of as a general and therefore domain-free variable dates back at least to Spearman (1904). However, the idea that fluid and crystallized intelligence were separable was proposed by Spearman’s student Raymond Cattell (1941) and elaborated by Cattell and his student John Horn (Horn and Cattell, 1966a, 1966b). As described in Cattell’s biography by the website Human Intelligence:¹

Fluid intelligence (Gf) is important for reasoning and novel problem solving, and there is strong and emerging evidence that it represents the heritable and biological aspect of intelligence (Plomin et al., 2008; Wright et al., 2007). Longitudinal and cross-sectional studies across the life span have repeatedly shown that, while crystallized intelligence—the culturally derived knowledge aspect of intelligence—remains high and even increases

_________________

¹ Available: http://www.intelltheory.com/rcattell.shtml [January 2015].

over the life span, Gf declines over age (Horn and Cattell, 1967). In addition, individual differences in fluid intelligence (i.e., rank-order differences) appear to be quite stable over the life span (Deary et al., 2009, 2012). For example, Deary and his colleagues in the Lothian cohort studies made use of the fact that over 150,000 11-year-olds in the Lothian region of Scotland were tested for intelligence (IQ scores) more than 50 years ago and many of those individuals have been available for testing in recent years. Recently, Deary and colleagues (2012) conducted a genome-wide complex trait analysis on this sample and found a genetic correlation of 0.62 between intelligence in childhood and in old age. Furthermore, it appears that this relationship is higher for the lower quartile of abilities than for the upper quartile, which suggests that a more complete understanding of this relationship would be important for the selection and assignment of enlisted personnel.

The validity of fluid measures has been demonstrated for military-related tasks such as air traffic control (Ackerman and Cianciolo, 2002) and multitasking (Hambrick et al., 2010, 2011). The long-term stability and validity of fluid measures have been demonstrated in a sustained program of studies by David Lubinski and Camilla Benbow (2000, 2006). They started with a sample of 13-year-olds identified as being in the top 1 percent of individuals on measures of verbal and mathematical reasoning and tracked those individuals into middle adulthood (Lubinski and Benbow, 2006). Scores on these measures substantially predicted accomplishments in a wide array of domains in middle adulthood. Even at the highest levels, the scores obtained at age 13 predicted the number of patents, academic publications, and achievement in science and business at later ages.

The distinction between fluid and crystallized abilities becomes critically important in selection for the military. Recent papers have suggested that the Armed Services Vocational Aptitude Battery (ASVAB) is largely crystallized and that incremental validity can be added with measures of working memory capacity and fluid intelligence. The ASVAB does include a spatial ability subtest (Assembling Objects) which reflects a fluid ability in the typical examinee population (see Chapter 4, Spatial Abilities, for further discussion). Roberts and colleagues (2000) reported two studies, with a total of 7,100 subjects, showing that the ASVAB largely reflects acculturated learning and minimally reflects fluid abilities (Gf). Hambrick and colleagues (2011) had Navy sailors perform a synthetic work task that simulated the multitasking demands of many different jobs. While the ASVAB did predict performance on this task, the ability to update working memory accounted for even more variance in the prediction of multitasking and synthetic work. Future research will be important to improve understanding of the mechanisms underlying fluid abilities and the differences between the

mechanisms of working memory and fluid intelligence, including measures of these constructs as potential supplemental tests to the ASVAB.

There is ongoing military interest in and research on measures of fluid abilities. An expert panel charged with a review of the ASVAB recommended consideration of existing and new measures of fluid abilities as potential additions to the ASVAB (Drasgow et al., 2006). Alderton and colleagues (1997) examined a battery of tests in the spatial ability and working memory domains, administered in conjunction with the ASVAB. Their data show that Assembling Objects has a substantial loading on a general factor, as well as loading on a specific spatial ability factor. Thus, although it does indeed reflect a measure in the fluid abilities domain, it is likely not the best measure of fluid intelligence. Nonverbal reasoning tests, such as matrix tests, commonly produce very high general factor loadings, and a matrix test will be administered to all military applicants starting in April 2015 (see Russell et al., 2014).

The psychological and biological mechanisms reflected in standard tests of fluid intelligence and responsible for individual differences in the construct have been largely ignored in the psychometric literature and only recently have been addressed in the cognitive psychology and neuroscience literature. This lack of understanding of the specific cognitive abilities and the underlying biomarkers reflected in fluid intelligence is a gap in knowledge that it is important to fill to maximize the benefits of such assessments. If, for example, fluid intelligence is a composite of several underlying specific cognitive abilities it would be extremely useful to know whether those abilities are differentially related to various criterion measures and whether they might interact in some way that would be important to assess.

WORKING MEMORY CAPACITY AND EXECUTIVE ATTENTION

Measures of memory span (short-term memory) have been used to study memory abilities since Ebbinghaus (see Dempster, 1981). The first publication of a study using memory span as a measure (Jacobs, 1887) reported a strong relationship between a child’s memory span and rank in class, and Francis Galton himself (1887) observed that few mentally deficient individuals could recall more than two items in a span test. Simple memory span tasks have been included in most large-scale tests of intelligence. Thus, from the beginning, what came to be called short-term memory appeared to reflect important individual differences in higher-order cognitive functions. The emergence of short-term memory as a major construct in cognitive psychology was predicated largely on research using span-like tasks, meaning that most of the work was done using serial recall of short lists of digits, letters, or words and with the same pool of items used over and over across lists. Crowder (1982), in a paper titled “The De-

mise of Short-term Memory,” argued against two separate memory stores, and one of his arguments was based on the lack of relationship between measures of short-term memory and measures of real-world cognition. If short-term memory was important to real-world cognition, then individual differences in measures of that memory should correspond to individual differences in reading, learning, decision making, etc., and there was little evidence supporting that conclusion.

The picture clarified substantially when complex span measures were shown to have quite substantial correlations with reading and listening comprehension (Daneman and Carpenter, 1980; Engle and Kane, 2004). Examples of two complex spans alongside a simple letter span task, all of which require manipulation and remembering of verbal materials, are shown in Figure 2-1. In the reading span task, the subject is to read aloud the sentence and decide whether the sentence makes sense. That is followed by a letter to recall. In the operation span task, the subject is to calculate whether the equation is correct and then see a letter to recall. After two to seven such items, the subject is shown a set of question marks and asked to recall the to-be-remembered items.

Complex tasks may also involve the manipulation and remembering of nonverbal information such as the tasks in Figure 2-2. These tasks require the subject to make a decision about a pattern such as whether the rotated

FIGURE 2-1 Example of a simple span task, a reading span task, and an operation span task.
NOTE: WMC = working memory capacity.
SOURCE: Engle, Randall W. (2010). Role of working memory capacity in cognitive control. Current Anthropology, 51(S1):S17–S26. Reproduced by permission of and published by The University of Chicago Press.

FIGURE 2-2 Three different spatial tasks.
NOTE: WMC = working memory capacity
SOURCE: Kane et al. (2004, p. 196).

block letter would be a correct letter when upright or whether the figure is symmetrical around a vertical axis. Each decision is followed by an item to be remembered such as the arrow pointing in one of eight directions and being one of two lengths, or a cell in a matrix.

One might think that tasks that differ as widely as those in Figures 2-1 and 2-2 would yield very different predictive validity for higher level tasks, but that is not the case. As shown in Figure 2-3, a huge array of such tasks has been shown to reflect a coherent latent factor. Further, that latent factor, typically called “working memory capacity” (WMC), has a very high relationship to the construct for fluid intelligence.

The wide array of WMC tasks have been shown to be quite valid in predicting performance on a huge variety of real-world cognitive tasks. Quoting from Engle and Kane (2004, p. 153):

FIGURE 2-3 Path model for structural equation analysis of the relation between working memory capacity and reasoning factors.
SOURCE: Kane et al. (2004, p. 205).

The expert panel charged with a review of the ASVAB, described in the previous discussion of fluid abilities, also recommended consideration of working memory measures as potential additions to the ASVAB (Drasgow et al., 2006). Previously, Alderton and colleagues (1997) examined a battery of tests that included working memory measures, administered in conjunction with the ASVAB. Sager and colleagues (1997) offered evidence of the validity of working memory measures in this battery for predicting military training outcomes. Furthermore, a working memory test from this battery is currently being administered to Navy applicants (see Russell et al., 2014). Working memory measures were also explored in Project A, the Army’s large-scale exploration of the relationship between a broad array of individual-differences constructs and various performance domains (Russell and Peterson, 2001; Russell et al., 2001).

Although the construct under discussion here is typically referred to as working memory capacity, there is strong and emerging evidence that the critical factor for regulation of thought and emotion is the ability to control one’s attention, often referred to as executive attention (EA). EA refers to the ability to prevent attention capture by both endogenous and exogenous events (Engle and Kane, 2004). Individuals assessed to have lower EA are thought to be more likely to allow internally or externally generated events to capture their attention from tasks currently being performed. Thus, studies will often use the same tasks developed to measure WMC but will refer to the construct as Executive Attention.

There is a strong connection between the measures of WMC described above and measures of attention such as the Stroop task, antisaccade task, dichotic listening, and the flanker task. In an example of the antisaccade task, subjects stare at a fixation point on a computer screen while there are two boxes 11 degrees to each side of the fixation. At some point, one of the boxes will flicker and the subject is to look at the box on the opposite side of the screen. The flickering box affords movement, and evolution has predisposed us to look at that box since things that move have possible survival consequences. Performance can be measured either by eye movement analysis or by having the subject identify a briefly presented item in the box opposite to the flickering box (Kane et al., 2001; Unsworth et al., 2004); in both cases low WMC individuals are nearly twice as likely to make an error and glance at the flickering box. In the dichotic listening task, low WMC individuals are more than three times more likely than high WMC individuals to hear their name in the to-be-ignored ear.

The strong relationship of performance on these low-level attention tasks to the WMC tasks suggests that EA is likely to play a crucial role in both types of tasks. We do note that although EA is conceptualized as a cognitive ability, the pattern of relationships among various WMC tasks may also result from differences across participants in the degree of en-

gagement with the tasks. Attributing relationships to EA differences alone requires the assumption of a common level of task motivation (usually a high level is assumed).

The concept of individual differences in WMC/EA has been used in explanations of psychopathologies such as alcoholism and schizophrenia. For example, Finn (2002) proposed a cognitive-motivational theory of vulnerability to alcoholism in which one key factor is WMC/EA. He argued that greater WMC allows an individual to better manipulate, monitor, and control the behavioral tendencies resulting from alcoholism, and that this directly affects the ability to resist a prepotent behavior such as taking a drink in spite of being aware that such behavior is ultimately maladaptive. Individual differences in WMC/EA have also been shown to be important in emotion regulation (Hofmann et al., 2011). Thus, assessment of whether individuals are likely to be more or less able to control impulses and self-destructive thoughts would benefit from inclusion of WMC measures.

The linkages between EA and impulse control suggest that examinations of EA may benefit from examining relations with self-control measures in the personality domain to determine the degree of overlap and potential incremental validity of one over the other. Recent studies have shown that the tendency to mind-wander during performance of a critical task is highly associated with measures of WMC (McVay and Kane, 2009, 2012a, 2012b). These researchers used a variety of techniques to measure what they called task-unrelated thoughts during performance of complex tasks. In one study (Kane et al., 2007), subjects carried a Palm Pilot² and were alerted eight random times over the course of their day to answer questions about the tasks they were currently performing, their level of concentration, how challenging the task was, how much effort they were expending, and whether their mind had wandered in the last few minutes. The results in Figure 2-4 show clearly that low and high WMC individuals differed greatly in their tendency to mind-wander and that the differences grew as more concentration was required in the task and the task became more challenging. Low WMC individuals are more likely to mind-wander as a task increases in challenge and effort level required. One question that could be investigated through future research would be the cause or effect related to whether mind wandering is a consequence of task difficulty and WMC or a predictor of WMC (suggesting that mind wandering is a consequence rather than a cause of WMC performance). These differences in performance would seem to be generalizable to a wide range of tasks performed in the Army across the full spectrum of operations from peacetime to combat situations.

_________________

² Palm Pilot was an early personal digital assistant that could be set up with multiple alarms and short interactive response-entry actions.

FIGURE 2-4 High versus low WMC individuals and task-unrelated thoughts in daily life.
NOTE: Values on the y-axis represent the mind wandering dependent variable, scored on each questionnaire as either 1 (for mind wandering) or 2 for on-task thoughts; lower values thus indicate more mind wandering. Values on the x-axis represent group-centered ratings for (a) concentration (“I had been trying to concentrate on what I was doing”), (b) challenge (“What I’m doing right now is challenging”), and (c) effort (“It takes a lot of effort to do this activity”).
SOURCE: Kane, J.J., L.H. Brown, J.C. McVay, I. Myin-Germeys, P.J. Silva, and T.R. Kwapil. (2007). For whom the mind wanders, and when: An experience-sampling study of working memory and executive control in daily life. Psychological Science, 18(7):167. Reproduced by permission of SAGE Publications.

While a general mental abilities (i.e., Gf) approach is useful and has been considered the gold standard for predicting job performance (Schmidt and Hunter, 1998), recent work in this area suggests the importance of WMC in such predictions. In particular, WMC has been found to capture specific aptitudes beyond general mental abilities (Bosco and Allen, 2011; Hambrick et al., 2010; König et al., 2005). A recent study by König and colleagues (2005) testing 122 college students found that WMC was the best predictor of multitasking (similar conclusions were supported by Damos, 1993; Hambrick et al., 2010, 2011; and Stankov et al., 1989). These studies also showed WMC remained predictive of multitasking performance after controlling for fluid intelligence. In hierarchical regression analyses, WMC demonstrated the highest correlations with several measures of multitasking and predicted the most unique variance (Hambrick et al., 2010, 2011). Other research has found that WMC and Gf are distinct but strongly related (Kane et al., 2005).

Another perspective on assessments of WMC and EA is that, although they have great validity in predicting performance in real-world job situations, some research indicates they produce smaller mean racial/ethnic

group differences than do measures of crystallized ability. Subgroup differences contribute to adverse impact, a violation of Title VII of the 1964 Civil Rights Act. Under that statute, a violation of Title VII³ may be demonstrated by showing that an employment practice or policy has a disproportionately adverse effect on members of the protected class as compared with nonmembers of the protected class. Such impact is only acceptable to the extent that the practice is proven to be germane to the job being selected for. In other words, a test that has good validity and low adverse impact against a protected class is preferred over one that has good validity but has higher adverse impact.

A series of studies (Bosco and Allen, 2011) compared the EA battery developed by the Engle lab (Engle and Kane, 2004) with the Wonderlic test in terms of ability to predict job performance and associated adverse impact due to race (i.e., different mean scores for the two racial groups on the test). In three different studies, respectively involving college students, MBA students, and individuals working in a large financial firm, Bosco and Allen found that the EA battery accounted for greater variance in task or job performance than the Wonderlic test and had substantially less adverse impact. The EA battery predicted an additional 7.2 percent of the variance beyond the Wonderlic on the job simulation task, as well as an additional 5.2 percent of the variance in supervisor ratings of job performance. The reduced adverse impact for the EA battery was also found for supervisory ratings of managers in the workplace environment.

These findings are intriguing enough to mention; however, they are based on modest sample sizes, and additional replication is needed to solidify the basis of these findings. Verive and McDaniel (1996) report a meta-analysis of short-term memory tests on nearly 28,000 subjects and found that the black-white difference was less than half what it is on typical general cognitive ability tests, and yet the validity estimates remained high: .41 for job performance and .49 for training performance. Again, although interesting, the committee does not view these results as definitive. For example, the meta-analysis relies on untested assumptions about the degree of range restriction in the samples, and there is variance associated with these meta-analytic mean estimates that deserves to be understood.

Because short-term memory tests have been shown to be relatively unreliable and have reduced validity compared to measures of working memory capacity and executive attention (Engle et al., 1999a, 1999b), one might expect the latter measures to be even more resistant to adverse impact. This is consistent with recent work by Redick and colleagues (2012) in which gender differences were shown to be minimal on working memory complex span tasks over a sample size of 6,000 young adults.

_________________

³ See 42 U.S.C. § 2000e-2. Available: http://www.eeoc.gov/laws/statutes/titlevii.cfm [December 2014].

Thus, the WMC/EA approach to assessment appears to provide substantial incremental validity for specific job situations and yet is less influenced by race or ethnic group. This tentative finding would seem to be particularly important for the modern Army situation but clearly needs further study and development, including research into cost-effective large-scale testing mechanisms suitable for administration in mobile or other non-laboratory settings without compromising validity, reliability, or test security. (See Section 5 of this report, Methods and Methodology, for further discussion of research topics to facilitate such developments.) In developing a future research program, it is important to recognize that although much research has been conducted on the constructs of fluid intelligence, WMC, and EA, research on the relationship between WMC and fluid intelligence is a relatively new and incomplete endeavor that combines two typically parallel research approaches: experimental and differential. Bringing these research approaches under one roof will improve the identification and understanding of the mechanisms responsible for the constructs of WMC, fluid intelligence, and EA, thus making significant contributions to the basic understanding of individual differences.

Research Recommendation: Fluid Intelligence, Working Memory Capacity, and Executive Attention

1. A. Research should be conducted to ascertain whether these constructs reflect a common mechanism or are highly related but distinct mechanisms.
2. B. Assessments reflecting the results of research into the commonality versus distinctness of these constructs should be developed for purposes of validity investigations.
3. C. Ultimately, the basic research results from items A and B above should be used to inform research into time-efficient, computer-automated assessment(s).

INHIBITORY CONTROL

The research on WMC/EA described above illustrates how measures based on tasks conducted in the laboratory (“lab task measures”) can be used to index individual differences in cognitive control or executive

capacity that contribute to performance in various contexts. This body of cognitive-performance work represents an important extension of traditional personality-oriented research on variations in the tendency to restrain versus express impulses and emotions—research reflected in psychological constructs ranging from “ego control” (Block and Block, 1980) to “constraint” (Tellegen, 1985), “novelty seeking” (Cloninger, 1987), and “syndromes of disinhibition” (Gorenstein and Newman, 1980; Patterson and Newman, 1993). It would be useful to be able to predict with some accuracy those individuals who have difficulty controlling impulses for unacceptable behavior—that is, predicting cognitive, personality, and emotional characteristics that might lead to inappropriate or unacceptable behavior of the sort that has implications for an individual’s military career or mission success.

Variations in performance on WMC/EA tasks and personality scale measures of impulsivity versus restraint can be viewed as indexing a common individual-differences construct. As evidence for this, capacities associated with WMC/EA appear to play a crucial role in the blocking or inhibition of intrusive thoughts (Brewin and Holmes, 2003). For example, individual differences in WMC are related to the ability to prevent unwanted information from intruding into consciousness and negatively affecting task performance. Individuals with greater measured WMC are better at suppressing unwanted thoughts when instructed to do so under experimental conditions, whether these thoughts are neutral (Brewin and Beaton, 2002) or obsessional (Brewin and Smart, 2005). These findings may help to explain why low intelligence, which is strongly correlated with WMC, is a risk factor for posttraumatic stress disorder (PTSD; Brewin et al., 2000). This relationship is particularly important to understand better, given the increasing number of members of the military reporting PTSD.

Other recent research (Patrick et al., 2012, 2013a, 2013b) indicates that assessment of inhibitory control can be extended to include physiological response measures, which may be of value for understanding processes underlying effective performance as well as adding to prediction of performance outcomes. Anterior brain structures, including regions of prefrontal cortex (Blumer and Benson, 1975; Damasio et al., 1990) and the anterior cingulate cortex, appear crucial for inhibitory control. The prefrontal cortex is theorized to be important for “top-down” processing, that is, guidance of behavior by internal representations of goals or states (Cohen and Servar-Schreiber, 1992; Miller, 1999; Wise et al., 1996). The anterior cingulate cortex has been conceptualized as a system that invokes the control functions of the prefrontal cortex as needed to successfully perform a task, either by detecting errors as they occur (Gehring et al., 1995; Scheffers et al., 1996), by monitoring conflict among competing response tendencies (Carter et al., 1998), or by estimating the likelihood of committing an er-

ror at the time a response is called for (Brown and Braver, 2005). Given the evidence for a brain basis to executive capacity, it should be possible to quantify individual differences in inhibitory control through brain response measures as well as through personality scale or lab performance measures.

However, some major challenges exist to incorporating physiological measures into assessment of individual-differences constructs like inhibitory control. In particular, while prominent models of personality include reference to neurobiological systems, the models themselves are based primarily on self-report personality data, with ideas about their connections to neurobiology formulated subsequently. As a consequence of this: (1) physiological variables tend to correlate only modestly with personality scale scores, as expected of measures from differing domains (cf. Campbell and Fiske, 1959), and (2) existing conceptions of individual differences tend to persist unaltered, rather than being reshaped by neurobiological findings.

A strategy for addressing these challenges as related to assessment of individual differences pertinent to performance in real-world contexts is the psychoneurometric approach (Patrick and Bernat, 2010; Patrick et al., 2012, 2013a, 2013b). This approach is grounded in classic perspectives on psychological assessment, which conceive of dispositional tendencies as constructs that transcend specific domains of measurement (Cronbach and Meehl, 1955; Loevinger, 1957). Viewed this way, ideas regarding the nature of a trait construct and how to measure it are considered provisional and subject to modification based on data.

Figure 2-5 depicts the psychoneurometric approach as applied to the individual-differences construct of inhibitory control, which can be operationalized psychometrically as disinhibition versus restraint (Krueger et al., 2007; see discussion below) or behaviorally (as discussed above) as performance on lab tasks that index cognitive control or executive capacity. The first step in the approach entails identifying reliable physiological indicators (Phys_var1, Phys_var2, etc., in Figure 2-5) of the target construct operationalized psychometrically—in the case of this illustration, as scores on a self-report measure of disinhibitory tendencies (i.e., disinhibition scale shown as Cont_DIS in Figure 2-5). The next step entails mapping the interrelations among physiological variables known to correlate with the disinhibition scale measure to (1) establish a statistically reliable neurometric measure of inhibitory control (shown as Cont_neurometric in Figure 2-5) and (2) develop understanding of brain circuits/processes associated with individual differences in inhibitory control. Knowledge gained about the convergence of multiple physiological indicators from different experimental tasks—and about brain mechanisms underlying this convergence—in turn feeds back into conceptualization and psychometric measurement of the target construct (large curved arrow on left side of Figure 2-5).

The Externalizing Spectrum Inventory (ESI) provides a comprehensive

FIGURE 2-5 The psychoneurometric approach as applied to the individual-differences construct of inhibitory control.
NOTES: Cont_DIS = construct of inhibitory control as assessed by self-report (i.e., disinhibition scale).
Cont_neurometric = construct of inhibitory control as assessed by a composite of interrelated neurophysiological variables.
Phys_var = physiological variable known to correlate reliably with inhibitory control as assessed by self-report.
SOURCE: Patrick, C.J., C.E. Durbin, and J.S. Moser. (2012). Reconceptualizing antisocial deviance in neurobehavioral terms. Development and Psychopathology, 24(3):1,064. Reproduced by permission of Cambridge University Press.

approach to assessing individual differences in inhibitory control through self-report (Krueger et al., 2007; Patrick et al., 2013a). It comprises 23 unidimensional subscales indexing tendencies toward impulsivity versus planful control, irresponsibility versus dependability, aggression in various forms versus empathic concern, fraudulence versus honesty, excitement seeking, rebelliousness and blame externalization, and use/abuse of alcohol and other drugs. As shown in Figure 2-6, the subscales of the ESI exhibit a

FIGURE 2-6 A schematic of the best fitting confirmatory bifactor model of the ESI (Krueger et al., 2007). The model is represented schematically because the 23 subscales of the ESI included in the model are too numerous to depict effectively in full.
NOTE: ESI = externalizing spectrum inventory; EXT = externalizing; S = scale, where the subscript numbers represent differing subscales.
SOURCE: Patrick, C.J., C.E. Durbin, and J.S. Moser. (2012). Reconceptualizing antisocial deviance in neurobehavioral terms. Development and Psychopathology, 24(3):1,050. Reproduced by permission of Cambridge University Press.

bifactor structure, with all scales loading on a general factor (externalizing, or disinhibition), and certain scales also loading on separate subfactors reflecting callous aggression and addiction proneness. Variations in general tendencies toward impulsiveness versus restraint associated with the broad disinhibition factor can be assessed using a brief scale consisting of 20 ESI items, referred to as DIS-20. This disinhibition scale does not include any aggression- or substance-related items from the ESI, but it nonetheless strongly predicts tendencies toward antisocial-aggressive behavior and substance problems (Patrick et al., 2012, 2013a, 2013b). It is not known whether the ESI will yield the same results in high-stakes testing situations, and the validity and susceptibility to faking or coaching is unknown.

The construct of inhibitory control has well-established brain correlates. According to Patrick and colleagues (2006), the best known indicator of this type is reduced amplitude of the P3 (or P300) brain potential response to task-relevant stimuli in the widely used ‘oddball’ task. They presented evidence that reduced P3 amplitude reflects general externalizing proneness (maladaptive acting out) as indexed by disorder symptoms. Differences between subjects high and low in disinhibition have also been shown for error-related negativity (ERN), the brain potential response that occurs when subjects make an error on cognitive tasks. Hall and colleagues (2007) demonstrated a negative relationship between amplitude of the ERN

in a flanker task and levels of disinhibition as indexed by the ESI. This finding was replicated in subjects assessed for disinhibitory tendencies using the DIS-20 scale (Patrick et al., 2012); Figure 2-7 depicts average ERN waveforms for high versus low DIS-20 scorers based on a median split. Importantly, variations in inhibitory control assessed in these ways show correlations with lab task measures of executive capacity as well as with brain response measures. For example, in a study of twins, Young and colleagues (2009) reported a genetic correlation of -0.6 between disinhibitory tendencies as assessed by personality-trait and clinical-symptom measures and executive capacity as indexed by performance on WMC/EA tasks (i.e., heritable variance in disinhibitory tendencies was associated inversely, to a substantial degree, with heritable variance in executive capacity). Further-

FIGURE 2-7 Mean error-related negativity (ERN) waveform for individuals high as compared to low in disinhibitory tendencies (i.e., above versus below the median on a 20-item disinhibition scale). The ERN (circled) reflects self-recognition of erroneous responses within a performance task (in this case, a speeded stimulus discrimination procedure).
SOURCE: Patrick, C.J., C.E. Durbin, and J.S. Moser. (2012). Reconceptualizing antisocial deviance in neurobehavioral terms. Development and Psychopathology, 24(3):1,057. Reproduced by permission of Cambridge University Press.

more, the likely overlap between inhibitory control capacity and individual differences in WMC/EA would be important to examine through future research to identify ways in which they are correlated or distinct.

Extending work on brain correlates of inhibitory control, Patrick and colleagues (2013b) demonstrated the effectiveness for predicting criterion variables across domains of clinical diagnosis (e.g., symptoms of antisocial and substance-related disorders) and neurophysiology (e.g., separate brain event-related potential [ERP] measures) of a composite psychometric-neurophysiological (psychoneurometric) index of trait disinhibition. This composite index consists of two brain-ERP indicators and scores on the DIS-20 disinhibition scale, along with another self-report measure of trait disinhibition. The psychoneurometric index was developed using data from one large participant sample (N = 393) and evaluated for predictive validity in a separate cross-validation sample (N = 60). Figure 2-8 depicts results for the cross-validation sample. The purple bars (with their tops circled) represent the correlations between scores on the four-indicator psychoneurometric (disinhibition-scale/brain-ERP) factor and criterion variables consisting of (1) a composite of separate ERP variables (i.e., target stimulus P3 from an oddball task, feedback stimulus P3 from a choice-feedback task, and response-locked ERN from a flanker task) and (2) symptoms of differing impulse-control disorders as assessed by clinical interview. Depicted in the figure for purposes of comparison are correlations for the ESI Disinhibition scale indicator of the DIS/ERP factor alone (gray bars) and the mean of the two ERP indicators alone with the composite ERP and diagnostic criterion variables (pink bars). A minus sign (–) above certain bars denotes a negative correlation coefficient for the variable indicated.

The data summarized by Figure 2-8 show that the psychoneurometric factor predicted criterion variables in the diagnostic and brain response domains to comparable robust degrees: the correlations for this factor with ERP composite scores and diagnostic composite scores (purple bars) both exceeded 0.6. By contrast, ESI-Disinhibition scores alone (gray bars) predicted criterion variables in the diagnostic domain very effectively but predicted criteria in the brain response domain only modestly. The ERP indicators alone (pink bars) predicted criterion variables in the brain response domain very effectively but predicted criteria in the diagnostic domain only modestly.

Research Recommendation: Inhibitory Control

FIGURE 2-8 Associations with independent composite indices of brain response (left bars) and diagnostic symptoms (right bars) for three measures of disinhibitory tendencies: (1) scores on a 20-item disinhibition (DIS) scale (gray bars), (2) mean of two P3 brain responses (ERP) indicators of disinhibition (pink bars), and (3) composite of two self-report and two P3 brain indicators of disinhibition (purple bars). (-) = direction of correlational association is negative. Similar magnitude of rs for DIS/ERP predictor with brain and diagnostic criteria (circled purple bars) indicates that this psychoneurometric measure predicts effectively across these two domains of measurement.
SOURCE: Patrick et al. (2013b, p. 913).

1. Develop time-efficient, computer-automated self-report and behavioral assessments of inhibitory control capacity that demonstrate convergence with neurophysiological indices, as well as differentiation from constructs considered distinct from inhibitory control.
2. Examine the extent to which inhibitory control—as assessed through self-report, task-behavioral, and physiological response measures—predicts performance outcomes of interest (e.g., accidents, disciplinary incidents) and understand the common and unique aspects of the different assessment approaches in terms of underlying processes tapped by each and how these processes relate to performance.

REFERENCES

Ackerman, P.L., and A.T. Cianciolo. (2002). Ability and task constraint determinants of complex task performance. Journal of Experimental Psychology: Applied, 8(3):194–208.

Alderton, D.L., J.H. Wolfe, and G.E. Larson. (1997). The ECAT battery. Military Psychology, 9(1):5–37.

Barrouillet, P. (1996). Transitive inferences from set-inclusion relations and working memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22(6): 1,408–1,422.

Benton, S.L., R.G. Kraft, J.A., Glover, and B.S. Plake. (1984). Cognitive capacity differences among writers. Journal of Educational Psychology, 76(5):820–834.

Block, J., and J.H. Block. (1980). The role of ego-resiliency and ego-control in the organization of behavior. In W.A. Collins, Ed., Minnesota Symposium on Child Psychology (pp. 39–101). Hillsdale, NJ: Erlbaum.

Blumer, D., and D. Benson. (1975). Personality changes with frontal and temporal lesions. In D.F. Benson and F. Blumer, Eds., Psychiatric Aspects of Neurologic Disease (pp. 151–170). New York: Grune and Stratton.

Bosco, F.A., and D.G. Allen. (August 2011). Executive attention as a predictor of employee performance. Academy of Management Annual Meeting Proceedings, 2011(1):1–6.

Brewin, C.R., and A. Beaton. (2002). Thought suppression, intelligence, and working memory capacity. Behaviour Research and Therapy, 40(8):923–930.

Brewin, C.R., and E.A. Holmes. (2003). Psychological theories of posttraumatic stress disorder. Clinical Psychology Review, 23(3):339–376.

Brewin, C.R., and L. Smart. (2005). Working memory capacity and suppression of intrusive thoughts. Journal of Behavior Therapy and Experimental Psychiatry, 36(1):61–68.

Brewin, C.R., B. Andrews, and J.D. Valentine. (2000). Meta-analysis of risk factors for posttraumatic stress disorder in trauma-exposed adults. Journal of Consulting and Clinical Psychology, 68(5):748–766.

Brown, J., and T. Braver. (2005). Learned predictions of error likelihood in the Anterior Cingulate Cortex. Science, 307(5,712):1,118–1,121.

Campbell, D.T., and D.W. Fiske. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2):81–105.

Carter, C.S., T.S. Braver, D.M. Barch, M.M. Botvinick, D.N. Noll, and J.D. Cohen. (1998). Anterior cingulate cortex, error detection, and the online monitoring of performance. Science, 280(5,364):747–749.

Cattell, R.B. (1941). Some theoretical issues in adult intelligence testing. Psychological Bulletin, 38:592.

Clarkson-Smith, L., and A.A. Hartley. (1990). The game of bridge as an exercise in working memory and reasoning. Journal of Gerontology, 45(6):233–238.

Cloninger, C.R. (1987). A systematic method for clinical description and classification of personality variants. Archives of General Psychiatry, 44(6):573–588.

Cohen, J.D., and D. Servan-Schreiber. (1992). Context, cortex, and dopamine: A connectionist approach to behavior and biology in schizophrenia. Psychological Review, 99(1):45–77.

Cronbach, L.J., and P.E. Meehl. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4):281–302.

Crowder, R.G. (1982). The demise of short term memory. Acta Psychologica, 50(3):291–323.

Damos, D.L. (1993). Using meta-analysis to compare the predictive validity of single- and multiple-task measures of flight performance. Human Factors, 35(4):615–628.

Damasio, A.R., D. Tranel, and H. Damasio. (1990). Individuals with sociopathic behavior caused by frontal damage fail to respond automatically to social stimuli. Behavioral Brain Research, 41(2):81–94.

Daneman, M., and P.A. Carpenter. (1980). Individual differences in working memory and reading. Journal of Verbal Learning and Verbal Behavior, 19(4):450–466.

Daneman, M., and P.A. Carpenter. (1983). Individual differences in integrating information between and within sentences. Journal of Experimental Psychology: Learning, Memory, and Cognition, 9(4):561–584.

Daneman, M., and I. Green. (1986). Individual differences in comprehending and producing words in context. Journal of Memory and Language, 25(1):1–18.

Deary, I.J., L.J. Whalley, and J.M. Starr. (2009). A Lifetime of Intelligence: Follow-up studies of the Scottish mental surveys of 1931 and 1947. Washington, DC: American Psychological Association.

Deary, I.J., J. Yang, G. Davies, S.E. Harris, A. Tenesa, D. Liewald, M. Luciano, L.M. Lopez, A.J. Gow, J. Corley, P. Redmond, H.C. Fox, S.J. Rowe, P. Haggarty, G. McNeill, M.E. Goddard, D.J. Porteous, L.J. Whalley, J.M. Starr, and P.M. Visscher. (2012). Genetic contributions to stability and change in intelligence from childhood to old age. Nature, 482(7,384):212–215.

Dempster, F.N. (1981). Memory span: Sources of individual and developmental differences. Psychological Bulletin, 89(1):63–100.

Drasgow, F., S.E. Embretson, P.C. Kyllonen, and N. Schmitt. (2006). Technical Review of the Armed Services Vocational Aptitude Battery (ASVAB) (FR-06-25). Alexandria, VA: Human Resources Research Organization.

Engle, R.W. (2010). Role of working-memory capacity in cognitive control. Current Anthropology, 51(S1):S17–S26.

Engle, R.W., and M.J. Kane. (2004). Executive attention, working memory capacity, and a two-factor theory of cognitive control. In B. Ross, Ed., The Psychology of Learning and Motivation, Vol. 44 (pp. 145–199). New York: Academic Press.

Engle, R.W., J.J. Carullo, and K.W. Collins. (1991). Individual differences in working memory for comprehension and following directions. Journal of Educational Research, 84(5): 253–262.

Engle, R.W., S.W. Tuholski, J.E. Laughlin, and A.R.R. Conway. (1999a). Working memory, short-term memory and general fluid intelligence: A latent variable approach. Journal of Experimental Psychology: General, 128(3):309–331.

Engle, R.W., M.J. Kane, and S.W. Tuholski. (1999b). Individual differences in working memory capacity and what they tell us about controlled attention, general fluid intelligence and functions of the prefrontal cortex. In A. Miyake and P. Shah, Eds., Models of Working Memory: Mechanisms of Active Maintenance and Executive Control (pp. 102–134). London, UK: Cambridge University Press.

Feldman-Barrett, L., M.M. Tugade, and R.W. Engle. (2004). Individual differences in working memory capacity and dual-process theories of the mind. Psychological Bulletin, 130(4):553–573.

Finn, P.R. (2002). Motivation, working memory, and decision making: A cognitive-motivational theory of personality vulnerability to alcoholism. Behavioral and Cognitive Neuroscience Reviews, 3:183–205.

Galton, F. (1887). Supplementary notes on prehension in idiots. Mind, 12(45):79–82.

Gehring W.J., M.G.H. Coles, D.E. Meyer, and E. Donchin. (1995). A brain potential manifestation of error-related processing. In G. Karmos, M. Molnar, V. Csepe, I. Czigler, and J.E. Desmedt, Eds., Perspectives of Event-Related Potentials Research (pp. 267–272). Amsterdam, the Netherlands: Elsevier.

Gorenstein, E.E., and J.P. Newman (1980). Disinhibitory psychopathology: A new perspective and a model for research. Psychological Review, 87(3):301–315.

Hall, J.R., E.M. Bernat, and C.J. Patrick. (2007). Externalizing psychopathology and the error-related negativity. Psychological Science, 18(4):326–333.

Hambrick, D.Z., F.L. Oswald, E.S. Darowski, T.A. Rench, and R. Brou. (2010). Predictors of multitasking performance in a synthetic work paradigm. Applied Cognitive Psychology, 24(8):1,149–1,167.

Hambrick, D.Z., T.A. Rench, E.M. Poposki, E.S. Darowski, D. Roland, R.M. Bearden, F.L. Oswald, and R. Brou. (2011). The relationship between the ASVAB and multitasking in Navy sailors: A process-specific approach. Military Psychology, 23(4):365–380.

Hofmann, W., M. Friese, B.J. Schmeichel, and A.D. Baddeley. (2011). Working memory and self-regulation. In K.D. Vohs and R.F. Baumeister, Eds., Handbook of Self-Regulation: Research, Theory, and Applications, 2nd Edition (pp. 204–225). New York: Guilford Press.

Horn, J.L. (1967). Intelligence: Why it grows, why it declines. Society, 5(1):23–31.

Horn, J.L., and R.B. Cattell. (1966a). Refinement and test of the theory of fluid and crystallized general intelligences. Journal of Educational Psychology, 57(5):253–270.

Horn, J.L., and R.B. Cattell. (1966b). Age differences in primary mental ability factors. Journal of Gerontology, 21(2):210–220.

Horn, J.L., and R.B. Cattell. (1967). Age differences in fluid and crystallized intelligence. Acta Psychologica, 26:107–129.

Jacobs, J. (1887). Experiments on “prehension.” Mind, 12(45):75–79.

Kane, M.J., K.M. Bleckley, A.R.A.Conway, and R.W. Engle. (2001). A controlled-attention view of working-memory capacity. Journal of Experimental Psychology: General, 130:169–183.

Kane, M.J., D.Z. Hambrick, S.W. Tuholski, O. Wilhelm, T.W. Payne, and R.W. Engle. (2004). The generality of working-memory capacity: A latent-variable approach to verbal and visuo-spatial memory span and reasoning. Journal of Experimental Psychology: General 133(2):189–217.

Kane, M.J., D.Z. Hambrick, and A.R.A. Conway. (2005). Working memory capacity and fluid intelligence are strongly related constructs: Comment on Ackerman, Beier, and Boyle. Psychological Bulletin, 131(1):66–71.

Kane, M.J., L.H. Brown, J.C. McVay, I. Myin-Germeys, P.J. Silvia, and T.R. Kwapil. (2007). For whom the mind wanders, and when: An experience-sampling study of working memory and executive control in daily life. Psychological Science, 18(7):614–621.

Kiewra, K.A., and S.L. Benton. (1988). The relationship between information processing ability and notetaking. Contemporary Educational Psychology. 13(1):33–44.

King, J., and M.A. Just. (1991). Individual differences in syntactic processing: The role of working memory. Journal of Memory and Language 30(5):580–602.

Kyllonen, P.C., and R.E. Christal. (1990). Reasoning ability is (little more than) working-memory capacity? Intelligence, 14(4):389–433.

Kyllonen, P.C., and D.L. Stephens. (1990). Cognitive abilities as determinants of success in acquiring logic skill. Learning and Individual Differences, 2(2):129–160.

König, C.J., M. Buhner, and G. Murling (2005). Working memory, fluid intelligence, and attention are predictors of multitasking performance, but polychronicity and extraversion are not. Human Performance, 18(3):243–266.

Krueger, R.F., K.E. Markon, C.J. Patrick, S.D. Benning, and M.D. Kramer. (2007). Linking antisocial behavior, substance use, and personality: An integrative quantitative model of the adult externalizing spectrum. Journal of Abnormal Psychology, 116(4):645–666.

Loevinger, J. (1957). Objective tests as instruments of psychological theory. Monograph supplement 9. Psychological Reports, 3:635–694.

Lubinski, D., and C.P. Benbow. (2000). States of excellence. The American Psychologist, 55(1):137–150.

Lubinski, D., and C.P. Benbow. (2006). Study of mathematically precocious youth after 35 years: Uncovering antecedents for the development of math-science expertise. Perspectives on Psychological Science, 1(4):316–345.

McVay, J.C., and M.J. Kane. (2009). Conducting the train of thought: Working memory capacity, goal neglect, and mind wandering in an executive-control task. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35(1):196–204.

McVay, J.C., and M.J. Kane. (2012a). Drifting from slow to “D’oh!” Working memory capacity and mind wandering predict extreme reaction times and executive-control errors. Journal of Experimental Psychology: Learning, Memory, and Cognition, 38(3):525–549.

McVay, J.C., and M.J. Kane. (2012b). Why does working memory capacity predict variation in reading comprehension? On the influence of mind wandering and executive attention. Journal of Experimental Psychology: General, 141(2):302–320.

Miller, E.K. (1999). The prefrontal cortex: Complex neural properties for complex behavioral. Neuron, 22(1):15–17.

Patrick, C.J., and E.M. Bernat. (2010). Neuroscientific foundations of psychopathology. In T. Millon, R.F. Krueger, and E. Simonsen, Eds., Contemporary Directions in Psychopathology: Toward the DSM-V (pp. 419–452). New York: Guilford Press.

Patrick, C.J., E.M. Bernat, S.M. Malone, W.G. Iacono, R.F. Krueger, and M.K. McGue. (2006). P300 amplitude as an indicator of externalizing in adolescent males. Psychophysiology, 43(1):84–92.

Patrick, C.J., C.E. Durbin, and J.S. Moser. (2012). Reconceptualizing proneness to antisocial deviance in neurobehavioral terms. Development and Psychopathology, 24(3):1,047–1,071.

Patrick, C.J., M.D. Kramer, R.F. Krueger, and K.E. Markon. (2013a). Optimizing efficiency of psychopathology assessment through quantitative modeling: Development of a brief form of the Externalizing Spectrum Inventory. Psychological Assessment, 25(4):1,332–1,348.

Patrick, C.J., N.C. Venables, J.R. Yancey, B.M. Hicks, L.D. Nelson, and M.D. Kramer. (2013b). A construct-network approach to bridging diagnostic and physiological domains: Application to assessment of externalizing psychopathology. Journal of Abnormal Psychology, 122(3):902–916.

Patterson, C.M., and J.P. Newman. (1993). Reflectivity and learning from aversive events: toward a psychological mechanism for the syndromes of disinhibition. Psychological Review, 100(4):716–736.

Plomin, R., J.C. Defries, G.E. McClearn, and P. McGuffin. (2008). Behavioral Genetics. (5th Ed.). New York: Worth.

Roberts, R.D., G.N. Goff, F. Anjoul, P.C. Kyllonen, G. Pallier, and L. Stankov. (2000). The Armed Services Vocational Aptitude Battery (ASVAB): Little more than acculturated learning (Gc)!? Learning and Individual Differences, 12(1):81–103.

Redick, T.S., J.M. Broadway, M.E. Meier, P.S. Kuriakose, N. Unsworth, M.J. Kane, and R.W. Engle. (2012). Measuring working memory capacity with automated complex span tasks. European Journal of Psychological Assessment, 28(3):164–171.

Russell, T.L., and N.G. Peterson. (2001). The experimental battery: Basic attribute scores for predicting performance in a population of jobs. In J.P. Campbell and D.J. Knapp, Eds., Exploring the Limits in Personnel Selection and Classification (pp. 269–306). Hillsdale, NJ: Lawrence Erlbaum.

Russell, T.L., N.G. Peterson, R.L. Rosse, J.T. Hatten, J.J. McHenry, and J.S. Houston. (2001). The measurement of cognitive, perceptual and psychomotor abilities. In J.P. Campbell and D.J. Knapp, Eds., Exploring the Limits in Personnel Selection and Classification (pp. 71–110). Hillsdale, NJ: Lawrence Erlbaum.

Russell, T.L., L. Ford, and P. Ramsberger, Eds. (2014). Thoughts on the Future of Military Enlisted Selection and Classification. Alexandria, VA: Human Resource Research Organization.

Sager, C.E., N.G. Peterson, S.H. Oppler, R.L. Rosse, and C.B. Walker. (1997). An examination of five indexes of test battery performance: Analysis of the ECAT battery. Military Psychology, 9(1):97–120.

Scheffers, M., M.G.H. Coles, P. Bernstein, W. Gehring, and E. Donchin. (1996). Event-related brain potentials and error-related processing: An analysis of incorrect responses to go and no-go stimuli. Psychophysiology, 33(1):42–53.

Schmidt, F.L., and J.E. Hunter. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124(2):262–274.

Shute, V.J. (1991). Who is likely to acquire programming skills? Journal of Education Computing Research, 7(1):1–24.

Spearman, C. (1904) “General intelligence,” objectively determined and measured. The American Journal of Psychology, 15(2):201–292.

Stankov, L., G. Fogarty, and C. Watt. (1989). Competing tasks: Predictors of managerial potential. Personality and Individual Differences, 10(3):295–302.

Tellegen, A. (1985). Structures of mood and personality and their relevance to assessing anxiety, with an emphasis on self-report. In A.H. Tuma and J.D. Maser, Eds., Anxiety and the Anxiety Disorders (pp. 681–706). Hillsdale, NJ: Lawrence Erlbaum.

Unsworth, N., J.C. Schrock, and R.W. Engle. (2004). Working memory capacity and the antisaccade task: Individual differences in voluntary saccade control. Journal of Experimental Psychology: Learning, Memory, and Cognition, 30:1,302–1,321.

Verive, J.M., and M.A. McDaniel. (1996). Short-term memory tests in personnel selection: Low adverse impact and high validity. Intelligence, 23(1):15–32.

Wise, S.P., E.A. Murray, and C.R. Gerfen. (1996). The frontal-basal ganglia system in primates. Critical Reviews in Neurobiology, 10(3-4):317–356.

Wright, S.B., B.J. Matlen, C.L. Baym, E. Ferrer, and S.A. Bunge. (2007). Neural correlates of fluid reasoning in children and adults. Frontiers in Human Neuroscience, 1:8. Available: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2525981/ [September 2014].

Young, S. E., N.P. Friedman, A. Miyake, E.G. Willcutt, R.P. Corley, B.C. Haberstick, and J.K. Hewitt. (2009). Behavioral disinhibition: Liability for externalizing spectrum disorders and its genetic and environmental relation to response inhibition across adolescence. Journal of Abnormal Psychology, 118(1):117–130.