Measuring Human Capabilities: An Agenda for Basic Research on the Assessment of Individual and Group Performance Potential for Military Accession (2015)

1

Overview

This report represents the second phase of a larger project on measuring human capabilities in the context of developing a basic research agenda aimed at identifying possible predictors of first-term soldier performance in the U.S. Army that could usefully supplement measures currently used as screening devices in the enlistment process. In Phase 1, committee members identified promising research topics, identified experts in these topic areas, and convened a workshop at which invited experts presented their research. That workshop is summarized in a report entitled New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary (National Research Council, 2013). In Phase 2, the committee was enhanced with additional members and charged with further exploration, building on the workshop, in order to develop a recommended future research program for the Foundational Science Research Unit of the U.S. Army Research Institute for the Behavioral and Social Sciences (ARI). Box 1-1 provides the specifics of the committee charge.

The committee interpreted this charge as requesting recommendations for a basic research agenda aimed at identifying ways to supplement the Army’s enlisted soldier accession system with additional predictors that go beyond the currently used cognitive and personality measures (described in the following section) and that have the potential to improve the already high quality of accession decisions. To accomplish this task, the committee has focused on potential determinants of individual and collective (e.g., team) performance.

SETTING THE STAGE: THE CURRENT ARMY ENLISTED SOLDIER ACCESSION SYSTEM

The committee’s focus is on the initial selection process that determines eligibility for entry-level positions within the Army. Individuals with an interest in enlisting must meet standards in a number of areas. These include two formal testing vehicles: The Armed Services Vocational Aptitude Battery (ASVAB) is a cognitive knowledge, skill, and ability battery of tests. The Tailored Adaptive Personality Assessment System (TAPAS) is

a computer-administered personality measure. In addition, there are three nontest screens: educational attainment, an examination of moral character, and an examination of physical and medical readiness to serve. A concise review of the U.S. military’s selection and assessment system is provided by Sellman and colleagues (2010).

The ASVAB is primarily conducted as the CAT-ASVAB, a computer-administered adaptive version of the ASVAB test battery comprising 10 tests. Four of the tests (word knowledge, paragraph comprehension, arithmetic reasoning, and mathematics knowledge) are combined into a composite known as the Armed Forces Qualifying Test (AFQT), which is used as a basic entry screen. The other tests (general science, electronics information, mechanical comprehension, auto information, shop information, and assembling objects) are used for determining qualification for specific occupational assignments once the entry screen has been passed. Extensive research supports the predictive capability of ASVAB performance for subsequent training and first-term job performance (see Armor and Sackett, 2004, for a review).

The TAPAS is a personality measurement system that can be configured to deliver nonadaptive and adaptive personality tests based on Item Response Theory (IRT). The TAPAS tests typically measure 12-18 of a possible 28 narrow personality factors. TAPAS factor scores are used to form composites, currently called “can do,” “will do,” and “persistence,” which are used to predict job performance and attrition criteria. While the ASVAB is part of the enlistment decision for all applicants, the TAPAS is administered to those scoring below the 50th percentile on the AFQT. Those below the 50th percentile have a higher risk of failure to meet standards and adjust successfully to military life, and the TAPAS is used to identify and screen out high-risk candidates.

The Army also classifies applicants on the basis of educational attainment, as extensive research shows that possession of a high school diploma is the best single predictor of successful adjustment to military life (Strickland, 2005; Trent and Laurence, 1993). The rate of noncompletion of a tour of duty is markedly higher for high school dropouts and for holders of other credentials such as a General Education Development certificate or high school completion via home schooling than for diploma holders (Strickland, 2005). Thus, applicants are put into one of three tiers (tier 1: diploma; tier 2: alternate credential; tier 3: dropout), with more stringent AFQT standards applied below the first tier.

Applicants with a qualifying AFQT score receive a physical examination covering a range of features including blood pressure, pulse, visual acuity, hearing, blood testing, urinalysis, and drug and HIV testing. Some conditions require medical treatment before enlistment; others are disqualifying, though applicants can apply for a waiver in some circumstances.

Finally, applicants must meet moral character standards. Some criminal activities are immediately disqualifying. In other cases, applicants can apply for a waiver, which prompts a review of the specific circumstances and a case-specific determination as to whether the applicant will be permitted to enlist.

RESEARCH APPROACH

Constraints

The committee operated under a number of constraints as it examined possible additional screening tools that could usefully augment the current testing in the cognitive ability and personality domains. These constraints were determined via instructions from and discussions with representatives of the Foundational Science Research Unit, ARI, and the committee has taken them as conjoint conditions (acting together) on what should be included in the focus of our report and what should be minimized or excluded altogether.

The first constraint was that the committee should focus on attributes broadly useful for first-term enlisted soldiers. We thus excluded attributes that are relevant to just a single occupational specialty or to a select set of occupational specialties. This constraint is consistent with the cognitive and personality measures currently used by the Army: cognitive problem solving skill and a pattern of personality attributes reflecting ability to adjust to military life are broadly relevant regardless of occupational specialty.

Second, the committee was instructed to focus on measures that can be administered pre-accession and in a cost-effective manner to large numbers of individual candidates without requiring special skills to administer the measure or to evaluate performance on the measure, and without requiring elaborate equipment. These constraints preclude consideration of predictor measures such as:

• Measures involving complex work sample measures (with extensive sampling of skill performance and work competencies possibly requiring days or weeks to complete);
• Measures involving interpersonal interaction (e.g., role plays, team performance tasks);
• Neurological measures (including invasive procedures);
• Physical ability and fitness measures; and
• Assessments that can play a useful role for the Army but that do not take place pre-accession, such as mid-career assessments or post-injury return-to-work assessments.

The neurological measures category listed above merits amplification. While the constraint regarding a restriction to measures that do not require special skill and/or equipment to administer precludes neurological measures for present use as a routine part of entry level screening, we do, consistent with the charge to the committee, consider potential roles for neurological measures (see Chapter 10). First, we consider them for use as criterion measures against which other measures (e.g., self-reports) can be evaluated. Second, we consider them for use in settings other than mass screening of candidates. For example, there may be roles for neurological measures in follow-up assessment of limited subsets of candidates, such as those producing a particular and difficult to interpret pattern of results on other measures used in the screening process. Finally, we consider the possibility for measurement developments that may in the future make large scale screening with one or more neurological measures logistically possible.

The third constraint was to focus on attributes for which there is a theoretical foundation for the measurement of the attribute of interest and for the relevance of the attribute to important military outcomes. This constraint precluded consideration of approaches based on brute empiricism, such as the use of empirical biographical data keys in which various background characteristics are assigned weights based on the degree to which they prove to differentiate between soldiers who score high versus low on outcomes of interest. Similarly, consideration of features such as birth order were excluded because a strong theoretical foundation for their use is lacking.

Finally, we received specific instruction that genetic screening was outside the committee’s purview. Thus, we did not include genetic testing in our review.

Note that, within these constraints, measures excluded from consideration as possible supplements to the accession system may still be recommended for use as criterion measures in the evaluation for operational utility of other measures that are consistent with the constraints. For example, measures involving team performance tasks, excluded as a testing measure under the second constraint above, may nonetheless be useful as criteria against which individual measures of propensity for effectiveness as a team member may be evaluated. Furthermore, as the committee developed an agenda for future research, some topics were considered based upon the committee’s expectations of the impact of future technology or other capabilities that could significantly change the feasibility for operational use in the long term.

Terminology

To understand potential improvements in human capability measurement, several important terms need to be understood, as they define what and how measurements are conducted. The committee uses the term “construct” to refer to the attribute label attached to a measure (e.g., arithmetic reasoning, fluid intelligence, conscientiousness). Other terms used in various places to refer to individual-differences measures include “trait” and “factor.” “Trait” connotes a reasonably stable attribute (as opposed to a “state” such as mood or emotion, which is expected to change frequently). “Factor” denotes an attribute in common among a set of trait measures, identified through application of the technique of factor analysis.

Identifying Topics for Research

At early meetings during Phase 1 of the project, the committee identified a lengthy list of possible topics for research. Committee discussion led to identifying a subset of this list as worthy of further investigation. A sizable number of the selected topics were the focus of a workshop held in April 2013. Some selected topics could not be covered in the workshop due to time constraints or unavailability of the targeted speakers. Additional speakers were invited to subsequent committee meetings to address such topics. As Phase 2 of the project involved an expanded committee, we revisited the initial topic list from Phase 1, amending it to include input from new committee members. Committee membership included individuals with broad expertise in personnel selection in both civilian and military contexts, individual differences, performance measurement, teamwork, psychometrics, and neuroscience. Initial topic identification relied on the expertise and judgment of committee members. We asked questions such as “what is in operational use in other employment settings?”, “what looks promising in the selection literature on new predictor constructs and/or new predictor methods?”, and “what looks promising in the individual-differences literature that might prove applicable to personnel selection settings?”

We had available to us useful summaries of work addressing these questions. For example, the Annual Review of Psychology commissions systematic and thorough reviews of developments in the area of personnel selection on a recurring basis. The two most recent reviews at the time of the committee’s work were by Hough and Oswald (2000) and by Sackett and Lievens (2008). Hough, Oswald, and Sackett serve on the present committee.

The committee used workshop content, subsequent presentations to the committee, review of publically available research and data, and discussions within the committee to identify the set of topics discussed in this report.

(For more details of the topics considered in developing this report, see the workshop agenda and selection of topics considered for the workshop agenda in Appendix A and the list of the data gathering presentations delivered to the committee during the study’s second phase in Appendix B.) We do not offer an assessment of the topics that were considered but not included in the recommended research agenda. We acknowledge that the set of topics selected represent the collective judgment of the committee. It is possible that a differently constituted committee would identify some additional topics or would choose not to focus on some of the topics covered here. The committee membership does reflect broad and varied expertise relevant to our charge, and we are confident that we have identified a promising, even if not exhaustive, research agenda.

In evaluating research topics, we applied the following decision process. First, could we identify a conceptual basis for a linkage between a particular predictor construct and a criterion construct that can be expected to be of interest to the Army? Success of ARI’s basic research program is largely determined by the feasibility of developing foundational science into applied research programs and ultimate implementation to affect U.S. Army policy and procedure. If the committee could not identify potential utility in the basic research results to improve prediction of soldier success, the topic was not considered further. Note that we did not view our task as limited to existing operational Army criteria (e.g., criteria used to assess training performance or attrition). A conceptually meaningful criterion construct, such as team effectiveness, could be considered even if a measure based on that construct is not currently in operational use.

Second, could we identify settings where we could see analogs to military performance, such as job performance in the civilian workforce, where measures of particular predictor constructs have been (a) successfully developed, (b) shown to be linked to criteria of interest, and (c) shown to have incremental validity over measures in the ability and personality domain? Although topics were not discarded from further consideration solely on the basis of failure to meet all three conditions, the committee weighed topics against each other, and topics included in this final report were judged to meet an appropriate minimum threshold given the prior research and data available on the particular topic.

Third, we sought to identify research developments that suggest a reconsideration of long-standing research domains that may have been rejected in the past for a variety of possible reasons. In particular, the committee sought constructs with proven predictive capability but that were not conducive to testing through standard paper and pencil tests. For example, we considered whether there are new measurement developments that could potentially overcome obstacles to the measurement of a particular predictor construct (e.g., the development of measurement methods more resistant

to faking and coaching). The committee also evaluated research domains that may have been stymied due to lack of funding or due to misunderstood research results that may have deterred further research programs, as well as domains that may have been considered high-risk (and potentially high-payoff) compared to other research domains.

Fourth, we considered whether there are constructs for which a promising research base is developing but which have not been investigated in the context of personnel selection. This involved considering the broader individual-differences literature, rather than focusing solely on the personnel selection literature.

Fifth, one key feature that might easily be overlooked is that the charge to the committee focused on identifying a basic research agenda that might in time lead to improvements in the Army enlisted soldier selection process. Thus our charge excluded possible methods of improving selection that were, in the committee’s judgment, beyond the basic research stage. Perhaps the most vivid example of this is the domain of vocational interest measurement. Vocational interest measures have for some time been viewed as useful to individuals for career guidance but of limited value for personnel selection. There has been a recent resurgence of research on the relationship between vocational interests and subsequent performance outcomes (e.g., Nye et al., 2012; Van Iddekinge et al., 2011), suggesting stronger interest-outcome relationships than had been seen in the past. The committee gave careful attention to this domain, including an invited presentation to the committee on the topic. After extensive discussion, however, the committee concluded that what was needed was a program of criterion-related validation research to determine whether this positive pattern of relationships would also be found in Army settings. Such work is essentially operational, as well-developed measures exist ready for tryout. Thus, while the committee is cautiously optimistic that vocational interest measurement has the potential to improve selection, the consensus was that this was not a basic research issue.

As the committee deliberated the list of possible topics, these questions were carefully considered to determine whether a possible topic satisfied a minimum threshold for inclusion in the final recommended research agenda. They also contributed to the decision process whereby topics were evaluated against each other so as to select the strongest candidates, by the committee’s judgment, to be most likely to have the largest impact on improving the military personnel testing, selection, and assignment process. No single research topic was a perfect fit to all the criteria. Furthermore, large variations in prior research volume, strategy, and results were found between topics, and this is reflected in the presentation of those topics in the individual chapters of this report.

A Taxonomic Structure for Ways to Improve Selection Systems

Sackett and Lievens (2008) offered a taxonomy of ways that a selection system can be improved, and Sackett presented a version of this taxonomy at the workshop convened as part of Phase 1 of the current project. In particular, Sackett and Lievens (2008) proposed that a selection system can be improved by one or more of the following:

1. Identification and measurement of new predictor constructs;
2. Identification and prediction of new outcomes;
3. Improved measurement of existing predictor constructs; or
4. Identification of features that moderate predictor-criterion relationships (e.g., identifying circumstances under which predictor-criterion relationships are stronger or weaker).

The committee used this taxonomic system in considering potential research investments. While Sackett and Lievens used the terms “new constructs” and “existing constructs” in the context of the entire field of personnel selection, we view them in terms of constructs currently in use for Army enlisted soldier selection. For example, while spatial ability is included in the ASVAB, it is not currently in use for enlisted selection, and thus we view spatial ability as a new construct for consideration. Our recommendations fall into all four of the categories in the above taxonomy, and we structure the report in terms of these categories.

What emerged as the most prominent of the categories in this taxonomy is the identification and measurement of new predictor constructs. Thus, following this introductory chapter, Section 2 of the report contains chapters that describe fluid intelligence, working memory capacity, executive attention, inhibitory control, cognitive biases, and spatial abilities. Each of these domains is described in more detail below.

Another prominent category is the identification and prediction of new outcomes. Although we identify three new performance domains that are conceptually relevant for a broad range of Army enlisted soldier positions, Section 3 presents only the first of these: teamwork behavior. Note that investigations into the prediction of new outcomes may result either in a determination that these outcomes are well predicted by currently used predictor measures or in a determination that a new predictor or predictors are needed to predict these outcomes. Two chapters contain elements that cut across aspects of the previous two sections, and therefore Section 4 contains hybrid topics with joint focus on new predictor constructs and prediction of new outcomes. The first chapter in that section, hot cognition, describes the two constructs, defensive reactivity and emotion regulation, and one outcome, performance under stress. The second chapter in Section 4 also

presents two closely linked topics— adaptability and inventiveness—which can be conceptualized either as an outcome variable to be predicted or as a predictor construct.

Section 5 contains single chapters linked to other domains in the taxonomy. A chapter on psychometrics focuses on both ways of measuring existing constructs better (e.g., using new developments in IRT to further improve the ASVAB) and on potential new measurement methods (e.g., gaming). A chapter on situational judgment discusses a measurement method that can potentially be used for improved measurement of existing constructs (e.g., measuring personality constructs) and measuring new constructs not currently part of the Army’s enlisted soldier selection system. Finally, a chapter on neuroscience focuses broadly on the potential use of neuroscience-based measures as markers of psychological states (e.g., undue anxiety while completing existing Army selection instruments). These states may moderate predictor-criterion relationships, as candidates exhibiting undue anxiety may produce test scores that are systematically lower than their true standing on the construct of interest.

The report concludes with a single chapter in Section 6, The Research Agenda. For the convenience of the reader, Chapter 11 includes a consolidated list of the committee’s conclusions and recommendations, which together comprise the recommended research agenda for ARI’s Foundational Science Research Unit. This final chapter also presents the committee’s assessment of the funding level needed to implement the recommended research agenda.

Considerations in Choosing Criteria

A widely accepted principle within the field of personnel selection is that to develop a selection system, one must begin by specifying the criterion of interest. Using a simple example, if told “we want a selection system for supermarket cashiers,” the response is to question the organization further: do you want cashiers who are fast in scanning groceries, friendly in dealing with customers, or reliable in their attendance? Some firms may emphasize speed and efficiency; others may emphasize friendliness. Some may want a balance between speed, efficiency, and friendliness. This has implications for the subsequent selection system: the individual attributes that predict who will be quick in scanning groceries are likely to be very different from those that predict warm and friendly customer interactions.

Importantly, the choice to, for instance, focus on predicting speed and efficiency versus friendly customer interaction is a matter of organizational values. It is not appropriate for the selection researcher to assert that the organization should value one outcome versus the other. The researcher can inform the organization about the degree to which a given outcome

is predictable, but the choice of the outcome(s) of interest is ultimately a matter of organizational strategy.

These ideas have major implications for the recommendations developed in this report. The charge to the committee was to identify a research agenda with the potential of improving the Army’s enlisted soldier selection system. This is a very broad charge. The committee would have acted very differently had it been presented with a charge that focused on a single specific criterion: for example, improve soldier’s technical proficiency or reduce the rate of discharge for disciplinary reasons or reduce the rate of attrition due to lack of adjustment to military life. We also would have acted differently had our charge been to focus on selection criteria for classification of individuals into occupational specialties or specific jobs. However, absent this advance specification, we considered prospects for improving the selection system regarding a wide range of criteria.

That there is interest in multiple criteria in military selection is reflected in the currently used selection tools. At a high level of abstraction, the job performance domain can be subdivided into “can do” and “will do” domains. The ASVAB focuses on the “can do” domain: it is an effective predictor of the degree to which an enlistee will become technically proficient following training. It is not a particularly effective predictor of the typical degree of effort an enlistee will exert, or of the degree to which an enlistee will avoid behaviors that would result in disciplinary action. In contrast, the personality domains measured by the TAPAS includes a focus on the “will do” domain, and the TAPAS is predictive of avoiding disciplinary action and effective adjustment to military life. (For a recent discussion of the broad array of individual-differences constructs relevant to the military, see Rumsey and Arabian, 2014.)

Thus, the Army has interest in multiple criteria. Army research on the use of individual-differences measures that predict outcomes of interest has examined a wide range of criteria, including task proficiency, effort, maintaining military discipline, adjustment to military life, and attrition, among others. Therefore, the committee cast a broad net in developing recommendations for research. The requirement that we set for ourselves was that we could see a conceptual or empirical link between an attribute under consideration and one or more outcomes that constitute a component of overall individual or team effectiveness.

In considering outcomes of interest, we were informed by ongoing conceptual and empirical work about the underlying structure of individual and team effectiveness. A variety of scholars have advocated for differing representation of the underlying dimensionality of individual and team effectiveness. Campbell (2012) summarizes and integrates a variety of perspectives in the structure of behavior, performance, and effectiveness in contemporary organizations. We drew from a number of these perspectives,

rather than embracing a single approach. We outline here a set of outcome variables that we believe are broadly relevant for organizations in general and the Army in particular.

_________________

¹ Recently, Army leaders, such as Lt. Gen. Robert Brown, commander of the Army Combined Arms Center, have referred to the need for soldiers who “improve and thrive in conditions of chaos” (see Army Times article on “The Human Dimension” panel during the 2014 Association of the United States Army convention, available at http://www.armytimes.com/article/20141015/NEWS/310150065/Wanted-Soldiers-who-thrive-chaos [October 2014]).

In Table 1-1, we present a grid that pairs each of the research domains for which we offer recommendations with this set of outcomes. For each research domain, we identify the outcome or outcomes for which we view a linkage as plausible. We do not view this as etched in stone; arguments that a domain may be linked to additional outcomes are possible. One reason for providing this grid is to show that each domain is linked to one or more outcomes, which is the basis for that domain being included as part of our proposed research agenda.

There is a second critical implication of this grid. Some may ask why we do not prioritize our recommendations (e.g., rank them 1-10). The reason is linked to the point developed earlier in this section that the choice of the outcome measure(s) on which to focus is a matter of organizational values, rather than a scientific question. Should the Army decide that any one of the outcomes in the grid is strategically of greatest value to its mission(s), then research domains linked to those outcomes would become higher in priority. Furthermore, particular occupational specialties might place greater value on different outcomes, thereby giving certain research domains priority for both selection and classification purposes. Put another way, the research domains we identify could be prioritized very differently depending on the value that the Army assigns to each outcome domain.

AN OVERVIEW OF THE RESEARCH AGENDA

In Chapters 2 through 10, divided into four sections, the committee presents a summary of available research and the committee’s assessment of that research in consideration of a future research agenda to improve selection and retention of successful soldiers. The research domains presented in each chapter are outlined below.

Section 2. Identification and Measurement of New Predictor Constructs

The report’s second section includes three chapters that present future research opportunities in the identification and measurement of new predictor constructs.

TABLE 1-1 Grid Showing Links Between Research Domains and Outcomes

Chapter 2. Fluid Intelligence, Working Memory Capacity, Executive Attention, and Inhibitory Control

Chapter 2 discusses fluid intelligence, working memory capacity, executive attention, and inhibitory control in relation to an individual’s emotional, behavioral, and impulse control. Many intelligence measures focus on crystallized intelligence: the learned and acquired skills and knowledge component of intelligence. Assessment of fluid intelligence could potentially reveal more about an individual’s reasoning and novel problem-solving abilities. Working memory capacity and executive attention assessments, which are relatively short and easy to administer, have been found to be valid in predicting performance on a large variety of real-world cognitive tasks.

Chapter 3. Cognitive Biases

Chapter 3 describes cognitive biases that can produce errors in judgment or decision making. For example, projection (assuming others share one’s own feelings, attitudes, and values) can interfere with soldiers’ abilities to accurately judge the motives of others, such as host-nation citizens or international coalition military members. Cognitive biases operate in both everyday reasoning and decision making and also may play a role in life-and-death disasters; therefore, learning about individuals’ susceptibility or proneness to cognitive biases may be useful for informing assignment decisions. One important question in this area is the degree to which cognitive biases can be mitigated by training.

Chapter 4. Spatial Abilities

Chapter 4 considers the spectrum of skills in the domain of spatial abilities. Research suggests that spatial abilities may be an important predictor of performance, particularly in scientific and technical fields. Multiple facets of spatial abilities have been identified or proposed, all of which relate to the many different ways individuals understand their own spatial relationship to and within surroundings and also the way individuals understand representations of multidimensional figures in one-dimensional displays. Although one spatial ability measure (Assembling Objects) is included in the ASVAB, this chapter presents evidence of the potential value of other approaches to the measurement of spatial abilities that may yield more useful information for military selection and classification.

Section 3. Identification and Prediction of New Outcomes

The third section consists of a single chapter on Teamwork Behavior, one of three new outcomes the committee identified with potential for identification and prediction in military assessment settings. The other two outcomes are discussed as part of hybrid chapters in Section 4.

Chapter 5. Teamwork Behavior

Chapter 5 considers individual and team factors that may be of use in predicting successful teamwork behavior. The chapter focuses on how selection and classification of entry-level enlisted soldiers can improve unit performance and mission success. The Input-Process-Outcome model can serve as a loose framework to identify future research objectives. Starting with the end goal, the committee first discusses team outcomes to define the criteria domain for selection and classification. Next, we examine team processes and emergent states as more proximal criteria of collective capacity. Finally, we examine how future research on individual-level inputs to teams might help understand who is best suited for teamwork and how individuals might be better classified into specific Army small units, to include teams, squads, and platoons.

Section 4. Hybrid Topics with Joint Focus on New Predictor Constructs and Prediction of New Outcomes

The report’s fourth section includes two hybrid topics with aspects that cut across the two previous sections and thereby represent both new predictor constructs and the prediction of new outcomes.

Chapter 6. Hot Cognition: Defensive Reactivity, Emotional Regulation, and Performance Under Stress

Chapter 6 examines “hot cognition”: how individuals perform in situations that elicit strong emotions (in contrast to cognition under circumstances of cool or moderate emotions, or “cold cognition”). Hot cognition is responsible for such behaviors as defensive reactivity: the degree to which one is prone to negative emotional activity (particularly fear) in threatening situations. Fear is often an unproductive emotion, especially for combat soldiers, whereas fearlessness or boldness can be a productive emotion. However, when taken too far, fearlessness might be maladaptive, contributing to a soldier’s disregard for safety procedures or operational protocol. Research opportunities in this domain include, for example, investigating

whether there may be an optimal level of defensive reactivity for performance in particular conditions or by a particular individual.

Chapter 7. Adaptability and Inventiveness

Chapter 7 discusses the potential of measuring individuals’ adaptability and inventiveness, an important attribute for soldiers who routinely face unexpected and unique environments, situations, challenges, and opportunities. Adaptability involves the ability to adjust and accommodate to changing and unpredictable physical, interpersonal, cultural, and task environments. Inventiveness refers to the ability to think innovatively and produce novel high-quality and task-appropriate ideas, incorporating an orientation toward problem solving. Research on these two constructs suggests they increment predictive validity over other cognitive ability and personality measures for important outcomes such as performance and career continuance and progression.

Section 5. Methods and Methodology

This section includes three single chapters linked to the other research domains as methods and methodology for implementation.

Chapter 8. Psychometrics and Technology

Chapter 8 examines a variety of areas that show promise for improvements in measurement, including the application and modeling of forced-choice measurement methods, development of serious gaming, pursuit of Multidimensional Item Response Theory (MIRT), Big Data analytics, and other modern statistical tools. One example of the potential benefit is the likelihood that MIRT models can yield information about examinees’ performance beyond what has been possible with traditional unidimensional IRT models. MIRT models may also offer improvements to test efficiency.

Chapter 9. Situations and Situational Judgment Tests

Chapter 9 focuses on the use of situations and situational judgment tests to measure and assess individuals’ judgment abilities to interpret, evaluate, and weigh alternate courses of action appropriately and effectively. The chapter considers a variety of approaches to, and formats for, these tests; it discusses possible advantages of various presentation formats. For example, situational judgment tests administered in a video format may

reduce the impact of lower verbal ability on test results and may provide a more immersive and engaging testing experience.

Chapter 10. Assessment of Individual Differences Through Neuroscience Measures

Chapter 10 examines neuroscience measures that may warrant consideration for testing applications in the near term, particularly as measures of anxiety, attention, and motivation in test takers. To illustrate, some level of anxiety is normal in test-taking situations. However, high levels of anxiety can have detrimental effects on test performance. Determining in real time through the use of the noninvasive technique of electroencephalography whether a candidate is experiencing such detrimental anxiety affords test administrators the opportunity to offer mitigation strategies to such candidates, thereby improving the degree to which assessment results offer an accurate representation of such candidates’ abilities. Understanding candidates’ levels of attention and motivation during testing can similarly yield better understanding of the credibility of test results.

REFERENCES

Armor, D., and P. Sackett. (2004). Manpower quality in the all-volunteer force. In B.A. Bicksler, C.L. Gilroy, and J.T. Warner, Eds., The All-Volunteer Force: Thirty Years of Service (pp. 90–108). Washington, DC: Brassey’s.

Campbell, J.P. (2012). Behavior, performance and effectiveness in the twenty-first century. In S.W.J. Kozlowski, Ed., The Oxford Handbook of Organizational Psychology, Volume 1 (pp. 159–194). New York: Oxford University Press.

Hough, L.M., and F.L. Oswald. (2000). Personnel selection: Looking toward the future—remembering the past. Annual Review of Psychology, 51(1):631–664.

National Research Council. (2013). New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. R. Pool, Rapporteur. Committee on Measuring Human Capabilities: Performance Potential of Individuals and Collectives. Board on Behavioral, Cognitive, and Sensory Sciences, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.

Nye, C.D., R. Su, J. Rounds, and F. Drasgow. (2012). Vocational interests and performance: A quantitative summary of over 60 years of research. Perspectives on Psychological Science, 7(4):384–403.

Rumsey, M.G., and J.M. Arabian. (2014). Military enlistment selection and classification: Moving forward. Military Psychology, 26(3):221–251.

Sackett, P.R., and F. Lievens. (2008). Personnel selection. Annual Review of Psychology, 59:419–450.

Sellman, W.S., D.H. Born, W.J. Strickland, and J.J. Ross. (2010). Selection and classification in the U.S. military. In J.L. Farr and N.T. Tippens, Eds., Handbook of Employee Selection (pp. 679–704). London, UK: Routledge.

Strickland, W.J., Ed. (2005). A Longitudinal Examination of First Term Attrition and Reenlistment among FY 1999 Enlisted Accessions (Technical Report 1172). Arlington, VA: U.S. Army Research Institute for the Behavioral and Social Sciences.

Trent, T., and J.H. Laurence. (1993). Adaptability Screening for the Armed Forces. Washington, DC: Office of the Assistant Secretary of Defense (Force Management and Personnel).

Van Iddekinge, C.H., P.L. Roth, D.J. Putka, and S.E. Lanivich. (2011). Are you interested? A meta-analysis of relations between vocational interests and employee performance and turnover. Journal of Applied Psychology, 96(6):1,167–1,194.

This page intentionally left blank.