New Directions in Assessing Performance Potential of Individuals and Groups Workshop Summary (2013) / Chapter Skim
Currently Skimming:
2 Overview
2 Overview
Pages 5-20
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 5... ...
Well, [the military writer] Tom Ricks wrote recently that real insiders talk about personnel policy, because they know it provides the foundation for everything in the military." To set the stage for the workshop, the initial presentations provided invited participants and audience members with the past, present, and future context of military assessment and assessment science.
|
From page 6... ...
" Today, the Army's base assessment tool for cognitive assessment and vocational skills is the Armed Services Vocational Aptitude Battery, or ASVAB. It was originally developed in the 1970s, and in the 1990s it was updated for computerized adaptive testing as CAT-ASVAB (see Sands et al., 1999)
|
From page 7... ...
, as well as its applied program, in personnel assessment, and considered the dominant theories and measurements in use over the past 50 years or so. He then asked, "Given where we are, and the state of personnel testing in the military, can the committee provide recommenda tions for basic research in terms of scientific investment [to address the questions described in the following section]
|
From page 8... ...
Future Applications Finally, Goodwin explained that the Army is interested in moving beyond assessment for selection into the military and into assessment for a particular team or unit. "Right now," Goodwin said, "we predict indi vidual performance and individual potential fairly well" with respect to selecting recruits into the military and to assigning occupational job cat 2 Various presenters and participants at the workshop used the terms "construct," "factor," and "trait" in ways that appear to make them near-synonyms.
|
From page 9... ...
Is the goal to predict individual performance in a unit? It is not that simple, Goodwin observed, because individual performance in a unit depends on a number of other factors, including characteristics of other team members.
|
From page 10... ...
THE PRESENT AND FUTURE OF ASSESSMENT SCIENCE After Goodwin provided the past, present, and future context of military assessment, Fred Oswald, a professor of industrial and organi zational psychology at Rice University in Houston, Texas, provided an overview of the current state and likely future directions of the science of assessment, with an eye toward areas that might be of particular interest to the military. Domains of Performance There are countless determinants of performance that are relevant to success in one's job, Oswald noted.
|
From page 11... ...
Any reliable performance measure of interest to the Army, according to Oswald, should be a function of declarative knowledge, procedural knowledge, motivation, or a combination of these three determinants. Declarative knowledge includes knowing facts, rules, and strategies -- all things that can generally be tested with direct questioning.
|
From page 12... ...
A second general factor approach, which Oswald described as a con trolled heterogeneity approach, would sample deliberately from each of the four traits by conducting a factor analysis within each construct, identifying the items with the highest loadings within each of them, and then combining them into a single CSE scale. Oswald explained that CSE provides a general factor that predicts a wide variety of behavior or per formance outcomes, such as task performance, contextual performance, and job satisfaction.
|
From page 13... ...
The decision will also depend on various practical constraints, such as the amount of time available to develop or administer the measures, Oswald noted. Moving beyond this specific example, Oswald pointed out that simi lar measurement approaches are applicable to any domain represented by
|
From page 14... ...
It might require measures and technologies that go beyond classical test theory types of psychometric approaches and traditional approaches to measure development itself." As a partial answer to that question, Oswald described some of his own multitasking research. The criterion was a computer monitoring task in which the subject had to simultaneously monitor four quadrants on the computer screen (as illustrated in Figure 2-2)
|
From page 15... ...
Oswald added that at a normal pace, emotional stability was a strong predictor, such that "being calm and figuring out what was going on added incremental validity above ability." In the fast-paced condition, "everybody was a little bit anxious, and therefore emotional stability was less of a predictor as a trait." By contrast, conscientiousness had no cor relation with performance in the normal-pace condition and had a nega tive relationship with task accomplishment in the fast-paced condition, presumably because greater attention to detail would make it harder to speed up performance to keep pace with the task.
|
From page 16... ...
Echoing Gerald Goodwin's earlier observations, Oswald emphasized that, if it is more expensive to use the more complex tests and technologies of modern psychometrics, then one should expect there to be a valuable payoff. This payoff could take the form of increased validity, but it could also come in the form of improved reliability, tests that are easier to adapt, better item security, improved ability to analyze subgroup differences, and so on.
|
From page 17... ...
The Big Picture Continuing on, Oswald cautioned workshop participants that it is important to keep in mind that, whether the psychometrics being used are traditional or modern, the statistical concepts and practices underly ing test development focus on just a small part of the broader context that is involved in understanding and predicting performance. To point out some of the things that are often missed, Oswald spoke briefly about what he called "the big picture." For example, validity studies are generally static: "You develop a new measure, you see whether it predicts a criterion down the road," but often the data are all from one point in time.
|
From page 18... ...
Therefore, he concluded, invest ment in an expanded and integrated systems approach to personnel selec tion and classification could provide great value. DISCUSSION In the discussion that followed, committee member Randall Engle expressed the opinion that essentially all of the constructs that Oswald spoke about -- self-efficacy, working memory, and so on -- are more or less poorly understood in some ways.
|
From page 19... ...
. Personality Traits Related to Effectiveness of Junior and Senior Air Force Officers (USAF Personal Training Research, No.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.