New Directions in Assessing
Performance Potential
of Individuals and Groups

WORKSHOP SUMMARY

Robert Pool, Rapporteur

Committee on Measuring Human Capabilities:
Performance Potential of Individuals and Collectives

Board on Behavioral, Cognitive, and Sensory Sciences

Division of Behavioral and Social Sciences and Education

NATIONAL RESEARCH COUNCIL
OF THE NATIONAL ACADEMIES

THE NATIONAL ACADEMIES PRESS
Washington, D.C.
www.nap.edu



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page R1
New Directions in Assessing Performance Potential of Individuals and Groups WORKSHOP SUM MA RY Robert Pool, Rapporteur Committee on Measuring Human Capabilities: Performance Potential of Individuals and Collectives Board on Behavioral, Cognitive, and Sensory Sciences Division of Behavioral and Social Sciences and Education

OCR for page R1
THE NATIONAL ACADEMIES PRESS  500 Fifth Street, NW  Washington, DC 20001 NOTICE: The project that is the subject of this report was approved by the Govern­ ing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineer­ ing, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropri­ ate balance. This study was supported by Contract/Grant No. W-911NF-12-1-0504 between the National Academy of Sciences and the U.S. Department of the Army. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the organiza­ tions or agencies that provided support for the project. International Standard Book Number-13:  978-0-309-29044-9 International Standard Book Number-10:  0-309-29044-9 Additional copies of this report are available from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu. Copyright 2013 by the National Academy of Sciences. All rights reserved. Printed in the United States of America Suggested citation: National Research Council. (2013). New Directions in Assess- ing Performance Potential of Individuals and Groups: Workshop Summary. R. Pool, R ­ apporteur. Committee on Measuring Human Capabilities: Performance Potential of Individuals and Collectives, Board on Behavioral, Cognitive, and Sensory Sci­ ences. Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.

OCR for page R1
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal govern­ ment on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its mem­ bers, sharing with the National Academy of Sciences the responsibility for advis­ ing the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. C. D. Mote, Jr., is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in pro­ viding services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. C. D. Mote, Jr., are chair and vice chair, respectively, of the National Research Council. www.national-academies.org

OCR for page R1

OCR for page R1
COMMITTEE ON MEASURING HUMAN CAPABILIITES: PERFORMANCE POTENTIAL OF INDIVIDUALS AND COLLECTIVES Jack W. Stuster (Chair), Anacapa Sciences, Inc., Santa Barbara, CA Georgia T. Chao, Eli Broad Graduate School of Management, Michigan State University Randall W. Engle, School of Psychology, Georgia Institute of Technology Leaetta Hough, Dunnette Group, Ltd., Saint Paul, MN Patrick C. Kyllonen, Center for Academic and Workforce Readiness and Success, Educational Testing Service, Princeton, NJ John J. McArdle, Department of Psychology, University of Southern California Stephen Stark, Department of Psychology, University of South Florida Cherie Chauvin, Study Director Tina Winters, Associate Program Officer Renée L. Wilson Gaines, Senior Program Assistant v

OCR for page R1
BOARD ON BEHAVIORAL, COGNITIVE, AND SENSORY SCIENCES John T. Cacioppo (Chair), Department of Psychology, University of Chicago Linda M. Bartoshuk, Department of Community Dentistry and Behavioral Sciences, University of Florida Richard J. Bonnie, Institute of Law, Psychiatry and Public Policy, University of Virginia Jennifer S. Cole, Department of Linguistics, University of Illinois at Urbana-Champaign Alice H. Eagly, Department of Psychology, Northwestern University Susan T. Fiske, Department of Psychology and Woodrow Wilson School of Public and International Affairs, Princeton University Daniel R. Ilgen, Department of Psychology, Michigan State University Nina G. Jablonski, Department of Anthropology, Pennsylvania State University James S. Jackson, Institute for Social Research, University of Michigan, Ann Arbor Jonathan D. Moreno, Department of Medical Ethics and Health Policy and Department of History and Sociology of Science, University of Pennsylvania Michael I. Posner, Department of Psychology, University of Oregon (Emeritus) Marcus E. Raichle, Mallinckrodt Institute of Radiology, Washington University in St. Louis Valerie F. Reyna, Human Neuroscience Institute, Cornell University Richard M. Shiffrin, Department of Psychological and Brain Sciences, Indiana University Jeremy M. Wolfe, Brigham and Women’s Hospital, Departments of Ophthalmology and Radiology, Harvard Medical School Barbara A. Wanchisen, Director Jatryce Jackson, Program Associate vi

OCR for page R1
Acknowledgments T his workshop summary is based on the proceedings of a ­ ublic p workshop held on April 3-4, 2013, convened by the Board on Behavioral, Cognitive, and Sensory Sciences, and planned by the Committee on Measuring Human Capabilities: Performance Potential of Individuals and Collectives. The planning committee members identified research areas of interest and specific presenters, organized the agenda, and facilitated roundtable discussions; however, they did not participate in the writing of this summary. Its contents reflect their diligent planning efforts, the insightful presentations of invited experts, and the thought- provoking roundtable discussions of all invited participants as well as the general audience. The workshop, and the larger study for which it is a part, was spon­ sored by the U.S. Army Research Institute for the Behavioral and Social Sciences (ARI). In particular, the committee wishes to thank Gerald G ­ oodwin, chief of foundational science at ARI, and his entire research team, for their support and guidance throughout the planning process. Among the National Research Council (NRC) staff, special thanks are due to Barbara A. Wanchisen who provided oversight to the process of planning and hosting the workshop. Additionally, special thanks to Tina Winters, associate program officer, who was in many ways the life of this workshop. She dedicated countless hours to organizing this large public event, including working extensively with each presenter and panelist to refine their planned remarks in light of the event’s goals and the planning committee’s intentions. Renée L. Wilson Gaines, senior program assistant, vii

OCR for page R1
viii ACKNOWLEDGMENTS also provided critical logistical and administrative support to the event, including coordinating the details of the event’s special location. We also thank NRC consultants Robert Pool for his assistance in drafting the sum­ mary and Robert Katt for final editing of the manuscript. And finally we thank the executive office reports staff of the Division of Behavioral and Social Sciences and Education, especially Kirsten Sampson Snyder, who managed the review process, and Yvonne Wise, who oversaw the final publication process. This summary has been reviewed in draft form by individuals cho­ sen for their diverse perspectives and technical expertise, in accordance with procedures approved by the NRC’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published sum­ mary as sound as possible and to ensure that the summary meets institu­ tional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this summary: Jennifer S. Cole, Department of Linguistics, University of Illinois at Urbana-­ hampaign; C Neal W. Schmitt, Department of Psychology and Management, ­ ichigan M State University; Robert Sternberg, Department of Psychology, Yale University; William J. Strickland, president and chief executive officer, Human Resources Research Organization, Alexandria, Virginia; and Mary D. Zalesny, office of the Chief of Staff of the Army Strategic Studies Group, Arlington, Virginia. Although the reviewers listed above provided many constructive comments and suggestions, they were not asked to endorse the content of the report nor did they see the final draft of the report before its release. The review of this report was overseen by Nancy T. Tippins, senior vice president and managing principal, Valtera Corporation. Appointed by the NRC, she was responsible for making certain that an independent examination of this summary was carried out in accordance with insti­ tutional procedures and that all review comments were carefully consid­ ered. Responsibility for the final content of this summary rests entirely with the rapporteur and the institution.

OCR for page R1
Contents 1 INTRODUCTION 1 Reference, 4 2 OVERVIEW 5 The Context of Military Assessment, 6 The Present and Future of Assessment Science, 10 Discussion, 18 References, 19 3 NEW CONSTRUCTS FOR ASSESSING INDIVIDUALS 21 Neurobehavioral Constructs, 21 Working Memory Capacity and Executive Attention, 28 The Agentic Self: Action-Control Beliefs, 38 Discussion, 42 Interests, 44 References, 48 4 EMERGING UNDERSTANDINGS OF GROUP-RELATED CHARACTERISTICS 51 Collective Intelligence, 51 Predicting Team Performance, 59 Team Assembly, 63 Discussion, 72 References, 77 ix

OCR for page R1
x CONTENTS 5 NEW APPROACHES AND CAPABILITIES IN ASSESSMENT 79 A Taxonomy for Ways to Improve Selection Systems, 80 Psychometrics for a New Generation of Assessments, 87 Discussion, 94 References, 98 6 ETHICAL ISSUES RELATED TO PERSONNEL ASSESSMENT AND SELECTION 101 A Case Study of the Future, 101 Codes of Ethics, 102 Ethical Issues Related to Emerging Assessment Technologies, 104 References, 107 7 A WAY FORWARD 109 The Boring Box, 109 Final Thoughts, 114 References, 116 APPENDIX: Workshop Agenda and Participants 117