Appendix A
Workshop Agenda and Selection of Additional Topics Considered for Workshop Agenda
AGENDA
Workshop on New Directions in Assessing Individuals and Groups April 3-4, 2013
Workshop Goals
- Facilitate interdisciplinary dialogue on the current and future state-of-the-science in measurement of individual capabilities and the combination of individual capabilities to create collective capacity to perform.
- Inform the design of a maximally effective selection and assignment system.
Wednesday, April 3
| 8:00 am | Workshop Check-In |
| 9:00 | Welcome from the National Research Council |
|
Robert Hauser, Executive Director, Division of Behavioral and Social Sciences and Education |
| Overview of the Board on Behavioral, Cognitive, and Sensory Sciences | |
|
Barbara A. Wanchisen, Director, Board on Behavioral, Cognitive, and Sensory Sciences |
|
| Introductions | |
| 9:30 | Workshop Objectives and Study Overview |
|
Jack Stuster, Anacapa Sciences, Inc., and Chair, Committee on Measuring Human Capabilities |
|
| 10:00 | Sponsor’s Perspective |
|
Gerald (Jay) Goodwin, Chief, Foundational Science, U.S. Army Research Institute for the Behavioral and Social Sciences |
|
| 10:45 | Break |
| 11:00 | Setting the Stage: The Evolving Goals of Candidate Testing and Its Role in Personnel Selection |
|
Fred Oswald, Rice University |
|
| 12:00 pm | Keynote Address: Psychometrics for a New Generation of Assessments |
|
Alina von Davier, Research Director, Center for Advanced Psychometrics, Educational Testing Service |
|
| 12:30 | Working Lunch |
|
Jack Stuster, Chair |
|
|
Topic: Discussion of ideas presented in Keynote Address |
|
| 1:15 | Emerging Constructs and Theory |
| Part One: Invited Presentations | |
| A Psychoneurometric Approach to Individual-Differences Assessment | |
|
Christopher Patrick, Florida State University |
|
| The Emerging Cognitive Constructs of Working Memory Capacity and Executive Attention | |
|
Michael Kane, University of North Carolina, Greensboro |
|
| The Agentic Self: Action Control Beliefs | |
|
Todd Little, University of Kansas |
|
| Part Two: Roundtable Discussion with Committee Members and Invited Presenters | |
| 3:30 | Break |
| 3:45 | Ethical Implications of Future Testing Techniques and Personnel Selection Paradigms |
|
Rodney Lowman, Alliant International University |
|
| Reactions from Committee Members | |
| 4:45 | Conclude Day One |
| Thursday, April 4 | |
| 8:30 am | Day Two Workshop Check-In |
| 9:00 | Summary of Day One and Overview of Day Two |
|
Jack Stuster, Anacapa Sciences, Inc., and Chair, Committee on Measuring Human Capabilities |
|
| 9:15 | Measuring Individual Differences and Predicting Individual Performance |
| Part One: Invited Presentations | |
| Taxonomic Structure for Thinking About Ways to Improve the Quality of Selection Systems | |
|
Paul Sackett, University of Minnesota |
|
| Rethinking Interests | |
|
James Rounds, University of Illinois at |
|
|
Urbana-Champaign |
|
| Assessing Cognitive Skills: Case History, Diagnosis, and Treatment Plan | |
|
Earl Hunt, University of Washington |
|
| 10:15 | Break |
| 10:30 | Measuring Individual Differences and Predicting Individual Performance, Continued |
| Part Two: Roundtable Discussion with Committee Members and Invited Presenters | |
| 12:00 pm | Working Lunch |
|
Jack Stuster, Chair |
|
|
Topic: Continued roundtable discussion with committee members and invited presenters |
|
| 12:45 | Group Composition Processes and Performance |
| Part One: Invited Presentations | |
|
Team Composition: Theory, Practice, and the Future Scott Tannenbaum, Group for Organizational Effectiveness |
|
| Understanding and Enabling the Collective Capabilities of Teams | |
|
Leslie DeChurch, Georgia Institute of Technology |
|
| Collective Intelligence in the Performance of Human Groups | |
|
Anita Williams Woolley, Carnegie Mellon University |
|
| Part Two: Roundtable Discussion with Committee Members and Invited Presenters | |
| 3:15 | Break |
| 3:30 | Cross-cutting Links and Research Gaps: Roundtable Discussion with Committee Members and All Invited Presenters |
| 4:00 | Workshop Implications |
| Part One: Invited Presentation | |
| Summary of Emerging Themes | |
|
Randall Engle, Georgia Institute of Technology and Member, Committee on Measuring Human Capabilities |
|
| Part Two: Reactions from Invited Presenters and Committee Members | |
| 4:45 | Closing Comments |
|
Jack Stuster, Chair |
|
| 5:00 | Adjourn |
SELECTION OF ADDITIONAL TOPICS CONSIDERED FOR WORKSHOP AGENDA
The following list is an unprioritized selection of the topics developed by the committee through brainstorming and deliberation processes as potential topics for inclusion during the workshop. Many of the topics were included in the final workshop agenda, while others were not for a variety of reasons. Some potentially important topics were excluded due to reasons such as time limitations of the event, availability of key presenters, compatibility with broad categories selected for emphasis at the workshop, and the committee’s assessment of the likely value of discussion of particular topics over others. Some key topics not included in the workshop were included in later data gathering sessions of the committee during the study’s second phase, as listed in Appendix B. This list is not all-inclusive, and it does not document all of the topics considered through two years of in-person meetings, conference calls, emails, and other information sharing that occurred between committee members, invited experts, the study sponsor, and National Research Council staff in order to arrive at the contents of this final report.
Measurement Techniques
Unobtrusive testing methods
Bayesian modeling
Machine learning
Nonparametric analyses
Context
Quantitative group decision making
Measurement at an Individual Level
Constructs of cognition: knowledge, reasoning, memory, speed of processing, visualization
Biodata
Experience sampling
21st century skills
Vocational interest measurements
Situational judgment inventories
Situation awareness
Implicit biases
Modeling
Decision theoretic advances
Behavioral economics/game theory
Medical decision making
Information processing models
Group modeling
Methods
Asynchronous interviewing
Automatic scoring
Communication analysis
Computational linguistics
Latent semantic analysis
Data mining
Likert scales
Clinical interviews
Sociometry
Simulations and gaming
Synthetic validation
Unproctored tests
Neuroscience and Psychophysiology
Psychoneurometrics
Blood chemistry
Biomarkers
