National Academies Press: OpenBook

Field Evaluation in the Intelligence and Counterintelligence Context: Workshop Summary (2010)

Chapter: Appendix A: Workshop Agenda and Participants

« Previous: References
Suggested Citation:"Appendix A: Workshop Agenda and Participants." National Research Council. 2010. Field Evaluation in the Intelligence and Counterintelligence Context: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/12854.
×

Appendix A
Workshop Agenda and Participants

AGENDA

Workshop on Field Evaluation of Behavioral and Cognitive Sciences-Based Methods and Tools for Intelligence and Counterintelligence


Tuesday, September 22, 2009

8:00 a.m.

Workshop Check-In

8:20

Welcome

Barbara Wanchisen, director, Board on Behavioral, Cognitive, and Sensory Sciences, National Research Council

8:30

Background and Committee Genesis

Philip Rubin, committee chair and CEO, Haskins Laboratories

EXAMPLES OF BEHAVIORAL TOOLS AND METHODS

Moderator: Robert Fein, forensic psychologist, Harvard Medical School and committee member

8:45

Preliminary Credibility Assessment Screening System (PCASS)

Donald Krapohl, special assistant to the academy director, Defense Academy for Credibility Assessment

Suggested Citation:"Appendix A: Workshop Agenda and Participants." National Research Council. 2010. Field Evaluation in the Intelligence and Counterintelligence Context: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/12854.
×

9:15

Voice Stress Technologies

Philip Rubin

9:45

Break

10:00

APOLLO

Charles Twardy, research assistant professor, George Mason University

10:30

Alternative Competing Hypotheses (ACH)

Neil Thomason, senior lecturer (retired), University of Melbourne and committee member

11:00

Response and Discussion

Respondent Reactions

Steven Kleinman, consultant and strategist, Intelligence and National Security Policy

Paul Lehner, chief engineer, Information Technology Division, Center for Integrated Intelligence Systems, MITRE Corporation

General Discussion

12:30 p.m.

Working Lunch

MODELS OF EVALUATION IN OTHER FIELDS

Moderator: Philip Rubin

1:30

Challenges in Field Evaluation for Education

Grover (Russ) Whitehurst, senior fellow, Governance Studies, and director, Brown Center on Education Policy

2:00

Validation and Evaluation Practices in the Health Sciences

Lisa J. Colpe, senior scientist, Division of Services and Intervention Research, National Institute of Mental Health

2:30

Evidence-Based Practices in Criminal Justice

Cynthia Lum, deputy director, Center for Evidence-Based Crime Policy and assistant professor, Administration of Justice Department, George Mason University

3:00

Human Factors and Organizational Psychology: Application to Training Evaluation

Eduardo Salas, professor, Institute for Simulation and Training, University of Central Florida and committee member

3:30

Break

Suggested Citation:"Appendix A: Workshop Agenda and Participants." National Research Council. 2010. Field Evaluation in the Intelligence and Counterintelligence Context: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/12854.
×

3:45

Response and Discussion

Respondent Reaction: Possible Application to Intelligence and Counterintelligence Community Needs

Robert Boruch, professor, Graduate School of Education and Statistics, University of Pennsylvania and committee member

General Discussion

4:55

Closing Comments

Philip Rubin

5:00

Adjourn

Wednesday, September 23, 2009

9:00 a.m.

Workshop Check-In

9:10

Welcome

Barbara Wanchisen

9:20

Background and Review of Day One

Philip Rubin

ETHICAL, REGULATORY, AND CULTURAL CONSIDERATIONS

Moderator: Philip Rubin

9:30

Ethical Challenges in Translating Psychophysiology and Neuroscience to Technology for Intelligence and Counterintelligence

Adil Shamoo, editor-in-chief, Accountability in Research, University of Maryland School of Medicine

Commentary

Jonathan Moreno, David and Lyn Silfen University Professor, Department of Medical Ethics and Department of History and Sociology of Science, University of Pennsylvania and committee member

Discussion

10:30

Break

INTERNATIONAL PERSPECTIVES AND APPLICATION OF LESSONS FROM POLICING

Moderator: Philip Rubin

10:45

A UK Perspective

George Brander, Ministry of Defence, United Kingdom

11:15

Canadian Defense Validation Efforts

Suggested Citation:"Appendix A: Workshop Agenda and Participants." National Research Council. 2010. Field Evaluation in the Intelligence and Counterintelligence Context: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/12854.
×

 

David Mandel, defence scientist and group leader, Defence Research and Development Canada, Toronto

11:45

Application of Lessons from Forensics

Christian Meissner, associate professor and chair, Institutional Review Board, Department of Psychology, University of Texas at El Paso

12:15 p.m.

Working Lunch

1:15

Response and Discussion

Committee Reaction

General Discussion

SUMMATION AND IMPLICATIONS

Moderator: Philip Rubin

2:45

Summation, Discussion, and Suggestions for Future Evaluations

Summative Comments

Robert Fein

Committee Reactions

General Discussion

3:45

Closing Comments

Philip Rubin

4:00

Adjourn

PARTICIPANTS

Planning Committee Members:

Robert F. Boruch, University of Pennsylvania

Robert A. Fein, Harvard Medical School

Jonathan D. Moreno, University of Pennsylvania

Philip E. Rubin (Chair), Haskins Laboratories

Eduardo Salas, University of Central Florida

Neil Thomason, University of Melbourne

Carol H. Weiss, Harvard University


Other Workshop Presenters:

George Brander, Defence Science and Technology Laboratory, Cyber & Influence Centre; UK Ministry of Defence Human Factors Team

Lisa Colpe, National Institute of Mental Health

Steven M. Kleinman, consultant and strategist, Intelligence and National Security Policy

Donald Krapohl, Defense Academy for Credibility Assessment

Suggested Citation:"Appendix A: Workshop Agenda and Participants." National Research Council. 2010. Field Evaluation in the Intelligence and Counterintelligence Context: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/12854.
×

Paul Lehner, MITRE Corporation

Cynthia Lum, Center for Evidence-Based Crime Policy, George Mason University

David R. Mandel, Defence Research and Development Canada

Christian Meissner, Department of Psychology, University of Texas at El Paso

Adil E. Shamoo, School of Medicine, University of Maryland

Charles R. Twardy, George Mason University

Grover “Russ” Whitehurst, Brown Center on Education Policy


National Research Council Staff:

Laudan Aron, Division of Behavioral and Social Sciences and Education

Dan Melnick, National Research Council

Patricia Morison, Center for Education

Mary Ellen O’Connell, Board on Behavioral, Cognitive, and Sensory Sciences

Robert Pool, consultant

Miron Straf, Division of Behavioral and Social Sciences and Education

Barbara Wanchisen, Board on Behavioral, Cognitive, and Sensory Sciences

Renée L. Wilson Gaines, Board on Behavioral, Cognitive, and Sensory Sciences


Registered Attendees:

Hal Arkes, Ohio State University

Zunair Ashfaq, University of Pennsylvania

Emma Barrett, UK Ministry of Defence

Monique E. Beaudoin, National Defense University, Ottawa

Ronald Benefield, Office of the Naval Counterintelligence Executive

Sujeeta Bhatt, Defense Intelligence Agency

Anthony Boemio, Booz Allen Hamilton

Susan Brandon, Defense Counterintelligence and Human Intelligence Center, Defense Intelligence Agency

Troy Brown, Defense Academy for Credibility Assessment

James Bruce, RAND Corporation

Dennis Buede, Innovative Decisions, Inc.

Michael Cassidy, Marymount University

Ron Ceasar, Ron Ceasar Photography

Paul Chatelier, Naval Postgraduate School

Michael Cheek, ExecutiveBiz

A. Egon Cholakian, Interdisciplinary Research Development Fund Project

Brian Colder, National Geospatial-Intelligence Agency

Suggested Citation:"Appendix A: Workshop Agenda and Participants." National Research Council. 2010. Field Evaluation in the Intelligence and Counterintelligence Context: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/12854.
×

Keith Devereaus, U.S. Department of Homeland Security-Citizenship and Immigration Services

Ivy Estabrooke, Office of Naval Research

Pamela Flattau, Institute for Defense Analyses Science and Technology Policy Institute

Angelyn Flowers, University of the District of Columbia

Melanie Goodrich, Office of Naval Research

Leslie Goodyear, National Science Foundation

Julie Gravallese, MITRE Corporation

Hal Greenwald, MITRE Corporation

Joe Heaps, National Institute of Justice

Kristin Heckman, MITRE Corporation

Cheryl Hendrickson Caster, American Institutes for Research

Georgia Holmer, Pherson Associates, LLC

Alexis Jeannotte, Avian Engineering

David Kamien, Mind-Alliance Systems, LLC

Eric Kaufman, National Counterterrorism Center

Heather Kelly, American Psychological Association

A.T. Kendall, social research consultant

Mary Lee Kingsley, M.L. Kingsley, LLC

Adam Korobow, Logistics Management Institute

Robert Krikorian, U.S. Department of State

Howard Kurtzman, American Psychological Association

Thomas LaHann, Science Applications International Corporation

Eric Lang, Defense Personnel Security Research Center

Thomas Lawrence, Pennsylvania Army National Guard

Richard Lempert, Department of Homeland Security Science and Technology

Heather Leon, Navy Marine Corps Intelligence Training Center

Tod Levitt, George Mason University

Tiffany Lightbourn, Department of Homeland Security–U.S. Citizenship and Immigration Services

Deborah Loftis, Defense Intelligence Agency

Martha Lorber, MITRE Corporation

Kel McClanahan, National Security Counselors

Gary McClelland, University of Colorado

Robert McCreight, George Washington University

Nancy Merritt, National Institute of Justice

Erica Michael, University of Maryland Center for Advanced Study of Language

Avra Michelson, MITRE Corporation

Thomas Moore, Defense Advantage Research Projects Agency

Dwayne Norris, American Institutes for Research

Suggested Citation:"Appendix A: Workshop Agenda and Participants." National Research Council. 2010. Field Evaluation in the Intelligence and Counterintelligence Context: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/12854.
×

William Norris, Defense Academy for Credibility Assessment

J. O’Connor, Department of Homeland Security Human Factors/Behavioral Sciences Division

Jason Ogden, Customs and Border Protection

Randolph Pherson, Pherson Associates, LLC

Amy Pollick, Association for Psychological Science

Matthew Potts, Marine Corps Intelligence Association, Inc.

Jim Powlen, Logos Technologies, Inc.

Joseph Psotka, U.S. Army Research Institute

Leslie Richards, University of the District of Columbia

Steven Rieber, Office of the Director of National Intelligence

Michael Rugnetta, Center for American Progress

Adam Russell, Intelligence Advanced Research Projects Activity

Christina Saylor, U.S. Special Operations Command

Grace Scarborough, Pherson Associates, LLC

Dylan Schmorrow, Office of the Secretary of Defense

Rachael Scholz, Booz Allen Hamilton

Nathan Schwade, Sandia National Laboratories

Alan Schwartz, PolicyFutures, LLC

Dan Schwartz, Schwartz and Schwartz, LLC

Allison Smith, Department of Homeland Security Science and Technology Directorate

David L. Smith, Navy Marine Corps Intelligence Training Center

Robert A. Smith, University of Maryland

Jonathan Snider, Defense Threat Reduction Agency

Kevin Spence, Department of Homeland Security

Barry Spodak, Action Training Institute

Frank Stech, MITRE Corporation

Scott Tousley, MITRE Corporation

Edwin Urie, Henley-Putnam University

Anthony B. Veney, U.S. Department of Defense

Jeremy Wacksman, U.S. Department of the Navy, U.S. Department of Defense

Mark Weiss, National Science Foundation

Joe Wholey, University of Southern California

Deborah York, Phoenix Consulting Group

Laura Zimmerman, Applied Research Associates

Suggested Citation:"Appendix A: Workshop Agenda and Participants." National Research Council. 2010. Field Evaluation in the Intelligence and Counterintelligence Context: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/12854.
×

This page intentionally left blank.

Suggested Citation:"Appendix A: Workshop Agenda and Participants." National Research Council. 2010. Field Evaluation in the Intelligence and Counterintelligence Context: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/12854.
×
Page 91
Suggested Citation:"Appendix A: Workshop Agenda and Participants." National Research Council. 2010. Field Evaluation in the Intelligence and Counterintelligence Context: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/12854.
×
Page 92
Suggested Citation:"Appendix A: Workshop Agenda and Participants." National Research Council. 2010. Field Evaluation in the Intelligence and Counterintelligence Context: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/12854.
×
Page 93
Suggested Citation:"Appendix A: Workshop Agenda and Participants." National Research Council. 2010. Field Evaluation in the Intelligence and Counterintelligence Context: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/12854.
×
Page 94
Suggested Citation:"Appendix A: Workshop Agenda and Participants." National Research Council. 2010. Field Evaluation in the Intelligence and Counterintelligence Context: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/12854.
×
Page 95
Suggested Citation:"Appendix A: Workshop Agenda and Participants." National Research Council. 2010. Field Evaluation in the Intelligence and Counterintelligence Context: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/12854.
×
Page 96
Suggested Citation:"Appendix A: Workshop Agenda and Participants." National Research Council. 2010. Field Evaluation in the Intelligence and Counterintelligence Context: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/12854.
×
Page 97
Suggested Citation:"Appendix A: Workshop Agenda and Participants." National Research Council. 2010. Field Evaluation in the Intelligence and Counterintelligence Context: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/12854.
×
Page 98
Next: Appendix B: Relevant Readings »
Field Evaluation in the Intelligence and Counterintelligence Context: Workshop Summary Get This Book
×
Buy Paperback | $41.00 Buy Ebook | $32.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

On September 22-23, 2009, the National Research Council held a workshop on the field evaluation of behavioral and cognitive sciences—based methods and tools for use in the areas of intelligence and counterintelligence. Broadly speaking, the purpose of the workshop was to discuss the best ways to take methods and tools from behavioral science and apply them to work in intelligence operations. More specifically, the workshop focused on the issue of field evaluation—the testing of these methods and tools in the context in which they will be used in order to determine if they are effective in real-world settings.

This book is a summary and synthesis of the two days of presentations and discussions that took place during the workshop. The workshop participants included invited speakers and experts from a number of areas related to the behavioral sciences and the intelligence community. The discussions covered such ground as the obstacles to field evaluation of behavioral science tools and methods, the importance of field evaluation, and various lessons learned from experience with field evaluation in other areas.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!