INTELLIGENCE ANALYSIS FOR TOMORROW

Advances from the Behavioral and Social Sciences

Committee on Behavioral and Social Science Research to Improve Intelligence Analysis for National Security

Board on Behavioral, Cognitive, and Sensory Sciences

Division of Behavioral and Social Sciences and Education

NATIONAL RESEARCH COUNCIL
OF THE NATIONAL ACADEMIES

THE NATIONAL ACADEMIES PRESS

Washington D.C.
www.nap.edu



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page R1
INTELLIGENCE ANALYSIS FOR TOMORROW Advances from the Behavioral and Social Sciences Committee on Behavioral and Social Science Research to Improve Intelligence Analysis for National Security Board on Behavioral, Cognitive, and Sensory Sciences Division of Behavioral and Social Sciences and Education

OCR for page R1
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, N.W. Washington, DC 20001 NOTICE: The project that is the subject of this report was approved by the Govern- ing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineer- ing, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropri- ate balance. This study was supported by Grant No. 2008*1199327*000 between the National Academy of Sciences and the Office of the Director of National Intelligence. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the organiza- tions or agencies that provided support for the project. International Standard Book Number-13: 978-0-309-16342-2 International Standard Book Number-10: 0-309-16342-0 Additional copies of this report are available from National Academies Press, 500 Fifth Street, N.W., Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area); Internet, http://www.nap.edu. Copyright 2011 by the National Academy of Sciences. All rights reserved. Printed in the United States of America Suggested citation: National Research Council. (2011). Intelligence Analysis for Tomorrow: Advances from the Behavioral and Social Sciences. Committee on Behavioral and Social Science Research to Improve Intelligence Analysis for National Security, Board on Behavioral, Cognitive, and Sensory Sciences, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.

OCR for page R1
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Acad- emy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineer- ing programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is presi- dent of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Insti- tute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sci- ences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Coun- cil is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council. www.national-academies.org

OCR for page R1

OCR for page R1
COMMITTEE ON BEHAVIORAL AND SOCIAL SCIENCE RESEARCH TO IMPROVE INTELLIGENCE ANALYSIS FOR NATIONAL SECURITY Baruch Fischhoff (Chair), Department of Social and Decision Sciences and Department of Engineering and Public Policy, Carnegie Mellon University Hal R. Arkes, Department of Psychology, Ohio State University Bruce Bueno de Mesquita, Department of Politics, New York University and Hoover Institution, Stanford University Thomas Fingar, Freeman Spogli Institute for International Studies, Stanford University Reid Hastie, Chicago Booth Business School, University of Chicago Edward H. Kaplan, School of Management, School of Public Health, and School of Engineering and Applied Science, Yale University Steven W.J. Kozlowski, Department of Psychology, Michigan State University Gary H. McClelland, Department of Psychology, University of Colorado Kiron K. Skinner, Department of Social and Decision Sciences, Carnegie Mellon University and Hoover Institution, Stanford University Barbara A. Spellman, Department of Psychology and School of Law, University of Virginia Philip E. Tetlock, Department of Psychology and Wharton School of Business, University of Pennsylvania Catherine H. Tinsley, McDonough School of Business, Georgetown University Amy Zegart, School of Public Affairs, University of California, Los Angeles, and Hoover Institution, Stanford University Cherie Chauvin, Study Director Robert Pool, Consultant Matthew McDonough, Senior Program Assistant (through April 2010) Gary Fischer, Senior Program Assistant (from April 2010) v

OCR for page R1
BOARD ON BEHAVIORAL, COGNITIVE, AND SENSORY SCIENCES Philip E. Rubin (Chair), Haskins Laboratories and Yale University Lisa Feldman Barrett, Department of Psychology, Northeastern University Linda M. Bartoshuk, College of Dentistry, University of Florida Richard J. Bonnie, Institute of Law, Psychiatry and Public Policy, University of Virginia John T. Cacioppo, Department of Psychology, The University of Chicago Susan E. Carey, Department of Psychology, Harvard University Susan T. Fiske, Department of Psychology, Princeton University Nina G. Jablonski, Department of Anthropology, Pennsylvania State University Patricia K. Kuhl, Department of Speech and Hearing Sciences, University of Washington Jonathan D. Moreno, Departments of Medical Ethics and History and Sociology of Science, University of Pennsylvania Richard E. Nisbett, Department of Psychology, University of Michigan Michael I. Posner, Department of Psychology, University of Oregon (Emeritus) Valerie F. Reyna, Departments of Human Development and Psychology, Cornell University Richard M. Shiffrin, Psychology Department, Indiana University Brian A. Wandell, Department of Psychology, Stanford University Barbara A. Wanchisen, Director Mary Ellen O’Connell, Deputy Director Christie R. Jones, Program Associate vi

OCR for page R1
Acknowledgments This study was sponsored by the Office of the Director of National Intelligence (ODNI). The committee is grateful to the many ODNI staff who made valuable presentations and provided informative materials to the committee. The committee also benefited from presentations and comments from experts in intelligence analysis who spoke at committee meetings: James Bruce, The RAND Corporation; Grey Burkhart, Defense Intelligence Agency (DIA); Roger George, Central Intelligence Agency (CIA); Richards Heuer, Jr., CIA (Ret.); Robert Levine, Sherman Kent School for Intelligence Analysis at CIA University; Robert Morris, DIA. To further its understanding of the realities of the IC, the committee engaged in classified panel discussions with working analysts from across the IC. The panelists represented a wide range of experiences, skills, agen- cies, and length of service. Their comments were critical in allowing the committee to understand the greatest needs of working analysts and thus how the behavioral and social sciences can enhance their strengths and meet their challenges. We thank panel discussants from the ODNI, CIA, Depart- ment of Homeland Security (DHS), DIA, National Geospatial Agency, and Pherson Associates. The committee appreciates the information on training and tradecraft provided by the Sherman Kent School for Intelligence Analysis, DIA’s Joint Military Intelligence Training Center and Directorate for Analysis Research, and administrators of the ODNI’s Analysis 101 training program. We thank the following individuals for their presentations at the public workshop on May 15, 2009: Mark Chassin, The Joint Commission; Kay vii

OCR for page R1
viii ACKNOWLEDGMENTS Dickersin, U.S. Cochrane Center; Donald Hanle, National Defense Intelli- gence College; David Mandel, Defense Research and Development Canada; and Stephen Marrin, Mercyhurst College. For his lively and insightful keynote address, we also thank Lt. Gen. Brent Scowcroft, U.S. Air Force (Ret.). Among the National Research Council (NRC) staff, special thanks are due to Barbara Wanchisen and Mary Ellen O’Connell who provided oversight and support of the study. Two senior program assistants, Matt McDonough and Gary Fischer provided administrative and logistic support over the course of the study. We also thank an NRC consultant, Robert Pool, for his extensive assistance in editing many drafts of the report. And finally we thank the executive office reports staff of the Division of Behav- ioral and Social Sciences and Education, especially Eugenia Grohman, who provided valuable help with the editing and production of the report, and Kirsten Sampson Snyder, who managed the report review process. This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with pro- cedures approved by the NRC’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report: Lynn R. Eden, Freeman Spogli Institute for Interna- tional Studies, Center for International Security and Cooperation, Stanford University; Susan T. Fiske, Department of Psychology, Princeton University; Carl W. Ford, Jr., National Intelligence Council Associate, Office of the Director of National Intelligence and School of Foreign Service, George- town University; Katherine J. Hall, Global Analysis, BAE Systems; Thomas Hammond, Department of Political Science, Michigan State University; Thom J. Hodgson, Industrial and Systems Engineering Department, North Carolina State University; Frederick S. Kaplan, Division of Orthopaedic Molecular Medicine, Department of Orthopaedic Surgery, Hospital of The University of Pennsylvania; J. Keith Murnighan, Kellogg School of Manage- ment, Northwestern University; and Robert M. Oliver, Operations Research and Engineering Science Emeritus, University of California, Berkeley. Although the reviewers listed above provided many constructive com- ments and suggestions, they were not asked to endorse the content of the report nor did they see the final draft of the report before its release. The review of this report was overseen by Richard J. Bonnie, Institute of Law, Psychiatry, and Public Policy, University of Virginia and R. Stephen Berry, Gordon Center for Integrative Studies, Department of Chemistry and

OCR for page R1
ix ACKNOWLEDGMENTS James Franck Institute, University of Chicago. Appointed by the National Research Council, they were responsible for making certain that an inde- pendent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully con- sidered. Responsibility for the final content of this report rests entirely with the author and the institution. Baruch Fischhoff, Chair Cherie Chauvin, Study Director Committee on Behavioral and Social Science Research to Improve Intelligence Analysis for National Security

OCR for page R1

OCR for page R1
Contents Preface xiii Executive Summary 1 1 Challenges for the Intelligence Community 5 Structure of the Intelligence Community 5 Mission-Related Challenges 9 Challenges for Complex Organizations 13 Charge to the Committee 17 References 19 2 Learning and Evaluation 23 Learning 23 Evaluation 28 A Realistic Agenda for Change 29 References 29 3 Analysis 33 Expert Judgment 34 Structured Analytic Techniques 35 Probability Theory 36 Decision Analysis 39 Statistics and Data Analysis 40 Signal Detection Theory 40 Game Theory 41 xi

OCR for page R1
xii CONTENTS Operations Research 42 Qualitative Analysis 43 Summary 44 References 44 4 The Workforce 49 Recruitment and Selection 50 Development 51 Evaluation 56 References 57 5 Collaboration 61 Forms and Dimensions 62 Benefits 63 The Right Level 64 Barriers 66 Evaluation 68 References 70 6 Communication 73 Obstacles to Effective Communication 74 Communicating Analytical Results 74 Communicating Analytical Needs 76 Organization and Evaluation 77 References 78 7 Conclusions and Recommendations 81 Historic Context 83 A Behavioral and Social Sciences Foundation 84 Analytic Methods 85 Workforce Development 87 Collaboration 89 Communication 90 Conclusion 91 References 92 Appendixes A Contents: Intelligence Analysis: Behavioral and Social Scientific 95 Foundations B Biographical Sketches of Committee Members and Staff 97

OCR for page R1
Preface In 2008, the Office of the Director of National Intelligence (ODNI) asked the National Research Council (NRC) to establish a committee to synthesize and assess evidence from the behavioral and social sciences rel- evant to analytic methods and their potential application for the U.S. intel- ligence community (IC). The NRC thanks the Central Intelligence Agency’s Sherman Kent School for Intelligence Analysis, which supported a planning meeting early in the development of this study. Valuable insights, informa- tion, and questions resulting from those preliminary discussions greatly contributed to the study’s success. In response to the request from ODNI, the NRC established the Com- mittee on Behavioral and Social Science Research to Improve Intelligence Analysis for National Security, under the oversight of the Board on Behav- ioral, Cognitive, and Sensory Sciences. This report is the work of that committee. As specified in its charge, the committee restricted its focus to the analytic component of the IC, including development of the analytic workforce. Although the committee recognizes that analysts’ work depends on that of collectors and support personnel, these relations are beyond the scope of this report. Consistent with its charge, the committee has focused on the behavioral and social science related to “critical problems of individual and group judgment.” We note that the behavioral and social sciences can make contributions to other aspects of the IC’s mission (e.g., understanding deception, paths to terrorism, field operations). Members of the committee were volunteers, carefully selected by the NRC to cover a spectrum of relevant academic specialties and to bring expertise in both basic research and practical applications in diverse set- xiii

OCR for page R1
xiv PREFACE tings including private organizations, government, and the military. Several committee members have had significant experience with national security issues, including work with the IC. The study extended over a 30-month period. During its initial phase, the committee hosted three data-gathering meetings and a 1-day public workshop. At the workshop, committee members heard from speakers in several parallel endeavors, including the application of the behavioral and social sciences in Canadian intelligence and the emergence of evidence- based decision making in medicine. The committee also received briefings from current and former intelligence officers, as well as from consultants to the IC. These briefings provided the committee with critical context for assessing applications of the behavioral and social sciences to the unique needs, challenges, and circumstances of the IC. As a foundation for the deliberations summarized in this consensus report, each committee member authored a paper (in two cases with coauthors) that reviewed the research literature on a topic that the committee identified as central to fulfilling its charge. These papers are published as a companion volume, Intelligence Analysis: Behavioral and Social Scientific Foundations. Throughout its deliberations, the committee considered the realities of the IC as it developed the recommendations presented in the last chapter of this report. As a result, the committee’s recommendations focus on changes that are both important and feasible. These recommendations offer practi- cal ways to apply the behavioral and social sciences, which will bring the IC substantial immediate and longer-term benefits with modest costs and minimal disruption. In the course of preparing this report, each committee member took an active role in drafting chapters, leading discussions, and reading and commenting on successive drafts. The committee deliberated all aspects of this report, and its final content is the result of their tremendous effort, vision, and determination. Baruch Fischhoff, Chair Cherie Chauvin, Study Director Committee on Behavioral and Social Science Research to Improve Intelligence Analysis for National Security