Executive Summary
The intelligence community (IC) plays an essential role in the national security of the United States, and its success has always depended on being smarter and more agile than America’s adversaries. Today’s threat environment presents intense pressures to retain this edge through timely assessments and rapid adaptation.
The IC deserves great credit for its commitment to self-scrutiny and improvement, including its investments in lessons-learned, training, and collaboration procedures. Yet these efforts have been only weakly informed by the behavioral and social sciences. At the same time, post-9/11 changes in the IC have created unprecedented demands for that knowledge. In this context, the Office of the Director of National Intelligence (ODNI) asked the National Research Council to conduct a study to
synthesize and assess the behavioral and social science research evidence relevant (1) to critical problems of individual and group judgment and of communication by intelligence analysts and (2) to kinds of analytic processes that are employed or have potential in addressing these problems.
The study charge also asked for recommendations on analytic practices “to the extent the evidence warrants” and for future research, including the identification of impediments to implementation.
CONCLUSIONS
One of the most important things that the IC can learn from the behavioral and social sciences is how to characterize and evaluate its analytic assumptions, methods, technologies, and management practices. Behavioral and social scientific knowledge can help the IC to understand and improve all phases of the analytic cycle: how to recruit, select, train, and motivate analysts; how to master and deploy the most suitable analytic methods; how to organize the day-to-day work of analysts, as individuals and teams; and how to communicate with its customers. The knowledge presented in this report has evolved through scientific processes that have given it well-understood strengths and limitations. With modest material investment and strong leadership, the IC can derive significant benefit from that knowledge. The committee offers a strategy to first exploit what is already known and then proceed to new programs of basic research that address the IC’s unique needs.
The first element involves assessing how well current and proposed analytical methods are supported by scientific evidence. The IC should not rely on analytical methods that violate well-documented behavioral principles or that have no evidence of efficacy beyond their intuitive appeal.
The second element is to rigorously test current and proposed methods under conditions that are as realistic as possible. Such an evidence-based approach to analysis will promote the continuous learning needed to keep the IC smarter and more agile than the nation’s adversaries.
RECOMMENDATIONS AND IMMEDIATE ACTIONS
The committee makes five broad recommendations and, for each, specific actions that the IC can adopt immediately with relatively little cost or disruption. Those recommendations and actions presented in Chapter 7 are summarized here.
Use Behavioral and Social Science
The committee’s first recommendation calls on the Director of National Intelligence (DNI) to apply the principles, evidentiary standards, and findings of the behavioral and social sciences to the IC’s analytic methods, workforce development, collaborations, and communications.
To implement this recommendation, the committee offers five immediate actions: (1) use the Intergovernmental Personnel Act for expertise on a short-term basis; (2) give IC analysts short-term academic assignments to deepen their methodological and subject matter expertise; (3) develop specialized behavioral and social science expertise cells across the IC to provide
methodological assistance and to network with outside scientists; (4) use behavioral and social science expertise in the IC Associates Program; and (5) create and widely disseminate an Analytical Methods Resource Guide.
Use Scientific Analytical Methods
The committee’s second recommendation calls on the DNI to ensure that the IC adopts scientifically validated analytical methods and subjects its methods to performance evaluation.
To implement this recommendation, the committee offers three immediate actions: (1) institutionalize an “Analytical Olympics” to test competing analytic methods and foster a culture that values continuous improvement; (2) begin to assess how well-calibrated individual analysts are and provide them with appropriate feedback; and (3) create a research program that reviews current and historic analyses comparing alternative methods under real world conditions.
Use Scientific Methods for Workforce Development
The committee’s third recommendation calls on the DNI to ensure that IC agencies use evidence-based methods to recruit, select, train, motivate, and retain an adaptive workforce able to achieve the performance levels required by IC missions.
To implement this recommendation, the committee offers four immediate actions: (1) create a course for IC analysts and managers on the full range of analytical methods with strong scientific foundations; (2) create an inventory of psychometrically validated measures to study which abilities are related to analytical performance and use the results in workforce hiring; (3) set up an independent review of all workforce practices; and (4) develop training programs that engage the entire workforce as teachers and students.
Use Scientific Collaboration Methods
The committee’s fourth recommendation calls on the DNI to require systematic empirical evaluation of current and proposed procedures for enhancing the collaboration that is essential to fulfilling the IC’s mission.
To implement this recommendation, the committee offers three immediate actions: (1) conduct field evaluations of at least two frequently used collaborative methods; (2) rigorously evaluate collaborative tools such as A-Space to enhance their utility; and (3) develop a database, or modify the Library of National Intelligence, to characterize collaborative analyses in terms of features that might be related to their effectiveness.
Use Scientific Communication Methods
The committee’s fifth recommendation calls on the DNI to implement scientific evidence-based protocols for ensuring that analysts and customers understand one another.
To implement this recommendation, the committee offers three immediate actions: (1) develop and evaluate standard protocols for communicating the confidence to place in analytic judgments; (2) evaluate the efficacy of current methods for requesting analyses in terms of how well they convey customers’ intentions to analysts; and (3) evaluate the impact of internal review processes on how well the resulting reports convey analysts’ intended meaning.