6
Summative Comments
The workshop planning committee asked two members of the Intelligence Community (IC) to provide closing remarks. Both speakers have spent their entire career working with the IC and have extensive experience with intelligence analysis, moderator Carmen Medina, MedinAnalytics, explained. Following their remarks, the floor was opened for questions and discussion.
REMARKS FROM KAREN MONAGHAN
Karen Monaghan, Central Intelligence Agency (retired), explained that, although she has never focused directly on the study of narrative, she has always considered understanding cultural narratives to be a crucial component of intelligence analysis. To understand a country and its leaders, she elaborated, an analyst must immerse himself or herself in that country’s culture, adding that as a manager, she encouraged her analysts to listen to a country’s music and read its literature to gain insight into its culture.
The workshop discussions spurred Monaghan’s thinking about historical narratives and their important role in intelligence analysis. For example, she observed, historical narrative is key to knowing when and why a particular leader might change policies or take some other action, and can also shed light on how citizens might react to changes in government policies or to toxic narratives that are introduced into mainstream society. To illustrate this point, she noted that while many anthropologists were not surprised by the Rwandan genocide of 1994, many in the IC were. It was easier for anthropologists to anticipate the violent event, she explained, because they were familiar with Rwanda’s historical narrative.
Monaghan is also interested in learning how to identify when a narrative is an indicator of a change in policy. She cited the example of Secretary of the Treasury Steve Mnuchin, who had made the statement earlier that day that a weaker dollar is good for the U.S. trade balance. However, she continued, prior administrations have always advocated for a strong dollar policy. Thus, she said, knowing whether Mnuchin’s statement was just an off-the-cuff comment or part of a larger narrative signaling an official change in policy would be useful information for an analyst.
Referring to Mark Turner’s comment that he is more interested in understanding the mental process of narratives than their content, Monaghan wondered whether the IC might be missing something when it focuses its attention on how specific words are used in messages. She noted that when examining Jihadi messages, for example, analysts try to predict future acts of violence by focusing on the terms used and what they might mean. She suggested that analysts might learn more if they thought more broadly about narrative.
At the same time, however, Monaghan raised the question of whether researchers collect data on what might be less obvious indicators of societal dissent, such as humor and jokes. Often, she suggested, people will express their thoughts and beliefs through humor rather than definitive statements, and collecting data on this type of communication could help identify sentiments that may otherwise be unapparent.
Monaghan identified as another area worth studying the universality of narratives, addressing such questions as “Do [Jihadi narratives] have universal appeal? How much are they being edited in order to appeal to different groups of Muslims in different countries?” She questioned whether there is a way to identify why some groups are more receptive to particular narratives and what might motivate a group to take or not take action. Answers to these types of questions, she argued, could be helpful in countering violent extremism.
In closing, Monaghan suggested that narrative research could also be an asset to analysts trying to supplement the predictions available from political polling outcomes, pointing to cases in which polling prior to an event did not forecast the actual outcome, such as Brexit or the 2016 U.S. presidential election. In particular, she suggested that studying narrative might provide insight as to why a particular narrative resonates with people and what effect it may have on how events unfold.
REMARKS FROM JOSH KERBEL
As a former analyst and analytical methodologist, Josh Kerbel, National Intelligence University, said he appreciated learning more about narrative research and how it could benefit the IC. When people think
about intelligence analysis, he noted, they often think about the parsing of collected data in search of answers. In his view, however, a more important aspect of analytical work—especially in a highly complex environment—is sensemaking (i.e., the creative development of holistic perspectives). To make sense of the stories that are held within the data collected, he elaborated, analysts must ask the right questions. He suggested that as a sensemaking tool, studying and understanding narratives could help analysts ask better questions, which in turn could help them provide more insightful findings to their clients (e.g., members of Congress, military officials, the President).
Although narrative is a part of analysis, Kerbel continued, it is not a term that analysts commonly use in relation to their own thinking. Instead, he said, they generally use the term “analytic lines,” which refers to stories analysts tell each other and themselves to make sense of the areas they study. When information is received, he explained, it is usually measured against the prevailing analytic line to determine whether it is useful. To most analysts, in contrast, narratives are the stories told by adversaries, such as Al-Qaeda or ISIS.
Kerbel observed that studying narrative appears to be a more synthetic approach to analysis than the prevailing analytic approach typically used in the IC, in which the focus is on breaking down complex problems into more workable components. The synthetic approach, he explained, makes sense of information by pulling it together and shaping it into some kind of order, so it has the potential to be a very powerful tool for dealing with complexity. Thus, he argued, although it is not usually an explicit part of current analytic tradecraft and methodology, it should be, adding that the analytic culture and mindset may create a barrier to making that change. Narrative is inherently seen as not objective, he noted, because it is the expression of a story about something that happened, told from a certain perspective, yet objectivity, at least in theory, is central to the work of an analyst. Furthermore, he added, while narratives can be very rich and diffuse, analysts tend tomore write in a very clear, direct, and linear way. He noted as well that analytical work is guided by a set of core principles known as the Analytic Tradecraft Standards,1 which require analysts to bring objectivity to their work and to express findings in a clear and logical way.
Kerbel mentioned two areas in which narrative could be particularly useful. The first is imaginative visualization. While the IC is accustomed to using visualization on the “back end” of analysis in its products and presentations, Kerbel explained, narrative may help them better visualize their work on the “front end” and thereby be a useful cognitive tool for making
___________________
1 For more information on the Analytic Tradecraft Standards, see https://www.dni.gov/files/documents/ICD/ICD%20203%20Analytic%20Standards.pdf [March 2018].
sense of information. Second, he suggested, narrative could be helpful in making sense of the open-source social media or transmedia data discussed in earlier panels.
In closing, Kerbel suggested that narrative can be especially useful to analysts because it introduces a more human element into their thinking. He noted that analysts tend to observe humans from a distance, as if they were a “more physical kind of particles almost.” If analysts could use narrative effectively as an analytic tool, he argued, it might help them think in more human terms, which might in turn improve the way they observe and understand others. The addition of this human element, he suggested, could then lead to more insightful conclusions.
DISCUSSION
Medina opened the discussion by asking the panelists how they approach policy makers who enter an intelligence briefing with a set narrative that conflicts with the one they are about to present. Kerbel responded with a reminder that analysts often do not enter a briefing with a narrative in place; while they usually come equipped with collected data and facts, those facts are often not tied to a convincing narrative. However, he suggested, based on earlier panel discussions indicating that the best counter to a narrative is another narrative, analysts could be more persuasive if they prepared a narrative in advance of a briefing. Monaghan agreed, adding that analysts should know how their narratives compare with those of policy makers and where they may agree or differ. She suggested that parsing policy makers’ narratives for them could help to identify holes in the narratives and open the door for analysts to introduce their findings.
One participant added that the National Geospatial-Intelligence Agency (NGA) has started to use narrative as a tool for delivering imagery analysis products in a more dynamic way. Kerbel agreed that some segments of the IC, such as NGA, are doing valuable research in this area, but observed that the findings of this research are not yet being applied in the day-today work of analysts across the IC, particularly in all-source agencies. Monaghan added that the Analytic Tradecraft Standards were written in a way that is somewhat antithetical to narrative.
One participant agreed with the earlier point that, instead of thinking of narrative as being opposed to objectivity, it might be helpful for analysts to think in terms of competing narratives: one may be better than another, but they both can exist as possible outcomes for the question at hand. Thus, the participant explained, the analyst is not giving up on the idea of objectivity, just shifting the view a bit. Kerbel agreed and responded that the IC is exploring the idea of competing narratives. However, he added, there is still a great deal of pressure to simplify the message. He noted that the role
of an analyst is debated in the IC. Some believe, he observed, that it is an analyst’s job to predict, stating that people who hold this view often reject the idea of a competing narrative because not only is their narrative based on what they believe will happen, but they feel competing narratives would not help clients who are seeking greater certainty from the IC. The other view, Kerbel continued, is that an analyst’s job is not so much to predict but to inform the client of what is possible so the client can make a more informed decision. Those like himself who hold this latter view, he said, would likely agree that presenting competing narratives is an important part of an analyst’s job.
Kerbel went on to point out that, regardless of differing views of competing narratives, analysts face pressure to present information in a short and simple format. A participant cautioned that to suggest there are no competing narratives could be dangerous because it implies there is only one correct narrative. Kerbel agreed and suggested that because analysts are increasingly dealing with complex phenomena, there is always more than one narrative for any given situation.
One participant suggested that topic modeling2 might be helpful to the IC. Topic modeling, he explained, is an effective way of examining large amounts of data and extracting popular themes that are beginning to develop within a culture. He suggested that by using this technology to sort through the vast amounts of available open-source data, the IC might be able to identify evolving themes in subpopulations that might otherwise be overlooked. Another participant added that she recently had used a multilingual topic model to examine the French, English, and Kinyarwanda narratives from the Rwandan genocide. Running these types of computational linguistic models, she explained, can help researchers see the bigger picture that exists within the data, and qualitative methods can then be used to further clarify those findings. She added that the broad set of worldviews that emerges from the combination of these methods can then be used to develop theory from the ground up.
Kerbel seconded these comments and acknowledged that, because analysts frequently come from the more qualitative, social science disciplines, they often shy away from quantitative research. However, he added, the discussions that had taken place during the workshop might open the door for analysts’ use of more quantitative approaches in their analyses. Medina suggested that one area of analysis in which quantitative methods might be particularly helpful is identifying triggers for violence. Currently, she noted, analysts tend to rely on their sources to inform them about events that are about to unfold. Even if a communication is intercepted, she said,
___________________
2 For more information on topic modeling, see http://journalofdigitalhumanities.org/2-1/topic-modeling-a-basic-introduction-by-megan-r-brett [April 2018].
it must be a declarative statement for it to be recognized as identifying an upcoming event. Monaghan added that at times in her career, the information being collected about a country has appeared to suggest that a major change has occurred in the country’s overall identifying narrative. However, she noted, changing the organizational view of that narrative is not easy; it does not get switched off like a light switch. Perhaps, she commented, with these methods there is a way to identify triggers that point to a shift in the narrative.
Another participant suggested that it would be helpful for researchers to have access to members of the IC so they could help shape the research questions that are addressed. He asked whether the panelists ever had opportunities to engage with researchers in this way. Monaghan and Kerbel agreed that this idea has merit, but noted both time and bureaucratic constraints, as well as concerns about the potential release of classified information, as reasons why analysts do not often collaborate with outside researchers. In the world of intelligence analysis, they added, the rewards are greater for those who focus on analytical tradecraft rather than time-consuming research. Kerbel said he believes the IC has only recently become more introspective about the process of analytical work, but this has not traditionally been an important focus for intelligence analysts.
One participant noted that he has had experience with trying to train members of the military in how to collect data and knows that people can find it difficult to adapt to new methods. He asked the panelists whether it would be helpful if members of the IC were trained in new techniques for working with narrative. Kerbel pointed out that obtaining buy-in from the IC on a new methodology would require understanding the IC culture, to which analysts have a strong connection. Monaghan suggested that a good way to introduce a new methodology that is tied to the IC culture is to obtain buy-in from leadership and identify someone willing to champion the new approach. She added that a new methodology is more likely to be accepted if it is applied to an area that is not related to secrets. For example, she suggested, analysts are more likely to need help finding ways to make sense of open-source data or to study a closed country, such as Cuba, where they lack personal sources of information. Another participant pointed out that using the term “mental models” instead of “narrative” might improve receptivity. Kerbel agreed and pointed out that many analysts may well be resistant to anything tied to the term “narrative.” They work with narratives, he explained, but do not necessarily call them by that name. For example, he noted, “analytic lines” are narratives. He suggested that knowing how to phrase ideas is important and that this may be one of the most important lessons from the discussion.
