Appendix C
Committee Methodology

With the context and scope of its assignment established, the committee turned its attention to defining a robust methodology for technology warning that would be suitable for the diverse inquiries likely to stem from ongoing engagement between a standing committee of the National Research Council (NRC) and the intelligence community’s (IC’s) technology warning components. The proposed methodology is described in this chapter and tested through application in subsequent chapters.

KEY FEATURES OF THE METHODOLOGY

A robust methodology for technical inquiry should have four key features. First, to be accepted, it must be presented in a lexicon and structure appropriate for the user’s culture—in this case, for the culture in which the Defense Intelligence Agency (DIA) Technology Warning Division (the sponsor of this study) operates. Any communication of findings, conclusions, or recommendations offered by the committee must be expressed accordingly. The division makes use of weather-forecasting terminology (Futures, Watch, Warning, Alert)1 in the issu-

NOTE: This appendix is reprinted from Avoiding Surprise in an Era of Global Technology Advances (Washington, D.C.: The National Academies Press, 2005), pp. 20-27.

1

The definitions used by the DIA for these terms are as follows: Futures—Create a technology roadmap and forecast; identify potential observables to aid in the tracking of technological advances. Technology Watch—Monitor global communications and publications for breakthroughs and integrations. Technology Warning—Positive observables



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 162
Appendix C Committee Methodology With the context and scope of its assignment established, the commit- tee turned its attention to defining a robust methodology for technology warning that would be suitable for the diverse inquiries likely to stem from ongoing engagement between a standing committee of the National Research Council (NRC) and the intelligence community’s (IC’s) tech- nology warning components. The proposed methodology is described in this chapter and tested through application in subsequent chapters. KEY FEATURES OF THE METHODOLOGY A robust methodology for technical inquiry should have four key features. First, to be accepted, it must be presented in a lexicon and structure appropriate for the user’s culture—in this case, for the culture in which the Defense Intelligence Agency (DIA) Technology Warning Division (the sponsor of this study) operates. Any communication of findings, conclusions, or recommendations offered by the committee must be expressed accordingly. The division makes use of weather- forecasting terminology (Futures, Watch, Warning, Alert)1 in the issu- NOTE: This appendix is reprinted from Avoiding Surprise in an Era of Global Technology Advances (Washington, D.C.: The National Academies Press, 2005), pp. 20-27. 1The definitions used by the DIA for these terms are as follows: Futures—Create a technology roadmap and forecast; identify potential observables to aid in the tracking of technological advances. Technology Watch—Monitor global communications and publi- cations for breakthroughs and integrations. Technology Warning—Positive observables 

OCR for page 162
 AppENDIX C ance of technology assessments, making the overall warning message regarding all products readily interpretable by any reader. The committee adopted and adapted the Defense Intelligence Agency (DIA)’s vocabu- lary to characterize the relative status—and recommended action—for each technology. The second key feature is that, to be relevant, the study method- ology must be tied in a fundamental way to top-level Department of Defense (DOD) strategies. For example, the committee reviewed Joint Vision 2020 (JCS, 2000) to validate its selection of the technology topics addressed in this report. In future studies, to facilitate integration into the larger body of intelligence materials, the committee proposes that technology selections be derived through a more disciplined, RED team2 review of top-level strategy documents (e.g., Joint Vision 2020) with an eye to identifying technologies that could be used to deny a BLUE3 capability deemed critical to U.S. military success. The third key feature is that, to maintain focus and ensure timeliness, the study methodology must yield assessments built on a solid under- standing of the technical feasibility of potential technology-based threats. This requirement leads to a capability-based approach for investigating and categorizing candidate technologies. Furthermore, the technical peer review process to which all NRC reports are subjected provides additional assurance of the technical quality of committee assessments. Lastly, to be enduring, the methodology should accommodate evolv- ing realities of science and technology (S&T) leadership, driven by the synergistic trends of globalization and commercialization described in Chapter 1. Traditionally, the United States has assumed that it leads the world in S&T. This perspective leads the technology warning commu- nity to look for indications that external actors are trying to “catch up,” or to exploit known technologies in new ways. Projected trends suggest that it should no longer automatically be assumed that the United States will lead technological advances in all relevant technologies. This reality imposes a new burden on the technology warning community, generat- ing the need for it to search in different places and in different ways for the information needed to warn against technology surprise. indicate that a prototype has been achieved. Technology Alert—An adversary has been identified and operational capability is known to exist. 2“RED” is used in this report to denote the adversary or an adversarial perspective (e.g., “RED team”). 3 “BLUE” is used in this report to denote U.S. military forces.

OCR for page 162
 EMErGING COGNITIVE NEUrOSCIENCE AND rELATED TECHNOLOGIES FOUNDATION OF THE METHODOLOGY The committee believes that the Technology Warning Division can most effectively prioritize its limited resources by utilizing a capabilities- based approach with respect to assessing technologies. The landscape of potentially important emerging technologies is both vast and diverse. Ideally, the division should assess whether a given technology has the potential to pose a viable threat prior to commissioning in-depth analyses. Since the division is keenly interested in when specific tech- nologies may mature to the point that they pose a threat to U.S. forces, a functional decomposition from an adversarial, or RED, perspective is most useful. The methodology defined by the committee begins with the following focus question: What capabilities does the United States hae that, if threatened, impact U.S. military preeminence? In general, U.S. capabilities could be threatened either through direct denial of or disruption of BLUE capabilities or via RED capabili- ties that negate or significantly diminish the value of BLUE capabilities (e.g., improvised explosive devices (IEDs) being employed by insurgent forces in Iraq). Joint Vision 2020 was used to define the basic framework for U.S. military capabilities deemed vital to sustained success (JCS, 2000). The overarching focus of this vision is Full Spectrum Dominance—achieved through the interdependent application of four operational concepts (Dominant Maneuver, Precision Engagement, Focused Logistics, and Full Dimensional Protection) and enabled through Information Superior- ity, as illustrated in Figure [C]-1 (JCS, 2000). The committee selected the four operational concepts, together with Information Superiority, as the foundation for its assessment methodol- ogy. Joint Vision 2020 provides the definitions presented in Box [C]-1. The committee also noted the importance of technology warning with respect to the “Innovation” component of Joint Vision 2020 shown in Box [C]-2, since “leaders must assess the efficacy of new ideas, the potential drawbacks to new concepts, the capabilities of potential adver- saries, the costs versus benefits of new technologies, and the organiza- tional implications of new capabilities” (JCS, 2000). From this foundation the committee then identifies specific capa- bilities in accordance with the previously defined focus question—What capabilities does the United States hae that, if threatened, impact U.S. military preeminence? While the U.S. military has devoted significant time to the definition of vital capabilities in alignment with Joint Vision 2020, the committee made no effort in this first report to synchronize its derivations or defini- tions, or to provide a complete decomposition of the operational con- cepts and enablers into their underlying capabilities. Rather, committee

OCR for page 162
 AppENDIX C FIGURE [C]-1 Concepts constituting the basic framework for U.S. military C-1.eps capability as defined by Joint Vision 2020. (See Box [C]-1.) SOURCE: JCS (2000). members selected a few evolving technologies and assessed the potential for those technologies to threaten important U.S. capabilities. Given that the committee’s proposed basic methodology is adopted, future studies will analyze more comprehensively the threats to a taxonomy of U.S. military capabilities that derives from the operational concepts envisioned by Joint Vision 2020. The basic methodology developed by the committee is summarized in Box [C]-1 and is described in greater detail in subsequent sections. IDENTIFY The next step of the proposed assessment methodology is performed from the RED perspective. The central question here is as follows: What are the eoling technologies that, in the hands of U.S. adersaries, might be used to threaten an important U.S. military capability? A corollary question is, What technologies, if rapidly exploited by the U.S. military, are likely to yield sustained technological superiority? However, this issue was addressed only peripherally, given the division’s focus on technology warning. Having identified a technology of potential interest, the next chal- lenge becomes the derivation of “indicators” or “observables” that may

OCR for page 162
 EMErGING COGNITIVE NEUrOSCIENCE AND rELATED TECHNOLOGIES BOX [C]-1 Relevant Definitions from Joint Vision 2020 Serving as Foundation for Assessment Methodology Information Superiority is the capability to collect, process, and dis­ seminate an uninterrupted flow of information while exploiting or denying an adversary’s ability to do the same. Information superiority is achieved in a noncombat situation or one in which there are no clearly defined adversaries when friendly forces have the information necessary to achieve operational objectives. Dominant Maneuver is the ability of joint forces to gain positional advantage with decisive speed and overwhelming operational tempo in the achievement of assigned military tasks. Widely dispersed joint air, land, sea, amphibious, special operations and space forces, capable of scaling and massing force or forces and the effects of fires as required for either combat or noncombat operations, will secure advantage across the range of military operations through the application of information, deception, engagement, mobility and counter­mobility capabilities. Focused Logistics is the ability to provide the joint force the right per­ sonnel, equipment, and supplies in the right place, at the right time, and in the right quantity, across the full range of military operations. This will be made possible through a real­time, web­based information system providing total asset visibility as part of a common relevant operational picture, effectively linking the operator and logistician across Services and support agencies. Precision Engagement is the ability of joint forces to locate, surveil, discern, and track objectives or targets; select, organize, and use the correct systems; generate desired effects; assess results; and reengage with decisive speed and overwhelming operational tempo as required, throughout the full range of military operations. Full Dimensional Protection is the ability of the joint force to protect its personnel and other assets required to decisively execute assigned tasks. Full dimensional protection is achieved through the tailored selec­ tion and application of multilayered active and passive measures, within the domains of air, land, sea, space, and information across the range of military operations with an acceptable level of risk. SOURCE: JCS (2000).

OCR for page 162
 AppENDIX C BOX [C]-2 Proposed Methodology for Technology Warning Foundation Joint Vision 2020 Operational Concepts and Information S uperiority • Focus What capabilities does the United States have that, if threatened, impact U.S. military preeminence? • Identify What are the evolving technologies that, in the hands of U.S. adversaries, might be used to threaten an important U.S. military capability? What are the observables that may indicate adversarial adoption or exploitation of such technologies? • Assess Accessibility: How difficult would it be for an adversary to exploit the technology? Maturity: How much is known about an adversary’s intentions to exploit the technology? Consequence: What is the impact on U.S. military capability should the technology be employed by an adversary? • Prioritize Identify: What are the relative resources to be applied to each emerging technology to support the technology warning process? • Task Establish and assign intelligence-information-collection requirements. aSOURCE: JCS (2000). suggest adversarial adoption or exploitation of that technology. Although targeted intelligence-collection methods remain important, in this report the committee focuses on observables that may be derived from open source analysis—leveraging the effects of the information revolution and acknowledging that the twin forces of globalization and commercialization provide new sources of relevant information. At the same time, however, the committee recognizes the difficulty of discerning when technological advances portend emerging threats rather than societal benefits.

OCR for page 162
 EMErGING COGNITIVE NEUrOSCIENCE AND rELATED TECHNOLOGIES CHART [C]-1 Example of Technology Assessment Chart Technology Observables Brief description of technology Brief description of observables Accessibility Maturity Consequence Level 1, 2, or 3 Technology Futures Short characterization Technology Watch Technology Warning Technology Alert A sample chart—Chart [C]-1—exemplifies how each technology is assessed. ASSESS The committee’s assessment methodology involves characterization of a technology in terms of three variables: Accessibility, Maturity, and Consequence. Priorities for more detailed analyses may derive from any individual variable or any combination of the three. Accessibility The Accessibility variable focuses on the question How difficult would it be for an adersary to exploit the technology? It addresses the ability of an adversary to gain access to and exploit a given technology. This assessment is divided into three levels: • Leel . The technology is available through the Internet, being a commercial off-the-shelf item; low sophistication is required to exploit it. • Leel . The technology would require a small investment (hun- dreds to a few hundred thousand dollars) in facilities and/or expertise. • Leel . The technology would require a major investment (mil- lions to billions of dollars) in facilities and/or expertise. In general, Level 1 technologies are those driven by the global com- mercial technology environment; they are available for exploitation by a diverse range of potential adversaries. Level 3 technologies, by contrast, are typically accessible only to state-based actors. The indicators likely to be of value in determining an adversary’s actual access to a given technology vary by level as well as by the type of technology.

OCR for page 162
 AppENDIX C Maturity The Maturity variable focuses on the question How much is known about an adersary’s intentions to exploit the technology? It integrates what is known about an adversary’s actions, together with an evaluation of the state of play with respect to the technology of interest. At the highest level, called Technology Alert, an adversary has been identified and an operational capability has been observed. At the lowest level, Technology Futures, the potential for a technology-based threat has been identified, but no positive indicators have been observed. The Maturity assessment is divided into four categories: the first two (the lower levels) suggest further actions for the technology warning community; the other two indicate the need for immediate attention by military leadership: • futures. Create a technology roadmap and forecast; identify potential observables to aid in the tracking of technological advances. • Technology Watch. Monitor (global) communications and publi- cations for breakthroughs and integrations. • Technology Warning. Positive observables indicate that a proto- type has been achieved. • Technology Alert. An adversary has been identified and opera- tional capability is known to exist. Given the potential for disruptive advances through technological breakthroughs or innovative integration, as well as the difficulty of identifying and tracking meaningful indicators, any particular technol- ogy is unlikely to progress sequentially through the various categories of Maturity listed above. As indicated at the beginning of this chapter, the committee adopted and adapted the DIA’s terminology in defining these categories. The definitions are likely to evolve as the process matures. The committee sees significant value in this basic approach, however, since it divorces the challenge of technology warning from the discrete time lines associ- ated with “prediction,” which are almost invariably inaccurate. Consequence Characterization of a technology in terms of the Consequence variable involves addressing the question What is the impact on mili­ tary capability should the technology be employed by an adersary? It involves assessing the impact of the postulated RED technology on the capability of BLUE forces. This impact can range from denial or nega- tion of a critical capability to the less-consequential level of annoyance or nuisance. A corollary assessment may be made as to the locus of

OCR for page 162
0 EMErGING COGNITIVE NEUrOSCIENCE AND rELATED TECHNOLOGIES impact—that is, whether the technology affects a single person, as in the case of an assassination, or creates a circumstance of mass casualty and attendant mass chaos. PRIORITIZE The objective of the prioritization step of the methodology is to respond to the question What are the relatie resources to be applied to each emerging technology to support the technology warning process? This step is intended to harmonize the distinct nodes of observed capa- bility, demonstrated intent, resources available, and the inherent cost of inaction. Prioritization is key to the technology warning methodology, since the Technology Warning Division lacks the resources to fully analyze every conceivable evolving technology. It is equally important to recognize that prioritization is an integral part of each methodology parameter. The prioritization of individual parameters is based on the levels of change detection and potential impact. By prioritizing the parameter, the division can focus subsequent analyses over a smaller subset of an assigned change detection domain. Priority assignment is essential to enable the focusing of more sophisticated information- gathering tools and analytic techniques on the areas of highest potential concern. The prioritization methodology lends itself to any number of com- mercially available tools and techniques designed for assistance in estab- lishing and maintaining a logical and consistent focus as well as the flexibility to react to the dynamics of technology change and country-of- interest variability. During the prioritization process, it will be important to establish measures of performance to allow critical analysis as well as change management in order to improve the overall process. The end result of the prioritization process is to provide for actionable awareness with which to influence analysis and tasking, the last of the methodol- ogy parameters. The committee envisions that prioritization would be accomplished in close consultation with the technology warning community. It made no attempt to further develop the prioritization process in this report. TASK The Technology Warning Division will inevitably have unmet needs for additional information and/or intelligence relating to the prioritized list of evolving technologies. Although some needs may be met through division-chartered research, others will require the assistance of the broader intelligence community.

OCR for page 162
 AppENDIX C The task step—Establish and assign intelligence­information­ collection requirements”—involves the dissemination of collection requirements to other IC components and subordinate agencies. Such requirements must provide sufficient specificity to enable interpretation by collectors who are not necessarily literate in the specific technology. The requirements may include general instructions for accomplishing the mission. It is envisioned that some of the observables postulated in the Identify step of the methodology will provide a useful basis for such tasking. The results from collection efforts will be integrated back into the assessment step in order to refine, reprocess, and update the division’s understanding of a given technology. This analysis may stimulate the issuance of a new report to the division’s customers to inform them of changes in the assessed maturity of that technology. USING THE METHODOLOGY IN THIS REPORT To test the robustness of the proposed technology warning method- ology, the committee applied it in order to assess four key areas in this initial report. It should be noted that this initial exercise was necessarily circumscribed by the domain expertise represented in the committee members and by the shortness of time for broader outreach to the tech- nical community at large. Furthermore, since the methodology emerged in parallel with the committee’s technology assessments, the approaches taken were not entirely consistent. The foundation provided by Joint Vision 2020 and augmented by the military and professional backgrounds of committee members was used to select the following four key capabilities to assess: • Information superiority (Chapter 3), • Air superiority (Chapter 4), • Discrimination between friends/foes/neutrals (Chapter 5), and • Battle readiness and communications superiority (Chapter 6). Chapters 3 through 6 each address the “Identify” activity with examples of evolving technologies that may threaten the capability and potential indicators that such technology development is under way. The “Assess” activity then examines opportunity and motivation for adver- sarial technology development and/or employment, posits change detec- tion relative to the indicators, and assesses likely impact. Preliminary characterizations of accessibility, maturity, and consequence are pro- vided for most evolving technologies, although the level of specificity is variable.

OCR for page 162
 EMErGING COGNITIVE NEUrOSCIENCE AND rELATED TECHNOLOGIES Subsequent steps (i.e., “Prioritize” and “Task”) of the proposed methodology require customer inputs and actions and are left to future study efforts. REFERENCE JCS (Joint Chiefs of Staff). 2000. Joint Vision 2020. Director for Strategic Plans and Policy, J5; Strategy Division. U.S. Government Printing Office, Washington, D.C. June.