10
Complex and Interdependent Systems

INTRODUCTION

The previous chapters of this report call for the use of systems analysis and systems engineering in countering terrorism. This effort can draw on bodies of knowledge already available in the United States and applicable immediately to dealing with terrorist threats. For example, DOD, NASA, and various intelligence agencies have directed systems techniques to highly complex, but ultimately successful, military and aerospace applications dating all the way back to the Apollo and Strategic Submarine programs and continuing to this day. Additionally, the Environmental Protection Agency has developed a methodology for risk analysis and prioritization of environmental threats, and private-sector analysts—in the financial services (www.riskmetrics.com) and geophysical exploration (MacKay, 1999) industries, for example—have used a wide variety of risk modeling and other systems methodologies to manage large-scale global operations.

While none of these techniques is an exact match for the counterterrorism challenges described in this report, there appears to be considerable near-term potential to extend these techniques to provide a strategic framework for addressing these threats. One near-term example is in the integration of current infrastructure models. DOD’s Modeling and Simulation Office and DARPA, for example, have developed procedures and algorithms for distributed simulations that take advantage of multiple existing simulation capabilities in different organizations and include them in a broader framework to address questions that could not have been answered by any one of the simulation systems individually. These techniques have the potential to be adapted to a variety of models (cur-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 287
Making the Nation Safer: The Role of Science and Technology in Countering Terrorism 10 Complex and Interdependent Systems INTRODUCTION The previous chapters of this report call for the use of systems analysis and systems engineering in countering terrorism. This effort can draw on bodies of knowledge already available in the United States and applicable immediately to dealing with terrorist threats. For example, DOD, NASA, and various intelligence agencies have directed systems techniques to highly complex, but ultimately successful, military and aerospace applications dating all the way back to the Apollo and Strategic Submarine programs and continuing to this day. Additionally, the Environmental Protection Agency has developed a methodology for risk analysis and prioritization of environmental threats, and private-sector analysts—in the financial services (www.riskmetrics.com) and geophysical exploration (MacKay, 1999) industries, for example—have used a wide variety of risk modeling and other systems methodologies to manage large-scale global operations. While none of these techniques is an exact match for the counterterrorism challenges described in this report, there appears to be considerable near-term potential to extend these techniques to provide a strategic framework for addressing these threats. One near-term example is in the integration of current infrastructure models. DOD’s Modeling and Simulation Office and DARPA, for example, have developed procedures and algorithms for distributed simulations that take advantage of multiple existing simulation capabilities in different organizations and include them in a broader framework to address questions that could not have been answered by any one of the simulation systems individually. These techniques have the potential to be adapted to a variety of models (cur-

OCR for page 287
Making the Nation Safer: The Role of Science and Technology in Countering Terrorism rently or soon to be available) in areas of threat modeling and critical infrastructure to provide near-term improvements in developing vulnerability assessments and risk mitigation strategies. Currently, however, the U.S. government’s departments and agencies are in no position to make optimal use of available modeling and simulation technologies to support the creation of an overall strategy for their counterterrorism activities. They are not organized to assess terrorist threats, infrastructure vulnerabilities, and mitigation strategies from a systems perspective. Thus, although many initiatives have been proposed since September 11, and some—such as improving airport security, local emergency response, and seaport operations—are in early stages of implementation, they are often proceeding without the benefit of a systems approach. Specific examples of the value of systems approaches are also described in other chapters of this report, particularly Chapter 7, on transportation, and in Chapter 11, on crosscutting challenges. While an overall systems approach is particularly important in the development of a national strategy for counterterrorism, there exist today models for particular infrastructures within the United States that have been produced by various government agencies and private industrial organizations. Aspects of energy distribution, power grids, air traffic control, and military support infrastructures have been analyzed and modeled to varying degrees of fidelity. In the near term, these models must be extended and expanded to provide better representation of specific critical infrastructures, and the models must be tested and evaluated against real-world data. A program to measure the interactions between various infrastructures must also be established. This effort will rely on determining the connectivity between infrastructures through analyses, model development, data collection, experiments, and model validation. At the same time a more detailed understanding of the implications of various threat scenarios for critical U.S. infrastructures must be established. The committee recognizes that it will never be possible to model the entire U.S. system in finite detail, but we can determine which components of our critical infrastructures are least robust; how an attack on one component of a particular infrastructure affects other systems; and which identified vulnerabilities within critical infrastructures are most vulnerable to a wide range of postulated threat scenarios. A FRAMEWORK FOR A SYSTEMS APPROACH TO COUNTERTERRORISM When modeling terrorist networks and homeland systems, knowledge of the associated “architectural” framework—including its characteristic state variables1—is essential. In Figure 10.1, threats from terrorist networks constitute a 1   It should be noted that use of the term “state variable” is somewhat different from its use in, say, mathematics and control theory.

OCR for page 287
Making the Nation Safer: The Role of Science and Technology in Countering Terrorism FIGURE 10.1 A framework for a systems approach to counterterrorism. SOURCE: Haimes (2002), p. 37. Adapted from Figure 2 in “A Roadmap for Modeling the Risks of Terrorism to the Homeland,” ASCE Journal of Infrastructure Systems, Vol. 8, No. 2, June. Copyright 2002 by the American Society of Civil Engineers (ASCE). Reprinted by permission of the publisher, ASCE.

OCR for page 287
Making the Nation Safer: The Role of Science and Technology in Countering Terrorism key input. These threats can be understood and modeled only when we identify and understand the societal environment, the geopolitical dynamics within which the terrorist networks are energized and operate, and the characteristics and limitations of the threatened infrastructure. The causal relationships among these inputs and outputs then enable the building of models that predict the efficacy of risk-management policy options. Understanding counterterrorism as a “system of systems” is essential, because the outputs of the terrorism network, as shown in Figure 10.1, are the same four types of risk that constitute the inputs to the homeland security system. Terrorism network characteristics, the resulting potential sources of risk, and homeland infrastructure and security characteristics all contribute to the comprehensive effort needed to identify conceivable types of risk. Additional characteristics, such as the funding sources of the terrorist groups, the level of sophistication of these groups, and the driving forces that feed them (such as unfavorable socioeconomic conditions, geopolitical considerations, cultural conflicts, and racial and religious issues), are also essential to the risk analysis. Four major risk classes to homeland security can be identified, as shown in Figure 10.1: Risks to human lives and to individual property, liberty, and freedom; Risks to organizational and societal infrastructures and to the continuity of government operations, including the military and intelligence-gathering infrastructure; Risks to critical cyber and physical infrastructures; and Risks to socioeconomic sectors. An essential factor for sound decision making is identification of these and other sources of homeland risk at a sufficient level of detail. This will enable effective strategic and tactical planning. SYSTEMS MANAGEMENT ISSUES While systems engineering is essential to the successful design, development, and deployment of a complex system (Sage and Rouse, 1999), how well a system is operated once it is deployed is the concern of systems management. The new Office of Homeland Security (OHS) is developing a strategic plan for the United States that will include the participation of many public and private organizations. To support the development of its plan, the OHS will need an overall management system that takes into account many of the governance, decision-making, and information systems and tools discussed below.

OCR for page 287
Making the Nation Safer: The Role of Science and Technology in Countering Terrorism Governance and Decision Making Lack of a shared understanding of the elements of governance by key stakeholders, especially in cross-organizational decision-making situations, can result in conflicts and possibly stalemates. Worse yet is when, because of ill-specified governance practices, there are critical decisions that no one sees the need for or that no one is responsible for making. Several governance issues are of particular importance: What types of decisions must be made? Who can make which types of decisions? Who can delegate decision-making authority? How is decision making supported? Often, the overriding question is, “Who decides who decides?” Who can resolve inevitable decision conflicts when multiple organizations perceive responsibility for a particular decision? On the other hand, when there are gaps between organizations, who should assure that key decisions are not lost in those gaps? This question leads to an obvious issue: the respective responsibilities of various federal departments and agencies on threats to the U.S. infrastructure and, of course, to U.S. citizens and residents. Carter (2001-2002) has outlined the nature of the federal “architecture” for addressing terrorism. He concludes that the U.S. government lacks a managerial category for catastrophic terrorism per se—as opposed to its well-established categories for war, crime, or natural disaster. Further, state and local governments lack the resources and specialized knowledge to combat terrorism. Thus, government at all levels lacks a framework for bringing responsibility, accountability, and resources together to deliver homeland security against terrorism. As Carter notes, “The federal government disperses executive authority so thoroughly that few individuals believe they are accountable for any of the government’s key security outputs.” The responsibilities of state and local organizations are similarly dispersed and fragmented. These issues were recognized long before the terrorist acts of September 11, 2001. Studies of infrastructure vulnerabilities led to Presidential Decision Directive No. 63 (PDD63),2 which describes these national vulnerabilities and provides guidelines for addressing and eliminating them. Lead agencies and tasks are specified for each of the components of the overall national infrastructure. 2   For the full text of the white paper containing the Clinton administration’s Policy on Critical Infrastructure Protection: Presidential Decision Directive 63, May 22, 1998, see <http://www.nipc.gov/about/pdd63.htm>.

OCR for page 287
Making the Nation Safer: The Role of Science and Technology in Countering Terrorism This sets the stage for addressing governance issues and needs but does not fully resolve them. Information Systems and Tools Beyond the important guidance provided by PDD63, as well as its classified companion PDD62, there is a substantial need for management information systems and tools. First of all, as noted by Carter, allocation of responsibilities must be finer-grained than specified in PDD63. The approach to allocation he proposes is characterized in Table 10.1. Nevertheless, decision-making problems remain. These include overlapping organizational seams, which tend to produce conflicted decisions. Gaps between seams are much more subtle, because they can result in “lost” decisions—those that are simply unrecognized as needing to be addressed and resolved. Problems of overlapping seams, and of gaps between them, suggest several opportunities for applying science and technology. But solutions should not eliminate those organizational seams, which provide valuable levels of resiliency—the inherent distribution of authority across federal, state, and local agencies, after all, is central to American life. Thus, complete integration is not only quite difficult, it is politically impossible and culturally undesirable. Means for assessing current and emerging states of distributed responsibility are needed, and computer-based tools can be useful in modeling the decision-forcing phenomena. Such tools can also enable teams to access, create, or manipulate computational models—for example, of relationships and flows. Model building provides a good way to span the organizational boundaries often associated with complex decisions. And because technology currently exists for monitoring, cluster analysis, and portrayal of the nature of e-mails and attachments flowing in large organizations, this idea is by no means far-fetched. The information systems and tools suggested above are portrayed in terms of technology-enabled capabilities. However, it is important to emphasize the essential need for scientific research, both to provide the knowledge upon which TABLE 10.1 Agencies and Organizations versus Responsibilities Detection Prevention Protection Interdiction Containment Attribution Analysis and Invention Org 1   … Org N

OCR for page 287
Making the Nation Safer: The Role of Science and Technology in Countering Terrorism these capabilities may be based and to assess the consequences (including the behavioral, social, and economic impacts) of deploying them. Systems Expertise for the OHS As stated above, the U.S. government lacks the structure and framework for bringing responsibility, accountability, and resources together to secure the homeland against catastrophic terrorism. Moreover, a federal architect and national systems integrator across all departments and all levels of government is needed to develop and validate operating models in order to provide the prioritized decision making, planning, and training needs of governmental counterterrorism programs. Recommendation 10.1: In order to define critical infrastructure vulnerabilities and enable better decision making within the federal government on priorities related to counterterrorism, the OHS should utilize a dedicated core of systems engineering and research expertise to conduct systems analyses, systems engineering, risk modeling and assessment, and related model development. This core of expertise should reside in the proposed Homeland Security Institute, the capabilities of which are described in Chapter 12. OHS is in need of a range of services, including the development or integration of models and databases necessary for critical decision making and possibly the coordination of the design and development of data-acquisition networks to provide the inputs for these models and databases. An organization with all of the relevant expertise—expertise in policy analysis; intelligence collection; research and analysis of terrorist behavior; risk modeling, assessment, and management; threats from information, chemical, and biological warfare; critical infrastructures (such as electric power, communications, finance, water resources, health, food, and other major systems); and database standards and integration—does not exist. Current modeling capabilities in priority areas for counterterrorism activities need to be assessed such that an overall modeling architecture for modeling and simulation can be developed. This architecture could be based on DOD’s previous R&D efforts in this area, and it could be used to determine whether current models should be adapted for the counterterrorism mission or whether new models are required. The outputs from these models would help government agencies answer questions about strategic counterterrorism issues. For example, when the models are used to identify key critical infrastructure risks (e.g., information-security risks in the control systems (SCADA) for the power grid), information would be disseminated to the agencies so that they could undertake programs to mitigate these risks. Finally, gaps and seams in the overall counterterrorism effort need to be identified such that better interfaces exist among federal, state, and local govern-

OCR for page 287
Making the Nation Safer: The Role of Science and Technology in Countering Terrorism ment agencies, as well as the many public and private organizations that have operational responsibilities and relevant information and expertise. Systems analysis should be used as one tool to help identify these gaps and seams. COUNTERTERRORISM THREAT MODELING3 The analysis of terrorist threats is a major input, as shown in Figure 10.1, to the risk analyses that must be performed to establish homeland security priorities. Currently, a large volume of pertinent information is collected by the U.S. intelligence community, but there is much work to do in organizing and integrating the information so it can be used for counterterrorism activites. This section discusses some of the factors involved in the development of an appropriate risk analysis model. It sketches an illustrative model based on systems analysis, probability theory, and game theory—one that can be used to set priorities among the various threats and threat-reduction measures. These measures include short-term actions such as restricting access to an airplane cockpit; medium-term actions such as the manufacture and stockpiling of vaccines; and long-term actions such as investing in specific areas of scientific research and in the development of new technologies. Comparison of such options is complicated because of massive uncertainty, but investment decisions must be made nonetheless. To inform such decisions, we first need a system framework that embraces the various infrastructures within the United States, the terrorist system, and their interactions. The committee lays out such a framework, sketches a model that represents it, and describes ways of dealing with uncertainty in the model’s variables. The System to Be Strengthened An overall system description must describe connections between infrastructures, people, the national economy, and social values. All of these are vulnerable, in part from the myriad of interdependencies and in part from the openness of American society (Gilmore Commission, 2001). This modeling effort clearly must be approached in stages, with continuing improvements in scope and level of detail. And as the key threats become better understood, the evolution of this system description will lead to many near-term actions. Ultimately, there is little we can do to avoid some level of discrete vulnerability. Still, we can seek to ensure to the degree possible that U.S. infrastructure systems as a whole, and certain critical subsystems, are robust, adaptive, and 3   This section is based in part on a working paper of the Stanford Department of Management Science and Engineering (Guikema and Paté-Cornell, 2002) and a RAND working paper (Davis, 2002) being used in a project for the Defense Advanced Research Projects Agency.

OCR for page 287
Making the Nation Safer: The Role of Science and Technology in Countering Terrorism resilient against a wide variety of terrorist attacks. This is akin to a “capabilities-based” approach to defense planning.4 In the face of massive uncertainty, a common impulse is to think of across-the-board defense improvements. But given our finite wealth, time, and ability to concentrate, we must make choices. Doing so requires using probability estimates or other methods for dealing with uncertainty. If probabilities are used—e.g., the probability of a given type of attack, of an installation’s vulnerability, or of the capacity to rebuild or substitute for a damaged node—they typically cannot be obtained from empirical frequency distributions; the events are too uncommon or hypothetical. Instead, the probabilities must be derived using a combination of modeling, gaming, and analysis—all with a good deal of subjectivity. Further, the probabilities should change over time as our experience grows and our knowledge improves. The Threat System to Be Weakened In parallel with strengthening defenses, we can reduce the likelihood of various threats by destroying terrorist organizations where possible and, in some cases, by deterring elements of the terrorist organizations’ larger systems. A terrorist network has numerous parts, each with different vulnerabilities and receptiveness to influence. A Bin Laden may not be deterrable, but other parts of the system—for example, an organization’s financiers and state supporters—may well be. The segments of society from which the terrorists are drawn could be influenced by international actions and by attacks on terrorism ideology and tactics. Within the United States, those who assist terrorists may be dissuaded or caught. Finally, the terrorist actors themselves are often concerned about operational risk—they may be willing to risk their lives, but not in futile attacks. Thus, better defensive measures can help to deter or deflect.5 A Simple Game-Structured View The overall system the committee is describing is dynamic. U.S. actions affect the terrorist system, and terrorist actions affect the United States. It is thus appropriate to view the problem analytically as a game, a simple version of which is shown in Figure 10.2. (It is simple by virtue of its not treating other countries or organizations explicitly.) The state of the real world changes as both sides take actions and have 4   See Rumsfeld (2001). 5   For recent discussions of terrorist behaviors, see Lesser et al. (1999), Talbott and Chanda (2001), Tucker (2000), Moodie (1998), Roberts (1997), and the Monterey Institute’s online bibliography at <http://cns.miis.edu/research/cbw/biblio.htm#terror>.

OCR for page 287
Making the Nation Safer: The Role of Science and Technology in Countering Terrorism FIGURE 10.2 Perceptions and decisions on both the U.S. and the terrorist sides. SOURCE: Paté-Cornell and Guikema (2002), by permission of the authors. reactions. Thus, Figure 10.2 applies over and over again, for each iteration in time. The state of the single node related to terrorist decision making (bottom left in the figure), for example, is the result of a complex process that can be modeled through multiple levels of resolution. The same is true for U.S. decision making. Both sides make decisions, in part on the basis of their beliefs about the other side. A System Model for Counterterrorism Defense Figure 10.3 gives highlights of a prototype model that was recently built for analyzing counterterrorism defense in such a dynamic system. Although it is not a finished product, it illustrates a global approach that could be extended and used in real time to support protection decisions. Table 10.2 summarizes its variables and the values of those variables considered in the pilot study. The model of Figure 10.3 is presented in terms of Bayesian-net influence diagrams.6 Such a diagram includes not only a (directed) network of boxes and 6   In other contexts, an influence diagram merely indicates what elements of a modeled system affect one another, without use of Bayesian nets. Such diagrams are a core element of the System Dynamics method introduced by MIT’s Jay Forrester in the 1960s (see Sterman, 2000). Variants, sometimes referred to as cognitive maps, can characterize the reasoning of adversaries, for example.

OCR for page 287
Making the Nation Safer: The Role of Science and Technology in Countering Terrorism FIGURE 10.3 Single-period influence diagrams for terrorist and U.S. decisions. SOURCE: Paté-Cornell and Guikema (2002), by permission of the authors.

OCR for page 287
Making the Nation Safer: The Role of Science and Technology in Countering Terrorism The vulnerabilities of a system are not necessarily constant. The susceptibility of the electric power grid to disruption continually changes as loads ebb and flow and as generation resources come on line, are utilized, or made unavailable. There are also persistent vulnerabilities in both hardware and software. For example, the transmission bottlenecks of Paths 15 and 26 connecting Northern and Southern California are well known (Houseman and Martin, 2001), and, as shown by Project Eligible Receiver, computers controlling electric power grids are accessible, and subject to manipulation, by anyone with hacker knowledge (Gertz, 1998; Myers, 1998). Similar vulnerabilities apply to telecommunications—the public telephone and Internet systems, for instance, and the dependence of one on the other (the Internet uses leased telephone lines for much of its physical network). This story is told with remarkable clarity in the NRC report Trust in Cyberspace (Computer Science and Telecommunications Board, 1999). A direct way to address vulnerable transmission bottlenecks and make the grid more robust is to build additional transmission capacity, but there are indications that redundancy has a dark side (in addition to increased costs). The likelihood of hidden failures in any large-scale system increases as the number of components increases. Modeling techniques are only now emerging for the analysis of such hidden failures (see, for example, Wang and Thorp, 2001). Still, the interdependencies among any of these systems are not well understood, and few models exist to bolster our understanding. Moreover, though underlying features common to these networks suggest the possibility of a unifying mathematical theory, in the current stage of knowledge we must admit that the similarity between, say, a blackout in interconnected power networks and a meltdown on the Internet is metaphorical rather than structural. Models of discipline-specific phenomena rely on corresponding domain experts, who generally do a respectable job. But while it is at the seams, or interfaces, between disciplines where modeling must address terrorist threats, these experts often find it difficult to work at the seams. Communicating across disciplines requires domain experts to learn one another’s languages in order to pose significant questions and usefully interpret the answers. Computational simulation is commonly needed, especially for modeling at the seams, and this will most often demand the use of high-performance computers. Given the necessary fidelity of the simulations, the complexity of the models, their usually stochastic nature (which requires much repetition), the massive amounts of data, and the need for advanced visualization and software-management tools to validate the results, only high-performing computing will do. Consequently, development and operation of the most extensive models can only be performed using a few specialized facilities. DOD, notably the Defense Modeling and Simulation Office, has a good deal of experience in this area, particularly with large federations of models and distributed operations. This experience, as well as that of other organizations,

OCR for page 287
Making the Nation Safer: The Role of Science and Technology in Countering Terrorism can be applied immediately to government efforts on developing and validating simulation systems specifically aimed at counterterrorism applications. However, seasoned practitioners would be the first to acknowledge the challenges in this domain. Additional research is needed in the validation area, but salvation—to the extent that it is possible—will depend on refining the theory and science of modeling, not on after-the-fact testing procedures. There is broad consensus among system modelers that quality must be designed in from the outset and established module-by-module during development. Advanced computational techniques, identification of interdependencies among infrastructure elements, and development of software and data-analysis capabilities that make use of the latest developments in computer hardware are also required. The ability to test models against real-world data to determine model fidelity for particular infrastructures and the interdependencies between different infrastructures is critical. Models must be developed and verified using real-world data. Data Issues for Infrastructure Modeling Modeling of the U.S. critical infrastructure requires significant capabilities for integrating data measured in diverse units of space and time, and it must address the limitations of many current data sources that were developed originally for stand-alone systems. Integration is limited, however, not only by the frequent incompatibility of different data sets but also by imprecision in the definitions that embrace them all (i.e., lack of adequate metadata). This situation will persist into the foreseeable future unless there is a significant data-management effort—and particularly the development of tools and methodologies for effective database integration on a large scale. The commonly cited example of the failure of the NASA Mars mission in 1999 (Madnick, 2001), caused by an erroneous attempt to integrate two databases (one with English units and the other with metric units), is just one example of inadequate database integration. Counterterrorism applications will require the integration of data from Web sources, fielded instrumentation, legacy applications in database-management systems, and many other sources. Clearly, most of these systems were not designed to work together, and much effort will thus be required to establish the data definitions for successful integration and model use. Many efforts are currently under way to develop the necessary metadata and associated standards. They include projects of the International Organization for Standardization, the FAA, and sections of the U.S. national security communities—notably the intelligence community Metadata Standard program7 and the DOD Modeling and Simulation Knowledge and Data Integration initiative. These 7   See <www.xml.saic.com/icml/> for more information.

OCR for page 287
Making the Nation Safer: The Role of Science and Technology in Countering Terrorism projects are proceeding rapidly, but similar efforts have not been widely implemented in other federal, state, and local agencies regarding databases that are highly relevant for critical infrastructure modeling and counterterrorism programs. Much more work is required, then, to develop and implement viable metadata standards that are sufficiently robust to enable the required database integration. Extending Modeling and Analysis Capabilities and Enabling Interoperation Among Databases There are two primary conclusions of this section: First, the requirements of R&D programs for protecting the U.S. critical infrastructure from catastrophic terrorism, as well as the related needs of agency operations, will not be met by the current generation of models. Furthermore, no central organization is charged with the development and implementation of the necessary models and staffed with the appropriate domain experts. Current models are designed to analyze individual systems and thus are unable to provide realistic, decision-quality information about the likely effects of terrorist acts on the overall critical infrastructure. Improving the situation should be a high priority for federal counterterrorism programs since many of the disciplines underlying large-scale systems-modeling issues will require further basic research. (The National Science Foundation (NSF) is a logical home to foster such research needs, many of which are discussed in the next section of this chapter.) Second, although there are many private and public databases that contain information potentially relevant to counterterrorism programs, they lack the necessary context definitions (i.e., metadata) and access tools to enable interoperation with other databases and the extraction of meaningful and timely information. Although elsewhere in the U.S. government efforts are under way to develop metadata standards that would greatly improve integration and interoperability, national homeland security efforts will require that programs establishing the relevant databases are supported and funded to ensure that metadata standards for counterterrorism applications are implemented. Homeland security needs should be a driver for new efforts in this area that address the current limits in data coverage, quality, timeliness, and supporting database-management technology. Recommendation 10.3: A governmentwide effort should be made to leverage existing modeling and analysis capabilities and, where appropriate, to develop new capabilities to model critical infrastructures and related interdependencies. In so doing, the federal government should collaborate with commercial organizations that have system models relevant to the homeland security missions—notably in the areas of threat assessment and critical infrastructure—in order to identify candidates for near-term model-integration initiatives.

OCR for page 287
Making the Nation Safer: The Role of Science and Technology in Countering Terrorism The capabilities and quality of existing models of government and commercial operations need to be assessed. New models may have to be built and validated in some areas. The results of these efforts, together with new methods, some of which may have to be developed, should be used to construct integrated models that can improve our understanding of the vulnerabilities of the infrastructures and their interdependencies, i.e., what needs to be defended. This understanding can be used in turn to develop sensor deployment and defensive strategies, the merit of which can be indicated by the model and validated by red-team efforts of the type described in Recommendation 10.2. Recommendation 10.4: The federal government, working with the various commercial organizations that have been identified with homeland-security-related missions, should identify counterterrorism-related databases and establish metadata standards and assess tools for integrating diverse bits of data. To conduct the analyses on which models are based, a rational data structure is needed. Efforts toward achieving this structure are under way in some government organizations. These efforts, however, have pointed to the need for additional funding, some for the organizations operating the various databases in order to establish the necessary metadata standards, and some to develop access tools for database interoperation. MODELING CHALLENGES FOR COUNTERTERRORISM The preceding sections have emphasized the critical importance of models in the systems approach to counterterrorism, and they have also noted some of the deficiencies of current modeling technologies. This section describes two methods of model development and operation that appear to offer significant potential for analyzing the complexities of counterterrorism applications. As such, they should be a significant part of the research agenda. Complex Adaptive Systems and Agent-Based Models Complex adaptive systems involve phenomena that may be characterized by the interactions of numerous individual agents or elements, which tend to self-organize at increasingly higher levels. This process results in evolutionary, emergent, and adaptive properties that are not exhibited by the individual agents themselves. For example, an animal may be an agent in a formation of a herd of animals, and herds of animals may become a species, and the species may be part of a particular ecosystem. There is a clear analogy here to the characteristics of our society’s critical infrastructure and its associated adaptation, emergence, and evolution. Complex adaptive systems obtain data and information from their internal

OCR for page 287
Making the Nation Safer: The Role of Science and Technology in Countering Terrorism and external environments alike. They find patterns, and ultimately process and represent them as internal models; these can then used by analysts to predict the potential outcome of future decisions. Further, in complex adaptive systems these internal models are subjected to revision as the impacts of decisions provide feedback. This often results in self-organization into a higher-level complex system (Axelrod and Cohen, 1999). A general rule for complex systems is that we cannot create a model that accurately predicts the outcomes of the actual system. However, we can create a model that accurately simulates the processes that the system will use in order to create a given output. Awareness of the potential for such models has profound implications for organizational efforts that are intended for such purposes as homeland security. System Dynamics-Based Models System dynamics models are used for representing whole federations (systems) of systems. They take a top-down approach to system analysis by compressing the many variables of a large, complicated system of systems into a relatively small number of overall attributes, called aggregate state variables. According to Sterman (2000), characteristics of these models include the following: State-determined. The aggregate state variables span and define all key variables within the system. Feedback-driven dynamics. The dynamics of the overall system arise not from exogenous shocks but rather from feedback between the state variables. Nonlinear. The model structure can produce highly nonlinear responses, giving rise to complex, even chaotic, dynamics. Boundary defined by system-level problem or issue. Only the system states and feedback paths necessary to replicate and investigate a given system-level problem are modeled. Emphasis on policy design for system control. Because these systems are focused on social or organizational problems, system dynamics models emphasize the design and implementation of policies that can improve problematic system performance. Risk Modeling, Assessment, and Management Process The entire process of risk assessment and management (both of which stem from risk modeling) is a synthesis of the empirical and the normative, of the quantitative and the qualitative, and of objective and subjective evidence. In risk assessment, according to Kaplan and Garrick (1981), the analyst often attempts to answer the following three questions:

OCR for page 287
Making the Nation Safer: The Role of Science and Technology in Countering Terrorism What can go wrong? What is the likelihood that it will go wrong? What are the consequences? Answers to these questions help risk analysts to identify, measure, quantify, and evaluate risks and their consequences. Risk management builds on the risk-assessment process by seeking answers to a second set of three questions: What can be done and what options are available? What are the trade-offs in terms of all costs, benefits, and risks? What are the impacts of current management decisions on future options? A systems-based risk-management approach that harmonizes overall system management must also address the four sources of failure: organizational, human, hardware, and software (Haimes, 1998). This chapter has largely addressed possible ways of reducing failure in the latter two, but doing so in the first two could well be the greater challenge. A central quandary facing the development of system-level counterterrorism modeling is the large gap between what a systems analysis says stakeholders should do, and what they actually do. Simply put, assume that the systems modelers get it exactly right and produce models that capture important dynamics and indicate important policies that should (and should not) be followed. Three layers of issues then compromise the results of these models and the effective policies and actions presumably based on them: Sterman (1989) has used the term “misperception of feedback” for the cluster of problems pertaining to a single human actor who, ideally, first comes to understand and then effectively manages system-level complexity. Repeatedly, in case and experimental situations alike, human actors manage complex systems at suboptimal levels, even when perfect information concerning the system and its dynamic complexity is available to them through system simulations and analyses. Similarly, individual government officials could well make ineffective or even counterproductive choices on antiterrorism actions. Senge (1990) and others have focused attention on a related set of issues centering on how organizations do (or do not) learn about system-level complexity. For over a decade, corporate America and key divisions of federal, state, and local governments have been striving to become more effective learning organizations. But their behaviors, even with perfect information, parallel that of the individual. Perhaps the threat of massive damage inflicted by deliberate terror will motivate government agencies to learn by means other than direct trial-and-error experimentation. Finally, we need to address the cognitive and organizational issues in-

OCR for page 287
Making the Nation Safer: The Role of Science and Technology in Countering Terrorism volved when multiple, networked agencies operate in a complex intergovernmental bureaucracy to address a problem such as terrorism. Long-Term Systems Engineering and Research Needs Federal agencies, industry consortia, and other groups addressing counterterrorism will need to develop systems-level approaches for evaluating the costs, benefits, and risks associated with homeland protection. In addition, a significant new research program in systems analysis and systems engineering for counterterrorism will be needed to develop the modeling concepts and implementations that are essential for understanding critical U.S. infrastructures. The need for improved modeling concepts applies not only to infrastructure, however, but also to the entire spectrum of science and technology for counterterrorism. The National Science Foundation (NSF) is a logical home for such a research program. Studies will be needed in the spirit of Figure 10.1, which depicts the myriad perspectives of the homeland’s system of systems (its governance, economy, and infrastructures) as well as those of the terrorist networks. Such studies will enable a greater understanding of the nature of external threats, along with the strengths and weaknesses of the U.S. critical infrastructure, so that effective policies can be formulated. The development of concepts and computational methodologies to enhance system-of-systems research and integration would allow us to address the organizational abilities needed to execute high-level systems management. In particular, approaches for agent-based and equation-based (e.g., system dynamics) modeling would enable representation of terrorist and critical infrastructure characteristics. Research would also support the development of methodologies to improve understanding of the interconnectedness and interdependencies among critical infrastructures and to better understand, model, assess, and manage the risks to homeland security from physical, economic, social, and psychological perspectives. Finally, the development and use of simulators and learning environments will be key supports for the analysis of counterterrorism policy. System-level insights are often counterintuitive, are not easily learned by trial and error, and have outcomes that may only be known in the long run. Simulators linked to learning environments can help systems managers develop and implement robust policies without experiencing costly system failures. Recommendation 10.5: To support the necessary S&T, federal agencies should establish new mechanisms for funding counterterrorism research and pilot projects at various research institutions in order to support efforts at the national, regional, and local levels. In particular, the federal government should establish a long-term, multidisciplinary systems engineering and research agenda to support future modeling challenges, educational

OCR for page 287
Making the Nation Safer: The Role of Science and Technology in Countering Terrorism opportunities, and projects aimed at developing an overall systems approach to counterterrorism. The agenda should include the following: System-of-systems perspectives for homeland security; Agent-based and system dynamics modeling; Analysis of risk assessment and management from multiple perspectives, including the risk of potentially extreme and catastrophic events; Modeling of interdependencies among critical infrastructures; and Development of simulators and learning environments. Research projects should involve many domains of expertise; a single disciplinary perspective should not dominate the agenda. NSF would be an appropriate lead agency for such a research effort, but other federal research agencies, such as the Defense Threat Reduction Agency, DARPA, and the Intelligence Community’s Advanced Research and Development Activity, have relevant expertise and should develop companion programs to support the long-term research agenda. IMPLICATIONS FOR EDUCATION The development of effective counterterrorism strategies relies on the pursuit of specific science and technology goals as well as on a systems approach (including study of those who would attack the United States) within which to apply the results. This suggests a need for systems-level thinking in education and, more specifically, a provision for educational degrees focused on systems, to help create a cadre of people who understand the interconnectedness of our society’s many parts. Degree programs at the graduate level are needed to produce leaders fully cognizant of the issues of systems and their complexity—people who can operate at the interfaces and offer an integrated vision of, say, engineering and political systems. Such degree programs will be characterized by a highly interdisciplinary course of study, which can be difficult to organize within the departmental structure of universities.8 In addition to people who have received an education specifically focused on 8   This issue was framed by Kennedy (1997) in his insightful book Academic Duty. In the final chapter, he asks: “Can the universities really make a difference with respect to the Big Problems facing us?” His list of challenges ranges from disarmament to genetic testing, but although these problems are intellectually exciting and analytically demanding, they do not come in disciplinary packages. Instead, these real, complex, and large-scale problems demand the involvement of graduates—possibly the product of “re-engineered” university departments, according to Kennedy—who are not only well trained in their fields but also skillful in systems thinking and comfortable working in a highly interdisciplinary environment.

OCR for page 287
Making the Nation Safer: The Role of Science and Technology in Countering Terrorism systems-level tools and thinking, tackling counterterrorism problems will require the people who come out of other graduate programs to be exposed to a broad background of ideas. Graduates of law and public policy programs, for example, will need to be better prepared to apply their skills in areas with substantial scientific and technological content, while science and engineering professionals will have to learn how to identify policy constraints and possibilities and devise political strategies that take the interests of all stakeholders into account. The education of future leaders is important, but existing leaders will also need to embrace systems approaches to today’s problems in order to make deep contributions to the nation’s holistic responses to the threat of terrorism. Business and military leaders are traditionally required to engage in continuing education courses, which could provide opportunities to update them on advances in systems analysis and on the types of problems that will benefit from systems-level thinking and tools. Recommendation 10.6: Government agencies that fund university research should enhance their support of research projects that feature systems analysis and systems engineering, in part to help produce new integrative departments and future leaders who think across the traditional academic boundaries and who can address the complex scientific and technological issues discussed above. REFERENCES Advisory Panel to Assess Domestic Response Capabilities for Terrorism Involving Weapons of Mass Destruction (Gilmore Commission). 2001. Third Annual Report to the President and the Congress, prepared for the Department of Defense, RAND, Santa Monica, Calif., December 15. Available online at <www.rand.org/nsrd/terrpanel/terror3-screen.pdf>. Axelrod, Robert, and Michael D. Cohen. 2000. Harnessing Complexity: Organizational Implications of a Scientific Frontier, Free Press, New York. Carter, Ashton B. 2001-2002. “The Architecture of Government in the Face of Terrorism,” International Security, Vol. 26, No. 3, pp. 5-23, Winter. Casti, John L. 1996. Would-Be Worlds: How Simulation Is Changing the Face of Science, John Wiley & Sons, New York. Chankong, Vira, and Yacov Y. Haimes. 1983. Multiobjective Decision Making: Theory and Methodology, Elsevier-North Holland, New York. Computer Science and Telecommunications Board, National Research Council. 1999. Trust in Cyberspace, National Academy Press, Washington, D.C. Critical Infrastructure Assurance Office. 1998. The Clinton Administration’s Policy on Critical Infrastructure Protection: Presidential Decision Directive 63, The White House, Washington, D.C., May 22. Available online at <http://www.info-sec.com/ciao/paper598.pdf>. Davis, Paul K. 2002. “Key Elements of a Framework for the ‘Influencing Component’ of Counter-Terrorism,” working paper in a project for the Defense Advanced Research Projects Agency (DARPA), RAND, Santa Monica, Calif., May 24. Davis, Paul K., James H. Bigelow, and Jimmie McEver. 2001. Exploratory Analysis and a Case History of Multiresolution, Multiperspective Modeling, RP-925, RAND, Santa Monica, Calif. (This is a volume of reprinted journal articles.)

OCR for page 287
Making the Nation Safer: The Role of Science and Technology in Countering Terrorism Gertz, Bill. 1998. “Infowar Game Shut Down U.S. Power Grid, Disabled Pacific Command,” Washington Times, April 17, p. A3. Haimes, Yacov Y. 1998. Risk Modeling, Assessment, and Management (Wiley Series in Systems Engineering), John Wiley & Sons, New York. Haimes, Yacov Y. 2002. “A Roadmap for Modeling the Risks of Terrorism to the Homeland,” ASCE Journal of Infrastructure Systems, Vol. 8, No. 2, June, pp. 35-41. Haimes, Yacov Y., and Pu Jiang. 2001. “Leontief-Based Model of Risk in Complex Interconnected Infrastructures,” ASCE Journal of Infrastructure Systems, Vol. 7, No. 1, March, pp. 1-12. Heller, Miriam. 2002. “Critical Infrastructure Interdependencies: A System Approach to Research Needs,” Infrastructure and Information Systems, National Science Foundation, presentation to the National Academies Committee on Science and Technology for Countering Terrorism: Panel on Systems Analysis and Systems Engineering, February 14. Hillestad, Richard, and Paul K. Davis. 1998. Resource Allocation for the New Defense Strategy: The DynaRank Decision Support System, MR-996-OSD, RAND, Santa Monica, Calif. Available online at <www.rand.org/publications/MR/MR996/index.html>. Houseman, R., and E. Dee Martin. 2001. Protecting America’s Critical Energy Infrastructure from Terrorist Attack, Bracewell and Patterson, L.L.P., Houston, Tex., November 7. Howard, R.A. 1999. “From Influence to Relevance and Knowledge,” Influence Diagrams, Belief Nets, and Decision Analysis (Wiley Series in Probability and Mathematical Statistics), R.M. Oliver and J.Q. Smith, eds. John Wiley & Sons, New York. Kaplan, Stan, and B. John Garrick. 1981. “On the Quantitative Definition of Risk,” Risk Analysis, Vol. 1, No. 1, pp. 11-27. Keeney, Ralph L., and H. Raiffa. 1993. Decisions with Multiple Objectives: Preferences and Value Trade-Offs, Cambridge University Press, New York. Keeney, Ralph L. 1976. Decisions with Multiple Objectives: Preferences and Value Tradeoffs (Wiley Series in Probability and Mathematical Statistics), John Wiley & Sons, New York. Kennedy, Donald. 1997. Academic Duty, Harvard University Press, Cambridge, Mass. Lempert, Robert J., Michael E. Schlesinger, and Steven C. Bankes. 1996. “When We Don’t Know the Costs or the Benefits: Adaptive Strategies for Abating Climate Change,” Climatic Change, Vol. 33, No. 2, pp. 235-274. Lesser, Ian O., Bruce Hoffman, John Arquilla, David Ronfeldt, and Michele Zanini. 1999. Countering the New Terrorism, prepared for the U.S. Air Force, RAND, Santa Monica, Calif. MacKay, James A., Ian Lerche, and John A. MacKay. 1999. Economic Risk in Hydrocarbon Exploration, Academic Press, San Diego, Calif. Madnick, Stuart E. 2001. “The Misguided Silver Bullet: What XML Will and Will NOT Do to Help Information Integration,” Proceedings of the Third International Conference on Information Integration and Web-Based Applications and Services (IIWAS2001), held in Linz, Austria, September 10-12. Moodie, Michael. 1998. Chemical and Biological Weapons: Will Deterrence Work? (The Deterrence Series), Chemical and Biological Arms Control Institute, Washington, D.C., March. Myers, Laura. 1998. “Pentagon Has Computers Hacked,” Associated Press, April 17. Available online at <www.connectingpointlv.com/pentagon_has_computers_hacked.htm>. Neumann, Peter G. 1995. Computer Related Risks, Addison-Wesley, Reading, Mass. Paté-Cornell, M. Elisabeth, and Seth D. Guikema. 2002. “Probabilistic Modeling of Terrorist Threats: A Systems Analysis Approach to Setting Priorities Among Countermeasures,” submitted for publication in Military Operations Research; working paper of the Department of Management and Science and Engineering, Stanford University, Stanford, Calif. Renaldi, Steven M., James P. Peerenboom, and Terry K. Kelly. 2001. “Identifying, Understanding, and Analyzing Critical Infrastructure Interdependencies,” IEEE Control Systems Magazine, Vol. 21, No. 6, December, pp. 11-25.

OCR for page 287
Making the Nation Safer: The Role of Science and Technology in Countering Terrorism Roberts, Bradley, ed. 1997. Terrorism with Chemical and Biological Weapons: Calibrating Risks and Responses, Chemical and Biological Arms Control Institute, Washington, D.C. Rumsfeld, Donald H. 2001. Quadrennial Defense Review Report, Department of Defense, Washington, D.C., September 30. Available online at <www.comw.org/qdr/qdr2001.pdf>. Sage, Andrew P., and William B. Rouse, eds. 1999. Handbook of Systems Engineering and Management, John Wiley & Sons, New York. Senge, Peter M. 1990. The Fifth Discipline: The Art and Practice of the Learning Organization, Currency/Doubleday, New York. Shachter, Ross D. 1986. “Evaluating Influence Diagrams,” Operations Research, Vol. 34, No. 6, November-December, pp. 871-882. Sterman, John D. 1989. “Misperceptions of Feedback in Dynamic Decision Making,” Organizational Behavior and Human Decision Processes, Vol. 43, No. 3, pp. 301-335. Sterman, John D. 2000. Business Dynamics: Systems Thinking and Modeling for a Complex World, Irwin Professional Publishing Co./McGraw-Hill Professional Books, New York. Talbott, Strobe, and Nayan Chanda, eds. 2001. The Age of Terror: America and the World After September 11, Basic Books, New York. Tucker, Jonathan B., ed. 2000. Toxic Terror: Assessing Terrorist Use of Chemical and Biological Weapons, MIT Press, Cambridge, Mass. U.S. Environmental Protection Agency. 2002. Lessons Learned on Planning and Scoping for Environmental Risk Assessments, Washington D.C., January. Vatis, Michael A. 2001a. Combating Terrorism: A Compendium of Recent Counter Terrorism Recommendations from Authoritative Commissions and Subject Matter Experts, Institute for Security Technology Studies at Dartmouth College, Hanover, N.H., September 16. Available online at <http://www.ists.dartmouth.edu/ISTS/counterterrorism/cbt_ter1.pdf>. Vatis, Michael A. 2001b. Cyber Attacks During the War on Terrorism: A Predictive Analysis, Institute for Security Technology Studies at Dartmouth College, Hanover, N.H., September 22. Available online at <http://www.ists.dartmouth.edu/ISTS/counterterrorism/cyber_a1.pdf>. Wang, Hongye, and James S. Thorp. 2001. “Optimal Locations for Protection System Enhancement: A Simulation of Cascading Outages,” IEEE Transactions on Power Delivery, Vol. 16, No. 4, October, pp. 528-533.