Cover Image

PAPERBACK
$39.75



View/Hide Left Panel

Appendix D
Bioterrorism Risk Analysis with Decision Trees

Gregory S. Parnell, Ph.D.

Professor, Department of Systems Engineering

United States Military Academy, West Point, New York

INTRODUCTION

The foundational risk analysis method used by the Department of Homeland Security (DHS) Biological Threat Risk Assessment (BTRA) methodology is event trees. Event trees are a proven probabilistic risk analysis technique that has been effectively used for risk analysis of natural and man-made hazards (Dillon-Merrill, Parnell, and Buckshaw, 2007). The body of this report has shown weaknesses in the use of event trees to model terrorist actions since event trees do not model the actions of an intelligent adversary.

To address these concerns, we convert the DHS bioterrorist event tree to a bioterrorist decision tree by changing terrorist decisions to decision nodes, removing two nodes that are problematic and unnecessary, dramatically reducing the complexity by assessing probabilities for each arc for each event instead of probability distributions for each arc for each event. In addition, we describe several alternatives for consequence modeling including separate and aggregated consequences.

MANY BTRA MODELING ALTERNATIVES EXIST

Several risk analysis modeling decisions must be made to provide effective and efficient risk analyses that support national homeland security decision-makers. Figure D.1 is a strategy generation table (Parnell, Driscoll, and Henderson, 2008) used to identify possible modeling decisions. The column titles of Figure D.1 identify some of the most important modeling decisions. The analysis responsiveness (model run time) determines the flexibility of the model and the usefulness to support risk assessment and risk management decision making. The model’s transparency increases the understanding and credibility of the model to stakeholders and decision makers. The assumed time period significantly impacts the data collection. The longer the time period, the more challenging it will be to provide credible data assessments. The next three columns (terrorist decisions, U.S. decisions, and uncertain events) are the decisions and events that must be modeled. The types of consequences are major modeling decisions since models will need to be developed for each type of consequence. Finally, the consequences can be modeled individually or combined. Combining enables an integrated assessment but takes more modeling and analysis to credibly combine the consequences.

The columns below the modeling decisions identify several possible techniques for each modeling decision. For example, analysis responsiveness can be real-time, hours, days, weeks, or months. Years are possible but probably not very useful. Using the strategy generation table, we can shade one (or more) box(es) in each column to describe or develop a BTRA modeling alternative. Figure D.1 describes the BTRA of 2006 and Figure D.2 describes the Bioterrorist Decision Model developed in this appendix.

The shading in Figure D.1 shows the committee’s understanding of the 2006 BTRA modeling. Battelle developed its own software instead of usually commercially available software to perform the event tree analysis. Due to the complexity, the BTRA model runs in days and requires special software and specially trained analysts to perform the analysis. Some sensitivity analysis capability has been developed and performed. The BTRA model is not transparent. The model is very complex and uses a mixture of best available existing models and new, unvalidated models. The first event in the BTRA event tree is the frequency of attacks. This approach requires specification of a time period and the prediction of the number of attacks with each agent. BTRA event tree models terrorist decisions, U.S. decisions, and uncertain events as probabilities. The methodology greatly increases its complexity and data requirements by assessing probability distributions on each branch of the event tree. The primary consequence modeling was on mortality but some modeling of morbidity and economics was done. The consequences were analyzed individually and not combined.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 85
Appendix D Bioterrorism Risk Analysis with Decision Trees Gregory S. Parnell, Ph.D. Professor, Department of Systems Engineering United States Military Academy, West Point, New York INTRODUCTION ments. The next three columns (terrorist decisions, U.S. decisions, and uncertain events) are the decisions and events The foundational risk analysis method used by the De- that must be modeled. The types of consequences are major partment of Homeland Security (DHS) Biological Threat modeling decisions since models will need to be developed Risk Assessment (BTRA) methodology is event trees. Event for each type of consequence. Finally, the consequences can trees are a proven probabilistic risk analysis technique that be modeled individually or combined. Combining enables an has been effectively used for risk analysis of natural and integrated assessment but takes more modeling and analysis man-made hazards (Dillon-Merrill, Parnell, and Buckshaw, to credibly combine the consequences. 2007). The body of this report has shown weaknesses in the The columns below the modeling decisions identify use of event trees to model terrorist actions since event trees several possible techniques for each modeling decision. For do not model the actions of an intelligent adversary. example, analysis responsiveness can be real-time, hours, To address these concerns, we convert the DHS bio- days, weeks, or months. Years are possible but probably terrorist event tree to a bioterrorist decision tree by changing not very useful. Using the strategy generation table, we can terrorist decisions to decision nodes, removing two nodes shade one (or more) box(es) in each column to describe or that are problematic and unnecessary, dramatically reducing develop a BTRA modeling alternative. Figure D.1 describes the complexity by assessing probabilities for each arc for the BTRA of 2006 and Figure D.2 describes the Bioterrorist each event instead of probability distributions for each arc Decision Model developed in this appendix. for each event. In addition, we describe several alternatives The shading in Figure D.1 shows the committee’s under- for consequence modeling including separate and aggregated standing of the 2006 BTRA modeling. Battelle developed consequences. its own software instead of usually commercially available software to perform the event tree analysis. Due to the MANY BTRA MODELING ALTERNATIVES EXIST complexity, the BTRA model runs in days and requires special software and specially trained analysts to perform Several risk analysis modeling decisions must be made the analysis. Some sensitivity analysis capability has been to provide effective and efficient risk analyses that support developed and performed. The BTRA model is not transpar- national homeland security decision-makers. Figure D.1 is a ent. The model is very complex and uses a mixture of best strategy generation table (Parnell, Driscoll, and Henderson, available existing models and new, unvalidated models. The 2008) used to identify possible modeling decisions. The col- first event in the BTRA event tree is the frequency of attacks. umn titles of Figure D.1 identify some of the most important This approach requires specification of a time period and the modeling decisions. The analysis responsiveness (model prediction of the number of attacks with each agent. BTRA run time) determines the flexibility of the model and the event tree models terrorist decisions, U.S. decisions, and usefulness to support risk assessment and risk management uncertain events as probabilities. The methodology greatly decision making. The model’s transparency increases the increases its complexity and data requirements by assessing understanding and credibility of the model to stakeholders probability distributions on each branch of the event tree. The and decision makers. The assumed time period significantly primary consequence modeling was on mortality but some impacts the data collection. The longer the time period, the modeling of morbidity and economics was done. The conse- more challenging it will be to provide credible data assess- quences were analyzed individually and not combined. 

OCR for page 85
 DEPARTMENT OF HOMELAND SECURITY BIOTERRORISM RISK ASSESSMENT Analysis Model Terrorist Combining Responsiveness Time Period U.S. Decisions Uncertain Events Consequences Consequencesa Transparency Decisions (Run-Time) Transparent, Analyzed indi- Real-time simple models Time until first Scenarios Scenarios Not modeled Mortality vidually and not (Minutes) tailored to avail- attack combined able data Transparent us- Fixed time ing metamodels period with Probability Probability Deterministic Converted to Hours developed for best Morbidity potential for distributions distributions (parameter) dollars available national multiple attacks models Black box with Decision Decision models that are Multiple attacks Combined with made to made to Probability Days mixture of best in a specified Economic multiattribute maximize some maximize some distribution available and un- time period value function objective(s) objective(s) validated models Black box with Multiple attacks Probability Combined with unvalidated, un- Weeks in an unspeci- Game theory models distributions on Psychological multiattribute verified, and unac- fied time period probabilities utility function credited models Distributed mod- eling using best Months Not applicable Attacker-defender models Not applicable Environmental Not applicable available national models aKirkwood (1997) discusses the technical assumptions for multiattribute value and utility functions. FIGURE D.1 BTRA modeling alternatives. This figure provides a bioterrorism risk assessment modeling alternative generation table (Parnell, Driscoll, and Henderson, 2008) to help identify the BTRA modeling alternatives available to DHS. The column headings are the modeling decisions that must be made by DHS. The column cells identify the modeling techniques we considered for each modeling decision. The gray shading depicts the committee’s understanding of 2006 BTRA methodology. USING DECISION ANALYSIS TO ANALYZE decision trees has been used since 1968 (Raiffa, 1968; THE TERRORIST’S ATTACK DECISION Clemen, 1996). Multiple objective decision analysis has been used since 1976 (Keeney and Raiffa, 1976; Kirkwood, 1997). Based on the committee’s assessment, several improve- Maxwell (2006) summarizes the large selection of commer- ments are needed. First and foremost, the methodology must cially available decision and risk analysis software. consider the terrorist as an intelligent adversary that will Figure D.2 uses the format of Figure D.1 and shows select the best attack strategy to maximize their strategic the modeling techniques that would be used in a decision objectives. Second, the methodology must be transparent. analysis method. The darker shaded cells define one potential A key goal should be the use of commercially available decision analysis method used to maximize the achievement software that has built-in sensitivity analysis features to im- of terrorist objectives. The lighter shaded cells describe al- prove understanding and transparency. The method should ternative decision analysis methods. The goal would be to eliminate unnecessary complexity and demands for data that use commercially available tools and keep the models small will have no meaning if one bioterrorism attack is made on enough to have reasonable run times. Using commercially the United States, e.g., the attack frequency for each agent. available software helps make the models transparent and Finally, the methodology should be easily modified to sup- allows the use of standard decision analysis and sensitivity port the analysis of risk management alternatives. analysis that provide insights and improve transparency. Decision analysis offers the potential to make many of The decision tree would model the terrorist’s decision to use the improvements we have discussed. Decision analysis is biological agents to achieve his or her strategic objectives closely related to probabilistic risk analysis (Paté-Cornell by maximizing consequences to the United States. All of and Dillon, 2006). Single objective decision analysis with the terrorist decisions would be modeled as decision nodes.

OCR for page 85
 APPENDIX D Analysis Model Terrorist Combining Responsiveness Time Period U.S. Decisions Uncertain Events Consequences Transparency Decisions Consequences (Run-Time) Transparent, Analyzed indi- Real-time simple models Time until first Scenarios Scenarios Not modeled Mortality vidually and not (Minutes) tailored to avail- attack combined able data Transparent us- Fixed time ing metamodels period with Probability Probability Deterministic Converted to Hours developed for best Morbidity potential for distributions distributions (parameter) dollars available national multiple attacks models Black box with Decision Decision models that are Multiple attacks Combined with made to made to Probability Days mixture of best in a specified Economic multiattribute maximize some maximize some distribution available and un- time period value function objective(s) objective(s) validated models Black box with Multiple attacks Probability Combined with unvalidated, un- Weeks in an unspeci- Game theory models distributions on Psychological multiattribute verified, and unac- fied time period probabilities utility function credited models Distributed mod- eling using best Months Not applicable Attacker-defender models Not applicable Environmental Not applicable available national models FIGURE D.2 BTRA modeling using decision analysis. This figure provides an alternative generation table developed in Figure D.1. However, instead of showing the 2006 BTRA modeling alternative, the dark gray shading highlights a decision analysis method for BTRA. The light gray shading identifies possible variations to the proposed decision analysis methodology. For example, instead of combining the consequences using a multiattribute value model, the consequences could be analyzed individually and not combined or be converted to dollars. Since they are uncertain to the terrorists, U.S. decisions (e.g., subject matter experts. However, in order to use as much as interdiction) and uncertain events (e.g., detection) would be possible of the existing 2006 BTRA event tree method, we modeled using probability distributions. Any of the conse- directly converted the event tree to a decision tree. Using quences that have credible models could be used. Decision a format similar to Figure 3.4 in Chapter 3 of this report, trees can be used to find the terrorist strategy (a sequential Figure D.3 lists one possible set of assumptions that could set of decisions) that maximizes the terrorist objectives by be used to convert the DHS event tree to the bioterrorist deci- averaging out and rolling back the decision tree. The decision sion tree. The figure adds new node numbers, type of node, tree can be solved multiple times for each single objective or rationale, average branches, and probability distributions to can be solved once with combined consequences (Parnell, be assessed. The phases are the same but are not included 2007). There are at least three ways of combining the con- due to space limitations on the page. sequences: converting each consequence to dollars, using a Several assumptions were made in Figure D.3. First, the multiple attribute value model to normalize and weight the old nodes numbers 1 (frequency of attack) and 16 (potential consequences, or using a multiple attribute utility model to for multiple attacks) were deleted for the reasons discussed normalize and weight the consequences. Each of the tech- above. Second, we converted all terrorism decisions to decision nodes.1 That left six chance nodes: four interdic- niques has different assumptions and data requirements. All have been used on major national studies. tion nodes, one detection node, and one consequence node. Each of these would be uncertain to the bioterrorist. Third, AN ILLUSTRATIVE BIOTERRORIST DECISION MODEL USING DECISION TREES 1While agent selection is an obvious decision, some of the later decisions could be modeled as uncertain nodes early in the terrorist planning cycle. The 18 node event tree (with consequences) could be The actual nodes that would be decision or chance nodes would depend on simplified especially if credible data are not available from the knowledge of subject matter experts.

OCR for page 85
 DEPARTMENT OF HOMELAND SECURITY BIOTERRORISM RISK ASSESSMENT Old New Depends Maximize Probability Type of Max Average Paths Stage Stage Rationale Decision/Event on Paths Distributions to Assess Node Branches Branches (cumulative) No. No. Events (cumulative) (additive) All probabilities will Frequency of Initiation by Ter- Not 1 Deleted change after first rorist Group Applicable bioattack. Terrorists will consider 3 1 Decision the bioagents they can Bioagent Selection 28 28 28 28 0 obtain. Target will be selected to 2 2 Decision Target Selection 1 8 3 84 224 0 maximize consequences. Mode of Dissemination (also Mode will be selected to 4 3 Decision determines wet or dry disper- 1, 2 9 3 252 2,016 0 maximize consequences. sal form) Mode will be selected to 5 4 Decision Mode of Agent Acquisition 1 4 4 1,008 8,064 0 maximize consequences. Can be changed by U.S. Interdiction during 6 5 Chance 1, 4 2 2 2,016 16,128 112 actions. Acquisition Terrorist selects Location of Production and 7 6 Decision 1 2 2 4,032 32,256 0 location. Processing 8 7 Decision Depends on agent. Mode of Agent Production 1 3 3 12,096 96,768 0 Preprocessing and 9 8 Decision 1, 2, 3, 7 3 3 36,288 290,304 0 Concentration 10 9 Decision Drying and Processing 1, 2, 3 3 3 108,864 870,912 0 11 10 Decision Additives 1, 2, 3 2 2 217,728 1,741,824 0 Can be changed by U.S. Interdiction During 12 11 Chance 6 2 2 435,456 3,483,648 56 actions. Production and Processing Mode of Transport and 13 12 Decision Terrorist decision. 1, 2, 3 3 3 1,306,368 10,450,944 0 Storage Interdiction During Transport 14 13 Chance Depends on U.S. actions. 6 2 2 2,612,736 20,901,888 56 and Storage 15 14 Chance Depends on U.S. actions. Interdiction During Attack 2 2 5,225,472 41,803,776 1 Terrorist can always do Not 16 Potential for Multiple Attacks 1 0 multiple attacks. Applicable Can be changed by U.S. 17 15 Chance Event Detection 1, 2, 3 3 3 15,676,416 125,411,328 252 actions. FIGURE D.3 This figure describes one possible set of assumptions that would generate a decision tree that could be solved for a bioterror- ist to maximize the consequences of damage to the United States. The figure uses the format of Figure 3.4 in Chapter 3 of this report and adds new node numbers, type of node, rationale, average branches, and probability distributions to be assessed. All terrorist decisions are converted to decision nodes. THE BIOTERRORIST DECISION MODEL CAN we added the consequence model to the decision tree as PROVIDE RISK ASSESSMENT RESULTS the end node. In decision analysis software, this would be AND SENSITIVITY ANALYSIS implemented using an equation in the end node that uses scenario parameters common to all agents and parameters The decision analysis model that we have described (agent decision and chance node outcomes) that depend on would identify the terrorist’s best strategy to maximize the path through the decision tree. If the consequences are the consequences of an attack. Senior decision makers and not combined, a decision tree would be created for each stakeholders would be provided a one to n list of the agents consequence using a different consequence model. that have the potential to create the most harm to the United

OCR for page 85
 APPENDIX D States. Since decision analysis also calculates the cumula- solved using commercially available software using com- tive consequence distribution for each strategy, absolute risk plete enumeration or Monte Carlo simulation. Third, the new could easily be displayed for each agent. challenge is how to develop consequence models that use the Decision analysis models are transparent. Commercial decision parameters in the decision tree that will allow for decision analysis tools provide a range of powerful sensitiv- rapid evaluation of the decision tree for each path. Fourth, ity analysis tools (Clemen, 1996) to increase understanding further opportunities exist to simplify the decision tree. For and improve credibility. The model can be quickly resolved example, if a decision does not impact the consequences, it if any stakeholder provides an alternative set of data assump- can be removed from the decision tree. tions. Sensitivity analysis bar charts (Tornado diagrams) can be used to show the most significant data assumptions. Value THE BIOTERRORIST DECISION MODEL of information calculations can be performed to find out what EFFECTIVELY ADDRESSES THE FUNDAMENTAL uncertainties have the most impact on the agent risk. CONCERNS OF THE BTRA OF 2006 In the introduction we listed the most fundamental con- THE BIOTERRORIST DECISION MODEL ALSO cerns with the 2006 BTRA methodology: not considering SUPPORTS RISK MANAGEMENT DECISION MAKING intelligent adversary decision making, huge data demands, So far we have focused on the use of decision analysis as more complexity than the available data support, lack of a modeling framework to support bioterrorism risk assess- transparency for decision makers/stakeholders (see Chapter ments. The Bioterrorist Decision Model would provide the 3), and lack of a clear linkage to DHS risk management de- baseline risk for the bioagents analyzed. Since the model can cision making. The Bioterrorist Decision Model effectively be run quickly, it could be a very useful tool to support DHS addresses each of these concerns. risk management decision making. The Bioterrorist Decision Model solves the problem of The bioterrorism risk is impacted by the U.S. ability to modeling an intelligent adversary by selecting the bioagents reduce the threat (prevent an attack or interdict an attack in that will maximize the objectives of the terrorists. The progress), reduce the nation’s vulnerabilities, and mitigate model greatly reduces the huge data demands by convert- the consequences given that an attack has occurred. Govern- ing terrorist decisions to decision nodes, deleting the two ment agencies, including the intelligence community, the most problematic nodes—frequency of attack and multiple Department of Homeland Security, and the Department of attacks—and not using probability distributions for each arc Health and Human Services, expend significant resources on each node. Finally, the model improves transparency by each year to increase security against attacks on our nation, using commercially available software with built-in sensitiv- including bioterrorist attacks. In the Bioterrorist Decision ity analysis capabilities. Model, U.S. capabilities are reflected in the probabilities assigned to the uncertain nodes (the interdiction, detection, REFERENCES and consequence nodes). To assess the risk reduction of risk Clemen, R. 1996. Making Hard Decisions, 2nd edition. Belmont, Calif.: management alteratives we can modify the model to change Duxbury Press. the probabilities for each risk management alternative or Dillon-Merrill, R.L., G.S. Parnell, and D.L. Buckshaw. 2007. “Logic Trees: set of alternatives. Due to the complexities of risk assess- Fault, Success, Attack, Event, Probability, and Decision Trees.” In John ment mentioned in Chapter 2 of this report, the results may G. Voeller (ed.), Wiley Handbook of Science and Technology for Home- be initially non-intuitive. For example, a large reduction in land Security. Hoboken, N.J.: Wiley and Sons. Forthcoming. Keeney, R.L. 1992. Value-Focused Thinking: A Path to Creatie Decision- the consequences of the highest-risk bioagent may not have making. Cambridge, Mass.: Harvard University Press. a large reduction in overall risk since the second-highest- Keeney, R.L., and H. Raiffa. 1976. Decision Making with Multiple Objec- agent consequences might not be affected. In some cases, we ties Preferences and Value Tradeoffs. New York: Wiley. would have to consider sets of alternatives since, in general, Kirkwood, C.W. 1997. Strategic Decision Making: Multiobjectie Decision the risk reduction would not be additive. Some risk manage- Analysis with Spreadsheets. Belmont, Calif.: Duxbury Press. Maxwell, D.T. 2006. “Improving Hard Decisions.” OR/MS Today, pp. 51- ment alternatives may be synergistic (impact greater than the 61. [Biannual survey of decision analysis software] sum of their individual benefits) or complementary (impact Parnell, G.S. 2007. “Multi-objective Decision Analysis.” In John G. Voeller less than the sum of their individual benefits). (ed.), Wiley Handbook of Science and Technology for Homeland Secu- rity. Hoboken, N.J.: Wiley & Sons. Forthcoming. Parnell, G.S., P.J. Driscoll, and D.L. Henderson (eds.). 2008. Decision Mak- INSIGHTS FROM THE BIOTERRORIST ing for Systems Engineering and Management. Wiley Series in Systems DECISION MODEL APPROACH Engineering, Andrew P. Sage (ed.). Hoboken, N.J.: Wiley and Sons. Paté-Cornell, E.E., and R.L. Dillon. 2006. “The Respective Roles of There are several important insights from the analysis Risk and Decision Analysis in Decision Support.” Decision Analysis presented in this appendix. First, converting the event tree to 3(4):220-232. a decision tree greatly simplifies the probability assessment Raiffa, H. 1968. Decision Analysis: Introductory Lectures on Choices Under Uncertainty. Boston, Mass.: Addison-Wesley. tasks. Second, the decision tree should allow the tree to be