Department of Homeland Security Bioterrorism Risk Assessment: A Call for Change (2008)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Appendix D Bioterrorism Risk Analysis with Decision Trees Gregory S. Parnell, Ph.D. Professor, Department of Systems Engineering United States Military Academy, West Point, New York INTRODUCTION ments. The next three columns (terrorist decisions, U.S. decisions, and uncertain events) are the decisions and events The foundational risk analysis method used by the De- that must be modeled. The types of consequences are major partment of Homeland Security (DHS) Biological Threat modeling decisions since models will need to be developed Risk Assessment (BTRA) methodology is event trees. Event for each type of consequence. Finally, the consequences can trees are a proven probabilistic risk analysis technique that be modeled individually or combined. Combining enables an has been effectively used for risk analysis of natural and integrated assessment but takes more modeling and analysis man-made hazards (Dillon-Merrill, Parnell, and Buckshaw, to credibly combine the consequences. 2007). The body of this report has shown weaknesses in the The columns below the modeling decisions identify use of event trees to model terrorist actions since event trees several possible techniques for each modeling decision. For do not model the actions of an intelligent adversary. example, analysis responsiveness can be real-time, hours, To address these concerns, we convert the DHS bio­ days, weeks, or months. Years are possible but probably terrorist event tree to a bioterrorist decision tree by changing not very useful. Using the strategy generation table, we can terrorist decisions to decision nodes, removing two nodes shade one (or more) box(es) in each column to describe or that are problematic and unnecessary, dramatically reducing develop a BTRA modeling alternative. Figure D.1 describes the complexity by assessing probabilities for each arc for the BTRA of 2006 and Figure D.2 describes the Bioterrorist each event instead of probability distributions for each arc Decision Model developed in this appendix. for each event. In addition, we describe several alternatives The shading in Figure D.1 shows the committee’s under- for consequence modeling including separate and aggregated standing of the 2006 BTRA modeling. Battelle developed consequences. its own software instead of usually commercially available software to perform the event tree analysis. Due to the MANY BTRA MODELING ALTERNATIVES EXIST complexity, the BTRA model runs in days and requires special software and specially trained analysts to perform Several risk analysis modeling decisions must be made the analysis. Some sensitivity analysis capability has been to provide effective and efficient risk analyses that support developed and performed. The BTRA model is not transpar- national homeland security decision-makers. Figure D.1 is a ent. The model is very complex and uses a mixture of best strategy generation table (Parnell, Driscoll, and Henderson, available existing models and new, unvalidated models. The 2008) used to identify possible modeling decisions. The col- first event in the BTRA event tree is the frequency of attacks. umn titles of Figure D.1 identify some of the most important This approach requires specification of a time period and the modeling decisions. The analysis responsiveness (model prediction of the number of attacks with each agent. BTRA run time) determines the flexibility of the model and the event tree models terrorist decisions, U.S. decisions, and usefulness to support risk assessment and risk management uncertain events as probabilities. The methodology greatly decision making. The model’s transparency increases the increases its complexity and data requirements by assessing understanding and credibility of the model to stakeholders probability distributions on each branch of the event tree. The and decision makers. The assumed time period significantly primary consequence modeling was on mortality but some impacts the data collection. The longer the time period, the modeling of morbidity and economics was done. The conse- more challenging it will be to provide credible data assess- quences were analyzed individually and not combined. 85 

86 DEPARTMENT OF HOMELAND SECURITY BIOTERRORISM RISK ASSESSMENT Analysis Model Terrorist Combining Responsiveness Time Period U.S. Decisions Uncertain Events Consequences Transparency Decisions Consequencesa (Run-Time) Transparent, Analyzed indi- Real-time simple models Time until first Scenarios Scenarios Not modeled Mortality vidually and not (Minutes) tailored to avail- attack combined able data Transparent us- Fixed time ing metamodels period with Probability Probability Deterministic Converted to Hours developed for best Morbidity potential for distributions distributions (parameter) dollars available national multiple attacks models Black box with Decision Decision models that are Multiple attacks Combined with made to made to Probability Days mixture of best in a specified Economic multiattribute maximize some maximize some distribution available and un- time period value function objective(s) objective(s) validated models Black box with Multiple attacks Probability Combined with unvalidated, un- Weeks in an unspeci- Game theory models distributions on Psychological multiattribute verified, and unac- fied time period probabilities utility function credited models Distributed mod- eling using best Months Not applicable Attacker-defender models Not applicable Environmental Not applicable available national models aKirkwood (1997) discusses the technical assumptions for multiattribute value and utility functions. FIGURE D.1  BTRA modeling alternatives. This figure provides a bioterrorism risk assessment modeling alternative generation table (Parnell, Driscoll, and Henderson, 2008) to help identify the BTRA modeling alternatives available to DHS. The column headings are the modeling decisions that must be made by DHS. The column cells identify the modeling techniques we considered for each modeling decision. The gray shading depicts the committee’s understanding of 2006 BTRA methodology. USING DECISION ANALYSIS TO ANALYZE decision trees has been used since 1968 (Raiffa, 1968; THE TERRORIST’S ATTACK DECISION Clemen, 1996). Multiple objective decision analysis has been used since 1976 (Keeney and Raiffa, 1976; Kirkwood, 1997). Based on the committee’s assessment, several improve- Maxwell (2006) summarizes the large selection of commer- ments are needed. First and foremost, the methodology must cially available decision and risk analysis software. consider the terrorist as an intelligent adversary that will Figure D.2 uses the format of Figure D.1 and shows select the best attack strategy to maximize their strategic the modeling techniques that would be used in a decision objectives. Second, the methodology must be transparent. analysis method. The darker shaded cells define one potential A key goal should be the use of commercially available decision analysis method used to maximize the achievement software that has built-in sensitivity analysis features to im- of terrorist objectives. The lighter shaded cells describe al- prove understanding and transparency. The method should ternative decision analysis methods. The goal would be to eliminate unnecessary complexity and demands for data that use commercially available tools and keep the models small will have no meaning if one bioterrorism attack is made on enough to have reasonable run times. Using commercially the United States, e.g., the attack frequency for each agent. available software helps make the models transparent and Finally, the methodology should be easily modified to sup- allows the use of standard decision analysis and sensitivity port the analysis of risk management alternatives. analysis that provide insights and improve transparency. Decision analysis offers the potential to make many of The decision tree would model the terrorist’s decision to use the improvements we have discussed. Decision analysis is biological agents to achieve his or her strategic objectives closely related to probabilistic risk analysis (Paté-Cornell by maximizing consequences to the United States. All of and Dillon, 2006). Single objective decision analysis with the terrorist decisions would be modeled as decision nodes. 

APPENDIX D 87 Analysis Model Terrorist Combining Responsiveness Time Period U.S. Decisions Uncertain Events Consequences Transparency Decisions Consequences (Run-Time) Transparent, Analyzed indi- Real-time simple models Time until first Scenarios Scenarios Not modeled Mortality vidually and not (Minutes) tailored to avail- attack combined able data Transparent us- Fixed time ing metamodels period with Probability Probability Deterministic Converted to Hours developed for best Morbidity potential for distributions distributions (parameter) dollars available national multiple attacks models Black box with Decision Decision models that are Multiple attacks Combined with made to made to Probability Days mixture of best in a specified Economic multiattribute maximize some maximize some distribution available and un- time period value function objective(s) objective(s) validated models Black box with Multiple attacks Probability Combined with unvalidated, un- Weeks in an unspeci- Game theory models distributions on Psychological multiattribute verified, and unac- fied time period probabilities utility function credited models Distributed mod- eling using best Months Not applicable Attacker-defender models Not applicable Environmental Not applicable available national models FIGURE D.2  BTRA modeling using decision analysis. This figure provides an alternative generation table developed in Figure D.1. However, instead of showing the 2006 BTRA modeling alternative, the dark gray shading highlights a decision analysis method for BTRA. The light gray shading identifies possible variations to the proposed decision analysis methodology. For example, instead of combining the consequences using a multiattribute value model, the consequences could be analyzed individually and not combined or be converted to dollars. Since they are uncertain to the terrorists, U.S. decisions (e.g., subject matter experts. However, in order to use as much as interdiction) and uncertain events (e.g., detection) would be possible of the existing 2006 BTRA event tree method, we modeled using probability distributions. Any of the conse- directly converted the event tree to a decision tree. Using quences that have credible models could be used. Decision a format similar to Figure 3.4 in Chapter 3 of this report, trees can be used to find the terrorist strategy (a sequential Figure D.3 lists one possible set of assumptions that could set of decisions) that maximizes the terrorist objectives by be used to convert the DHS event tree to the bioterrorist deci- averaging out and rolling back the decision tree. The decision sion tree. The figure adds new node numbers, type of node, tree can be solved multiple times for each single objective or rationale, average branches, and probability distributions to can be solved once with combined consequences (Parnell, be assessed. The phases are the same but are not included 2007). There are at least three ways of combining the con- due to space limitations on the page. sequences: converting each consequence to dollars, using a Several assumptions were made in Figure D.3. First, the multiple attribute value model to normalize and weight the old nodes numbers 1 (frequency of attack) and 16 (potential consequences, or using a multiple attribute utility model to for multiple attacks) were deleted for the reasons discussed normalize and weight the consequences. Each of the tech- above. Second, we converted all terrorism decisions to niques has different assumptions and data requirements. All decision nodes. That left six chance nodes: four interdic- have been used on major national studies. tion nodes, one detection node, and one consequence node. Each of these would be uncertain to the bioterrorist. Third, AN ILLUSTRATIVE BIOTERRORIST DECISION MODEL USING DECISION TREES   While agent selection is an obvious decision, some of the later decisions could be modeled as uncertain nodes early in the terrorist planning cycle. The 18 node event tree (with consequences) could be The actual nodes that would be decision or chance nodes would depend on simplified especially if credible data are not available from the knowledge of subject matter experts. 

88 DEPARTMENT OF HOMELAND SECURITY BIOTERRORISM RISK ASSESSMENT Old New Depends Maximize Probability Type of Max Average Paths Stage Stage Rationale Decision/Event on Paths Distributions to Assess Node Branches Branches (cumulative) No. No. Events (cumulative) (additive) All probabilities will Not Frequency of Initiation by Ter- 1 Deleted change after first Applicable rorist Group bioattack. Terrorists will consider 3 1 Decision the bioagents they can Bioagent Selection 28 28 28 28 0 obtain. Target will be selected to 2 2 Decision Target Selection 1 8 3 84 224 0 maximize consequences. Mode of Dissemination (also Mode will be selected to 4 3 Decision determines wet or dry disper- 1, 2 9 3 252 2,016 0 maximize consequences. sal form) Mode will be selected to 5 4 Decision Mode of Agent Acquisition 1 4 4 1,008 8,064 0 maximize consequences. Can be changed by U.S. Interdiction during 6 5 Chance 1, 4 2 2 2,016 16,128 112 actions. Acquisition Terrorist selects Location of Production and 7 6 Decision 1 2 2 4,032 32,256 0 location. Processing 8 7 Decision Depends on agent. Mode of Agent Production 1 3 3 12,096 96,768 0 Preprocessing and 9 8 Decision 1, 2, 3, 7 3 3 36,288 290,304 0 Concentration 10 9 Decision Drying and Processing 1, 2, 3 3 3 108,864 870,912 0 11 10 Decision Additives 1, 2, 3 2 2 217,728 1,741,824 0 Can be changed by U.S. Interdiction During ­ 12 11 Chance 6 2 2 435,456 3,483,648 56 actions. Production and Processing Mode of Transport and 13 12 Decision Terrorist decision. 1, 2, 3 3 3 1,306,368 10,450,944 0 Storage Interdiction During Transport 14 13 Chance Depends on U.S. actions. 6 2 2 2,612,736 20,901,888 56 and Storage 15 14 Chance Depends on U.S. actions. Interdiction During Attack 2 2 5,225,472 41,803,776 1 Not Terrorist can always do 16 Potential for Multiple Attacks 1 0 Applicable multiple attacks. Can be changed by U.S. 17 15 Chance Event Detection 1, 2, 3 3 3 15,676,416 125,411,328 252 actions. FIGURE D.3  This figure describes one possible set of assumptions that would generate a decision tree that could be solved for a bioterror- ist to maximize the consequences of damage to the United States. The figure uses the format of Figure 3.4 in Chapter 3 of this report and adds new node numbers, type of node, rationale, average branches, and probability distributions to be assessed. All terrorist decisions are converted to decision nodes. we added the consequence model to the decision tree as THE BIOTERRORIST DECISION MODEL CAN the end node. In decision analysis software, this would be PROVIDE RISK ASSESSMENT RESULTS implemented using an equation in the end node that uses AND SENSITIVITY ANALYSIS scenario parameters common to all agents and parameters The decision analysis model that we have described (agent decision and chance node outcomes) that depend on would identify the terrorist’s best strategy to maximize the path through the decision tree. If the consequences are the consequences of an attack. Senior decision makers and not combined, a decision tree would be created for each stakeholders would be provided a one to n list of the agents consequence using a different consequence model. that have the potential to create the most harm to the United 

APPENDIX D 89 States. Since decision analysis also calculates the cumula- solved using commercially available software using com- tive consequence distribution for each strategy, absolute risk plete enumeration or Monte Carlo simulation. Third, the new could easily be displayed for each agent. challenge is how to develop consequence models that use the Decision analysis models are transparent. Commercial decision parameters in the decision tree that will allow for decision analysis tools provide a range of powerful sensitiv- rapid evaluation of the decision tree for each path. Fourth, ity analysis tools (Clemen, 1996) to increase understanding further opportunities exist to simplify the decision tree. For and improve credibility. The model can be quickly resolved example, if a decision does not impact the consequences, it if any stakeholder provides an alternative set of data assump- can be removed from the decision tree. tions. Sensitivity analysis bar charts (Tornado diagrams) can be used to show the most significant data assumptions. Value THE BIOTERRORIST DECISION MODEL of information calculations can be performed to find out what EFFECTIVELY ADDRESSES THE FUNDAMENTAL uncertainties have the most impact on the agent risk. CONCERNS OF THE BTRA OF 2006 In the introduction we listed the most fundamental con- THE BIOTERRORIST DECISION MODEL ALSO cerns with the 2006 BTRA methodology: not considering SUPPORTS RISK MANAGEMENT DECISION MAKING intelligent adversary decision making, huge data demands, So far we have focused on the use of decision analysis as more complexity than the available data support, lack of a modeling framework to support bioterrorism risk assess- transparency for decision makers/stakeholders (see Chapter ments. The Bioterrorist Decision Model would provide the 3), and lack of a clear linkage to DHS risk management de- baseline risk for the bioagents analyzed. Since the model can cision making. The Bioterrorist Decision Model effectively be run quickly, it could be a very useful tool to support DHS addresses each of these concerns. risk management decision making. The Bioterrorist Decision Model solves the problem of The bioterrorism risk is impacted by the U.S. ability to modeling an intelligent adversary by selecting the bio­agents reduce the threat (prevent an attack or interdict an attack in that will maximize the objectives of the terrorists. The progress), reduce the nation’s vulnerabilities, and mitigate model greatly reduces the huge data demands by convert- the consequences given that an attack has occurred. Govern- ing terrorist decisions to decision nodes, deleting the two ment agencies, including the intelligence community, the most problematic nodes—frequency of attack and multiple Department of Homeland Security, and the Department of attacks—and not using probability distributions for each arc Health and Human Services, expend significant resources on each node. Finally, the model improves transparency by each year to increase security against attacks on our nation, using commercially available software with built-in sensitiv- including bioterrorist attacks. In the Bioterrorist Decision ity analysis capabilities. Model, U.S. capabilities are reflected in the probabilities assigned to the uncertain nodes (the interdiction, detection, REFERENCES and consequence nodes). To assess the risk reduction of risk management alteratives we can modify the model to change Clemen, R. 1996. Making Hard Decisions, 2nd edition. Belmont, Calif.: Duxbury Press. the probabilities for each risk management alternative or Dillon-Merrill, R.L., G.S. Parnell, and D.L. Buckshaw. 2007. “Logic Trees: set of alternatives. Due to the complexities of risk assess- Fault, Success, Attack, Event, Probability, and Decision Trees.” In John ment mentioned in Chapter 2 of this report, the results may G. Voeller (ed.), Wiley Handbook of Science and Technology for Home- be initially non-intuitive. For example, a large reduction in land Security. Hoboken, N.J.: Wiley and Sons. Forthcoming. the consequences of the highest-risk bioagent may not have Keeney, R.L. 1992. Value-Focused Thinking: A Path to Creative Decision- making. Cambridge, Mass.: Harvard University Press. a large reduction in overall risk since the second-highest- Keeney, R.L., and H. Raiffa. 1976. Decision Making with Multiple Objec- agent consequences might not be affected. In some cases, we tives Preferences and Value Tradeoffs. New York: Wiley. would have to consider sets of alternatives since, in general, Kirkwood, C.W. 1997. Strategic Decision Making: Multiobjective Decision the risk reduction would not be additive. Some risk manage- Analysis with Spreadsheets. Belmont, Calif.: Duxbury Press. ment alternatives may be synergistic (impact greater than the Maxwell, D.T. 2006. “Improving Hard Decisions.” OR/MS Today, pp. 51- 61. [Biannual survey of decision analysis software] sum of their individual benefits) or complementary (impact Parnell, G.S. 2007. “Multi-objective Decision Analysis.” In John G. Voeller less than the sum of their individual benefits). (ed.), Wiley Handbook of Science and Technology for Homeland Secu- rity. Hoboken, N.J.: Wiley & Sons. Forthcoming. Parnell, G.S., P.J. Driscoll, and D.L. Henderson (eds.). 2008. Decision Mak- INSIGHTS FROM THE BIOTERRORIST ing for Systems Engineering and Management. Wiley Series in Systems DECISION MODEL APPROACH Engineering, Andrew P. Sage (ed.). Hoboken, N.J.: Wiley and Sons. Paté-Cornell, E.E., and R.L. Dillon. 2006. “The Respective Roles of There are several important insights from the analysis Risk and Decision Analysis in Decision Support.” Decision Analysis presented in this appendix. First, converting the event tree to 3(4):220-232. a decision tree greatly simplifies the probability assessment Raiffa, H. 1968. Decision Analysis: Introductory Lectures on Choices Under tasks. Second, the decision tree should allow the tree to be Uncertainty. Boston, Mass.: Addison-Wesley.