|
||||||||||||||||
|
||||||||||||||||
The following HTML text is provided to enhance online readability. Many aspects of typography translate only awkwardly to HTML. Please use the page image as the authoritative form to ensure accuracy. Appendix D
|
||||||||||||||||
 |
 |
Page 85 |
 |
 |
 |
|
| ||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 85
Department of Homeland Security Bioterrorism Risk Assessment: A Call for Change
Appendix D
Bioterrorism Risk Analysis with Decision Trees
Gregory S. Parnell, Ph.D.
Professor, Department of Systems Engineering
United States Military Academy, West Point, New York
INTRODUCTION
The foundational risk analysis method used by the Department of Homeland Security (DHS) Biological Threat Risk Assessment (BTRA) methodology is event trees. Event trees are a proven probabilistic risk analysis technique that has been effectively used for risk analysis of natural and man-made hazards (Dillon-Merrill, Parnell, and Buckshaw, 2007). The body of this report has shown weaknesses in the use of event trees to model terrorist actions since event trees do not model the actions of an intelligent adversary.
To address these concerns, we convert the DHS bioterrorist event tree to a bioterrorist decision tree by changing terrorist decisions to decision nodes, removing two nodes that are problematic and unnecessary, dramatically reducing the complexity by assessing probabilities for each arc for each event instead of probability distributions for each arc for each event. In addition, we describe several alternatives for consequence modeling including separate and aggregated consequences.
MANY BTRA MODELING ALTERNATIVES EXIST
Several risk analysis modeling decisions must be made to provide effective and efficient risk analyses that support national homeland security decision-makers. Figure D.1 is a strategy generation table (Parnell, Driscoll, and Henderson, 2008) used to identify possible modeling decisions. The column titles of Figure D.1 identify some of the most important modeling decisions. The analysis responsiveness (model run time) determines the flexibility of the model and the usefulness to support risk assessment and risk management decision making. The model’s transparency increases the understanding and credibility of the model to stakeholders and decision makers. The assumed time period significantly impacts the data collection. The longer the time period, the more challenging it will be to provide credible data assessments. The next three columns (terrorist decisions, U.S. decisions, and uncertain events) are the decisions and events that must be modeled. The types of consequences are major modeling decisions since models will need to be developed for each type of consequence. Finally, the consequences can be modeled individually or combined. Combining enables an integrated assessment but takes more modeling and analysis to credibly combine the consequences.
The columns below the modeling decisions identify several possible techniques for each modeling decision. For example, analysis responsiveness can be real-time, hours, days, weeks, or months. Years are possible but probably not very useful. Using the strategy generation table, we can shade one (or more) box(es) in each column to describe or develop a BTRA modeling alternative. Figure D.1 describes the BTRA of 2006 and Figure D.2 describes the Bioterrorist Decision Model developed in this appendix.
The shading in Figure D.1 shows the committee’s understanding of the 2006 BTRA modeling. Battelle developed its own software instead of usually commercially available software to perform the event tree analysis. Due to the complexity, the BTRA model runs in days and requires special software and specially trained analysts to perform the analysis. Some sensitivity analysis capability has been developed and performed. The BTRA model is not transparent. The model is very complex and uses a mixture of best available existing models and new, unvalidated models. The first event in the BTRA event tree is the frequency of attacks. This approach requires specification of a time period and the prediction of the number of attacks with each agent. BTRA event tree models terrorist decisions, U.S. decisions, and uncertain events as probabilities. The methodology greatly increases its complexity and data requirements by assessing probability distributions on each branch of the event tree. The primary consequence modeling was on mortality but some modeling of morbidity and economics was done. The consequences were analyzed individually and not combined.
OCR for page 86
Department of Homeland Security Bioterrorism Risk Assessment: A Call for Change
FIGURE D.1 BTRA modeling alternatives. This figure provides a bioterrorism risk assessment modeling alternative generation table (Parnell, Driscoll, and Henderson, 2008) to help identify the BTRA modeling alternatives available to DHS. The column headings are the modeling decisions that must be made by DHS. The column cells identify the modeling techniques we considered for each modeling decision. The gray shading depicts the committee’s understanding of 2006 BTRA methodology.
USING DECISION ANALYSIS TO ANALYZE THE TERRORIST’S ATTACK DECISION
Based on the committee’s assessment, several improvements are needed. First and foremost, the methodology must consider the terrorist as an intelligent adversary that will select the best attack strategy to maximize their strategic objectives. Second, the methodology must be transparent. A key goal should be the use of commercially available software that has built-in sensitivity analysis features to improve understanding and transparency. The method should eliminate unnecessary complexity and demands for data that will have no meaning if one bioterrorism attack is made on the United States, e.g., the attack frequency for each agent. Finally, the methodology should be easily modified to support the analysis of risk management alternatives.
Decision analysis offers the potential to make many of the improvements we have discussed. Decision analysis is closely related to probabilistic risk analysis (Paté-Cornell and Dillon, 2006). Single objective decision analysis with decision trees has been used since 1968 (Raiffa, 1968; Clemen, 1996). Multiple objective decision analysis has been used since 1976 (Keeney and Raiffa, 1976; Kirkwood, 1997). Maxwell (2006) summarizes the large selection of commercially available decision and risk analysis software.
Figure D.2 uses the format of Figure D.1 and shows the modeling techniques that would be used in a decision analysis method. The darker shaded cells define one potential decision analysis method used to maximize the achievement of terrorist objectives. The lighter shaded cells describe alternative decision analysis methods. The goal would be to use commercially available tools and keep the models small enough to have reasonable run times. Using commercially available software helps make the models transparent and allows the use of standard decision analysis and sensitivity analysis that provide insights and improve transparency. The decision tree would model the terrorist’s decision to use biological agents to achieve his or her strategic objectives by maximizing consequences to the United States. All of the terrorist decisions would be modeled as decision nodes.
OCR for page 87
Department of Homeland Security Bioterrorism Risk Assessment: A Call for Change
FIGURE D.2 BTRA modeling using decision analysis. This figure provides an alternative generation table developed in Figure D.1. However, instead of showing the 2006 BTRA modeling alternative, the dark gray shading highlights a decision analysis method for BTRA. The light gray shading identifies possible variations to the proposed decision analysis methodology. For example, instead of combining the consequences using a multiattribute value model, the consequences could be analyzed individually and not combined or be converted to dollars.
Since they are uncertain to the terrorists, U.S. decisions (e.g., interdiction) and uncertain events (e.g., detection) would be modeled using probability distributions. Any of the consequences that have credible models could be used. Decision trees can be used to find the terrorist strategy (a sequential set of decisions) that maximizes the terrorist objectives by averaging out and rolling back the decision tree. The decision tree can be solved multiple times for each single objective or can be solved once with combined consequences (Parnell, 2007). There are at least three ways of combining the consequences: converting each consequence to dollars, using a multiple attribute value model to normalize and weight the consequences, or using a multiple attribute utility model to normalize and weight the consequences. Each of the techniques has different assumptions and data requirements. All have been used on major national studies.
AN ILLUSTRATIVE BIOTERRORIST DECISION MODEL USING DECISION TREES
The 18 node event tree (with consequences) could be simplified especially if credible data are not available from subject matter experts. However, in order to use as much as possible of the existing 2006 BTRA event tree method, we directly converted the event tree to a decision tree. Using a format similar to Figure 3.4 in Chapter 3 of this report, Figure D.3 lists one possible set of assumptions that could be used to convert the DHS event tree to the bioterrorist decision tree. The figure adds new node numbers, type of node, rationale, average branches, and probability distributions to be assessed. The phases are the same but are not included due to space limitations on the page.
Several assumptions were made in Figure D.3. First, the old nodes numbers 1 (frequency of attack) and 16 (potential for multiple attacks) were deleted for the reasons discussed above. Second, we converted all terrorism decisions to decision nodes.1 That left six chance nodes: four interdiction nodes, one detection node, and one consequence node. Each of these would be uncertain to the bioterrorist. Third,
1
While agent selection is an obvious decision, some of the later decisions could be modeled as uncertain nodes early in the terrorist planning cycle. The actual nodes that would be decision or chance nodes would depend on the knowledge of subject matter experts.
OCR for page 88
Department of Homeland Security Bioterrorism Risk Assessment: A Call for Change
FIGURE D.3 This figure describes one possible set of assumptions that would generate a decision tree that could be solved for a bioterrorist to maximize the consequences of damage to the United States. The figure uses the format of Figure 3.4 in Chapter 3 of this report and adds new node numbers, type of node, rationale, average branches, and probability distributions to be assessed. All terrorist decisions are converted to decision nodes.
we added the consequence model to the decision tree as the end node. In decision analysis software, this would be implemented using an equation in the end node that uses scenario parameters common to all agents and parameters (agent decision and chance node outcomes) that depend on the path through the decision tree. If the consequences are not combined, a decision tree would be created for each consequence using a different consequence model.
THE BIOTERRORIST DECISION MODEL CAN PROVIDE RISK ASSESSMENT RESULTS AND SENSITIVITY ANALYSIS
The decision analysis model that we have described would identify the terrorist’s best strategy to maximize the consequences of an attack. Senior decision makers and stakeholders would be provided a one to n list of the agents that have the potential to create the most harm to the United
OCR for page 89
Department of Homeland Security Bioterrorism Risk Assessment: A Call for Change
States. Since decision analysis also calculates the cumulative consequence distribution for each strategy, absolute risk could easily be displayed for each agent.
Decision analysis models are transparent. Commercial decision analysis tools provide a range of powerful sensitivity analysis tools (Clemen, 1996) to increase understanding and improve credibility. The model can be quickly resolved if any stakeholder provides an alternative set of data assumptions. Sensitivity analysis bar charts (Tornado diagrams) can be used to show the most significant data assumptions. Value of information calculations can be performed to find out what uncertainties have the most impact on the agent risk.
THE BIOTERRORIST DECISION MODEL ALSO SUPPORTS RISK MANAGEMENT DECISION MAKING
So far we have focused on the use of decision analysis as a modeling framework to support bioterrorism risk assessments. The Bioterrorist Decision Model would provide the baseline risk for the bioagents analyzed. Since the model can be run quickly, it could be a very useful tool to support DHS risk management decision making.
The bioterrorism risk is impacted by the U.S. ability to reduce the threat (prevent an attack or interdict an attack in progress), reduce the nation’s vulnerabilities, and mitigate the consequences given that an attack has occurred. Government agencies, including the intelligence community, the Department of Homeland Security, and the Department of Health and Human Services, expend significant resources each year to increase security against attacks on our nation, including bioterrorist attacks. In the Bioterrorist Decision Model, U.S. capabilities are reflected in the probabilities assigned to the uncertain nodes (the interdiction, detection, and consequence nodes). To assess the risk reduction of risk management alteratives we can modify the model to change the probabilities for each risk management alternative or set of alternatives. Due to the complexities of risk assessment mentioned in Chapter 2 of this report, the results may be initially non-intuitive. For example, a large reduction in the consequences of the highest-risk bioagent may not have a large reduction in overall risk since the second-highest-agent consequences might not be affected. In some cases, we would have to consider sets of alternatives since, in general, the risk reduction would not be additive. Some risk management alternatives may be synergistic (impact greater than the sum of their individual benefits) or complementary (impact less than the sum of their individual benefits).
INSIGHTS FROM THE BIOTERRORIST DECISION MODEL APPROACH
There are several important insights from the analysis presented in this appendix. First, converting the event tree to a decision tree greatly simplifies the probability assessment tasks. Second, the decision tree should allow the tree to be solved using commercially available software using complete enumeration or Monte Carlo simulation. Third, the new challenge is how to develop consequence models that use the decision parameters in the decision tree that will allow for rapid evaluation of the decision tree for each path. Fourth, further opportunities exist to simplify the decision tree. For example, if a decision does not impact the consequences, it can be removed from the decision tree.
THE BIOTERRORIST DECISION MODEL EFFECTIVELY ADDRESSES THE FUNDAMENTAL CONCERNS OF THE BTRA OF 2006
In the introduction we listed the most fundamental concerns with the 2006 BTRA methodology: not considering intelligent adversary decision making, huge data demands, more complexity than the available data support, lack of transparency for decision makers/stakeholders (see Chapter 3), and lack of a clear linkage to DHS risk management decision making. The Bioterrorist Decision Model effectively addresses each of these concerns.
The Bioterrorist Decision Model solves the problem of modeling an intelligent adversary by selecting the bioagents that will maximize the objectives of the terrorists. The model greatly reduces the huge data demands by converting terrorist decisions to decision nodes, deleting the two most problematic nodes—frequency of attack and multiple attacks—and not using probability distributions for each arc on each node. Finally, the model improves transparency by using commercially available software with built-in sensitivity analysis capabilities.
REFERENCES
Clemen, R. 1996. Making Hard Decisions, 2nd edition. Belmont, Calif.: Duxbury Press.
Dillon-Merrill, R.L., G.S. Parnell, and D.L. Buckshaw. 2007. “Logic Trees: Fault, Success, Attack, Event, Probability, and Decision Trees.” In John G. Voeller (ed.), Wiley Handbook of Science and Technology for Homeland Security. Hoboken, N.J.: Wiley and Sons. Forthcoming.
Keeney, R.L. 1992. Value-Focused Thinking: A Path to Creative Decisionmaking. Cambridge, Mass.: Harvard University Press.
Keeney, R.L., and H. Raiffa. 1976. Decision Making with Multiple Objectives Preferences and Value Tradeoffs. New York: Wiley.
Kirkwood, C.W. 1997. Strategic Decision Making: Multiobjective Decision Analysis with Spreadsheets. Belmont, Calif.: Duxbury Press.
Maxwell, D.T. 2006. “Improving Hard Decisions.” OR/MS Today, pp. 5161. [Biannual survey of decision analysis software]
Parnell, G.S. 2007. “Multi-objective Decision Analysis.” In John G. Voeller (ed.), Wiley Handbook of Science and Technology for Homeland Security. Hoboken, N.J.: Wiley & Sons. Forthcoming.
Parnell, G.S., P.J. Driscoll, and D.L. Henderson (eds.). 2008. Decision Making for Systems Engineering and Management. Wiley Series in Systems Engineering, Andrew P. Sage (ed.). Hoboken, N.J.: Wiley and Sons.
Paté-Cornell, E.E., and R.L. Dillon. 2006. “The Respective Roles of Risk and Decision Analysis in Decision Support.” Decision Analysis 3(4):220-232.
Raiffa, H. 1968. Decision Analysis: Introductory Lectures on Choices Under Uncertainty. Boston, Mass.: Addison-Wesley.