**Suggested Citation:**"Appendix E: Optimizing Department of Homeland Security Defense Investments: Applying Defender-Attacker (-Defender) Optimization to Terror Risk Assessment and Mitigation." National Research Council. 2008.

*Department of Homeland Security Bioterrorism Risk Assessment: A Call for Change*. Washington, DC: The National Academies Press. doi: 10.17226/12206.

**Suggested Citation:**"Appendix E: Optimizing Department of Homeland Security Defense Investments: Applying Defender-Attacker (-Defender) Optimization to Terror Risk Assessment and Mitigation." National Research Council. 2008.

*Department of Homeland Security Bioterrorism Risk Assessment: A Call for Change*. Washington, DC: The National Academies Press. doi: 10.17226/12206.

**Suggested Citation:**"Appendix E: Optimizing Department of Homeland Security Defense Investments: Applying Defender-Attacker (-Defender) Optimization to Terror Risk Assessment and Mitigation." National Research Council. 2008.

*Department of Homeland Security Bioterrorism Risk Assessment: A Call for Change*. Washington, DC: The National Academies Press. doi: 10.17226/12206.

**Suggested Citation:**"Appendix E: Optimizing Department of Homeland Security Defense Investments: Applying Defender-Attacker (-Defender) Optimization to Terror Risk Assessment and Mitigation." National Research Council. 2008.

*Department of Homeland Security Bioterrorism Risk Assessment: A Call for Change*. Washington, DC: The National Academies Press. doi: 10.17226/12206.

**Suggested Citation:**"Appendix E: Optimizing Department of Homeland Security Defense Investments: Applying Defender-Attacker (-Defender) Optimization to Terror Risk Assessment and Mitigation." National Research Council. 2008.

*Department of Homeland Security Bioterrorism Risk Assessment: A Call for Change*. Washington, DC: The National Academies Press. doi: 10.17226/12206.

**Suggested Citation:**"Appendix E: Optimizing Department of Homeland Security Defense Investments: Applying Defender-Attacker (-Defender) Optimization to Terror Risk Assessment and Mitigation." National Research Council. 2008.

*Department of Homeland Security Bioterrorism Risk Assessment: A Call for Change*. Washington, DC: The National Academies Press. doi: 10.17226/12206.

**Suggested Citation:**"Appendix E: Optimizing Department of Homeland Security Defense Investments: Applying Defender-Attacker (-Defender) Optimization to Terror Risk Assessment and Mitigation." National Research Council. 2008.

*Department of Homeland Security Bioterrorism Risk Assessment: A Call for Change*. Washington, DC: The National Academies Press. doi: 10.17226/12206.

**Suggested Citation:**"Appendix E: Optimizing Department of Homeland Security Defense Investments: Applying Defender-Attacker (-Defender) Optimization to Terror Risk Assessment and Mitigation." National Research Council. 2008.

*Department of Homeland Security Bioterrorism Risk Assessment: A Call for Change*. Washington, DC: The National Academies Press. doi: 10.17226/12206.

**Suggested Citation:**"Appendix E: Optimizing Department of Homeland Security Defense Investments: Applying Defender-Attacker (-Defender) Optimization to Terror Risk Assessment and Mitigation." National Research Council. 2008.

*Department of Homeland Security Bioterrorism Risk Assessment: A Call for Change*. Washington, DC: The National Academies Press. doi: 10.17226/12206.

**Suggested Citation:**"Appendix E: Optimizing Department of Homeland Security Defense Investments: Applying Defender-Attacker (-Defender) Optimization to Terror Risk Assessment and Mitigation." National Research Council. 2008.

*Department of Homeland Security Bioterrorism Risk Assessment: A Call for Change*. Washington, DC: The National Academies Press. doi: 10.17226/12206.

**Suggested Citation:**"Appendix E: Optimizing Department of Homeland Security Defense Investments: Applying Defender-Attacker (-Defender) Optimization to Terror Risk Assessment and Mitigation." National Research Council. 2008.

*Department of Homeland Security Bioterrorism Risk Assessment: A Call for Change*. Washington, DC: The National Academies Press. doi: 10.17226/12206.

**Suggested Citation:**"Appendix E: Optimizing Department of Homeland Security Defense Investments: Applying Defender-Attacker (-Defender) Optimization to Terror Risk Assessment and Mitigation." National Research Council. 2008.

*Department of Homeland Security Bioterrorism Risk Assessment: A Call for Change*. Washington, DC: The National Academies Press. doi: 10.17226/12206.

**Suggested Citation:**"Appendix E: Optimizing Department of Homeland Security Defense Investments: Applying Defender-Attacker (-Defender) Optimization to Terror Risk Assessment and Mitigation." National Research Council. 2008.

*Department of Homeland Security Bioterrorism Risk Assessment: A Call for Change*. Washington, DC: The National Academies Press. doi: 10.17226/12206.

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Appendix E Optimizing Department of Homeland Security Defense Investments: Applying Defender-Attacker (-Defender) Optimization to Terror Risk Assessment and Mitigation Gerald G. Brown Distinguished Professor, Department of Operations Research Naval Postgraduate School, Monterey, California W. Matthew Carlyle Associate Professor, Department of Operations Research Naval Postgraduate School, Monterey, California R. Kevin Wood Professor, Department of Operations Research Naval Postgraduate School, Monterey, California The U.S. Department of Homeland Security (DHS) is a Â ssess the risk to our populace from terrorist attacks of all kinds. investing billions of dollars to protect us from terrorist The work we report here is directly motivated by just one such attacks and their expected damage (i.e., risk). We present risk assessment: pursuant to Homeland Security Presidential prescriptive optimization models to guide these invest- Directive 10 (HSPD-10) (The White House, 2004), DHS has ments. Our primary goal is to recommend investments in conducted an extensive bioterrorism risk-Âassessment exercise, a set of available defense options; each of these options referred to here as the Biological Threat Risk Assessment can reduce our vulnerability to terrorist attack, or enable (BTRA) (DHS, 2006). BTRA estimates risks of many bioterror future mitigation actions for particular types of attack. Our attack possibilities, and classifies a list of particular bioterror models prescribe investments that minimize the maximum agents as most-, intermediate-, and least-threatening. risk (i.e., expected damage) to which we are exposed. Our The BTRA risk assessment depends upon subject-matter âDefend-Attack-Mitigate risk-minimization modelâ as- experts (SMEs) advising, with perfect knowledge, the prob- sumes that terrorist attackers will observe, and react to, any ability that the âattackerâ (terrorist or terrorist group), or strategic defense investment on the scale required to protect âdefenderâ (the federal government), will choose some our entire country. We also develop a more general tri-level particular option at each stage of an 18-stage probability âDefender-Attacker-Defender risk-minimization modelâ risk assessment tree. in which (a) the defender invests strategically in interdic- We contend that representing intelligent adversarial tion and/or mitigation options (for example, by inoculating decisions with static probabilities elicited from SMEs is an health-care workers, or stockpiling a mix of emergency untenable paradigm: Not only can experts make mistakes, vaccines), (b) the attacker observes those investments and but static probabilities make no sense when the attacker can attacks as effectively as possible, and (c) the defender then observe and react, dynamically, to any earlier decisions made optimally deploys the mitigation options that his investments by the defender. have enabled. We show with simple numerical examples We also hold that the business of DHS lies not just in some of the important insights offered by such analysis. As assessing risks, but also in wisely guiding investments of a by-product of our analysis we elicit the optimal attacker our nationâs wealth to reduce these risks. These are strategic behavior that would follow our chosen defensive investment, decisions that must be made now, in a deliberative fashion. and therefore we can focus intelligence collection on telltales Here, we try to adopt the same problem context as BTRA of the most-likely and most-lethal attacks. to recoup its estimable investment in risk modeling. But, we distinguish between (a) strategic investment decisions that DHS makes that are visible to terrorists, (b) the decision INTRODUCTION a terrorist makes to attempt an attack and, finally, (c) the Since September 11, 2001, the U.S. Department of Home- after-attack mitigation efforts that prudent DHS investments land Security (DHS) has marshaled significant resources to will have enabled. 90

APPENDIX E 91 Our work applies equally well to any category of threat interceptor platforms to protect against a theater bal- that concerns DHS enough to warrant investments so signifi- listic missile attack (Brown et al., 2005b). cant they cannot be hidden from our taxpayers, and thus not from terrorists, either. Such threats cover biological, tadioac- The message here is that, with experience, we have gained tive, chemical, and conventional attacks on our infrastructure confidence that these new mathematical methods produce and citizens, as well as sealing our borders against illegal results that exhibit the right level of detail, solve the right deci- immigration, and a host of military topics. sion problems, and convey useful advice and insight to policy The modeling presented here has been motivated and makers. Such capabilities have not been available before. validated by more than one hundred worldwide infrastructure vulnerability analyses conducted since 9/11 by the military- THE MODEL, âMXMâ officer students and the faculty of the Naval Postgraduate School (Brown et al., 2005a, 2006a). Some of these stud- The Biological Threat Risk Assessment (BTRA) uses ies have been developed into complete decision-support a descriptive model. Our focus is prescriptive, rather than systems: descriptive: our models suggest prudent investment and mitigation plans for biodefense, and we strive to provide a â¢ SalmerÃ³n et al. (2004) have received DHS and Depart- realistic representation of the attack decisions made by an ment of Energy support to create the Vulnerability of intelligent adversary. Electric Grids Analyzer (VEGA), a highly detailed, As the defender, we seek to allocate a limited budget optimization-based decision-support system. VEGA among biodefense investment options to form a defense can evaluate, on a laptop computer, the vulnerability strategy that minimizes the maximum risk from the actions and optimal defense of electrical generation and dis- of a terrorist attacker. We might define risk as the expected tribution systems in the United States, where risk is number of fatalities, or as the expected 95th percentile of measured as expected unserved demand for energy fatalities, or as any other gauge that appeals. Risk is a some- during any repair-and-recovery period. what ambiguous term when used to discuss our bilateral view â¢ We have developed a decision-support system to of conflict between intelligent adversaries, so we hereafter advise policy makers regarding the interdiction of a substitute âexpected damage to the defender.â We assume proliferatorâs industrial project to produce a first batch that an intelligent adversary will attempt to inflict maximum of nuclear weapons (Brown et al., 2006b, 2007). expected damage. The following, simplified model mini- â¢ The U.S. Navy has developed a decision-support mizes a reasonable upper bound on expected damage; we system to optimally pre-position sensor and defensive discuss generalizations later. â¢ Indices d â D defense strategy, e.g., stockpile vaccines A and B, but not C a â A attack alternative, e.g., release infectious agent V m â M after-attack, mitigation activity, e.g., distribute vaccine A m â Md mitigation activities enabled by defense option d, e.g., distribute vaccine A, distribute vaccine B d â Dm defense strategies that enable mitigation activity m k â K resource types used by mitigation activities, e.g., aircraft for distributing vaccine, personnel for administering vaccine â¢ Data damaged,a expected damage if defense strategy d and attack alternative a are chosen, given no mitigation mitigated,a,m expected damage reduction of after-attack mitigation effort m, given investment strategy d and attack a (assumes additive reduction and â mitigationd ,a, m â¤ damaged ,a) m rk,d total mitigation resource of type k available if defense strategy d is chosen qk,d,m consumption of mitigation resource k provided by defense option d for mitigation activity m

92 DEPARTMENT OF HOMELAND SECURITY BIOTERRORISM RISK ASSESSMENT â¢ Decision Variables wd if defense strategy d chosen, otherwise 0 1 xa probability attacker chooses attack alternative a (0 â¤ xa â¤ 1) yd,m fraction of defense strategy d effort devoted to mitigation activity type m â¢ Formulation: MIN-MAX-MIN (MXM) (Defender-Attacker-Mitigator) z* = min max min wd xa yd ,m â damaged ,a wd xa - â mitigated ,a, m xa yd ,m (D0) d ,a d ,a ,m â wd = 1 (D1) d â xa = 1 ( A1) a â qk ,d ,m yd ,m â¤ â rk ,d wd âk â K (M1) d ,m d yd ,m â¤ wd âd â D, m â M d (M2) wd â{0,1}, x a â¥ 0, yd ,m â¥ 0 âd â D, a â A, m â M d Description most naturally modeled using integer attacker variables, our model with continuous attack (ya) variables will at least pro- The order of appearance of the operators, min, followed vide a conservative estimate of the defenderâs objective; i.e., by max, followed by min, in the objective function (D0) the attackerâs abilities to inflict damage are over-estimated represents the sequential nature of the decisions we are by our model. modeling, from the outside to the inside. The coefficient damaged,a in the objective accounts for any interdiction ef- fects that strategy d has on attack a, effects that are indepen- Discussion of MXM dent of any mitigation activities. (For example, vaccinating Figure E.1 depicts a tree showing the sequential actions emergency and health-care providers falls under the category of the defender (selecting a defense strategy), the attacker of âinterdictionâ: after an attack, no follow-up mitigation (choosing attack alternatives), and the defender (mitigating efforts apply to this vaccination.) The right-most minimiza- damage with resources put in place by the defense strategy). tion term, over yd,m, subtracts from expected damage if a (We use the generic term âtreeâ to represent the sequence of mitigating effort has been enabled by the defense plan, and if defender and attacker decisions we model. The âdecision some amount of that mitigation is applied. For simplicity of treeâ of Raiffa [1968] pits a single decision maker against exposition, we assume that mitigation results are additive and Mother Nature, while here we have two opponents try- restricted to sum to some value not exceeding total expected ing to shape an outcome governed by Mother Nature. The damage. (See the definition of mitigated,a,m.) Constraint (D1) term âgame treeâ [Kuhn, 1953] is a more appropriate term simply limits the defender to choosing one defense strategy. for our bioterror situation.) Each defense strategy has an Constraint (A1) limits the attacker to choosing a mixed at- immediate effect on the maximum damage of any attack, tack strategy, which of course admits a pure attack as well. reflected in damaged,a; it can also enable the capability to Constraints (M1) are joint resource constraints on mitigation reduce after-attack damage by as much as mitigated,a,m, if the efforts; constraints (M2) stipulate that mitigation efforts are chosen defense strategy permits a full allocation of mitiga- permitted only if the enabling defense strategy has been tion resources to mitigation action m. Given a fixed defense chosen. Constraints (M1) subsume those of type (M2), but strategy, we assume the attacker will first observe this strat- we keep these separate for later clarity. The attack variables, egy and then respond with a mixed strategy over the set of xa, and the mitigation variables, yd,m, are continuous. If the possible attacks. As we have said, this might be a relaxation attacker variables are restricted to be integer (for instance, of the original optimization problem faced by the attacker, they might be binary variables indicating whether or not and therefore grants him or her more attack capability than the terrorists decide to fully develop and deploy a particular the attacker really has in this sequential decision-making. pathogen in an attack), then the resulting analysis becomes In general we cannot tell how weak this relaxation is, but significantly more complicated than that which we present for specific cases (especially those with a moderate number here. Although dealing with bioterrorist attacks might be

APPENDIX E 93 FIGURE E.1â This tree depicts, left-to-right, a leading . defense strategy choice wd, consisting of component . defense investment options, and visible to an attacker, . followed by attack alternative choice(s) xa that (each) inflict expected damage damaged,a. Square nodes indicate defender decisions, and circle nodes indicate attacker decisions. We only illustrate a mitigation sub- xa yd,m tree (yd,m decisions) for one (wd, xa) pair. For a given wd damaged,a â mitigated,a,m defense strategy wd =1, the optimization recommends . . a mixed attack strategy for the attacker and a mixed . . . mitigation response yd,m from the defender. The de- . . fense strategy establishes all mitigation resources that . can be used after an attack. That strategy is seen by the attacker when he or she develops the attack plan. Enabled mitigation resources can reduce expected . damage through -mitigated,a,mxayd,m. (Our conserva- tive model does not allow the defender to observe . the precise type of attack, however, so the mitigation . response may not be optimal.) . of feasible attacker decisions) we Figure E-1 R01268, can use enumeration to probability distributions yields an objective function that bound the effect of this relaxation on the optimal objective measures âexpected damage.â function value. A âmixed attack strategyâ means that the optimal at- Solving MXM tacker decision includes multiple attacks and then we choose mitigation responses, and this results in some damage that Temporarily fixing w = w in MXM, we take the linear- Ë can only be estimated, and some part of that estimation can programming dual (hereafter referred to simply as âthe dualâ) involve an expectation. (For example, the damage could of the innermost minimizing linear program, using dual vari- involve an expectation taken over a probability distribution ables ak for constraints (M1), and bd,m for constraints (M2). for the time between when an attack is launched to when it This converts the inner âmax-min problemâ into a âmax-max is discovered.) Thus, integrating damage over one or more problem,â which is a simple maximization: â¢ Formulation: MAX-ATTACKER-LP (w) Ë zmax = max x, â damaged ,a wd xa - â rk ,d Î± k - â Ë wd b d ,m Ë Î± ,b d ,a k d ,mâMd s.t. â xa = 1 ( A1) a â qk ,d ,m Î± k + bd ,m â¥ â mitigated ,a,m xa âd â D, m â M d (DM1) k a Î±k â¥ 0 âk â K b d ,m â¥ 0 âd â D, m â M d Now, leaving w = w as shown in MAX-ATTACKER-LP, Ë then release w to vary as before, to achieve the following we take the dual of this linear program, using dual variables integer linear program which is essentially equivalent to â for constraint (A1) and yd,m for constraints (DM1), and MXM:

94 DEPARTMENT OF HOMELAND SECURITY BIOTERRORISM RISK ASSESSMENT â¢ Formulation: MIN-ILP (Defender-Attacker-Mitigator) zmin = min â (DILP0) â, wd , yd ,m s.t. â â¥ â damaged ,a wd - â mitigated ,a,m yd ,m âa â A ( DILP1) d d ,m â wd = 1 (D1) d â qk ,d ,m yd ,m â¤ â rk ,d wd âk â K (M1) d ,m d yd ,m â¤ wd âd â D, m â M d (M2) wd â{0, 1}, yd ,m â¥ 0 âd â D, m â M d The optimal solution to MIN-ILP prescribes among other characterizing component investment options in defense things a choice for the defense strategy, w*, to be implemented strategies. immediately by the defender, before an attack occurs. Given Damage estimates in Table E.3 include any synergies optimal incumbent solution w*, we recover the attackerâs among or interference between component investment op- optimal strategy x* by solving MAX-ATTACKER-LP(w*). tions in each defense strategy preparing for each attack. This is key. BTRA makes a point of such dependencies, and we represent these in complete, realistic detail here. A Numerical Example of MXM Table E.4 represents estimated mitigation capabilities. We provide a small numerical example to illustrate the These mitigation estimates correspond to a single, âfull- features of MXM. strengthâ mitigation effort being applied to a single attack We introduce a number of defensive investment options, alternative. If the attacker chooses a mixed attack strategy, programs that can be composed in groups into defense we may need to spread mitigation effort across multiple ac- strategies. Table E.1 displays defensive investment options tivities, reducing the expected effectiveness of each activity and costs. accordingly. In our example, the defensive investment options are de- The choice of defense strategy is limited by a total bud- noted âi01,â âi02,â and âi03.â From this set, policy makers get, which we vary over the integers from 0 to 11. We allow have determined 6 combinations that comprise the subset full employment of either mitigation effort, or any convex of admissible defense strategies whose implementation will combination of them. depend on the available budget; see Table E.2. Table E.3 dis- Because the defender is minimizing the optimal objec- plays expected damage resulting from each defense strategy tive function value of a maximization problem, the optimal and each attack alternative, i.e., the terms damaged,a. Figure E.2 illustrates the generic relationship relating investment options to the ability to reduce expected dam- age from any terrorist attack before it is carried out, and/or TABLE E.2â Defensive investment options in each mitigate damage after an attack occurs. This is a complicated potential defense strategy. function, neither convex nor concave, but our sampling Investment options of representative points can be used to represent this in i01 i02 i03 Defensive d00 strategies d01 x d02 x d03 x TABLE E.1â Defensive investment options and costs. d04 x x i costi d05 x x i01 2 i02 3 Strategy âd00â makes no investment at all. Defense strategy âd05â includes i03 5 investment options âi02â and âi03.â Logical, political, or other consider- ations preclude some of the strategies, for example, {âi01,â âi03â}. The For example, option âi03â costs 5. Total budget, logical, and perhaps po- total available budget, not yet specified, can also preclude certain strate- litical considerations will limit the combinations of these options that can gies. For instance, {âi02â and âi03â} cannot be selected if the total budget comprise admissible defense strategies is less than 8.

APPENDIX E 95 12 10 Expected Damage 8 6 4 2 0 0 5 10 15 20 25 30 Investment Option Cost FIGURE E.2â The purpose of Department of Homeland Security defensive investment options is to reduce expected damage before an attack occurs, and/or allow mitigation of expected damage after one occurs. The generic relationship illustrated here conjectures little to no effect at low investment levels, followed by increased effectiveness, and eventually leveling off with diminishing returns. The triangles represent points we might use as alternate investment options to adequately represent the entire function. R01268, Figure E-2 TABLE E.3â Expected damage resulting from each defense solution invests to reduce the expected damage, given future strategy (row) and each attack alternative (column), mitigation capability, of the most-threatening mixed attack. accounting for interdiction but not mitigation. This requires that the defender invest in a defense strategy that enables him or her to mitigate several very-damaging a01 a02 a03 attacks, and not just the worst one. Figure E.3 shows minimized maximum expected damage d00 10 10 10 d01 10 5 7 as a function of total defense budget, and Table E.5 summa- d02 6 8 7 rizes the solutions for each budget break-point. For instance, d03 6 6 6 with a budget of 3, the optimal defense plan in MXM is to d04 4 3 5 choose defense option âd02.â The terroristsâ optimal attack d05 5 5 4 is a mixed strategy, with a probability of 0.50 of choosing (This table gives the values for damaged,a for MXM. We use integral data âa02â and probability 0.50 of choosing âa03.â The result- to permit reproduction of our results.) ing expected damage, after mitigation, is 6.5. Analysis of this simple case reveals that we have optimally allocated our mitigation effort among the two worst attacks, reduc- ing the expected damage in each attack to the same value, 6.5. We can do no better than this, given our conservative TABLE E.4â (A, left; B, right) Maximum expected damage approximation. reduction from a mitigation activity enabled (prior to an attack) by a defense strategy (and applied after an attack). Generalizing Beyond Tri-level Decision Problems m = m1 a01 a02 a03 m = m2 a01 a02 a03 The DHS biological threat risk assessment (BTRA) con- d00 0 0 0 d00 0 0 0 d01 1 0 0 d01 1 0 0 sists of an 18-stage probability risk assessment tree, where d02 0 1 1 d02 0 2 0 each decision has been replaced by an a priori probability, as d03 0 0 1 d03 0 0 1 shown in Chapter 3 of this report. In the case of the each op- d04 1 1 1 d04 0 1 2 ponent, these probabilities are determined by subject-matter d05 0 1 1 d05 0 0 2 experts assessing how terrorists might make each decision, These tables specify mitigated,a,m for MXM, for each of two mitigation and how well DHS will do thwarting a bioagent attack at options (Table E.4.A, âm = m1,â and Table E.4.B, âm = m2â), for each some intermediate stage of its development. combination of defense and attack. For example, with defense option âd04â and attack âa03,â if we choose mitigation âm = m1â we reduce the damage We could instead model the BTRA as a 19-stage defender- by one unit, but if we choose mitigation âm = m2â we reduce the expected attacker-defender model, with a new stage zero describ- damage by two units (circled values). ing how DHS can invest in strategic biological defense

96 DEPARTMENT OF HOMELAND SECURITY BIOTERRORISM RISK ASSESSMENT 12 10 8 Expected Damage 6 4 2 0 0 2 4 6 8 10 Defense Budget FIGURE E.3â Expected damage as a function of defense budget. This display is for policy makers: as we devote more and more defense budget, we achieve less and less expected damage. Because the defenderâs investment options here are discrete, each improvement appears as a staircase drop as soon as sufficient budget permits some new, improved cohort of investment defense options, i.e., a new defense strategy. The law of diminishing returns is evident: expected damage reduced by each budget dollar decreases as budget increases. Policy makers can usually put their finger on the spot that appeals in an illustration such as this, perhaps based on criteria not part of the underlying modeling. The uppermost, solid line displays the expected damage when all mitigationd,a,m values are set to zero (i.e., we have no mitigation capabil- R01268, Figure E-3 ity) and only consider the expected damage from adopting a defense strategy, and then suffer the worst-case attack per expected damage in Table E.3. The dashed line illustrates the expected damage from MXM, the tri-level optimization. strategies, and each of the intermediate stages represented investment Âdecisions to minimize expected damage assuming by a set of decision variables that prescribe attacker or de- each opponent makes the optimal decision at each node of the fender behavior, and solve a multi-stage defender-attacker- corresponding tree. To fully represent the sequential nature defender(-attacker-...) model to determine optimal stage-zero of these decisions, we would require all decisions (except maybe those in the final stage) to be modeled with integer variables. However, solving such a model for just two stages TABLE E.5â For each budget just sufficient to afford a of integer decisions is difficult. new defense strategy, we show the Defender-Attacker We do not have the technology to handle three, much less solution and expected damage (i.e., for MXM with y = 0), 18, stages of alternating integer decisions. Allowing continu- the Defender-Attacker-Defender solution (for MXM) and ous decision variables in each of the stages except stage zero expected damage. (our defense decision variables) would again be a relaxation of the restrictions on the attacker, and could, in some cases, MXM with y = 0 MXM Budget w x z* wx y z* yield extremely weak bounds on our defensive capability. 0 d00 a01 10 d00 a01 â 10 We now show in the case of a two-stage model how this 2 d01 a01 10 d01 a01 m01 9 relaxation from integer to continuous variables reduces the 3 d02 a02 8 d02 a02(.50) m01(.50) 6.5 a03(.50) m02(.50) sequential decision problem to a simultaneous two-person 5 d04 a03 5 d04 a01(.50) m01(.50) 3.5 zero-sum game. a03(.50) m02(.50) Consider the bi-level, attacker-defender, max-min optimi- For example, with a budget of 3, the optimal defense strategy in MXM is zation formulation: (ALDL), where the subscript âLâ denotes âd02.â The terroristsâ optimal attack is a mixed strategy, choosing alternative a linear program (i.e., continuous decision variables, and âa02â with probability 0.50, and âa03â with probability 0.50. We anticipate responding accordingly with âm01,â the optimal response to âa01,â with the objective and constraints that are linear in those decision same probability (0.50), and similarly with âm02â with probability 0.5. The variables): resulting expected damage, after optimal mitigation in each case, is 6.5.Â

APPENDIX E 97 â â (ALDL) We observe that (ALDL) and (DL AL) are linear program- ming duals of each other, and thus (assuming both are max min x y g T x + x T Qy + cT y [ dual variables ] feasible) have the same optimal objective-function values, which is the same as the optimal objective value of (ALDL). s.t. Ax â¤b [Ï ] ( B1) Therefore, the sequence in which the decisions are made Dy â¥d [Î¼] ( B2 ) (either attacker first, followed by defender, or defender first, xâ¥0 followed by attacker) has no impact on the optimal objective- function value. yâ¥0 We have therefore proved the following: (ALDL) is a more general version of the model used by F Â ulkerson and Harding (1977) and Golden (1978) for their Theorem 1: For any attacker-defender model in the work on continuous network interdiction models. form (ALDL), we can exchange the order of deci- sions without affecting the optimal objective function Take the dual of the inner (defender, âyâ) problem in value. (ALDL): â Theorem 1 is a simple extension of von Neumannâs (ALDL) (1928) minimax theorem for polyhedral feasible regions using a proof technique similar to Ville (1938), but using max gT x +d T Î¼ x ,Î¼ the more modern technology of linear programming duals s.t. Ax â¤b [Ï ] ( B1) directly. This exchange argument, along with the observation that any two consecutive decision stages controlled by the -Q T x DT Î¼ â¤c [ y] ( D2 ) same decision maker are equivalent to a single stage (since xâ¥0 both stages are either a maximization or both are a minimi- Î¼â¥0 zation over a set of decision variables, this is equivalent to a single maximization, or minimization, over all of those This is our standard way to convert a âmax-minâ problem, variables simultaneously), can be repeated for any number of for which there is no conventional optimization method, into consecutive stages with continuous decision variables. The an equivalent âmax-maxâ problem that is nothing more than final model obtained in this manner is a simple maximization a conventional linear program. or minimization problem. Now, reverse the order of play in (ALDL) to (DLAL): Specifically, if we were to apply this to the 18-stage BTRA model (i.e., the model we would solve for any fixed, (DLAL) known defense decision in stage zero), we would aggregate adjacent attacker stages (and adjacent defender stages, if min max g T x + x T Qy + cT y [ dual variables ] y x there are any) and reduce the 18-stage BTRA tree to 8 stages. s.t. Ax â¤b [Ï ] ( B1) We would then require that all decision variables be continu- ous, and then swap adjacent defender-attacker pairs of stages Dy â¥d [Î¼ ] ( B2 ) until we obtain a model having all of the attacker decisions xâ¥0 in stage 1 and all of the defender decisions in stage 2. This yâ¥0 resulting model is equivalent to model (ALDL), above, and hence is equivalent to a simultaneous game. This variation on (ALDL) is formulated as if the defender The optimal solution would prescribe mixed strategies makes a decision first. for the attacker and defender, eliminating the sequential Take the dual of the inner, attacker, (âxâ) problem in nature of the real decisions that must be made. In general, (DLAL): the results from such an analysis might not be very accu- â rate, as every relaxation of a block of integer variables to (DL AL) continuous and the subsequent interchange and aggregation min bT Ï +cT y of adjacent stages can result in a significant relaxation of y,Ï attacker restrictions; in some models these approximations s.t. AT Ï -Qy â¥g [x] ( D1) could get significantly less informative with each additional stage exchanged in this manner. Dy â¥d [Î¼ ] ( B2 ) However, if the sequencing of two adjacent attacker- Ïâ¥0 defender stages is not a critical component of the formula- yâ¥0 tion, then the optimal solution of the relaxation might not be This formulation is equivalent to (DLAL), and is also a linear far off from that of the original model. As a simple example, program. if the attacker chooses which pathogen to load into a truck,

98 DEPARTMENT OF HOMELAND SECURITY BIOTERRORISM RISK ASSESSMENT and the defender then chooses whether or not to emplace ning industrial and military operations that precisely mimic a transportation blockades, relaxing the decision variables and bioterror-agent production program, or a defense plan. exchanging these two stages might not be as significant a re- We recommend eliciting from SMEs an explicit assess- laxation as in a situation where the attacker decides whether ment of the resources and capabilities of each opponent, and or not to release a pathogen, and the defender then chooses the way and rate at which various alternate activities would whether or not to employ his stockpile of a certain vaccine consume these. This is, in fact, the way that the BTRA that can treat the attackerâs pathogen. In the former case reports that the SMEs explained their reasoning to support the blockades will work against the truck regardless of the probability assessments. We advise using these technologi- pathogen chosen, while in the second example committing cal estimates as explicit inputs, and letting MXM determine to use the vaccine before a pathogen is released is clearly a attacker mixed-strategy probabilities and expected conse- bad idea, and allows the attacker to cause significantly more quences as outputs. This would be much more transparent damage. modeling, provide better documentation, and be less likely to be influenced by poor SME guesses about high-dimensional decisions governed by complicated resource limitations. How to Generalize BTRA to a Decision Model This also avoids the current step where SMEs convert ca- Prescribing Defense Investments pabilities assessments into just a few discrete, qualitative If we are to leverage the considerable effort that went probability classes (e.g., ânot likelyâ = 0.2, âlikelyâ = 0.5, into the development of the BTRA, we must use the data âvery likelyâ = 0.8). obtained, and elicit subject-matter-expert input, to develop The initial linear integer program and subsequent pair of a two- or three-stage sequential decision model of defensive linear programs afford us a great deal of flexibility and fidel- investments, attacks, and mitigation responses such that the ity in describing the actions of each opponent, and we can relaxation obtained by allowing continuous attacker vari- solve these at very large scale with off-the-shelf optimization ables, as in MXM, is at least a reasonable approximation. software. Also, solutions to such optimization models can be If we are successful in our new modeling effort, then analyzed to discover the âwhyâ as well as the âwhatâ of each the decisions at each stage except our new stage-zero will plan. Powerful, effective sensitivity and parametric analysis be continuous (and, more specifically, interpreted as mixed techniques are well known for these optimization models. strategies), but now the values of these mixed-strategy prob- We represent defensive investment strategy selection abilities will be prescribed by the optimization model: for simply, as we think realistic and politically palatable during the stage under control of the terrorists, these will represent this early phase of homeland security capital planning. We the worst-case mix of attack decisions the terrorists can de- anticipate that this will eventually mature to more closely vise; in the mitigation stage, under DHS control, these will resemble classic military capital planning (e.g., Brown et represent the best response to each of the attackerâs possible al., 2004). decisions in the previous stages. We present a deterministic model that minimizes the max- It is not lost on us that some of the BTRA probabilistic imum expected risk. If stochastic evaluation proves essential, risk assessment treeâs probabilities exhibit dependence on our model can be used within a simulation. Banks and An- the outcomes of some prior stages in the tree. A reformu- derson (2006) demonstrate such exogenous simulation with lation to a two- or three-stage sequential decision model a two person, zero-sum game. Tintner (1960) shows this for would necessarily require some reworking of these data. For a linear program. Our integer linear program is amenable to brute-force permutation of (potentially aggregated) stages, such simulation. we could unwind the conditional probabilities with Bayesâ theorem (just as DHS already does when it splits the single Secrecy in Planning BTRA tree into 28 independent trees, one for each bioagent, where selection of bioagent is the third terrorist stage in the If, as the defender, we strongly believe that we are able to original tree). conceal some of our defensive capability from the attacker, However, we hope to move away from subject-matter- then the transparency of model MXM is likely to be inap- expert (SME) elicitations of highly dependent probabilities propriate for determining optimal defense decisions. Instead, as follows. These dependencies are presumably due to the we find ourselves in an asymmetric conflict: the attacker and influence of prior stages on the state of the terrorist (or DHS) the defender do not agree on the objective function. This in terms of exhaustion of limited resources. MXM would more general case falls in the domain of bilevel and multi- explicitly guide strategic defensive investment in stage zero, level programming (see, for example, Candler and Townsley and subsequently offer all the explicit resource-limiting [1982], Bard and Moore [1992], and Migdalas et al. [1998]), features of a linear program for all the attacker decisions, and the associated mathematical models are more difficult to and in parallel all the defenderâs mitigation decisions that solve than those we have presented here. consume the mitigation resources provided by stage zero. In an extreme case, for example, we might believe that Linear programming has long been widely applied to plan- even though the attacker can observe our strategic defensive

APPENDIX E 99 investments, he or she is completely unaware of our mitiga- the mitigation efforts do not produce drastically different tion capabilities. We could then assume that the attacker results from each other relative to the defense-and-attack will make decisions based only upon the damaged,a values, combination they are applied to), it makes no sense to take whereas, given that we are perfectly aware of our mitigation extreme measures to conceal our mitigation capability. In capabilities, we will make our investment decisions based fact, we should broadcast it widely, in hopes that it will deter on damaged,aâmitigated,a,m values. This would be formu- attacker efforts. lated as a tri-level integer programming model, the most However, in the case where our mitigation capabilities are general versions of which are difficult to solve. However, much more (or less) effective for one (or a small number of) a straightforward heuristic for solving our problem would attacks than for the rest, and this fact fundamentally changes solve an attacker-defender version of the problem with no the worst-case attack decision for each of our defense op- mitigation options (i.e., by fixing yd,m = 0), and then choose tions, then we conjecture that we should conceal this capabil- the optimal mitigation decision for whatever defense and ity to maintain our advantage (or conceal our weakness) for attack decisions are made. that attack, and hopefully âshapeâ the attackerâs decisions Clearly this can lead to a suboptimal defense investment, toward the attacks that we are more capable of handling. especially when there are defense options that do not directly However, every situation is different, and it is extremely reduce expected damage (i.e., damaged,a might be high hard to predict what the effect any given âsecrecy policyâ for those defenses) but that enable mitigation efforts that will have on the optimal outcome, much less on the actual are significantly more effective than those available under attacker behavior. More research in this area is required. other defensive investments. We can use the stockpiling of a vaccine as an example; creating the stockpile will not Solving MXM at Very Large Scale with Decomposition reduce the damage of any attack, but the mitigation activity of distributing the vaccine and inoculating the susceptible Although we have solved large attacker-defender models population can be extremely effective. In this case, the of the same form as MXM (Brown et al., 2005b), if instances optimal defense and the resulting worst-case attack damage of MXM become too large to solve using commercial off- can differ significantly from the myopic defense. There are the-shelf integer linear programming software, we can use other, more effective heuristics for multilevel optimization (and have used) a version of Benders decomposition (e.g., in the literature, the breadth of which is beyond the scope Bazaraa, Jarvis, and Sherali, 1990, pp. 366-367) to solve of this appendix. MIN-ILP, with integer stage-zero investment decisions and In the case where the âsecretâ objective values maintain continuous mitigation decisions in the master problem, and the same relative ranking between each pair of feasible the resulting attacker LP subproblems. Israeli and Wood defense and attack combinations as discloseded by the (2002) explicitly develop such a decomposition for the case âpublicâ objective function, then the optimal defense and of shortest-path network interdiction problems. resulting worst-case attack do not change. For example, if We modify MIN-ILP, replacing equations (DILP1) with the mitigation effects mitigated,a,m are always a fixed percent- a set of constraints (DILP-CUTS), and calling the resulting age of damaged,a, then the optimal defensive investments, model MIN-ILP-DECOMP({Ë N}), where {Ë N} represents x x and the corresponding worst-case attack, will be the same, the set of all attacker plans from completed decomposition and the overall expected damage will be reduced by that iterations: {Ë N} â¡ {Ë n, n = 1,â¦, N}. x x fixed percentage. In this case (and similar cases, in which â â¥ â damaged ,a wd xa - Ën â Ën mitigated ,a, m x a yd ,m n = 1,..., N (DILP-CUTS) . d ,a d ,a ,m The complete decomposition algorithm is as follows: â¢ Algorithm DHS-MXM-DECOMP Input: Data for bio-terror defense problem, optimality tolerance e â¥ 0; Output: Îµ-optimal (MXM) defender plan (w*,y*); Initialize best upper bound zUB â â, best lower bound zUB â 0, define the incumbent, null (MXM) defender 1)â plan (w* â w1 â¡ "d00",y* â y1 â 0) as the best found so far, and set iteration counter N â 1; Ë Subproblem: Using w = w N, solve the linear program subproblem MAX-ATTACKER-LP (Ë to determine the 2)â Ë Ë w) optimal attack plan x N; the bound on the associated total expected target damage is zmax(Ë N); Ë x 3)â (zUB > zmax (Ë N) ) set zUB â zmax (Ë N) and record improved incumbent MXM defender plan If x x (w*, y*) â (Ë N, y N); w Ë

100 DEPARTMENT OF HOMELAND SECURITY BIOTERRORISM RISK ASSESSMENT 4)â (zUB - zLB â¤ e) go to End; If Given attack plans {Ë N}, attempt to solve master problem MIN-ILP-DECOMP({Ë N}) to determine an 5)â x x Â ptimal defender plan (Ë N+1, y N+1). The bound on the total expected target damage is zmin(Ë , y ); o w Ë w Ë 6)â zLB < zmin(Ë , y ) set zLB â zmin(Ë , y ); If w Ë w Ë 7)â (zUB - zLB â¤ e) go to End; If 8)â N â N + 1 and go to step (2) (Subproblem); Set End: Print â(w*, y*) is an Îµ-optimal (MXM) defender solution,â and halt. 9)â The optimal attacker plan x* can be recovered by solving [18]. The first attacker event sequence addresses selection of MAX-ATTACKER-LP(w*). agent, target method of dissemination, and acquisition; the Each instance of MAX-ATTACKER-LP(Ë is a linear w) next attacker sequence involves details of agent production program of a form we expect to be easy to solve even at and processing; the following attacker sequence describes large scale. transport and storage; and the last estimates repeated attacks. MIN-ILP-DECOMP({Ë N}) is easy to solve, but might get x These attacker sequences are interrupted by opportunities more challenging if embellished with too many more linear for the defender to interdict. The last stage [18] represents constraints. For a difficult instance, or at very large scale, we Mother Nature influencing consequences. For our purposes, can solve MIN-ILP-DECOMP({Ë N}) with an approximate, x there are merely four alternations from attacker to defender, but very fast heuristic, and our decomposition is still valid. followed by one truly random event governed by Mother The iterative behavior of the decomposition is instruc- Nature at the end. tive. Set a defense plan, and observe the attack response. Set Second, we decide how to reckon damaged,a as a function another defense plan that is robust with respect to the attack of defense strategy d and attack alternative a. This is not a response observed, and then observe another attack response. glib statement, but rather a meta-design guide to return to the As such iterations continue, the defender learns more about foundations of BTRA and critically review the assumptions the attacker, and refines his defense plan accordingly. Ulti- of sequence-dependence and level of detail. mately, the defender learns enough to declare that his best In theory, this could be achieved by setting a defender defense plan is (e-) optimal against the best possible attacker option d, and estimating the consequences of this action on plan, and attains a mathematical certificate of the quality of BTRA for each pure attacker response. This is no harder than his defense preparations. (See Table E.6.) for BTRA, and if we concentrate on estimating damaged,a The decomposition mathematically represents two op- posed sets of subject-matter experts: a Blue Team (defender), and Red Team (attacker). The decomposition iterations mathematically mimic a wargame between these opponents, TABLE E.6â Decomposition iterations reveal learning by where the defender suffers the disadvantage of not being able opponents. Here, the defender starts with defense strategy to hide the defense strategy, but the players play the game âd00â (do nothing), the attacker responds with his most- again and again, honing their respective strategies, until damaging alternative âa03â inflicting damage 10. neither opponent can improve. At ultra-large scale, we can nest decompositions. We do Defense Lower Attack Upper not anticipate this will be necessary for this application. Iteration Strategy Bound Alternative Bound We have implemented MXM and our decomposition al- n MIN-ILP- zLB MAX- zUB Mitigation gorithm for solving it in GAMS (2007). All model instances DECOMP ATTACKER- have been solved optimally. The complete implementation LP 1 âd00â 0 âa03â 10 âm01â is available from the authors. 2 âd05â 2 âa01â 5 âm02â 3 âd04â 3.5 âa01â(0.5) 3.5 âm01â(0.5) âa03â(0.5) âm02â(0.5) How Do We Get Here from a Descriptive Risk Assessment (e.g., DHS BTRA)? Subsequent iterations adjust defense strategy based on elicited attacker b Â ehavior, until neither opponent can take another turn for any further First, we must recognize and accept that each event-tree improvement. Our subject-matter experts (SMEs) are now optimization path in the BTRA consists almost exclusively of a set of m Â odels. The last iteration yields the same optimal solution as shown in decisionsâthese are not random events. There are 18 succes- Table E.5. Instead of using a âdo-nothingâ solution to initialize the algo- sive âeventsâ in the National Research Council rendition of rithm, we can just as easily take any feasible incumbent proposed by any decision maker as our first attempt: the algorithm will evaluate this solution, BTRA (see Tables E.3 and E.4). From start to finish, we show and then either obtain a certificate of its optimality, or find a better incum- each event number, using parentheses to distinguish defender bent. This is the distinguishing advantage of viewing these decomposition actions, and brackets for Mother Nature at the end: the BTRA algorithms as âlearningâ methods that iteratively improve upon an incum- event sequence is 1-5; (6); 7-11; (12); 13; (14-15); 16; (17); bent, possibly suboptimal, solution.

APPENDIX E 101 as a function of defense option d and a more palatable (i.e., (c) Are part of an integrated weapons of mass destruction unlike BTRA, a less minutely-detailed and less overwhelm- consequence management approach informed by current risk ingly numerous) set of attack alternatives a, we would assessments of threats, vulnerabilities, and capabilities; and create a risk-calculation engine that is at once credible and (d) Include the development of effective, feasible, and prag- efficient. matic concepts of operation for responding to and recovering By whatever means, we must estimate damaged,a for each from an attack. (The White House, 2007) defense option d and each attack alternative a. If we cannot estimate risks at this fidelity, we have no business doing risk We can see from these policy directives that the highest- analysis. level DHS problem is planning investmentsâhuge invest- We would prefer to be able to choose a number of defense mentsâto prepare to mitigate the consequences of any strategies, rather than just one. But, current risk analysis pro- attack. duces a single damage estimate distribution for each attack The material presented here follows both the letter and scenario. We assume these damage estimates are neither ad- the spirit of this direction. ditive nor separable between and among attacks, so we must rely on the simplified risk analysis we have. Accordingly, REFERENCES we endow each defense strategy with the number of defense investment options reflected in each BTRA path. Banks, D., and S. Anderson. 2006. âCombining Game Theory and Risk Our attack alternatives have not specified any particular Analysis in Counterterrorism: A Smallpox Example.â Pp. 9-22 in A. Wilson, G. Wilson, and D. Olwell (eds.), Statistical Methods in Coun- agent. Our methods can accommodate attacks by classes terterrorism. New York: Springer. of agents that include engineered and future agents not yet Bard, J., and J. Moore. 1992. âAn Algorithm for the Discrete Bilevel Pro- known. gramming Problem.â Naval Research Logistics 39(3):419-435. Solving the tri-level model achieved here isolates an op- Bazaraa, M.S., J. Jarvis, and H.D. Sherali. 1990. Linear Programming and timal defense strategy, and all its component investment op- Network Flows. New York: Wiley. Brown, G., R. Dell, and A. Newman. 2004. âOptimizing Military Capital tions. Because this optimal strategy dominates every attack Planning.â Interfaces 34(6):415-425. by any agent, we have presented an intrinsic risk analysis Brown, G., M. Carlyle, J. SalmerÃ³n, and K. Wood. 2005a. âAnalyzing the that highlights the most-critical, achievable defense strategy. Vulnerability of Critical Infrastructure to Attack, and Planning De- We can trivially rule out this best strategy, and solve for the fenses.â In H. Greenberg and J. Smith (eds.), Tutorials in Operations second-best, and so forth. This renders an explicit, unam- Research: Emerging Theory, Methods, and Applications, Hanover, Md.: Institute for Operations Research and Management Science. biguous prioritization of defense strategies. Brown, G., M. Carlyle, D. Diehl, J. Kline, and K. Wood. 2005b. âA Two- Sided Optimization for Theater Ballistic Missile Defense.â Operations Research 53(5):745-763. Mere Probabilistic Risk Assessment Is Not Enough Brown, G., M. Carlyle, J. SalmerÃ³n, and K. Wood. 2006a. âDefending What we are proposing here responds directly to the Critical Infrastructure.â Interfaces 36(6):530-544. Brown, G., M. Carlyle, R. Harney, E. Skroch, and K. Wood. 2006b. explicit language of HSPD-10 (The White House, 2004): âAnatomy of a Project to Produce a First Nuclear Weapon.â Science âthe United States requires a continuous, formal process for and Global Security 14(2/3):163-182. conducting routine capabilities assessments to guide priori- Brown, G., M. Carlyle, R. Harney, E. Skroch, and K. Wood. 2007. âInter- tization of our on-going investments in biodefense-related dicting a Nuclear Weapons Project.â In review. research, development, planning, and preparedness.â Candler, W., and R. Townsley. 1982. âA Linear Two-Level Programming Problem.â Computers and Operations Research 9(1):59-76. Further, we could not agree more with this: âSuccessful DHS (Department of Homeland Security). 2006. Bioterrorism Risk Assess- implementation of our program requires optimizing critical ment. Biological Threat Characterization Center of the National Biode- cross-cutting functionsâ (The White House, 2004). fense Analysis and Countermeasures Center. Fort Detrick, Md. Recently, HSPD-18 (The White House, 2007) has further Fulkerson, D.R., and G.C. Harding. 1977. âMaximizing the Minimum clarified our direction: âoptimize the investments neces- Source-Sink Path Subject to a Budget Constraint.â Mathematical Pro- gramming 13(1):116-118. sary for medical countermeasures development, and ensure GAMS (General Algebraic Modeling System). 2007. âGeneral Algebraic that our activities significantly enhance our domestic and Modeling Language GAMS.â Available at http://www.gams.com/. Ac- international response and recovery capabilities.â Further: cessed January 12, 2007. âMitigating illness and preventing death are the principal Golden, B. 1978. âA Problem in Network Interdiction.â Naval Research goals of our medical countermeasure efforts.â Logistics Quarterly 25(4):711-713. Israeli, E., and R.K. Wood. 2002. âShortest-Path Network Interdiction.â Moving beyond mere descriptive risk analysis, we want Networks 40(2):97-111. to address: Kuhn, H. 1953. âExtensive Games and the Problem of Information.â Pp. 193-216 in H. Kuhn and A. Tucker (eds.), Contributions to the Theory (a) Target threats that have potential for catastrophic impact of Games, Vol. II. Princeton, N.J.: Princeton University Press. on our public health and are subject to medical mitigation; Migdalas, A., P.M. Pardalos, and P. Varbrand. 1998. Multilevel Optimiza- (b) Yield a rapidly deployable and flexible capability to ad- tion: Algorithms and Applications. Dordrecht, Germany: Kluwer. dress both existing and evolving threats; Raiffa, H. 1968. Decision Analysis. Reading, Mass.: Addison-Wesley.

102 DEPARTMENT OF HOMELAND SECURITY BIOTERRORISM RISK ASSESSMENT SalmerÃ³n, J., K. Wood, and R. Baldick. 2004. âAnalysis of Electric Grid The White House. 2004. Homeland Security Presidential Directive 10 Security Under Terrorist Threat,â IEEE Transactions on Power Systems [HSPD-10]: Biodefense for the 21st Century. Available at www.fas. 19(2):905-912. org/irp/offdocs/nspd/hspd-10.html. Accessed January 16, 2008. Tintner, G. 1960. âA Note on Stochastic Linear Programming.â Economet- The White House. 2007. Homeland Security Presidential Directive 18 rica 28(2):490-495. [HSPD-18]: Medical Countermeasures Against Weapons of Mass Ville, J. 1938. âSur la theorie generale des jeux ou intervient lâhabilite Destruction. Available at www.fas.org/irp/offdocs/nspd/hspd-18.html. des joueurs.â Pp. 105-113 in E. Borel et al. (eds.), Traite du calcul des Accessed January 16, 2008. probabilites et de ses applications, Vol. II. Paris: Gautheir-Villars. von Neumann, J. 1928. âZur Theorie der Gesellshaftsphiele.â Annals of Mathemetics 100:295-320.