Department of Homeland Security Bioterrorism Risk Assessment: A Call for Change (2008)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

6 Improving Bioterrorism Consequence Assessment . . . and the big picture is worrying about how do we protect the most people from the greatest risks most of the time. —Department of Homeland Security Secretary Michael Chertoff at a press conference, January 5, 2007 EXISTING KNOWLEDGE DOES NOT SUPPORT SEIR models can provide useful insights into the me- THE DETAIL IN DEPARTMENT OF HOMELAND chanics of many common infectious diseases and into the SECURITY CONSEQUENCE MODELS effectiveness of control strategies. However, SEIR and similar modeling approaches have limitations. Even the In the Department of Homeland Security’s (DHS’s) simplest model requires a minimum amount of parametric report on the Biological Threat Risk Assessment (BTRA) data: in particular, the attack rate or risk of transmission per of 2006 (DHS, 2006), three measures of consequences are contact, the incubation period of the disease, the number of determined for each scenario: fatalities, illnesses, and direct potentially infectious contacts that a person has per unit of economic costs. These three measures are dependent on in- time, and the duration of the transmissible period. Models trinsic properties of the pathogen, the details of the scenario, that assess the health consequences of the pathogens of and the hypothesized U.S. response to the event, accounting concern for bioterrorism are difficult to parameterize owing for the effect of current U.S. medical mitigation capacity. to the lack of an adequate empirical base. Many diseases Although in the BTRA of 2006 an analysis was conducted are relatively obscure and are associated with extremely for the three measures of consequences, the overall risk- limited clinical and epidemiologic data.  For example, for informed agent prioritization is based only on mortality. In many diseases caused by agents being considered by DHS, presentations to the committee, DHS reported that it intends little is known about the dose-response relationships (a to take into account indirect economic costs (e.g., medical major concern, as the size of a dose may determine whether mitigation, emergency response, cleanup, and business loss) symptoms occur), the duration of the incubation period, as well. Some projected improvements for future BTRAs are and the severity of infection. Thus, estimates of the key described in Chapter 3. parameters for most bioterrorism agents must reflect a very Assessing an infectious agent’s impact on a population large variance. The anthrax attacks in the United States in is challenging. In order to measure the health consequences, 2001 demonstrated low correlation between environmental currently defined as the number of fatalities and of ill people, exposure and infection risk. Given the great uncertainty as- DHS has implemented a susceptible, exposed, infected, and sociated with model inputs, it is important to acknowledge recovered (SEIR) model using an off-the-shelf software that generated predictions are rough approximations at package called STELLA, which is run for each scenario. best and, while useful in helping to understand a problem, SEIR is a deterministic, “compartmental” model; it cat- should not be regarded as more than rough approximations. egorizes individuals as being in one of four compartments, Exacerbating this lack of certainty in model outputs, the representing the susceptible, the exposed, the infected, and increasing availability of sophisticated computer software those who have recovered. The model parameters specify the allows researchers to create highly artificial models, some- transition rates between the compartments—for example, the times based on weakly defended assumptions, where the attack rate—as susceptible people become infected.   Note: The committee thanks Jason Matheny, Ellis McKenzie, Marc Lipsitch, and Michael Boechler for reading this chapter. 1  “Direct economic costs” here refer to the costs of hospitalization and   Marc Lipsitch, Harvard School of Public Health. “Notes to the National funerals and exclude the cost of decontamination, loss of worker productiv- Research Council Committee on Methodological Improvements to the ity, and so on. DHS’s Biological Agent Risk Analysis.” Written communication to the   See www.iseesystems.com. Accessed January 30, 2008. committee, February 2007. 47 

48 DEPARTMENT OF HOMELAND SECURITY BIOTERRORISM RISK ASSESSMENT complexity and precision of results can be mistaken for of severe acute respiratory syndrome (SARS) in 2003, or accuracy. potential unintentional contact-increasing behavior after the Even when data for well-studied pathogens (e.g., influ- occurrence of a widely publicized bioterrorism attack. enza viruses) are available, predicting the propagation of in- Soon after the terrorist attacks of September 11, 2001 fection in a population requires understanding how individu- (9/11), several prominent infectious-disease modelers un- als, as well as medical and public health teams, will respond dertook studies to assess the likely magnitude of smallpox to a threat. Again, there are limited empirical data to inform epidemics under various response strategies. The U.S. gov- models regarding medical and public response capacity and ernment was particularly interested in determining whether, human behavior in the setting of bioterrorism. As pointed in the aftermath of an attack, vaccination of likely contacts of out in Ferguson (2007), there are fundamental limitations in infected persons (“ring vaccination” or “traced vaccination”) how models can capture the key social parameters of human would be as effective in containing an outbreak as would behavior. The manner in which people alter their behavior mass vaccination. At that time, soon after 9/11, vaccine was in an attempt to reduce their risk when faced with lethal or in limited supply. The former strategy would require fewer novel pathogens is difficult to predict and may significantly vaccinations and, due to the capability of smallpox vaccine alter the consequences of an attack. Models are unlikely to to induce a reaction, would be associated with less morbid- capture behaviors that significantly reduce social contact, ity. Despite available quantitative data from past smallpox as seen in Hong Kong and Singapore during the epidemic epidemics, there was considerable disagreement about the likely adequacy of the various responses. It took several   years and considerable debate to understand that the differ- In May (2004), Lord May writes, “The increasing speed and sophistica- tion and ease of use of computers enables an increasingly large number of ences in models’ conclusions rested mainly on assumptions life scientists who have no substantial background in mathematics to explore about the timing of transmission relative to symptoms and ‘mathematical models’ and draw conclusions about them. Such activity about the likely speed of the public health response (i.e., the usually consists of representing sensible and evidence-based assumptions capacity of public health workers to enact targeted versus as the starting point for a complicated and usually nonlinear dynamical mass vaccination campaigns) (Cooper, 2006). Substantial system, assigning particular parameters (often in an arbitrary way), and then letting this complicated system rip. This represents a revolutionary change differences between public health capabilities in different in such theoretical studies. Until only a decade or two ago, anyone pursu- jurisdictions present more variability. Thus, the site of the ing this kind of activity had to have a solid grounding in mathematics. And attack may significantly influence its consequences. In sum- that meant that such studies were done by people who had some idea, at mary, response logistics matter just as much as epidemiology an intuitive level, of how the original assumptions related to the emerging in determining the outcome of a bioterrorism attack (Kaplan graphical display or other conclusions on their computer. Removing this link means that we arguably are seeing an increasingly large body of work et al., 2003). in which sweeping conclusions—‘emergent phenomena’—are drawn from Although consequence models are imperfect, they clearly the alleged working of a mathematical model, without clear understanding can contribute to planning and mitigation. An appropriate es- of what is actually going on. I think this can be worrying.” (p. 790) timate of the damage that the United States could experience   Lord May further substantiates his argument in favor of simpler models is critical to allocating resources and developing mitigation over complex ones by citing the example of HIV/AIDS models developed in the mid-1980s to estimate the likely demographic impact in some central strategies to the numerous possible different threats. “Intui- African countries: “The main unknown at that time was the probability, ß, tive judgment” alone is inadequate, as it focuses on only a that an infected individual would infect a susceptible partner. Available handful of salient cues (and not necessarily the right ones), data suggested that ß depended relatively little on the number of sexual acts often weighed in a simple linear fashion (Hammond, 2006). within a partnership. On this basis, we used a relatively simple model to However, models should take into account the intuitive judg- suggest that the future demographic impact of HIV/AIDS could be severe in some such countries. In contrast, the World Health Organization and ments of informed and experienced health professionals. A the Population Council in New York produced models that were much “structured discussion” approach is also useful in driving more complex, including very detailed demographic data, but where HIV consensus. However, structured discussion is subject to transmission probability was treated as if for measles, compounding inde- small-group dynamics and may reflect primarily the biases pendently and randomly for each individual sex act. Thus, in effect, their of the most vocal, argumentative, or influential individuals models assumed that, knowing nothing of the infective status of individuals, 1 sex act with each of 10 different sex partners was effectively equivalent to (Janis, 1989). Both intuitive judgment and structured judg- 10 acts with 1; our data-governed, but otherwise much simpler, model saw ment can be useful adjuncts to the modeling process but, like the former as roughly 10 times more risky. So it was not surprising that the modeling itself, are not sufficiently robust and free of bias later models, apparently ‘more realistic’ by virtue of their computational and error to stand alone. complexity, suggested a less gloomy view than ours. Sadly, but understand- ably, our predictions have proved more reliable.” (p. 793)   The two excerpts from May (2004) in this footnote are reprinted with Recommendation: The susceptible, exposed, infected, permission from AAAS. Readers may view, browse, and/or download this and recovered (SEIR) model adopted by DHS is more material for temporary copying purposes only, provided these uses are for complex than can be supported by existing data or noncommercial personal purposes. Except as provided by law, this material may not be further reproduced, distributed, transmitted, modified, adapted, performed, displayed, published, or sold in whole or in part, without prior   An encompassing summary of forecasting using expert judgment can written permission from the publisher. be found in Armstrong (2001). 

IMPROVING BIOTERRORISM CONSEQUENCE ASSESSMENT 49 knowledge. DHS should make its SEIR model as simple reports of persons opening a letter or package, finding a as possible consistent with existing knowledge. powdery substance (later found to be harmless), and having a psychological and physical reaction that required medical The complexity of the consequence models presented attention (Wessely et al., 2001). Silver et al. (2002) point by DHS seems too great given the data available. The use out that the long-term social and psychological effects of a of a complex model when adequate data are unavailable is biological attack may be as damaging as the acute ones, that probably detrimental to the quality of conclusions, and their they may remain high for years, and that they may exacerbate use may be dangerously misleading. The complexity com- preexisting psychiatric disorders and further heighten the risk promises the ability to elicit sensible estimates, uncertainty of mass sociogenic illness. A distrust of medical experts and ranges, and correlations in the uncertainty for all of these government officials, who cannot provide blanket assurances parameters obtained from subject-matter experts. Hence the of no lasting harm, may result. The response to bioterrorist uncertainty of the model will likely be incorrectly estimated. events may involve the distribution of medical therapeutics In addition, complex models do not lend themselves well to and vaccines, isolation of symptomatic individuals, observa- independent validation and verification by other modelers. tion of potentially exposed people by public health officials, and other actions that are guaranteed to generate anxiety in the population. The ongoing risk of exposure, possible OTHER CONSEQUENCES NEED TO BE MODELED evacuation from contaminated areas, and the perceived or Bioterrorist attacks create direct impacts, which occur real risk of death or permanent health consequences are all immediately after the event, and indirect impacts, which contributory. If government and public health officials do not may be much longer term in nature. More specifically, direct properly manage risk communication, there is potential for impacts are damage and losses that can be directly attributed civil disruption and further business and economic losses. to the attack, such as injuries, loss of life, and damage to DALY would be an appropriate measure of psychological property and infrastructure as well as to natural habitats and distress, as it is already used in mental health. fish and wildlife populations. Indirect or secondary impacts Bioattacks can also have serious environmental conse- occur over time and include, for instance, family trauma quences. For example, Gruinard Island in the United King- and social disruption, business interruptions, and shortages dom became contaminated with anthrax spores after testing of critical human services. From a societal point of view, occurred on the island in 1942; the island was quarantined for a large number of cases of a disease, perhaps with a long almost 50 years. Decontamination was finally accomplished period of latency, can have far more serious implications in 1986 when, after removal of topsoil, the 520-acre island than the consequences of a large number of deaths from the was soaked with 280 tons of formaldehyde diluted in 2,000 same disease. tons of seawater. In the 2001 anthrax attacks in the United Any measure of health consequences must, at a minimum, States, four major cleanups were required: at the American combine considerations of morbidity and mortality. Two Media, Inc. (AMI), building in Boca Raton, Florida; the such measures of “health utility” commonly used are the dis- National Broadcasting Company (NBC) offices in New York ability-adjusted life-year (DALY) and the quality-adjusted City; the U.S. Capitol complex in Washington, D.C.; and at life-year (QALY). The more commonly used DALY is two facilities of the U.S. Postal Service. computed as (N × L) + (D × DW × DD), where N = number Agricultural consequences also need to be considered. of deaths, L = life expectancy in years, D = number of dis- Economic activity of U.S. agriculture has been estimated to abilities, DW = disability weight, and DD = duration of dis- exceed $1 trillion annually, with exports valued in excess ability in years. The disability weight is defined by a panel of of $50 billion. Protecting U.S. agriculture is critical to the clinicians. Some DALY models also apply age-weights and global economy and to the ensuring of an adequate and safe discount rates. The scaling is such that 1 DALY represents food supply in the United States and other countries. Several the loss of 1 year of equivalent full health. assessments of agricultural consequences have shown that Estimates of morbidity should also include psychological livestock and poultry populations are vulnerable to biologic effects. Fear is a terrorist’s “force multiplier.” Fears about attack. The U.S. Department of Agriculture has identified biological agents, which are not readily identifiable and are viruses and bacteria capable of causing widescale morbid- generally misunderstood, heighten the real or perceived ity and mortality of livestock and poultry that would result threats of terrorism. After the anthrax letter attacks in the in a cessation of international trade and exports costing the United States in October 2001, millions of people were made United States billions of dollars. anxious opening their mail; and there have been numerous   For a more detailed discussion of the importance of incorporating in- direct or secondary impacts in evaluating alternative programs, see Heinz   See news.bbc.co.uk/2/hi/uk_news/scotland/1457035.stm. Accessed Center for Science, Economics, and the Environment (1999). January 31, 2008.   These and other measures of population health are discussed in NRC   See frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=2005_ (1998). register&docid=fr18mr05-20.pdf. Accessed January 31, 2008. 

50 DEPARTMENT OF HOMELAND SECURITY BIOTERRORISM RISK ASSESSMENT Recommendation: While human mortality and the consequences from different events. One then has to use magnitude and duration of morbidity should remain the other methods of analysis such as cost-effectiveness analysis primary focus of DHS bioterrorism risk analysis, DHS or multigoal analysis.10 should incorporate other measures of societal loss, in- cluding the magnitude and duration of first- and second- REFERENCES order economic loss and environmental and agricultural effects. Armstrong, J.S. 2001. Principles of Forecasting. Newell, Mass.: Kluwer Academic Publishers. Boardman, A., D. Greenberg, A. Vining, and D. Weimer. 2001. Cost- Some direct impacts of bioterrorist attacks are relatively Benefit Analysis: Concepts and Practice. Upper Saddle River, N.J.: easy to quantify because they are easy to measure in dollars: Prentice Hall. insured losses to homes, businesses, and industry; bridge and Cooper, B. 2006. “Poxy Models and Rash Decisions.” Proceedings of the highway repairs; equipment replacement or repairs; crop National Academy of Sciences 103(33):12221-12222. DHS (Department of Homeland Security). 2006. Bioterrorism Risk Assess- loss; and so on. The costs of other direct impacts and many ment. Biological Threat Characterization Center of the National Biode- indirect impacts are less easy to determine and quantify—for fense Analysis and Countermeasures Center. Fort Detrick, Md. example, psychological distress and family instability. Ferguson, N. 2007. “Capturing Human Behavior.” Nature 446(7137):733. Cost-benefit analysis (CBA) has been proposed as a way Hammond, K.R. 2006. Beyond Rationality: The Search for Wisdom in a of combining direct and indirect effects of alternative pro- Troubled Time. New York: Oxford University Press. Heinz Center for Science, Economics, and the Environment. 1999. The grams. If one undertakes CBA, it is necessary to monetize Hidden Costs of Coastal Hazards: Implications for Risk Assessment each of the direct and indirect impacts to provide a common and Mitigation. Washington, D.C.: Island Press. metric for ranking the risks of different bioagents. Monetiza- Janis, I.L. 1989. Groupthink: Psychological Studies of Policy Decisions and tion means assigning values in dollars. Mortality and morbid- Fiascoes. Boston: Houghton Mifflin. ity (including psychological distress) could be monetized by Kaplan, E.H., D.L. Craft, and L.M. Wein. 2003. “Analyzing Bioterror Response Logistics: The Case of Smallpox.” Mathematical Biosciences setting 1 DALY or 1 QALY equal to the “value of a statistical 185(1):33-72. life-year” or to 1 year of income (typically on the order of May, R.M. 2004. “Uses and Abuses of Mathematics in Biology.” Science $50,000). The value of environmental impacts is measured 303(5659):790-793. in terms of willingness to pay by using contingent valuation NRC (National Research Council). 1998. Summarizing Population Health: techniques and has been a source of debate by economists Directions for the Development and Application of Population Metrics. Washington D.C.: National Academy Press. over the years. Silver R.C., E.A. Holman, D.N. McIntosh, M. Poulin, and V. Gil-Rivas. The total social cost of a bioterrorist attack can be esti- 2002. “Nationwide Longitudinal Study of Psychological Responses mated by combining direct and indirect economic costs with to September 11.” Journal of the American Medical Association the monetization of mortality, morbidity, and environmental 288(10):1235-1244. costs. Some critics of CBA are unwilling to attach monetary Wessely S., K.C. Hyams, and R. Bartholomew. 2001. “Psychological Im- plications of Chemical and Biological Weapons.” BMJ 323(7318):878- values to life, environmental impacts, or other non-economic 879. 10  For more details on the concepts and practice of cost-benefit analysis and alternative analyses, see Boardman et al. (2001).