Cover Image

PAPERBACK
$39.75



View/Hide Left Panel

6
Improving Bioterrorism Consequence Assessment

… and the big picture is worrying about how do we protect the most people from the greatest risks most of the time.

—Department of Homeland Security Secretary Michael Chertoff at a press conference, January 5, 2007

EXISTING KNOWLEDGE DOES NOT SUPPORT THE DETAIL IN DEPARTMENT OF HOMELAND SECURITY CONSEQUENCE MODELS

In the Department of Homeland Security’s (DHS’s) report on the Biological Threat Risk Assessment (BTRA) of 2006 (DHS, 2006), three measures of consequences are determined for each scenario: fatalities, illnesses, and direct economic costs.1 These three measures are dependent on intrinsic properties of the pathogen, the details of the scenario, and the hypothesized U.S. response to the event, accounting for the effect of current U.S. medical mitigation capacity. Although in the BTRA of 2006 an analysis was conducted for the three measures of consequences, the overall risk-informed agent prioritization is based only on mortality. In presentations to the committee, DHS reported that it intends to take into account indirect economic costs (e.g., medical mitigation, emergency response, cleanup, and business loss) as well. Some projected improvements for future BTRAs are described in Chapter 3.

Assessing an infectious agent’s impact on a population is challenging. In order to measure the health consequences, currently defined as the number of fatalities and of ill people, DHS has implemented a susceptible, exposed, infected, and recovered (SEIR) model using an off-the-shelf software package called STELLA,2 which is run for each scenario. SEIR is a deterministic, “compartmental” model; it categorizes individuals as being in one of four compartments, representing the susceptible, the exposed, the infected, and those who have recovered. The model parameters specify the transition rates between the compartments—for example, the attack rate—as susceptible people become infected.

SEIR models can provide useful insights into the mechanics of many common infectious diseases and into the effectiveness of control strategies. However, SEIR and similar modeling approaches have limitations. Even the simplest model requires a minimum amount of parametric data: in particular, the attack rate or risk of transmission per contact, the incubation period of the disease, the number of potentially infectious contacts that a person has per unit of time, and the duration of the transmissible period. Models that assess the health consequences of the pathogens of concern for bioterrorism are difficult to parameterize owing to the lack of an adequate empirical base. Many diseases are relatively obscure and are associated with extremely limited clinical and epidemiologic data.3 For example, for many diseases caused by agents being considered by DHS, little is known about the dose-response relationships (a major concern, as the size of a dose may determine whether symptoms occur), the duration of the incubation period, and the severity of infection. Thus, estimates of the key parameters for most bioterrorism agents must reflect a very large variance. The anthrax attacks in the United States in 2001 demonstrated low correlation between environmental exposure and infection risk. Given the great uncertainty associated with model inputs, it is important to acknowledge that generated predictions are rough approximations at best and, while useful in helping to understand a problem, should not be regarded as more than rough approximations. Exacerbating this lack of certainty in model outputs, the increasing availability of sophisticated computer software allows researchers to create highly artificial models, sometimes based on weakly defended assumptions, where the

Note: The committee thanks Jason Matheny, Ellis McKenzie, Marc Lipsitch, and Michael Boechler for reading this chapter.

1

“Direct economic costs” here refer to the costs of hospitalization and funerals and exclude the cost of decontamination, loss of worker productivity, and so on.

2

See www.iseesystems.com. Accessed January 30, 2008.

3

Marc Lipsitch, Harvard School of Public Health. “Notes to the National Research Council Committee on Methodological Improvements to the DHS’s Biological Agent Risk Analysis.” Written communication to the committee, February 2007.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 47
6 Improving Bioterrorism Consequence Assessment . . . and the big picture is worrying about how do we protect the most people from the greatest risks most of the time. —Department of Homeland Security Secretary Michael Chertoff at a press conference, January 5, 2007 EXISTING KNOWLEDGE DOES NOT SUPPORT SEIR models can provide useful insights into the me- THE DETAIL IN DEPARTMENT OF HOMELAND chanics of many common infectious diseases and into the SECURITY CONSEQUENCE MODELS effectiveness of control strategies. However, SEIR and similar modeling approaches have limitations. Even the In the Department of Homeland Security’s (DHS’s) simplest model requires a minimum amount of parametric report on the Biological Threat Risk Assessment (BTRA) data: in particular, the attack rate or risk of transmission per of 2006 (DHS, 2006), three measures of consequences are contact, the incubation period of the disease, the number of determined for each scenario: fatalities, illnesses, and direct potentially infectious contacts that a person has per unit of economic costs.1 These three measures are dependent on in- time, and the duration of the transmissible period. Models trinsic properties of the pathogen, the details of the scenario, that assess the health consequences of the pathogens of and the hypothesized U.S. response to the event, accounting concern for bioterrorism are difficult to parameterize owing for the effect of current U.S. medical mitigation capacity. to the lack of an adequate empirical base. Many diseases Although in the BTRA of 2006 an analysis was conducted are relatively obscure and are associated with extremely for the three measures of consequences, the oerall risk- limited clinical and epidemiologic data. 3 For example, for informed agent prioritization is based only on mortality. In many diseases caused by agents being considered by DHS, presentations to the committee, DHS reported that it intends little is known about the dose-response relationships (a to take into account indirect economic costs (e.g., medical major concern, as the size of a dose may determine whether mitigation, emergency response, cleanup, and business loss) symptoms occur), the duration of the incubation period, as well. Some projected improvements for future BTRAs are and the severity of infection. Thus, estimates of the key described in Chapter 3. parameters for most bioterrorism agents must reflect a very Assessing an infectious agent’s impact on a population large variance. The anthrax attacks in the United States in is challenging. In order to measure the health consequences, 2001 demonstrated low correlation between environmental currently defined as the number of fatalities and of ill people, exposure and infection risk. Given the great uncertainty as- DHS has implemented a susceptible, exposed, infected, and sociated with model inputs, it is important to acknowledge recovered (SEIR) model using an off-the-shelf software that generated predictions are rough approximations at package called STELLA,2 which is run for each scenario. best and, while useful in helping to understand a problem, SEIR is a deterministic, “compartmental” model; it cat- should not be regarded as more than rough approximations. egorizes individuals as being in one of four compartments, Exacerbating this lack of certainty in model outputs, the representing the susceptible, the exposed, the infected, and increasing availability of sophisticated computer software those who have recovered. The model parameters specify the allows researchers to create highly artificial models, some- transition rates between the compartments—for example, the times based on weakly defended assumptions, where the attack rate—as susceptible people become infected. Note: The committee thanks Jason Matheny, Ellis McKenzie, Marc Lipsitch, and Michael Boechler for reading this chapter. 1 “Direct economic costs” here refer to the costs of hospitalization and 3 MarcLipsitch, Harvard School of Public Health. “Notes to the National funerals and exclude the cost of decontamination, loss of worker productiv- Research Council Committee on Methodological Improvements to the ity, and so on. DHS’s Biological Agent Risk Analysis.” Written communication to the 2 See www.iseesystems.com. Accessed January 30, 2008. committee, February 2007. 

OCR for page 47
 DEPARTMENT OF HOMELAND SECURITY BIOTERRORISM RISK ASSESSMENT complexity and precision of results can be mistaken for of severe acute respiratory syndrome (SARS) in 2003, or accuracy.4 potential unintentional contact-increasing behavior after the Even when data for well-studied pathogens (e.g., influ- occurrence of a widely publicized bioterrorism attack. enza viruses) are available, predicting the propagation of in- Soon after the terrorist attacks of September 11, 2001 fection in a population requires understanding how individu- (9/11), several prominent infectious-disease modelers un- als, as well as medical and public health teams, will respond dertook studies to assess the likely magnitude of smallpox to a threat. Again, there are limited empirical data to inform epidemics under various response strategies. The U.S. gov- models regarding medical and public response capacity and ernment was particularly interested in determining whether, human behavior in the setting of bioterrorism. As pointed in the aftermath of an attack, vaccination of likely contacts of out in Ferguson (2007), there are fundamental limitations in infected persons (“ring vaccination” or “traced vaccination”) how models can capture the key social parameters of human would be as effective in containing an outbreak as would behavior. The manner in which people alter their behavior mass vaccination. At that time, soon after 9/11, vaccine was in an attempt to reduce their risk when faced with lethal or in limited supply. The former strategy would require fewer novel pathogens is difficult to predict and may significantly vaccinations and, due to the capability of smallpox vaccine alter the consequences of an attack. Models are unlikely to to induce a reaction, would be associated with less morbid- capture behaviors that significantly reduce social contact, ity. Despite available quantitative data from past smallpox as seen in Hong Kong and Singapore during the epidemic epidemics, there was considerable disagreement about the likely adequacy of the various responses. It took several years and considerable debate to understand that the differ- 4 In May (2004), Lord May writes, “The increasing speed and sophistica- ences in models’ conclusions rested mainly on assumptions tion and ease of use of computers enables an increasingly large number of about the timing of transmission relative to symptoms and life scientists who have no substantial background in mathematics to explore about the likely speed of the public health response (i.e., the ‘mathematical models’ and draw conclusions about them. Such activity usually consists of representing sensible and evidence-based assumptions capacity of public health workers to enact targeted versus as the starting point for a complicated and usually nonlinear dynamical mass vaccination campaigns) (Cooper, 2006). Substantial system, assigning particular parameters (often in an arbitrary way), and then differences between public health capabilities in different letting this complicated system rip. This represents a revolutionary change jurisdictions present more variability. Thus, the site of the in such theoretical studies. Until only a decade or two ago, anyone pursu- attack may significantly influence its consequences. In sum- ing this kind of activity had to have a solid grounding in mathematics. And that meant that such studies were done by people who had some idea, at mary, response logistics matter just as much as epidemiology an intuitive level, of how the original assumptions related to the emerging in determining the outcome of a bioterrorism attack (Kaplan graphical display or other conclusions on their computer. Removing this et al., 2003). link means that we arguably are seeing an increasingly large body of work Although consequence models are imperfect, they clearly in which sweeping conclusions—‘emergent phenomena’—are drawn from can contribute to planning and mitigation. An appropriate es- the alleged working of a mathematical model, without clear understanding of what is actually going on. I think this can be worrying.” (p. 790) timate of the damage that the United States could experience Lord May further substantiates his argument in favor of simpler models is critical to allocating resources and developing mitigation over complex ones by citing the example of HIV/AIDS models developed strategies to the numerous possible different threats. “Intui- in the mid-1980s to estimate the likely demographic impact in some central tive judgment” alone is inadequate, as it focuses on only a African countries: “The main unknown at that time was the probability, ß, handful of salient cues (and not necessarily the right ones), that an infected individual would infect a susceptible partner. Available data suggested that ß depended relatively little on the number of sexual acts often weighed in a simple linear fashion (Hammond, 2006). within a partnership. On this basis, we used a relatively simple model to However, models should take into account the intuitive judg- suggest that the future demographic impact of HIV/AIDS could be severe ments of informed and experienced health professionals. A in some such countries. In contrast, the World Health Organization and “structured discussion” approach is also useful in driving the Population Council in New York produced models that were much consensus. However, structured discussion is subject to more complex, including very detailed demographic data, but where HIV transmission probability was treated as if for measles, compounding inde- small-group dynamics and may reflect primarily the biases pendently and randomly for each individual sex act. Thus, in effect, their of the most vocal, argumentative, or influential individuals models assumed that, knowing nothing of the infective status of individuals, (Janis, 1989). Both intuitive judgment and structured judg- 1 sex act with each of 10 different sex partners was effectively equivalent to ment can be useful adjuncts to the modeling process but, like 10 acts with 1; our data-governed, but otherwise much simpler, model saw modeling itself, are not sufficiently robust and free of bias the former as roughly 10 times more risky. So it was not surprising that the and error to stand alone.5 later models, apparently ‘more realistic’ by virtue of their computational complexity, suggested a less gloomy view than ours. Sadly, but understand- ably, our predictions have proved more reliable.” (p. 793) Recommendation: The susceptible, exposed, infected, The two excerpts from May (2004) in this footnote are reprinted with and recovered (SEIR) model adopted by DHS is more permission from AAAS. Readers may view, browse, and/or download this complex than can be supported by existing data or material for temporary copying purposes only, provided these uses are for noncommercial personal purposes. Except as provided by law, this material may not be further reproduced, distributed, transmitted, modified, adapted, 5An performed, displayed, published, or sold in whole or in part, without prior encompassing summary of forecasting using expert judgment can written permission from the publisher. be found in Armstrong (2001).

OCR for page 47
 IMPROVING BIOTERRORISM CONSEQUENCE ASSESSMENT knowledge. DHS should make its SEIR model as simple reports of persons opening a letter or package, finding a as possible consistent with existing knowledge. powdery substance (later found to be harmless), and having a psychological and physical reaction that required medical The complexity of the consequence models presented attention (Wessely et al., 2001). Silver et al. (2002) point by DHS seems too great given the data available. The use out that the long-term social and psychological effects of a of a complex model when adequate data are unavailable is biological attack may be as damaging as the acute ones, that probably detrimental to the quality of conclusions, and their they may remain high for years, and that they may exacerbate use may be dangerously misleading. The complexity com- preexisting psychiatric disorders and further heighten the risk promises the ability to elicit sensible estimates, uncertainty of mass sociogenic illness. A distrust of medical experts and ranges, and correlations in the uncertainty for all of these government officials, who cannot provide blanket assurances parameters obtained from subject-matter experts. Hence the of no lasting harm, may result. The response to bioterrorist uncertainty of the model will likely be incorrectly estimated. events may involve the distribution of medical therapeutics In addition, complex models do not lend themselves well to and vaccines, isolation of symptomatic individuals, observa- independent validation and verification by other modelers. tion of potentially exposed people by public health officials, and other actions that are guaranteed to generate anxiety in the population. The ongoing risk of exposure, possible OTHER CONSEQUENCES NEED TO BE MODELED evacuation from contaminated areas, and the perceived or Bioterrorist attacks create direct impacts, which occur real risk of death or permanent health consequences are all immediately after the event, and indirect impacts, which contributory. If government and public health officials do not may be much longer term in nature. More specifically, direct properly manage risk communication, there is potential for impacts are damage and losses that can be directly attributed civil disruption and further business and economic losses. to the attack, such as injuries, loss of life, and damage to DALY would be an appropriate measure of psychological property and infrastructure as well as to natural habitats and distress, as it is already used in mental health. fish and wildlife populations. Indirect or secondary impacts Bioattacks can also have serious environmental conse- occur over time and include, for instance, family trauma quences. For example, Gruinard Island in the United King- and social disruption, business interruptions, and shortages dom became contaminated with anthrax spores after testing of critical human services. From a societal point of view, occurred on the island in 1942; the island was quarantined for a large number of cases of a disease, perhaps with a long almost 50 years. Decontamination was finally accomplished period of latency, can have far more serious implications in 1986 when, after removal of topsoil, the 520-acre island than the consequences of a large number of deaths from the was soaked with 280 tons of formaldehyde diluted in 2,000 same disease.6 tons of seawater. In the 2001 anthrax attacks in the United Any measure of health consequences must, at a minimum, States, four major cleanups were required: at the American combine considerations of morbidity and mortality. Two Media, Inc. (AMI), building in Boca Raton, Florida; the such measures of “health utility” commonly used are the dis- National Broadcasting Company (NBC) offices in New York ability-adjusted life-year (DALY) and the quality-adjusted City; the U.S. Capitol complex in Washington, D.C.; and at two facilities of the U.S. Postal Service.8 life-year (QALY). The more commonly used DALY is computed as (N × L) + (D × DW × DD), where N = number Agricultural consequences also need to be considered. of deaths, L = life expectancy in years, D = number of dis- Economic activity of U.S. agriculture has been estimated to abilities, DW = disability weight, and DD = duration of dis- exceed $1 trillion annually, with exports valued in excess ability in years. The disability weight is defined by a panel of of $50 billion. Protecting U.S. agriculture is critical to the clinicians. Some DALY models also apply age-weights and global economy and to the ensuring of an adequate and safe discount rates. The scaling is such that 1 DALY represents food supply in the United States and other countries. Several the loss of 1 year of equivalent full health.7 assessments of agricultural consequences have shown that Estimates of morbidity should also include psychological livestock and poultry populations are vulnerable to biologic effects. Fear is a terrorist’s “force multiplier.” Fears about attack. The U.S. Department of Agriculture has identified biological agents, which are not readily identifiable and are viruses and bacteria capable of causing widescale morbid- generally misunderstood, heighten the real or perceived ity and mortality of livestock and poultry that would result threats of terrorism. After the anthrax letter attacks in the in a cessation of international trade and exports costing the United States billions of dollars.9 United States in October 2001, millions of people were made anxious opening their mail; and there have been numerous 6 For a more detailed discussion of the importance of incorporating in- 8 See direct or secondary impacts in evaluating alternative programs, see Heinz news.bbc.co.uk/2/hi/uk_news/scotland/1457035.stm. Accessed Center for Science, Economics, and the Environment (1999). January 31, 2008. 7These and other measures of population health are discussed in NRC 9 See frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=2005_ (1998). register&docid=fr18mr05-20.pdf. Accessed January 31, 2008.

OCR for page 47
0 DEPARTMENT OF HOMELAND SECURITY BIOTERRORISM RISK ASSESSMENT Recommendation: While human mortality and the consequences from different events. One then has to use magnitude and duration of morbidity should remain the other methods of analysis such as cost-effectiveness analysis or multigoal analysis.10 primary focus of DHS bioterrorism risk analysis, DHS should incorporate other measures of societal loss, in- cluding the magnitude and duration of first- and second- REFERENCES order economic loss and environmental and agricultural effects. Armstrong, J.S. 2001. Principles of Forecasting. Newell, Mass.: Kluwer Academic Publishers. Boardman, A., D. Greenberg, A. Vining, and D. Weimer. 2001. Cost- Some direct impacts of bioterrorist attacks are relatively Benefit Analysis: Concepts and Practice. Upper Saddle River, N.J.: easy to quantify because they are easy to measure in dollars: Prentice Hall. insured losses to homes, businesses, and industry; bridge and Cooper, B. 2006. “Poxy Models and Rash Decisions.” Proceedings of the National Academy of Sciences 103(33):12221-12222. highway repairs; equipment replacement or repairs; crop DHS (Department of Homeland Security). 2006. Bioterrorism Risk Assess- loss; and so on. The costs of other direct impacts and many ment. Biological Threat Characterization Center of the National Biode- indirect impacts are less easy to determine and quantify—for fense Analysis and Countermeasures Center. Fort Detrick, Md. example, psychological distress and family instability. Ferguson, N. 2007. “Capturing Human Behavior.” Nature 446(7137):733. Cost-benefit analysis (CBA) has been proposed as a way Hammond, K.R. 2006. Beyond Rationality: The Search for Wisdom in a Troubled Time. New York: Oxford University Press. of combining direct and indirect effects of alternative pro- Heinz Center for Science, Economics, and the Environment. 1999. The grams. If one undertakes CBA, it is necessary to monetize Hidden Costs of Coastal Hazards: Implications for Risk Assessment each of the direct and indirect impacts to provide a common and Mitigation. Washington, D.C.: Island Press. metric for ranking the risks of different bioagents. Monetiza- Janis, I.L. 1989. Groupthink: Psychological Studies of Policy Decisions and tion means assigning values in dollars. Mortality and morbid- Fiascoes. Boston: Houghton Mifflin. Kaplan, E.H., D.L. Craft, and L.M. Wein. 2003. “Analyzing Bioterror ity (including psychological distress) could be monetized by Response Logistics: The Case of Smallpox.” Mathematical Biosciences setting 1 DALY or 1 QALY equal to the “value of a statistical 185(1):33-72. life-year” or to 1 year of income (typically on the order of May, R.M. 2004. “Uses and Abuses of Mathematics in Biology.” Science $50,000). The value of environmental impacts is measured 303(5659):790-793. in terms of willingness to pay by using contingent valuation NRC (National Research Council). 1998. Summarizing Population Health: Directions for the Deelopment and Application of Population Metrics. techniques and has been a source of debate by economists Washington D.C.: National Academy Press. over the years. Silver R.C., E.A. Holman, D.N. McIntosh, M. Poulin, and V. Gil-Rivas. The total social cost of a bioterrorist attack can be esti- 2002. “Nationwide Longitudinal Study of Psychological Responses mated by combining direct and indirect economic costs with to September 11.” Journal of the American Medical Association the monetization of mortality, morbidity, and environmental 288(10):1235-1244. Wessely S., K.C. Hyams, and R. Bartholomew. 2001. “Psychological Im- costs. Some critics of CBA are unwilling to attach monetary plications of Chemical and Biological Weapons.” BMJ 323(7318):878- values to life, environmental impacts, or other non-economic 879. 10 For more details on the concepts and practice of cost-benefit analysis and alternative analyses, see Boardman et al. (2001).