Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 53
6
Evaluation of Epidemic Modeling
The use of computational models for epidemic forecasting is challeng-
ing. Epidemic models are constructed by narrowing down broad scientific
understandings to specific parameter estimates and assumptions. Gaps in
scientific knowledge, limitations on data-collection resources, and the com-
plexity of the transmission processes themselves all make it impossible to
precisely predict the consequences of an infectious disease outbreak. The
very process of model construction requires simplifying assumptions that
introduce more uncertainty. The use of models to inform disease control
policies in the face of animal disease epidemics has been the subject of con-
siderable debate (Anonymous, 2001; Kitching et al., 2005, 2006; Dickey
et al., 2008; Mansley et al., 2011; Smith, 2011). Kitching et al. (2005,
2006) and Mansley et al. (2011) comment that the misapplication of foot-
and-mouth disease (FMD) epidemic forecasting can be misleading and can
promote a false sense of security. Forecasts in most fields of natural sciences
are best viewed skeptically. Despite the limitations, epidemic modeling can
be a useful conceptual resource because it forces a systematic review of all
components of an infectious disease outbreak, including critical assessment
of knowledge and uncertainty about each component.
OVERVIEW OF METHODS AND ANALYSIS
Section 6 of the updated site-specific risk assessment (uSSRA) estimates
the consequences of a potential release of FMD virus (FMDv) from the
proposed National Bio- and Agro-Defense Facility (NBAF) in Manhattan,
Kansas. As in the 2010 site-specific risk assessment (SSRA), the uSSRA uses
53
OCR for page 54
54 NBAF UPDATED SITE-SPECIFIC RISK ASSESSMENT
the North American Animal Disease Spread Model (NAADSM) in conjunc-
tion with data, statistical methods, and references from scientific literature.
Simulation outputs from NAADSM were used to evaluate the impact of
FMD spread through Kansas and into six adjoining states in different
release events. The analysis estimated the consequences of large epidemics
and the potential effects of some mitigation measures on an epidemic. De-
pending on the risk scenarios, the outputs suggested that an epidemic in the
seven states could last 18 months or more and result in the loss of tens of
millions of animals. The 2010 SSRA results were criticized by the previous
National Research Council committee for a lack of transparency, structural
limitations in NAADSM, and some specific modeling choices (NRC, 2010).
The uSSRA makes a variety of changes and attempts to address all the pre-
viously identified shortcomings of the 2010 SSRA. The revised model in the
uSSRA now estimates an FMD epidemic in these seven states to last about
twice as long as and affect several times more animals than the 2010 SSRA.
SUMMARY ASSESSMENT
The overall methodology and presentation of epidemic modeling in the
uSSRA are substantially improved compared to those in the 2010 SSRA.
Part of the reason is the uSSRA’s better description of model limitations and
uncertainty. Issues of reliability, uncertainty, and sensitivity are acknowl-
edged at the beginning of Section 6 of the uSSRA and addressed again
throughout. The breadth of epidemiological material collected in the uSSRA
could make it a useful reference for future FMD research and planning.
However, the epidemic modeling in the uSSRA still provides only a
limited picture of the likely possibilities involved in an FMD epidemic
originating in Manhattan, Kansas. Some of the limitations result from
inadequacy of available tools, including NAADSM, some from lack of
data and incomplete scientific understandings, and some from incomplete
characterization of the resources and capacity for mitigation responses.
Practical considerations have imposed a number of those limitations, as
the uSSRA acknowledges. The committee finds that the modeling results
underestimate the absolute size and duration of epidemics, in part because
of a number of specific assumptions used in the uSSRA. Overly optimistic
assumptions were made about response resources and capacities anticipated
to be available by 2020, and these in turn would lead to an underestima-
tion of the magnitude, duration, and economic impact of an FMDv escape
from the NBAF in Manhattan, Kansas. The uSSRA underestimated contact
risks and used overly optimistic parameter values for diagnostic capabili-
ties, surveillance, contact rates, vaccination, and response. Consequently,
the uSSRA spread model results incorrectly indicate foreshortened spread
and low impact estimates. The incomplete data on interstate direct contacts,
OCR for page 55
55
EVALUATION OF EPIDEMIC MODELING
including illicit livestock movements and interstate indirect contacts (fomi-
tes), would inhibit simulated movement, including secondary and tertiary
spread of virus and infected animals from Kansas to the six other states.
Considering the aggregate of design, methods, data, and assumptions,
the committee finds that the methodology as a whole lacks the overall valid-
ity necessary to predict with reasonable confidence the outcome of an FMD
epidemic emanating from an FMDv release from the NBAF in Manhattan,
Kansas. Much of the lack of validity was unavoidable, due in large part
to many ill-defined or unknown factors. These factors lead to considerable
uncertainty stemming from an absence of quality data and the vagaries of
proposed mitigation policies on the outcome of an FMD outbreak. It is also
important to note that these limitations may well lead to underestimates
or misestimates of the consequences of an epidemic, which are carried
over into the economic analysis. However, the committee strongly agrees
with the uSSRA’s broad conclusion that negative consequences of an FMD
epidemic originating in Manhattan, Kansas, will probably be severe. The
committee therefore agrees that great emphasis needs to be placed on pre-
venting release of FMDv and detecting and containing FMDv if it escapes.
METHODOLOGICAL LIMITATIONS
Limitations of the Scope of Model
The committee noted two major shortcomings related to the geo-
graphical and outcome scopes. First, the spread model incorporated only
seven states. According to the uSSRA, no suitable model for nationwide
FMD prediction is yet available. Thus, absolute impacts reported in the
uSSRA are acknowledged to be underestimates. The committee concurs
that extension of the assessment to include spread through the contiguous
United States, Mexico, and Canada would require several-fold greater ef-
fort (GAO, 2002). Second, there was no scenario involving FMD becoming
endemic. Endemic FMD would require different long-term control strate-
gies, such as a vaccination-to-live strategy, extensive laboratory testing for
surveillance, and an expensive long-term eradication program.
Limitations of NAADSM
Like all models, NAADSM provides an imperfect representation of
FMD spread and control and is based on a variety of simplified assump-
tions. As pointed out repeatedly in the uSSRA, use of only NAADSM,
without application of support models, carries a number of structural
limitations that force many ad hoc approximations to transmission and
mitigation processes, resulting in a significant decrease in the reliability
OCR for page 56
56 NBAF UPDATED SITE-SPECIFIC RISK ASSESSMENT
of simulation results under at least a subset of important conditions. The
limitations include the following:
• NAADSM can describe only regional transmission, in this case,
within a single state; it cannot account for bidirectional transmission across
state borders.
• NAADSM cannot address infection in wildlife, including feral
swine populations.
• NAADSM is not designed to include facilities that house multiple
animal species.
• The spread submodels between facilities make artificial assump-
tions about movement mechanisms and do not allow for accurate repre-
sentation of livestock movement patterns.
• There are no means of representing zoned movement controls in
response to an outbreak.
• The current implementation does not allow realistic modeling of
the livestock culling process, inasmuch as NAADSM cannot adequately ac-
count for handling times and logistic limitations (p. 451 of the uSSRA). Nor
does NAADSM allow options other than culling for the final disposition of
herds that are immune after infection (p. 478 of the uSSRA).
• The current implementation does not allow realistic modeling of
the distribution and use of vaccines during an outbreak; it does not allow
for simultaneous administration of vaccines directly by producers, and it
assumes an unlimited vaccine supply (p. 456 of the uSSRA).
• NAADSM allows users enormous latitude in defining the quali-
tative and quantitative components of transmission. This is one of the
strengths of NAADSM and also its major weakness, as it relies on expert
opinion to define components. Model outcomes are very sensitive to param-
eter assumptions, and even when expert opinions are used they can vary
and lead to wide probability distributions (Bates et al., 2003).
The uSSRA discusses those limitations and the ad hoc approximations
that they necessitated. Whereas these approximations likely prevent devel-
opment of accurate and nuanced understandings of the consequences of
variation in the logistics of mitigation, they serve as reasonable placeholders
for the broad-brush results obtained in the uSSRA. Resolving these limita-
tions will eventually require redesign of NAADSM or a switch to a more
flexible simulation platform.
OCR for page 57
57
EVALUATION OF EPIDEMIC MODELING
Limitations of Available Data
The uSSRA also points out many limitations in the data available for
use in NAADSM modeling, which add to the uncertainty of the results
presented. The limitations in the data include the following:
• The relationship between dose of FMDv and probability of an
infection of an individual animal in large (e.g., thousands of animals) and
in small (e.g., less than 100 animals) herds was not clarified. The relation-
ship is expected to vary, depending on FMDv serotype and strain, animal
species, and route of exposure, as well as on the size of the herd.
• Potential exposure of Kansas State University faculty, staff, and
students; NBAF employees; and Foreign Animal Disease Diagnostic School
participants to livestock.
• Distributions and movements of feral swine and of susceptible
wildlife, such as elk and deer that may have potential for transmitting
FMDv to livestock (Rhyan et al., 2008; Moniwa et al., 2012).
• Animal movement (direct contact) and fomite movement (indirect
contact) within and among states in the region modeled and long-range
movement of susceptible animals from the region to other states.
• Data on producers who are noncompliant with state and federal
regulations regarding veterinary inspection, animal identification, and per-
mitting and documentation of animal movement for those who buy and
sell through informal arrangements and who contribute to disease spread
through comingling of livestock at non-regulated events (such as swap
meets) or illegal animal movements.
• Some data sources used in setting model parameters are not pub-
licly available, which obstructs transparency and hinders independent rep-
lication of the uSSRA’s results.
• Although the uSSRA’s livestock database created for the Manhat-
tan area is a strong data contribution for a snapshot in time, such data can
become quickly outdated with changing numbers of animals, species, and
livestock movements. The uSSRA did not reference any state or federal
documents that would describe a mechanism for accurately identifying and
updating active premises. In the face of an FMD outbreak, it will be critical
to already have in place well-validated state animal health databases, active
surveillance, and premises identification.
Dose–Response Modeling and Minimum Infectious Dose
The uSSRA uses probit analysis to estimate the population probability
of infection associated with low doses of FMDv; the risk depends on the
probability of exposure to at least one viable virion when index cases are
OCR for page 58
58 NBAF UPDATED SITE-SPECIFIC RISK ASSESSMENT
simulated. Probit analyses can provide appropriate low-dose risk estimation
for some pathogens, but the committee has concerns about the development
and use of probit analyses for FMDv in the uSSRA.
First, use of the probit model instead of other dose–response models
merits more examination and justification than was included in the uSSRA.
The [log-]probit model can in some cases underestimate dose–response
(Gale, 2001) compared with the estimates produced with the exponential or
beta-Poisson models. The committee is aware that uSSRA Appendix Section
A6.1.2.1 states (p. A6-7) that “the exponential and beta-poisson [sic] model
were also considered; however, the potential of these models to characterize
the dose-response relationship of FMDv in cattle and sheep was previously
studied by French et al. and was found to be unsatisfactory, particularly at
low doses.” The cited dose–response modeling from French et al. (2002),
however, is at odds with the text in the uSSRA, and the uSSRA does not
adequately consider the French et al. analyses or earlier work by others
(Sutmoller and Vose, 1997; Cannon and Garner, 1999) that were cited by
French et al. (2002). The uSSRA should have provided a more accurate and
transparent analysis of the cited literature and provided further details to
compare results of an exponential analysis with those of a probit analysis.
Second, it appears that relevant data from experimental studies were
excluded, and their omission may limit the range of data used in the pro-
bit estimates. Specifically, the excluded data were related to animals that
seroconverted but did not show evidence of shedding in the once-daily
sampling schemes. Those animals could be the very ones that should be
included in the probit analysis. The animals had become infected by virtue
of the seroconversion, perhaps by a low dose that resulted in short-duration
shedding that was not detected in 12-hour or 24-hour sampling intervals.
Inclusion may have improved the probit-derived probability estimates of
low-dose infectivity.
Third, the committee is concerned about continued use of the “mini-
mum infectious dose” (MID) concept. The uSSRA states on p. 408 that
Many researchers have proposed that there is no risk of infection for doses
of FMDv lower than a certain amount, called the ‘minimum infectious
dose’.... These values might represent a phenomenon in which a minimum
number of pathogen particles are required to overcome host defenses and
establish an infection, or they could be an artifact of the use of a small
number of animals in infection experiments (i.e., if five animals were used,
identifying doses that cause less than a 20% probability of infection is
difficult).
The latter argument is legitimate in that these experiments have sample
sizes that are statistically inadequate to estimate the risk of infection at low
doses (Haas et al., 1999; NRC, 2005). However, the “minimum infectious
OCR for page 59
59
EVALUATION OF EPIDEMIC MODELING
dose” concept is not credible. The comment about researchers proposing
no risk below a particular threshold is related mainly to the older mi-
crobial risk-related literature. Recent work (e.g., Haas et al. 1999; Gale,
2001) typically applies dose–response modeling with the best available data
(possibly including meta-analyses) and extrapolates low-dose with probit,
beta-Poisson, exponential, or other dose–response models. For pathogens
on which reliable data for dose–response analyses are available, there is no
population threshold dose (NRC, 2005).
Fourth, the uSSRA does not provide an adequate discussion of the un-
certainties in the FMDv dose–response modeling using the probit model or
its alternatives nor does it provide an adequate discussion of their applica-
tion to predicting herd response. Failure to do so may leave the impression
that the dose–response predictions used in the probit model are highly
certain, and this is not the case. The statistical reliability of dose–response
modeling is briefly discussed, but its impact on the results is not adequately
analyzed. Results could be sensitive to uncertainties such as FMDv strain
differences, experimental dosing regimen (often bolus) compared to the
potential herd exposures resulting from a leak, and differences between
the experimental animals’ status and that of the target animal herds (e.g.,
species or breed, immune competency, concurrent infections, environmental
stresses). The direction and magnitude of these effects may be unknown for
FMDv, but they nevertheless remain as uncertainties in the extrapolation to
herd response that were not adequately addressed in the uSSRA.
Assumptions About Available Response Resources and Capacities
The uSSRA makes various assumptions about foreign animal and zoo-
notic disease response capabilities presumed to be in place at the time of
the anticipated NBAF opening in 2020. It will be important to have these
tested capabilities in place from day 1 to mitigate the effects of an acciden-
tal release of an infectious agent. The committee notes that many of these
assumptions are unrealistic today and that making them realistic would
require major investments and considerable political will before the NBAF
opens. Whereas the uSSRA does not discuss future investment require-
ments, it does acknowledge that capabilities will be changing over the next
8 years. Concerns about the assumptions related to capabilities include the
following:
• Vaccination would begin very early (on day 7) in an FMD epi-
demic. Also, once vaccination is initiated, single-dose, high-potency, 100%
efficacious emergency vaccine would be available in unlimited quantities.
It is further assumed that 100% of vaccinated animals would be protected
from infection. These assumptions would apply for all 7 serotypes of
OCR for page 60
60 NBAF UPDATED SITE-SPECIFIC RISK ASSESSMENT
FMDv and for all the strains within each serotype that could escape from
the NBAF. These assumptions are inconsistent with the current state of
knowledge.
• It would take 3–11 days to vaccinate all herds in Kansas (an aver-
age of 90 herds per day).
• 100% of cattle and swine producers will report a suspicious case
in less than 2 weeks following infection; this is unrealistic.
• Laboratory testing capacity for the presence of FMD virus (RRT-
PCR [Callahan et al., 2002]) and virus isolation and for the presence of
antibody is assumed to be unlimited, with a range in the turnaround time
for testing of only 0–2 days. Laboratories in the National Animal Health
Laboratory Network currently do not have the capability to conduct sero-
logical tests for FMD.
• Diagnostic laboratory tests are assumed to be of exceptionally high
accuracy and reliability, and perfect accuracy is assumed in detecting FMD
on 100% of infected premises.
Furthermore, the lack of real-time FMD surveillance, as acknowledged in
the uSSRA, diminishes the likelihood of early detection and control.
The uSSRA states that “economic estimates based on the outputs of
the economic model for the Updated SSRA will, again, underestimate the
absolute impact of the outbreak of FMD originating from the NBAF be-
cause the outbreak is artificially limited to the region modeled instead of the
whole of North America” (p. 405). Many of the limitations listed above are
also likely to result in underestimation of the extent and cost of a potential
release of FMDv from the NBAF in Manhattan, Kansas.
Other Sensitivity Analysis
The epidemic modeling section provides the only sensitivity analysis
that has any degree of rigor in the three volumes of the uSSRA. This section
of the uSSRA provides a correct and important caveat about the useful-
ness of the estimates (p. 534):
This analysis informs how much confidence can be placed in the results
as absolute reflections of the impact of an FMD outbreak given that some
of the modeling parameters are based on scanty evidence. As discussed,
epidemiological models are best used to understand relative risk and rela-
tive benefit of risk mitigation measures because inaccuracies in a model
are reflected in the baseline and experimental cases, largely cancelling each
other out.
The uSSRA further discusses that variation in the contact rate of less
than an order of magnitude (a factor of 0.5–2) changes the duration of an
OCR for page 61
61
EVALUATION OF EPIDEMIC MODELING
epidemic by over an order of magnitude (see pp. 537–538). However, many
parameter values have greater uncertainty—with ranges that span several
orders of magnitude. Distributions based on these wider ranges should
have been provided in the sensitivity analyses because that would provide
better information on the most important components of uncertainty in
the results.
REFERENCES
Anonymous. 2001. Lessons from an epidemic. Nature 411(6841):977.
Bates, T.W., M.C. Thurmond, and T.E. Carpenter. 2003. Description of an epidemic simulation
model or use in evaluating strategies to control an outbreak of foot-and-mouth disease.
Am J Vet Res 64(2):195-204.
Callahan, J.D., F. Brown, F.A. Osorio, J.H. Sur, E. Kramer, G.W. Long, J. Lubroth, S.J. Ellis,
K.S. Shoulars, K.L Gaffney, D.L. Rock, and W.M. Nelson. 2002. Use of a portable real-
time reverse transcriptase-polymerase chain reaction assay for rapid detection of foot-
and-mouth disease virus. J Am Vet Med Assoc 220(11):1636-1642.
Cannon, R.M., and M.G. Garner. 1999. Assessing the risk of wind-borne spread of foot-and-
mouth disease in Australia. Environ Int 25(6-7):713-723.
Dickey, B.F., T.E. Carpenter, and S.M. Bartell. 2008. Use of heterogeneous operation-specific
contact parameters changes predictions for foot-and-mouth disease outbreaks in complex
simulation models. Prev Vet Med 87(3-4):272-287.
French, N.P., L. Kelly, R. Jones, and D. Clancy. 2002. Dose-response relationships for foot and
mouth disease in cattle and sheep. Epidemiol Infect 128(2):325-332.
Gale, P. 2001. Developments in microbiological risk assessment for drinking water. J Appl
Microbiol 91(2):191-205.
GAO (U.S. Government Accountability Office). 2002. Foot and Mouth Disease: To Protect
U.S. livestock, USDA Must Remain Vigilant and Resolve Outstanding Issues. Available
online at http://www.gao.gov/new.items/d02808.pdf (accessed March 30, 2012).
Haas, C.N., J.B. Rose, and C.P. Gerba. 1999. Quantitative Microbial Risk Assessment. New
York: John Wiley.
Kitching, R.P., A.M. Hutber, and M.V. Thrusfield. 2005. A review of foot-and-mouth disease
with special consideration for the clinical and epidemiological factors relevant to predic-
tive modelling of the disease. Vet J 169(2):197-209.
Kitching, R.P., M. Thrusfield, and N.M. Taylor. 2006. Use and abuse of mathematical models:
An illustration from the 2001 foot and mouth disease epidemic in the United Kingdom.
Rev Sci Tech 25(1):293-311.
Mansley, L.M., A.I. Donaldson, M.V. Thrusfield, and N. Honhold. 2011. Destructive tension:
Mathematics versus experience—the progress and control of the 2001 foot and mouth
disease epidemic in Great Britain. Rev Sci Tech 30(2):483-498.
Moniwa, M., C. Embury-Hyatt, Z. Zhang, K. Hole, A. Clavijo, J. Copps, and S. Alexandersen.
2012. Experimental foot-and-mouth disease virus infection in white tailed deer. J Comp
Pathol Apr 18.
NRC (National Research Council). 2005. Reopening Public Facilities After a Biological
Attack: A Decision-Making Framework. Washington, DC: The National Academies
Press.
NRC. 2010. Evaluation of a Site-Specific Risk Assessment for the Department of Homeland
Security’s Planned National Bio- and Agro-Defense Facility in Manhattan, Kansas. Wash-
ington, DC: The National Academies Press.
OCR for page 62
62 NBAF UPDATED SITE-SPECIFIC RISK ASSESSMENT
Rhyan, J., M. Deng, H. Wang, G. Ward, T. Gidlewski, M. McCollum, S. Metwally, T.
McKenna, S. Wainwright, A. Ramirez, C. Mebus, and M. Salman. 2008. Foot-and-
mouth disease in North American bison (Bison bison) and elk (Cervus elaphus nelsoni):
Susceptibility, intra- and interspecies transmission, clinical signs, and lesions. J Wildl Dis
42(2):269-279.
Smith, G. 2011. Models of macroparasitic infections in domestic ruminants: A conceptual
review and critique. Rev Sci Tech 30(2):447-456.
Sutmoller, P., and D.J. Vose. 1997. Contamination of animal products: The minimum pathogen
dose required to initiate infection. Rev Sci Tech 16(1):30-32.
