National Academies Press: OpenBook

The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop (2004)

Chapter: Detection and Epidemiology of Bioterrorist Attacks

« Previous: Data Mining, Unsupervised Learning, and Pattern Recognition
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 135
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 136
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 137
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 138
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 139
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 140
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 141
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 142
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 143
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 144
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 145
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 146
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 147
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 148
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 149
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 150
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 151
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 152
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 153
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 154
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 155
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 156
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 157
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 158
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 159
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 160
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 161
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 162
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 163
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 164
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 165
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 166
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 167
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 168
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 169
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 170
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 171
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 172
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 173
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 174
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 175
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 176
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 177
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 178
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 179
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 180
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 181
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 182
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 183
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 184
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 185
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 186
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 187
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 188
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 189
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 190
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 191
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 192
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 193
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 194
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 195
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 196
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 197
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 198
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 199
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 200
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 201
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 202
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 203
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 204
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 205
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 206
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 207
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 208
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 209
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 210
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 211
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 212
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 213
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 214
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 215
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 216
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 217
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 218
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 219
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 220
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 221
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 222
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 223
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 224
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 225
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 226
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 227
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 228
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 229
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 230
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 231
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 232
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 233
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 234
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 235
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 236
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 237
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 238
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 239
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 240
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 241
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 242
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 243
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 244
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 245
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 246
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 247
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 248
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 249
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 250
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 251
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 252
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 253
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 254
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 255
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 256
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 257
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 258
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 259
Suggested Citation:"Detection and Epidemiology of Bioterrorist Attacks." National Research Council. 2004. The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10940.
×
Page 260

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

135 CIaire Broome "introduction by Session Chair" Transcript of Presentation Summary of Presentation Video Presentation Dr. Claire Broome serves as the senior advisor to the director for integrated health information systems at the Centers for Disease Control and Prevention (CDC). Dr. Broome oversees the development and implementation of CDC's National Electronic Disease Surveillance System, one of the highest priorities of CDC and the administration. Dr. Broome served as deputy director of the CDC and deputy administrator of the Agency for Toxic Substances and Disease Registry (ATSDR) from 1994 to 1999; as CDC's associate director for science from 1990 to 1994; and as chief of the Special Pathogens Branch in the National Center for Infectious Diseases from 1981 to 1990. Her research interests include epidemiology of meningitis and pneumonia; meningococcal, pneumococcal, and Haemophilus b vaccines; observational methods for vaccine evaluation; and public health surveillance methodology. Dr. Broome has received many professional awards, including the PHS Distinguished Service Medal, the Surgeon General's Medallion, the Infectious Disease Society of America's Squibb Award for Excellence of Achievement in Infectious Diseases, and the John Snow Award from the American Public Health Association. She was elected to membership in the Institute of Medicine in 1996. She graduated magna cum laude from Harvard University and received her M.D. from Harvard Medical School. She trained in internal medicine at the University of California, San Francisco, and in infectious diseases at Massachusetts General Hospital. 135

136 DR. BROOME: Good afternoon. Let me just go ahead and get started with the afternoon's first session. I am Claire Broome, medical epidemiologist at the Centers for Disease Control and Prevention where I have been involved in actually getting the data that you all would like to have as part of the targets for your modeling and simulations, and I have worked closely with a lot of the folks working on bioterrorism preparedness, and I will be moderating this session and what we will do is go through the first presentations and then try to pull some of this together and have a more interactive session during the discussant time period. We hope there will be some time for questions to the presenters as we go along but that will depend on how much the presenters keep to time. So, the first presenter is Ken Kleinman from the Harvard Medical School, and he will be talking ambulatory anthrax surveillance, an implemented system. 136

137 Introduction by Session Chair Claire Broome Dr. Broome introclucect herself as a mectica] epictemio~ogist at the Centers for Disease Control and Prevention and saint that she is invo~vect in obtaining a good amount of the data used in bioterrorism moclels and simulations. She is also involved with scientists who are working on bioterrorism preparedness. 137

138 Kenneth Kleinman "Ambulatory Anthrax Surveillance: An implemented System, with Comments on Current Outstanding Needs" Transcript of Presentation Summary of Presentation Power Point Slides Video Presentation Kenneth Kleinman is assistant professor in the Department of Ambulatory Care and Prevention, Harvard Medical School and Harvard Pilgrim Health Care. He serves as the main biostatistician on three CDC-funded projects to implement surveillance of health care system utilization in the Boston area and nationally, and he works with the national BioSense project. His interests include the analysis of longitudinal and other clustered data, epidemiologic methods, and missing data problems. 138

139 DR. KLEINMAN: Thank you. After the generality of this morning's talk I feel like this subtitle here is kind of grandiose, but I am a biostatistician and I am going to actually going to show you real data and describe a system that is running now in Boston, and there are a lot of people involved with getting the system running. It is actually a collaboration between the academic department, an HMO care provider and the State Department of Public Health. So, all these folks are involved in various aspects from those places. My two co-authors on the statistical part of what I am going to talk about today are Ralph Plasers and Rich Plott who are infectious disease epidemiologists. So, here is the outline for the talk. I am going to talk about why surveillance is important especially for anthrax and then I am going to talk about the data that we have and where we are, and I have tried to organize the talk around problems and our approaches to those problems that we encounter while trying to set up the surveillance system and where we should go in the future. So, why is surveillance for anthrax important? Anthrax is what they call a biphasic disease and what happens is you get exposed. You have no symptoms for a 139

140 while. Then you have symptoms that are very non-specific and they resemble a cold or flu and that happens within a couple of days and actually we don't know a whole lot about anthrax because there have only been about 40 cases in the United States this century including last October in humans I should say. So, there is not a whole lot known about it but it is supposed that most people have symptoms within a couple of days and then a day or two after you have those symptoms you start having really severe symptoms like severe sweating and breathing problems and eventually shock, and if you are not treated then there is death in 98 percent of cases and there is even death in some cases where there is treatment. So, what are you going to do if you have anthrax? Well, nothing when you get exposed and when you start having symptoms you might go see your doctor and if you don't go see your doctor then or if you don't get diagnosed correctly then when you start having the more severe symptoms in the second phase you probably go to the hospital and get ciprofloxacin or another approved treatment. So, there are a bunch of surveillance systems that are up and running now in the country that are based 140

141 on hospital surveillance, and what they do is they wait around in the hospital and they see if there are too many emergency room visits or too many diagnoses of anthrax and what can they do if they detect anthrax? Well, there are lots of people already in Phase II of the illness, and they are very sick and they are going to be arriving in the hospital in large numbers. So, at least you can be ready for them to come. So, you introduce some good if you do surveillance on that basis, but it would be better if we could detect people when they visit their doctor instead because that would be the earliest time that we would know about it, and also then we could just break out the drugs and prevent people from even entering Phase II and probably save them a lot of discomfort and problems and even lives. So, our data, our study, as I mentioned we are trying to do this. Our position is located between a health maintenance organization and a provider group and also we have, we are in an academic department. I am in an academic department. So, the people who are part of this group, this care group and HMO, there are about 250,000 people in a certain area of Massachusetts, and that is about 10 percent of the population in that area. So, what is very nice about this is that we can actually attempt to do that surveillance on the doctor's 141

142 visit before people get to the hospital because the provider group uses ambulatory medical records data and what that means is that every time you go to your doctor's office they actually have a PC in each examining room, and they will type in information about you including an ICD-9 diagnosis, and that is just some coding system for a diagnosis if you are not familiar with it, but it is symptoms and confirmed diagnosis, and they are continuously updated. They are centrally stored by the provider group, and we don't have to do anything different from what they are already doing, and it is standard practice to record all the diagnoses they make for each person who comes to see them,and that includes phone calls and nurse practitioners and physicians, and the system is actually a commercial system. So, it is relatively easy for us to take our system which is in Eastern Massachusetts and transport it to some other locale where they happen to use the same medical records system and there are a bunch of them. PARTICIPANT: Does this system store all the medical record? DR. KLEINMAN: It stores all the medical record. Things like pharmacy data and test results aren't always updated in the same system. So, it is all the direct 142

143 patient contact between the physician and the patient. Does that answer your question? PARTICIPANT: Lab results? DR. KLEINMAN: Lab results are not -- PARTICIPANT: No, would have been a good answer to my question. DR. KLEINMAN: Okay, I guess I wasn't clear enough on you question. Now, I am going to start talking about the problems we encountered when trying to set up the surveillance, and the first one is that it actually uses adult standard test for anthrax, and you have to get a chest x-ray, and I don't know enough about medicine to know what about this x-ray says that there is anthrax here, but if we actually waited for chests x-rays we wouldn't do any better than hospital surveillance because the physicians don't order chest x-rays for people who come in with cold symptoms. So, we can't actually do surveillance for diagnosis of anthrax, and so what we do instead is we define symptom clusters or syndromes, and what we do is we say that anything that your doctor might say if you came in with the first phase of anthrax problems we are going to try to collect that, and we call that lower respiratory illness or I might say lower respiratory infection later 143

144 because those are the symptoms that characterize the first phase of anthrax infections. That includes, I listed cough, pneumonia and bronchitis because about 90 percent of the diagnosis falls in one of those categories in this syndrome which includes about, I think it wrote it down here, 119 ICD counts go in there and that syndrome we actually borrowed from a Department of Defense product that was doing this before we were in a slightly different context. So, in our data set in about a 4-year period there are about 120,000 visits that found this syndrome, and if you think about a natural disease unlike anthrax if it doesn't get fixed the first time you go to your doctor you are going to go back again, and when we looked at our records very nearly about one-third of them were repeat visits that were shortly after other visits. So, based on the clinical expertise of the MDs working with us we were able to say that if it was within 6 weeks of a previous visit, the previous visit we weren't interested in it. It was probably the same illness, and so we throw those away, and what is left we call episodes of lower respiratory infection. PARTICIPANT: How many don't visit with the same illness? 144

145 DR. KLEINMAN: I have no idea. This is a very small portion of the doctor's visits. I don't know that information. It is a completely different order of magnitude. I don't know the answer. So, here is actual data and each dot here represents the count of visits for their respiratory infection on a given day in between January I, 1996 and December al, 1999 or January I, 2000, and I just want to point out a couple of interesting features here that will come into play later, and the first is that you can see that during the winter there are big peaks. People tend to go to their doctors for respiratory complaints more in the winter than they do in the summer presumably because they are more likely to have those symptoms and the other interesting feature is that you can see there are two trends here, and our clinics are actually open on the weekends. So, this lower band is the weekend visits and the upper band is the week day visits, and finally, I don't know how clear this is from here but you can see these points, really low here in winter. That is Christmas and New Yearls. So, people don't go to the doctor on holidays but they will go as often on the weekends, and they tend to go an awful lot more in the winter than they do in summer. 145

146 PARTICIPANT: What are the dots? DR. KLEINMAN: I am sorry. Each dot is the count of the number of visits for LRI on a given day. So, this is about 130. So, there are 130 visits among our 250,000 numbers for lower respiratory illness on I don't know what day this is, probably like May I, 1999. PARTICIPANT: Oh, I see. So, only one doctor per vertical slice. DR. KLEINMAN: Yes, that is right, at least in theory that is right. I am not sure that the resolution on the screen is up to that, but -- PARTICIPANT: Presumably the double banding begins in May? DR. KLEINMAN: Yes. So, if can see data like this you might be thinking I should do a time series analysis and I need to include a covariant for weekend versus week day and maybe some kind of yearly cycle for seasons, and I think there are a couple of problems with this, and the first one is that you are kind of constrained to get a threshold for the whole area under surveillance, and the real big problem is you can send cipro to 2-~/2 million people. It is going to break the bank and plus there are bad side effects of cipro so you don't want to dose that many people who don't need it, and there are other problems 146

147 with dosing that many people, and the other problem is it is not very sensitive at all, and the best, I know people who are doing this, and the best that I have heard that they can do is, you know, if there are 10 additional cases on a day they might detect it, and that is an awful lot. You don't want to have to have 10 initial people if you can avoid it. So, what we do to solve the problem is we actually geo code everyone, and geo coding just means that we take their address and we find out where on the map it lies, and it is actually remarkably easy to do, and accurate and cheap. We outsource it, and it costs us, I don't know 4 or 5 thousand dollars to do 250,000 addresses, and it comes back within a week. So, it is pretty slick. We only know the billing address. So, if people are exposed at the place where they work we are not going to know anything about that. We are working on trying to solve that problem, but it is kind of a feasibility issue because we don't really If the country that is the employs them. know where people work. they work for IBM they could be anywhere in instead of at some particular location, and kind of work address that we would get is who 147

148 So, the coding changes a little bit, and so it is not perfect but it is a lot better than not knowing where in the area they live. So, this is the kind of thing I get day and here I took the plot of the number of had LRI on a given day and their location and I kind of three-dimensional smootherometer(?) dimensional smootherometer and so, I am looking peaks, and you know there are probably one or two this location. PARTICIPANT: What are the axes? DR . KLEINMAN: Sorry, this is latitude and longitude. So, it is an arbitrary grid on space but these actually, if you were to look at this latitude and longitude point you would be able to find it in the middle of Massachusetts Bay. PARTICIPANT: Is it adjusted for population? DR. KLEINMAN: This is not adjusted for populations, no. This is just raw counts, but if I were adjusting for population the question would be then, if I got five bioterrorism I would say, "Is this point too high? Is this bump too big?" and that is kind of the job that we are trying to do now. PARTICIPANT: This was one day? 148 on a given people who used some or two- at these cases at

149 DR. KLEINMAN: That was just one day, yes. I have more slides that I could show about what happens. So, we have this data that is spread out in space and there is a whole bunch of spatial techniques. Most of them are developed for mining or for epidemiological exposures and so they don't actually think of time as an interesting dimension. They tend to summarize over time and say, "Is there a bump somewhere near say a contaminated well or near some sort of a high concentrated world where they are looking for something in it?" But we really need to know whether there is an event now as much as we need to know where it was, and there is a couple of so-called "special" techniques that are kind of intended for surveillance, and this touches on the same issue of the World Statistical Society, these two references and they are really designed to say, "Is there a bump right now?" and i don't think that either of these is really important for what we want to do either because they both assume that the time periods are similar to one another. So, they wouldn't be able to fit a covariate saying that this count happened during the winter; this count happened on Monday; this count happened on Christmas and be able to adjust for that, and they are, also, both designed for smaller data sets although I think that is probably not a big problem. 149

150 Now, the really big problem with these things though is that they are looking for clusters now and if you recall the problem that we have we are trying to say that we know there are clusters of flu because flu is contagious and flu symptoms are what we count. So, we know there is going to be a cluster on any given day especially during winter, but we know that there is going to be a cluster and we want to know whether the cluster we are seeing is bigger than the cluster we expect as opposed to whether there is a cluster, not whether there is a bump but is the bump bigger than the bump we expect based on the clustering there has been in the past. PARTICIPANT: So , the difference is anthrax is not transmitted. DR. KLEINMAN: That is true, yes. So, anthrax won't spread from person to person. So, what we have done here and I want to echo what some of the speakers were saying this morning, I am not saying that this is the best way to do this, but it is a way that seems to work, and that I was able to do and I think is easy to transport and those were qualities that I was looking for. What I am going to do is I am going to again more or less arbitrarily break up space into localities that make sense to me and the ones that make sense to me are

151 census tracts. Census tracts are defined by the Census Bureau, but they are meant to be somewhat meaningful. You know, they don't cross a highway in general. They are kind of neighborhoods as you might define a neighborhood near your house and they are actually defined by people locally, and so, what we have now is a bunch of longitudinally repeated data. We have a denominator and numerator for every census tract in our area over time, and what you do is use the generalized linear mixed model approach, logistic regression, and of course I could use a Poisson(?) regression instead, too, but I didn't do that. It is really just logistic regression, but it takes into account the correlations of counts on the same tract on different days. So, I could fit the model using this GLIMMIX macro that actually is distributed with SAS. It is not officially part of SAS. So, it is somewhere in limbo there, but it is part of your distribution. If you use SAS you can find it there and that means that it is easy to explain to someone else how to do it and see that they get the same results you would on a test data set for example, and with this kind of data of course I am not sure, as an aside but I could use generalized nesting equations to fit the data but I would to get a separate estimate for each tract for 151

152 purposes of explaining what the results are, and just because I am a statistician I have to show some notation and really what I am doing is just a logistic regression. I am trying to model the binomial distribution for tract I in time T and I take a model with some covariates usually on the time and a random effect that applies to each census tract, and usually assume that those random effects are distributed normally in some variance. That is all I am going to show you about that really. There is a lot of validity to the model that we get out of it. The observations for the winter months are much bigger than the ones for the summer months. They are bigger on Mondays and lowest on the weekends and the observations for holidays are less. So, what we are doing makes sense and the random effect might be features of each tract. So, if there are tracts where there are more people or there is more people that are prone to sickness or fewer people who are prone to sickness we are going to find that through these random effects and when we test for variance of those random effects we find out it is not zero. There really is a variability between the communities. So, it is worth doing that in the model, and if you are interested in interpretation you can actually estimate the random effect 152

153 that goes with each tract and those things are the odds ratios relative to the average tract. Now, another question that was brought up this morning is what are you going to do with the data once you have got it and what are you going to do with your model once you have it, and we are working with folks at the Mass Department of Public Health and we want to be able to communicate with them when we think something strange is happening and what we do with that is we can actually invert the estimate for each census tract and turn it into a PHAT(?) for each census tract on each day and then we can calculate the probability of seeing as many cases as we saw or more and the P value for the null hypothesis of the data comes from the binomial distribution we got from the model. Now, there is a problem with that. The big problem is that we have 520 census tracts in our area and the estimated P value for each one each day and that means we have IS0, 000 tests each year. So, that is a lot. The folks who were talking about data mining this morning, I don't know enough about data mining to know whether doing innumerable tests is a problem there. We are going to do, our proposal is to do a Bob Ferroni-like thing and what we do is actually we record the estimated number of years of testing required to see one P value as small as the one we 153

154 saw or smaller which is just the inverse of the number of tests we are doing times the P value that we got, and that is of course assuming that all the tests are independent which is not true, but it is better than nothing and one of the advantages of doing it this way is that when you get a big number it is bad, and we don't have to rely on people remembering that a small P value means something unusual. We can say that we would only expect this to happen once every 100 years. We think people will probably understand that pretty quickly. So, this is the point that actually we distribute it by means of a web site every day. We report the name of the town and the census tract number and I didn't mention earlier that those things are actually not determinants. Census tracts can bridge more than one town and of course multiple census tracts would be included entirely in one census tract. We also report the number of cases in that town or that census tract on that day, the number of insured people who live there and then the statistical -- I am actually going to work years between those counts and report the five most unusual counts and if it happens, if we expect those counts to happen once a day we just say that it has got a score of zero. It is really nothing to worry about at all. 154

155 Now, .004 years is like a day and one-half. So, we found a very interesting example, but there aren't more interesting days than that and actually the last time I did it it was something like this, I think it was March 21. So, I was able to say that a week ago we had this event that happens about once every 55 days, and that was infrequent enough that our department of public health is very interested and they sent a team, but that is a motion that we are trying to shorten where they contact people at the health plan or the care provider and figure out whether there is something really unusual about those cases or you know, if they all happen to be people in the same family with the same cold you probably wouldn't worry about it. So, they are able to investigate that. What I really want to point out about this is that this is the number of cases we saw in that census tract on that day and if there are only four of them we can be very sensitive here. We don't have to wait for 10 cases to show up the way you would have to do in a regular risk analysis. So, I am close to the end, and what I think is a really big unsolved problem for us and I have a very easy solution to it, but an event can take place over more than one census tract, and when it does our models aren't going 155

156 to know about that and so we rely on people looking at the following plot and this is automatically generated by the SAS program but we can say, "I think that these two census tracts are close enough together that I am concerned that the cause of that disease might be the same exposure to anthrax." It relies on someone actually looking at the plot every day. So, it is not a very good solution. So, now I am on to what I think are four things to do here and again this is a very different scale of importance compared to the coin toss but I think this question of what to do about cases in adjacent tracts is a big problem and I think that one approach would be to incorporate some kind of secondary initial test and I think that it would be easy to incorporate spatial correlation in the census tracts but you are not going to change the answer much. Flu is bad in some years and not bad in other years and it would be in theory easy to incorporate some measure of whether this season is a bad season for flu or a not so bad season for flu. Logistically it is hard to do that for various reasons, but I am not a cryptographer so I am sure that it is a great idea. We are using census tracts as our neighborhoods. We could use census block groups. We could use a truly 156

157 arbitrary grid on space that wouldn't take into account anything special about the boundaries and this is something that definitely should be assessed and I think that it would be good if we could figure out how to incorporate individual level covariates because when a 20-year-old person goes to their doctor complaining of respiratory infection it means something a lot different than when an BO-year-old person does that. Finally, I want to disclaim any collusion between me and the next speaker but I think that the biggest problem is figuring out how to simulate data because there is really no way to test the model sensitivity right now and we need to be, what we really need to be able to do is simulate the background noise which is flu and see whether we can detect and kind of solve the simulated data with anthrax and see whether we can detect additional cases. So, that is the end. I just have a summary which has the -- I don't think what we are doing is great. I think it solves some of the problems that other approaches that have already been developed don't solve, and I think that the most important thing to do is to try to assess the sensitivity of the model and we need to simulate a background in order to do that. Thank you. 157

158 (Applause.) PARTICIPANT: I just want to follow up on an earlier question about anthrax is not contagious. It didn't seem that the model incorporates that knowledge. DR. KLEINMAN: I think you are probably right. DR. BROOME: But what the model is trying to do is detect an increase that is different from your background expected. It doesn't matter, and in fact it is a good model for anthrax because if you got a point source for leads you will see cases all over the place and you want to pick them up as rapidly as possible. It doesn't matter whether they have then self-propagate. PARTICIPANT: But the lack of self-propagation is a significant signature that would help statistically distinguish anthrax attack from just a coincidental -- DR. BROOME: If you have to wait until you are picking up second waves that would be the case, but in fact the idea is to try to pick it up from that first release. PARTICIPANT: Oh, so, this is something I don't know. When people have the flu what is the cycle? DR. BROOME: One of the problems with using flu for modeling is it doesn't have, you know, well, I will tell you what. Rather than get into all the details why don't we wait until we have heard from Stephen and Sally 158

159 because I think they raise a number of the issues around disease modeling and we can get back to this question which is interesting but I think trying to put it in the context of what are we trying to do with these different models would be valuable. So, why don't we move ahead? Any other sort of clarification questions for Ken? Why don't we try to do that at this point. PARTICIPANT: Do you report your results to the disease center? DR. KLEINMAN: We don't do any of the reporting ourselves, meaning when I say, we, I mean the academic department. What happens to the data is it goes on the web site and I look at the plots and I send out an e-mail if I think there is something interesting about the plots, and the people at the Mass DPH decide what to do about it. So, they could decide to report it to the CDC or other organizations at that point. DR. BROOME: The real question is have you notified alerts to the health department and has anything been found that had any public health significance. What we are talking about here is just a statistical signal and assessing the meaning of that requires investigation by 159

usually either the health care providers or by the public health department neither of which has a lot of excess people sitting around waiting to investigate, and that is why the point Ken was making about being sensitive to small numbers of cases but at the same time minimizing the number of false alarms is really a critical characteristic. DR. KLEINMAN: And that is, also, why we don't send for the false alarm model, but I understand your question, but the answer to your question is there have been several times when a relatively unusual kind of event happened and that once every 55 days may have been the most extreme that has happened since we started running this in late October. That one may have been the most extreme but none of the ones that we have looked at have actually had public health ramifications, fortunately. So, we see about the number, approximately the number we expect of once-a- month events and because they are the expected events they are not interesting for public health. PARTICIPANT: The communication is actually not the focus indicated. DR. KLEINMAN: The communication has flaws. You know that is the most difficult part because it involves people on both ends and it requires someone at the DPH to be answering their e-mail and contacting someone at the 160

161 health care provider and that person to contact someone else to get the data on individual people. So, that is definitely something that we are working on, but it is not perfect yet. DR. BROOME: Does that answer your question? PARTICIPANT: No. DR. BROOME: What are you -- PARTICIPANT: I am looking at the time that an incident would take place for it to reach the Communicable Disease Center because if you saw this happening in one location you might be getting information from other locations. So, I am interested in this time line from a . · . SUSplClOUS evens. DR. BROOME: Well, to me that is an important question but it is secondary to knowing that this is actually useful information, and I think there is some very serious need for evaluation of whether this kind of very non-specific signal going off all the time is a useful thing to communicate. So, I mean we definitely want to hear about true signals, but in fact the critical need is to say does this have any potential real relevance. I mean just because you have got a whole bunch of flu patients who show up in a census tract, that happens a lot. 161

162 DR. LEVIN: So, if you are really interested in bioterrorism incidents wouldn't it be important to look at the genetic distance of the strains that are appearing from the conventional case from the year before because if the strains are only slight deviations from last year's in the area then it is more likely to have arisen naturally but if it has been an engineered strain -- DR. BROOME: I think there is a misperception here. We are not trying to find genetically engineered flu. We are trying to detect a non-specific febrile illness which might be misdiagnosed as flu. So, you are trying to separate out an anthrax, a febrile smallpox rash from the background noise of a normal influenza season and there is a whole lot more background noise than there is anthrax. Is that a fair summary? DR. KLEINMAN: Yes, obviously a fascinating question and one that we wouldn't be able to say anything about. DR. BROOME: I think people are really engaged and interested. Why don't we go ahead and get our presentations laid out because I think there is some very interesting information that will help frame the discussion, and Stephen Eubanks is going to discuss the 162

163 mathematics of planning . epidemiologic simulation for response He is at Los Alamos National Laboratory. 163

164 Ambulatory Anthrax Surveillance: An Implemented System, with Comments on Current Outstanding Needs Kenneth Kleinman Because the first symptoms of anthrax are similar to those of influenza, bronchitis, and other common illnesses, patients may or may not see their doctors at the onset of symptoms. However, with more severe symptoms occurring in the second phase of the disease, patients often seek treatment at a hospital. Currently, there are surveillance systems in hospitals to track anthrax occurrences. However, if patients could be ctiagnosect with anthrax upon seeing their ctoctors with the first symptoms, they couIcl be given treatment immecliately, be saved a good clear of discomfort, and never have to enter the second phase of the disease. The Harvard Medical School, an HMO, a care provider, and the Massachusetts Department of Public Health have impiementect a system to ctetect outbreaks of anthrax in the ctoctor's office instead of in the emergency room. In this system, mectical records are gathered by the provider for every patient visit and are continuously upciatect and centrally stored for use by the provider and the survei1Iance- system operators. The system tracks symptom clusters for lower respiratory illness, symptoms that characterize the first phase of anthrax infections. Of course, this inciuctes the symptoms of other, much more common illnesses as well. The symptom clusters are compared with symptom cluster records of the same census tract from the past to assess whether a given cluster is larger than what is expected. A report is issued daily on the Internet that gives each census tract, the number of patients who five there, and the number of years between patient counts of the given magnitude. For example, if a particular count is so common it can be expected to happen once a clay, it gets a score of zero. On the other banal, if we say that we wouicl only expect this to happen once every 100 years, people will probably understand that pretty quickly. In this way, the doctors communicate with the Department of Public Health when they think something strange Is happening. There are a few ways in which the system couIct be improved. One improvement wouIct be to ctetermine whether results from several census tracts are really just manifestations of the same overall exposure event. Another wouIct be to finct the optimal type of basic geographical unit census block groups, for example, or even an arbitrary grid and compare it to the census-tract groupings that are currently usecl. It wouIcl be good if we could figure out how to incorporate inctivicluaI-Ieve] covariance, because when a 20-year- oIct person goes to the doctor complaining of respiratory infection, it means something a lot different than when an 80-year-oicl person does that. do, do, . The biggest problem is figuring out how to simulate data, because there is still no other way to test a moclel's sensitivity. It is clesignect to detect outcomes that stiffer from the expected background, so the thing to do is to simulate the background noise that is, 164

165 cases of flu anct the like anct see whether the mocle! detects any simulatect anthrax data embecictect in that noise. 165

166 Stephen Eubank "Mathematics of Epidemio~ogica~ Simulations for Response Planning" Transcript of Presentation Summary of Presentation Power Point Slides Video Presentation Stephen Eubank received his B.A. in physics from Swarthmore College in 1979 and his Ph.D. in theoretical particle physics from the University of Texas at Austin in 1986. He has worked in the fields of fluid turbulence (at La Jolla Institute); nonlinear dynamics and chaos (at the Los Alamos Center for Nonlinear Studies); financial market modeling (as a founder of Prediction Company); and natural language processing (at ATR in Kyoto, Japan). He returned to Los Alamos as a staff member in 1987, where he has played a leading role in development of the traffic microsimulation for the Transportation Analysis and Simulation System (TRANSIMS) and has led the Epidemiology Simulation (EpiSims) project. His current interests include developing advanced technology for the study of large sociotechnical systems and understanding the dynamics and structure of social networks. 166

167 DR. EUBANK: Although I am very interested in providing a flu background for biosurveillance I haven't thought enough about the problem. What I am going to talk about is an individual simulation that is rather different from the classical kinds of epidemiological models. I am using simulation here not in the sense of solving ODEs or PBEs but as a more agent based kind of notation. I will describe the simulation that I am building. I will talk a little bit about why we use simulation in the first place and then I am going to try to give some mathematical questions that arise not just from epidemiological simulation but from any simulations of social technical systems we do, and then I would like to give an existence proof for a math program that has a very definite role in homeland defense in particular. So, to dive right in the kind of epidemiological simulation I am talking about requires three components. One is an estimate of the social network of a large region. Another is a model for the within-host progress of the disease once someone becomes infected and a third is a transmission model between people. So, very briefly I will go over where these things come from. I am not going to spend a lot of time on 167

168 the process of simulation because once you understand what the components are it is pretty obvious how you can use it to build an epidemiologic simulation. The reason we have gone in this direction of individual based simulation for response planning is that we have been asked to detect anomalous patterns which means looking at geographic distributions of disease by time for demographic distributions. With that kind of information we should be able to identify the critical path a disease takes through a population either geographically or demographically and if you can identify a critical path then you should be able to evaluate the effectiveness of countermeasures that are targeted geographically and demographically. So, we need demographic information and it just so happens that because of a transportation model that was developed at Los Alamo s we have an estimate for every household in a large city, in particular Portland, Oregon, 640,000 households and what everyone in the family does every day. So, this is a particular person. This parent takes a kid to child care, goes shopping, picks the kid back up and goes home. This is a completely trivial activity set. The actual activity sets we have are much more detailed. Since 168

169 we have this for every household in the city and there are only a limited number of locations where these activities can happen we can take a time slice in this picture and understand or gain an estimate of who is in contact with whom for how long, where, and what kind of activities they are doing. I can talk a little bit about where those estimates come from because I think it is interesting and very different from the kinds of data sets other people have been developing. Let me go ahead and explain the rest of the simulation models first. We have a very simple disease progression model, all we really need to know about the disease or what the effects on the individual who is sick are. So, this is a kind of a cartoon of viral load or how many spores you might have or something like that. It is time dependent. We can vary the rate to growth rates according to an individual's demographics and we can list the times what is happening to the person by place these thresholds on the disease. So, in this case someone can clear the disease completely if they don't receive a big dose. They become symptomatic at some level. You can change these thresholds. 169

170 You already have the thresholds around. It is very important for the actual dynamics of the epidemic and someone can either die or recover. This is to reinforce the notion that this is not some point estimate of what the disease looks like. PARTICIPANT: Is it true that the threshold is the same as the actors underneath? What is the load? DR. EUBANK: No, this is not really the actual load of virus in the body. So, if the threshold is not necessarily the same for entering as it is for leaving this is just the time rates so that you become recovered at the right time regardless of what the actual load in your body ~ s . We can assign different growth rates as I said for different people so we can get a distribution of effects of disease on a population. With a representation like this I can add in the effects say of post-exposure vaccination by shifting some of the margins around. This particular disease has several different, well, it is smallpox. So for a very old major infection you could have several different presentations of the disease and one of them is an early hemorrhagic one which often involves death much sooner than most. 170

171 PARTICIPANT: So, I guess the water supply is appropriate. DR. EUBANK: That is a lake in this water supply. That is the social network and the disease in the host. So, there is a transmission component that is very important. The traditional models for epidemiology require something called a reproductive number. I will talk about this a little bit more in a minute but it is basically what happens if you introduce an infected person into a susceptible population. How many cases is that index case going to cause? The reproductive number is an output of our simulation. It is very important to understand that. The probability is that given two people are in contact and one of them is infected you need to know the probability that the other person becomes infected, and it can be a function o how close the contact is, how long it takes, the demographics of the people involved, what kind of activity they are performing. Now, if you go talk to an epidemiologist they are going to want to know what this is in your simulation. They is quite a body of research into what that number should be, but if you talk to a clinician or more importantly if you talk to a patient this is what they want to know. I 171

172 want to know when I can send my kid back to school if he is still infectious. Should I go get a chickenpox vaccination, something like that? Then you compare a little bit more from these two approaches. Traditional epidemiological models, the next speaker may correct me on this but I class them all as basically coupled rate equations. The attempt to break down a population into very coarse groups, subgroups based on one or two demographic variables, and here I have shown age, and we assume some sort of mixing between these subgroups which is not really very well understood and then you make this assumption about the number that you have so many susceptible and so many infected, it is the rate. I am going to argue that reproductive number is - DR. CHENEY: Why are they broken up? I mean this is a basic question but why are they broken up into those age groups? What if you broke them up into different age groups, would it not be -- DR. EUBANKS: I picked these completely at random without looking at traditional models, but for a flu model, for instance kids may transfer flu among themselves more rapidly than older people. PARTICIPANT: The mixing rate is simulated? 172

173 DR. EUBANKS: The mixing rate would be, yes, how many times infected kids have an influence on the parents, but that means you know something about the structure of the population and how many times kids are in contact with older people. The reproductive number, it really involves two different things. It is person-to-person transmission characteristics and the disease and social mixing patterns, and you can't observe it in isolation. You can't go to a population and just say, "What will the reproductive number for this population be?" without an understanding of how the population mixes. So, in contrast to this approach what we are doing is to represent individuals each of whom carries some set of demographics which is much more finely resolved than in the other models. We are estimating contact rates between the individuals and the whole population and what may or may not be fairly important here is that these contact rates are estimated independently from the disease spreading problem. So, in the traditional epidemiological model you could say that we will fit the mixing rates and we will fit the reproductive numbers and we will reproduce a given epidemic that we have data for and generalize from that. 173

174 These contact rates were estimated for a transportation problem which means that the validation and verification issues are very different, but most importantly the reproductive number emerges as it does in the real world from transmission characteristics between individuals and the mixing of those individuals. PARTICIPANT: If I have a disease that only gets transmitted by kisses, I don't -- DR. EUBANK: There are some diseases this won't work for. This is not very good for HIV, but for something like influenza, colds and so on. I will tell you where the contacts come from and then you can make your own judgment about how well they represent the disease. PARTICIPANT: But I thought that this model would get at others. That was my question. The contact rate must depend on the disease. DR. EUBANK: Transmission during the contact depends on the disease. DR. LEVIN: But what constitutes a contact? DR. EUBANK: That is what I am about to tell you. What constitutes contact depends on the disease. Very often it depends on duration and proximity of physical contact. The next few slides are actually fairly apropos of the previous session's data mining, I think because they 174

175 illustrate a way to take data from very disparate sources and build some information of use to other processes like epidemiology. So, this is a description of where we get our contact patterns from. We take census data down to the block group level and we build a synthetic population. The property of this population that is important is that if you were to do a census on the synthetic population you would match the results of the actual census in an actual city. Also, there are no real people in our population. We have no problems with privacy because we don't have any real people to complain about it. If you did have, say, the NSA had data on everyone's movements during the course of a day and maybe we could get German data on everyone we could plug that in here just as easily, but we felt that it wasn't politically acceptable for Los Alamos to go tell people they wanted to know what they did all day. The next piece surveys that cities tend They send out diaries and day. of data we used is activity to collect on their population. ask people what they did every 175

176 We have about 2000 activity surveys for Portland, Oregon. We use those as templates and we matched households to households in the activity survey to find out which of the templates we want to use for that household. A template might be something like that cartoon I showed at the beginning of the talk. We remove from the survey all geographic information because 2000 templates is not enough to tell us where everyone is going. So, we make the assumption that transportation infrastructure through the city. It seems take travel constrains how people move like a fairly reasonable assumption. We take travel time estimates for the transportation network and generate possible locations for each of these activities to be carried out. The final step, what we set up then is a game. Everybody in the simulation is trying to minimize their travel time to fulfill all the constraints that their activity structure imposes on them, but they don't know what everyone else is doing except that everyone else is trying to minimize their travel time. So, we play this game on the computer and we simulate the traffic and results which updates the travel times which can update the activities and certainly update the routes that people choose. 176

177 When all this settles down we have an estimate for the contact panels. So, we put all that together and take the census data activity surveys and network the infrastructure data and we produce epidemic curves. Whether the curves are realistic or not depends on the assumptions we put in about the disease transmissions. This particular one shows vaccination response. We vaccinate people who are known to have been in contact with infected people. We can get rates of vaccination, numbers of people you would need to go around giving vaccinations, numbers of people you would need to go around as contact persons,numbers of people who become infected on a daily basis, all those kinds of things that you would expect have epidemiologically. DR. CHAYES: So, how come the vaccination bimodal? DR. EUBANKS: That is probably just a reflection of the incubation period. The disease has about a lO-day incubation period.. In addition to those kinds of traditional responses and results we can produce a demographically distributed picture. So, for instance this is the probability of distribution with age, given that you were Ro. how 177 come the vaccination is

178 infected during a specific week after an incident occurs, and this doesn't tell you as much as you might think. It is saying that in the first week the age distribution depends directly on exactly what the release scenario was. We happen to have a scenario where we release at a shopping mall. So, there are not many school- age kids there. There are some infants and there is a bunch of people you might expect to be at a shopping mall, and the distribution itself is relaxing as the weeks go by to a marginal distribution for the population, but there are some interesting question you could ask about this. One is what is the rate at which this distribution is relaxing and is it the same for all the different initial conditions. That would tell you something about when it is possible to apply these mixing models and just ignore all this detailed simulation. You could say that within 4 weeks the simulation Is indistinguishable from a model assuming uniform mix here of the population and that gives you a good time development on use of the simulation. We are, also, hoping to use this for targeting strategies for vaccinations, and it is not clear yet exactly how you would do that. 178

179 So, I won't spend more time on the simulation. I will briefly go through some of these points. Why is it that we are doing the simulation aside from the fact that my group is called basic and applied simulations lab. I think it is a very natural thing for homeland defense in particular. If you are talking about defending critical infrastructure you are really talking about having people and resources and interactions among people and resources are local. Even given telecommunications somebody has to interact locally with their telephone on their desk in order t o place that call to someone else. People move around and the resources often move with them and moreover defensive actions can change the way people move. So, what you really need to know of social networks is where the resources and people are and the function of time, possibly in hypothetical circumstances. So, just going out and measuring the way things are today is not going to help you understand what might happen if someone takes your advice and institutes a policy for defending the infrastructure. Another reason to use simulation I think is that it is a way to fuse, you know everyone knows about the data fusion problem, but there is another problem of fusing 179

information that is completely different in time. I am thinking here of something like demographic tables with procedural constraints that could be very vague like quality of life influences the decision or fairly specific like vehicles aren't going to physically pass through each other and it needs to take into account things that are completely asymmetric, irregular. The only reason for them being the way t hey are is some, it is contingent on some completely unknown past history. As an example I use the development of the social network that I went through briefly before. Suppose you have these demographic data tables and you had a land use network. I would like to know what the implication of this constraint that vehicles can't pass through one another is, and I really don't know of any statistical modeling technique that can tell you that. It is because the representation of the problem isn't quite large enough whereas this individual based simulation is a very natural representation and I can tell you that instead of the, like this represented the unweighted demographic tables. This represents the transportation infrastructure and the implication of this is that that person is going to arrive at a particular place at a particular time. ~0

181 Of course, it is not exactly clear what that means and so this is a simulation and that person doesn't exist. Simulation is really a deductive process that lets you untangle the entailments of hypotheses you make that may seem to be very unrelated. It is a very powerful technique, not in the statistical sense but in the sense of just what it can do, but the nature of the estimation that it makes is not so clear. What does it mean that that person on simulation arrived at that place at that time? I don't know. I think this is a very interesting mathematical question and what I would like to describe for you briefly is a few other interesting mathematical questions that arise in this simulation and then a program that attempts to address some of them. So, in addition to that kind of philosophical question about what simulation means there are very specific problems arising in our simulations of sociotechnical systems that I would to just very briefly mention. The first is we get questions in the form of well, here is what, you guys know what the social network looks like. Here is what the disease looks like. Tell me, how am I going to prioritize vaccination? To me that means 181

182 I have got some graph or network, and I need to identify the critical paths on this graph, and analyze what happens if I cut those patterns. I tell that to my computer scientist friends and they say, "Oh, that means this particular problem that I know about in computer science, although I don't know exactly how to apply it to your network." So, then we have to have a little discussion about how to take results that he knows about and produce something that is relevant to my needs. On the theoretical side if you think computer science is too much to worry about but I would like to know about results about very structured random graphs, we have a long history of theory on random graphs that have say a Poisson degree distribution where you pick links at random to turn on and off, and recently there has been a lot of work on small world methods that have power law degree distributions and these arise in places where first order clustering statistics are very important. The graphs that I am working with have a very different degree distribution, number of neighbors of each. This is a log-log plot and there is certainly some structure there but it is not structure that people have studied before. ~2

So, now, very briefly I am going to describe a math program. This is my existence. The first thing we had to understand was an axiomatic framework for what a simulation is. We derived a three-element object which is local functions on a network with a particular order of evaluation of the functions. So, we went through the whole network evaluating this. There is a computational theory associated with this and I am just glancing over this because again, like the other speakers have said I am not proposing this as the end all and be all of mathematical science. I am just trying to give you the flavor of a math program that has a very direct influence on homeland security. The kinds of research that go on in this program are characterization of dynamical systems, deriving morphisms between different dynamical systems, between different objects, inductions from one to another, algorithm development, specification simulations and extensions to other things as required by the applications that we are developing. One reason I agree on this point is that the morphisms and the relative relations between simulations is that it is very hard to go about validating these kinds of simulations and what we are trying to do is approach it at ~3 one to

a more basic level, understanding when two simulations are the same even if you are exploring different parts of the phase base. These things operate in huge dimensional probabilities bases and you can't possibly generate enough instances to explore the space base. So, you need to understand something about the structure of the simulation itself before you can examine your validation. So, focusing between the systems allows you to address knowledge and also to generalizability. The mathematical underpinnings let us know when it is even reasonable to suspect that we can build a simulation to do some of the things we want to do. A lot of our sociotechnical development depends on composing different simulations like the traffic simulation with an epidemiology simulation. We can understand something about the process of composing simulations with this approach, and it ha direct applications to some of the network and drafting problems that I mentioned. I think I am completely out of time. I can give you a long list of applications and theory to go with it but just to reassure you that it is possible to do this kind of work and not be completely shunned by the mathematical community this is a list of where we publish ~4

SDS papers but I would like to mention on the computer science side and the math side we have papers come back from a journal saying,"That is a wonderful paper, great results. Please remove all reference to applications." And if you want my opinion as to what mathematics can do for participating i homeland security change the culture a little bit. Math and society can have something to do with each other but the research must be driven by application, and those speakers before me have commented if you are just doing research in a vacuum it is not ever going to work, and the results have to be communicated back to people like me who can understand what the implications of your research are for very applied problems. Thank you. (Applause.) DR. BROOME: Any quick clarification questions? Thank you, Stephen, and I think that certainly the point of this workshop is to try to get some discussion amongst both mathematicians and those who might use their research. ~5

186 Mathematics of E:pidemiological Simulations for Response Planning Stephen Eubank An epiclemiological simulation can be used to detect anomalous patterns. It requires three components: an estimate of the social network of a large region; a mocte] for the within- host progress of a disease once someone becomes infected; and a mocle! of transmission between people. Dr. Eubank then spoke about each component of his simulation cleating with disease spread in PortIanct, Oregon, emphasizing the uniqueness of the transmission mocte] that he uses. Dr. Eubank and his colleagues essentially inherited the first component a database of the 640,000 househoIcts of PortIanct, Oregon, with estimates of what everyone in every family does every clay from a transportation mocte] that was cteve~opect at Los AIamos. For the second required component, the researchers have a very simple ctisease- progression mocle! that incorporates essentially all they really need to know about the disease or what the effects are on the inctiviclua] who is sick. Threshoicts that can be varied according to inctivicluals' demographics inclucle pathogen-growth rates in the hosts, whether or not they clear the disease (if the close is small enough), at what point they become symptomatic at some level, and whether or not they ultimately recover or die. The thirst component transmission between people is quite different from traditional moclels for epiclemiology, which require as input a "reprocluctive number." This number quantifies what happens if you introduce an infected person into a susceptible Copulation: how many cases is that inctex case going to cause? The reproductive number is an output of the simulation. The reproductive number involves two different things person-to-person transmission characteristics of the disease and sociaI-mixing patterns and the latter can't be cteterminect in isolation. So in contrast to the traditional approach, researchers are estimating contact rates between inctivicluais and the whole population, and what may or may not be fairly important is that these contact rates are estimated inclepenclently from the ctisease-spreacting problem. What is important is that the reproductive number emerges as it does in the read world from transmission characteristics between inctivicluals and the mixing of those inctivicluais. Dr. Eubank noted that while the simulation heat been describing obviously has a very direct influence on homeland security, it is also a vehicle for research. The elements of this program are characteristic of clynamica] systems, and from the program can be derived morphisms between different clynamical systems, between different objects, reductions from one to another, algorithm cteve~opment, forma] specifications of simulations, and extensions to other things as required by the applications being cteve~opect. ~6

187 In closing, Dr. Eubank noted that he and his colleagues have tract papers returned to them from mathematics and computer-science journals: " 'This is a wonclerful paper, with great results,' editors wouict say. 'But please remove all reference to applications.'. . . So if you want my opinion as to what mathematics can do for homeland security," he saint, "change the culture a little bit so that it's not bact to have applications. The speakers before me have also said that if you are just cloing research in a vacuum, it is not ever going to work." ~7

188 Sally Blower "Predicting the Unpredictable in an Age of Uncertainly" Transcript of Presentation Summary of Presentation Power Point Slides Video Presentation Sally Blower, Ph.D., is a professor of biomathematics at the University of California at Los Angeles. She is a mathematical and evolutionary biologist whose research focuses on developing models of transmission dynamics. She uses these models as health policy tools: to design epidemic control strategies for a variety of infectious diseases, to understand and predict the emergence of antibiotic and antiviral drug resistance, and to develop vaccination strategies. The main focus of her research is to develop the study of infectious diseases into a predictive science. Recently her work has focused on HIV, tuberculosis, and genital herpes. She has also pioneered the application of innovative uncertainty and sensitivity techniques (based upon Latin hypercube sampling) to the analysis of transmission models. These techniques enable transmission models to be used to predict the future with a degree of uncertainty and to identify which parameters are critical in determining which future outcome will actually occur. 188

189 DR. BLOWER: Thank you. So, I have been told to stand rigidly behind here and not move an inch. Okay, so in overview the material I want to cover then is to say a little bit about transmission models and their uses and to spend most of the time talking about uncertainty and sensitivity analysis to explain to you what it is, what it can be used for, some examples from work that we published on HIV and then to talk about the problem with risky vaccines, again, some work that we have done on HIV identifying vaccine perversity points, and you will have to wait to find out what those are and then talk a little bit about how that is related to smallpox, as we have a somewhat similar problem as we have currently a risky vaccine to deal with smallpox, and then I will go through as far as I know the complete literature of mathematical bioterrorism which is four publications at the moment. Next? So, what is a transmission model? It is a series of mathematical equations that specify the transmission dynamics of an infectious agent, and you can use either stochastic and obviously have probabilities in that versus deterministic, and both are useful. ~9

190 You can either have simple or complex and also one person's simple is another person's complex. The only thing to remember is that a complex model should always reduce down to a simple model. So, you end up with more results when you have a complex model, but they shouldn't be totally different results. So, the way that I use models in my group at UCLA is basically as health policy tools, and what we do is we predict the future with a degree of uncertainty by using uncertainty analysis. So, mathematical models can be used to look at the effects of current treatment conditions. PARTICIPANT: Could you speak a bit louder? DR . BLOWER: Okay, right, I will pretend I am talking to my children. So, to look at current treatment conditions and to look at what if conditions. So, what if we did something and what if we had a vaccine? Second use of them is to define perversity thresholds based upon drug resistance. When you treat individuals you get drug resistance. So, treatment has a good effect. It reduces drug sensitive strains. It has a bad effect. It increases drug resistance. So, you can look at trade-offs, and that is essentially what I mean by perversity thresholds when you make things worse than before you started. 190

191 So, you can look at changes in treatment and risk behavior and also risky vaccines, and then we can, also, identify the most effective prevention strategy for an epidemic by using sensitivity analysis. So, models can predict the future and mathematicians can write down equations fairly easily with a great deal of certainty. The problem becomes for all of these models we need specific parameter estimates and once you start to actually go to the literature and to talk to your infectious disease friends that is when you realize there is a problem and there is a great deal of problem with parameter estimation uncertainty even with diseases that we have been studying for hundreds of years like tuberculosis. So, when it comes to new agents there is going to be even more uncertainty. So, how we can deal with this is by using uncertainty and sensitivity analysis based upon something called Latin hypercube sampling and this allows us to predict the future with a degree of uncertainty, and I will show you some examples, but essentially all you need to know about this is the parameter -- can we go back? The parameter estimation uncertainty is being translated into prediction estimation uncertainty, okay? 191

192 So, just to talk a little bit about models and perversity thresholds when you are using treatment or vaccines there are approved by the FDA because they will help individuals. So, they work at the individual level but you can have individual good but population level harm and this is just referring to a paper that was published in 1996 in Science looking at tuberculosis and drug resistance and showing that the more you treat, if you have a high degree of drug resistance which actually happens in some parts of the world you are actually making things worse. Next? Another thing to be aware of, another trade-off is basically risk behavior and perversity. Again, this is another paper I published with Angele McLean in 1994, in Science and a paper we published in 2000 in Science, both looking at HIV and trade-offs between if risk behavior increases and you have got treatment or vaccine how you can actually end up making the epidemic worse. So, you need to have your epidemic interventions coordinated. So, what is uncertainty analysis? Uncertainty analysis is used to assess the variability which you can also call prediction imprecision in the outcome variable that is due to the uncertainty in our input parameters. 192

193 So, uncertainty analysis can be based upon Latin hypercube sampling. So, the question becomes what is Latin hypercube sampling, and that is a type of stratified Monte Carlo sampling. So, it is a statistical sampling technique that allows you to simultaneously vary all of the values of all of the input parameters. So, you don't just vary one at a time as with traditional methods or keep them all constant. You can vary everything at once and Latin hypercube sampling was first proposed by McKay, Conover and Beckman in 1979, to aid in the analysis of nuclear reactor safety. So, they were concerned with a problem of nuclear reactor meltdown, obviously a very important problem and they had very large models and the parameters were uncertain, so, a somewhat similar problem to what we are thinking about today. PARTICIPANT: When you mention hypercube and matching the color, is that the same thing? DR. BLOWER: Exactly but in hyperspace. So you just think in more dimensions. It is exactly the same conceptual principle. So, what is sensitivity analysis? We then couple the uncertainty analysis with sensitivity analysis and use it to identify the parameters in the model that are 193

194 increasing the outcome variables that we are most interested in. For example, if you are looking at the probability and the severity of an outbreak that is our outcome variable, and then by doing a sensitivity analysis we could tell which of the parameters are going to increase this. DR. CHAYES: Can we ask you why would you be using a Latin hypercube because in a Latin hypercube you don't want to have the same color occur in -- DR. BLOWER: Exactly. DR. CHAYES: -- direction. What does that correspond to? DR. BLOWER: It is actually a very efficient way of sampling the parameter space. So, instead of doing a full factorial in which you look at all the possible values of all the parameters and all the possible combinations it allows you as Latin squares does to do a much fewer number of simulations but makes sure that you have sampled the entire parameter space, and if you just did it randomly you could have very funny subset. So, it is a stratified random sampling method, okay? Could we go back? 194

195 So, to talk then about some specific examples so you can get a more concrete idea of this one thing that we have done is to predict the probability and the size of an outbreak occurring, and basically this is based on the basic reproductive number or R zero or R naught depending what side of the Atlantic you are from and so we have done an uncertainty analysis of R zero for HIV and basically if R zero is greater is than one an epidemic outbreak occurs and obviously for HIV this has already happened. So what we have done is basically the reverse of this saying that if we had enough treatment and could we actually make R zero less than one,. So, it is basically the same principle, okay? So, to do this uncertainty analysis we first of all used an HIV transmission model. You write down equations for the transmission of HIV. Then you analyze that to calculate an analytical expression for R zero. Then you estimate probability density functions for all of the parameters in the model. You basically get maximum and minimum upper and lower bounds for them. You then use Latin hypercube sampling and then calculate 1000 values for R zero and therefore you then have a measure for R zero as an uncertainty estimate. Next? 195

196 And this shows you what is on the Y axis here. So, on the Y axis is the value of R zero that we calculated and this is for three different conditions and these are box plots. So, the line in the middle shows you the median and then the main part of it is 50 percent the predicted data and then the whole thing is the 100 percent of it. So, you end up with an estimate of the outcome variable, but you can see exactly the prediction imprecision, the uncertainty in your estimate. Okay? Then we did sensitivity analysis to determine what is driving the value of R zero, and in this particular example it is the change in risk behavior is the most important parameter in the model. So, again the Y axis is R zero, the value of it and on the X axis this shows part of the sensitivity analysis. The parameter here is the change in risk behavior and you can see as that increases and decreases the value of R zero changes and that this is again uncertainty analysis. So, you have basically got a predicted cloud of data. Next? So, another example for you to see how uncertainty analysis can be used is what we did here is the title of the paper was predicting the unpredictable, the 196

197 transmission of drug-resistant HIV, and we published that in Nature Medicine last year. The idea here is that currently combination antiretroviral therapy is being used widely to treat the HIV epidemic in the United States. There is, also, a lot of drug resistance arising. At the moment there are very few data sets to actually say what is actually happening and so we did a model to predict what is likely to happen and therefore called it predicting the unpredictable. So, we predicted a variety of things, the number of HIV infections prevented over time, the number of new cases of drug-resistant HIV arising over time, the prevalence of drug-resistant cases over time, overall prevalence and then the number of infections prevented per drug-resistant case that arose. So, this is sort of a biological cost/benefit analysis, how many things that you have done that are good, the number of infections prevented versus sort of the number of things you have done bad which is generate drug resistance. So, you can use these models to work out biological cost/benefit, and then obviously you could add economics on top and do an economic cost/benefit analysis. 197

198 So, to see some of these results, this is the result showing number of HIV infections prevented if you started treating an HIV epidemic. So, on the Y axis is the number of new HIV infections that are prevented. On the X axis is the fraction that you treat and it is the uncertainty analysis. So, this parameter is an uncertain variable parameter and we allow this to vary between 50 and 90 percent, and therefore we predicted 1000 different epidemics over time. The blue predicted data are after 1 year. The yellow are after 5, and the red are after 10 years, and here you can see the more you treat, if you go along the X axis, the more HIV infections you prevent, and since it is an uncertainty analysis you get again the predicted clouds of data. This shows the predicted transmission that we predicted going from 1996 up to 2005, for what we predicted how much drug resistant HIV would be arising and as I said there are no data sets, no published data sets. We did this for San Francisco, and the red lines are the median. The predictions are box plots. So, you can see 50 percent of our predictions lie in the rectangles. After we made these predictions there was one datum point that arose which is the green. So, that 198

199 actually occurred after we had made the prediction. So, you could say that this made the predictions look fairly good, but there is only one piece of data, but it was collected afterwards. Anyway, next? This then shows again uncertainty analysis predicting the prevalence of HIV. Again, the treatment rate is an uncertain parameter varying between 50 and 90 and this is what would happen if you treated an HIV epidemic after one year of combination antiretrovial therapy. The drug sensitive strains come down, the white data at the top. The drug-resistant strains come up which are the red strains at the bottom, and it is a function of treatment and you can see these predictions are with uncertainty. This is what happens after 5 years. You can see that the treatment rate has the same effect but more so over time, and then finally after 10 years. Next? And actually this is what we are predicting the HIV epidemic will be like in the San Francisco gay community in 2005 which is pretty bad. Okay, this shows predictions for the prevalence of the HIV prevalence again as a function of the treatment 199

200 rate and you can see again what happens over time, year one, after 5 years, after 10. Then these are the results of the biological cost/benefit analysis in which we have predicted what would be again the effects of combination antiretroviral therapy treating an HIV epidemic, the number of HIV infections you prevent for each drug-resistant case you generate, so basically a trade-off, a biological trade-off, and these effects change over time. So, these are our predictions from 1996 to 2005. So, then the other part of this was to do the sensitivity analysis and this uses the results of the uncertainty analysis that we generate and then we calculate partial correlation coefficients and this basically shows that one of the key factors in increasing the amount of drug resistance was the treatment rate which is not surprising. This shows the unadjusted data but the partial correlation ren(?) coefficient is .9. So, these are the unadjusted data and then the other key factor revealed by the sensitivity analysis was the actual biological fitness of the drug-resistant strains and again these are the predicted unadjusted data. When you adjust this statistically for it then you get partial ren correlation coefficients of about .9. 200

201 So, how good does a vaccine have to be? If we worried about smallpox and other biological agents we fairly easily, this is sort of a classical result in literature, fairly easily work out how good a vaccine to be, and the three important components are vaccination coverage, how many people you need vaccinate. The E is the efficacy of the vaccine, how good is it and R zero is the basic reproduction number. So, basically how severe is the problem; how severe is the epidemic? So, if you have good estimates for those or again you can do an uncertainty analysis on it and work out for the size a basic reproduction number how good the vaccine has to be and what kind of coverage, how much, what percentage of the population do you need to get to take the vaccine . Live attenuated vaccines are actually used, what we have available for smallpox, polio and measles. They have many advantages. They generate a very high protective efficacy because they are essentially generally the same pathogen but attenuated. They are low cost and they are simple immunization schedules. So, live attenuated vaccines are very nice and are used a lot, but let us just think about some of the 201 are can the has the to

202 problems for that. The problem with risky vaccines, live attenuated vaccines is that they can actually cause the disease in some people. Therefore they have a benefit in which they are very, very effective, but they also have a risk. So, these type of vaccines, we call them risky vaccines because they are risky, and they can under some circumstances do more harm than good which is why people are worried about the smallpox vaccine at the moment. If you use mass vaccination with a smallpox vaccine there would be some very detrimental effects in large numbers of immunocompromised individuals, HIV infected individuals or immunocompromised because they have cancer in many of the urban centers, and there are hundreds of thousands of the people in the United States. So, it is a big problem. We looked at this in terms of HIV vaccine and this is a paper we published in PEAS last year looking at live attenuated HIV vaccines. Basically this is an ordinary differential equation model and allows us to look at the effect of a vaccine that would be very effective and protect against infection but in some people because they are vaccinated will actually cause AIDS though at a much slower rate than if they had gotten infected with the wild type. 202

203 So, if you had this live attenuated HIV vaccine would it be a good thing to use in a public health campaign? This was the question we addressed and you could actually, we have got web bush(?) into this model running and it is very user friendly. My children can actually run this, and so if you want to you can go to our web site and put in parameters and see how the model behaves for different coverage levels and different safety levels. So, that is something you may want to do. What we did is predict what would happen in Zimbabwe where there is a very severe HIV epidemic; 25 percent of the population are currently infected and in Thailand where very much fewer, much less severe epidemic and compare and contrast between these two countries using exactly the same hypothetical HIV vaccines to see what would happen. So, again, we used time-dependent uncertainty analysis and we specified the parameters such as efficacy, such as coverage levels by probability distribution function and then we made some predictions. If you actually look at analytical results, and we also looked at analytical results what you find is that all of the, we tested 1000 different HIV vaccines theoretically and on the computer. We tested them and found 203

204 if you looked at it analytically all of these thousand vaccines would lead to HIV eradication, and it would be a two-step process. You would replace the virulent strain, the wild type with the avirulent vaccine strain and then you would have to stop the vaccination program and you would have eradicated HIV. So, that is what you find out if you look at it analytically, and so equilibrium results are very important but it is important to look at the transient dynamics, too. So, we did that, and we found something that we have called the vaccine perversity point. Now, the vaccine perversity point is defined in terms of the vaccine safety level. So, in the case of HIV this is the fraction of vaccinated individuals who progress to AIDS as a result of the vaccine strain. So, it would be the same thing if you were looking at a smallpox vaccine. At the vaccine perversity point then the annual AIDS death rate with a live attenuated HIV vaccine in place is equivalent to the annual AIDS death rate without a live attenuated HIV vaccine and this shows our predicted results for what would happen if you put an HIV vaccine out in Thailand. On the Y axis is the total annual AIDS deaths for ~ per 100,000. On the X axis is the safety of the vaccines that we were looking at for HIV for them causing anywhere 204

205 between 1 percent and 10 percent of the vaccinated individuals to progress to AIDS in 25 years. So, these are fairly safe vaccines, and then again it is an uncertainty analysis. So, you have got predictive clouds of data. Sorry, can you go back? So, at time zero we start off with the sort of light green data, after 10 years the pink data and then after 50 years the yellow data and 200 years turquoise. We have just plotted the 200 years to show you that basically you are almost at the equilibrium after 50 years. Things don't change that much. What you can see here is there is very clearly a vaccine perversity point that is the vaccines that cause 5 percent or more of vaccinated individuals to progress to AIDS in 25 years actually raised the death rate. They make things worse. So, these vaccines in Thailand if you used them would actually make the HIV epidemic worse. If you had live attenuated vaccines that were 5 percent or less they would actually reduce the death rate and be a good idea. The same vaccines in Zimbabwe as you see there is no vaccine perversity point. The time zero data are much higher. We have a much more severe AIDS epidemic in Zimbabwe and these vaccines behaving in exactly the same way actually because the epidemic is so severe, their risky 205

206 effects basically aren't seen and therefore they are just seen to be beneficial, though obviously the safer the vaccine the more you lower the death rate, but there is no perversity point. You can't make things, frankly, any worse in Zimbabwe than they are already. DR. REINGOLD: Are you assuming this for now? DR. BLOWER: Yes, but you could also put it at any point and do the same analysis. DR. REINGOLD: So, you could make the epidemic worse without treatment over, without vaccine over time? DR. BLOWER: You can, yes, definitely. This is just to assess the effects of the vaccine. So, in Zimbabwe none of the vaccines resulted in perversity. In Thailand vaccine strains that caused more than 5 percent of vaccinated individuals to progress to AIDS in 25 years led to perversity and if you just want to think about testing these vaccines, HIV vaccines in clinical trials that is an enormous problem. So, whether or not a vaccine will cause a perverse effect will be situation specific and depend upon the risk of becoming infected and this really then is the big problem with small pox. Since we have a risky, dangerous vaccine, if it was used now in a mass vaccination campaign 206

207 in the United States, it would obviously do more harm than good. I think everyone is in agreement about that. It should only be used basically if there has been a major release of small pox and we are convinced that the number of death, therefore, are going to be greater than the number of deaths that would occur without using the vaccine. So, to go through the mathematical bioterrorism literature, actually the first paper is back in 1760 by Daniel Bernoulli. I think it is little read paper. It is written in French, but you can get a translation. I haven't read it in the French. I have read it in the English. So, it is an attempt at new analysis of the mortality caused by small pox and of the advantages of inoculation to prevent it. So, Bernoulli was very interested in doing health policy research back in 1760 and outcomes research and all the buzz words that are now used. The idea was to use a model to actually show that using vaccination against small pox was worthwhile. So, we are basically now doing -- it is always worthwhile to look back in the literature. The next paper that I came across -- there may be more out there, but these are the only ones I am aware of - 207

208 - is in 2001 by the CDC, Modeling Potential Responses to Small Pox as a Bioterrorism Weapon. Then there are two more papers. The paper by Ron Brookmeyer and Blade in Science in 2002, Prevention of Inhalation Anthrax in the U.S. Outbreak, and a recent paper that just went on line in PNAS 2002, looking at early statistical detection of anthrax outbreaks by tracking over-the-counter medication sales. So, just to summarize, even in an age of uncertainty, I believe prediction is possible. [Applause.] MS. BROOME: Clarification questions for Dr. Blower? MR. KAPER: I am Hans paper, National Science Foundation. You haven't given any specifics about your models, but I gather they are systems of ordinary -- DR. BLOWER: Yes, they are systems of ordinary -- and if you want actually any of the papers, they are downloadable from my web site and all the papers have the equations in them. MR. KAPER: The reason why I am asking is that in the mathematics community there is quite some experience 208

209 now that large systems -- I was wondering whether anything has been applied to your models? DR. BLOWER: To my models? Maybe Simon knows about that. MR. LEVIN: The person I would turn to Mac Kliman(?) because otherwise I haven't seen anything like that applied to any of these models. MR. CAPER: It seems to me that there is a fruitful area for applied mathematics to get involved. MR. LEVIN: I have a question, Sally, which is when you are looking at these cost benefit tradeoffs in terms of -- is there any notion of discounting them in terms of how long in the future you push the -- or how late in one's life? DR. BLOWER: I think that is very interesting and actually there was some work done actually by Hans Waller back in the seventies, who worked for the WHO until he got fired for doing some of those such things. But basically then you have to decide -- I mean, the reason I keep away from economics is you have to say basically how much life is worth, first of all, and then you have to discount, and future people are worth less than current people. So, you definitely could do that. But we haven't yet done that. 209

210 MR. LEVIN: Not just future people, but maybe delaying to later in one's life when the death occurs. DR. BLOWER: Oh, increasing life expectancy. MR. LEVIN: Yes. You may have the same number of deaths, but it may be when people last longer and die DR. BLOWER: We have looked at that, that basically that you avert because, yes, you don't prevent basically in the Science 2000 paper for HIV, you can show that you avert AIDS deaths and that that is a significant effect and, therefore, very worthwhile, as well as the number of infections prevented. 210

211 Predicting the Unpredictable in an Age of Uncertainty Sally Blower Mathematical transmission mocteis, whether cteterministic or stochastic, can be valuable in predicting the dynamics of an infectious agent. Dr. Blower and her group at UCLA use them as heaith-po~icy tools in two basic ways: to assess the effects of particular treatments and vaccines on current or what-if conditions and to clefine "perversity threshoicts" beyond which the use of a nominally beneficial] technology actually makes things worse, as in some cases of ctrug-causect ctrug resistance. In the latter circumstance, you can have inctiviclua] good but popuiation-Ieve] harm. For all these mocteis, mathematicians can write clown equations fairly easily with a great clear of certainty. But there is a problem with parameter-estimation uncertainty even with diseases that we have been studying for hunctrects of years, like tuberculosis; and with new agents there is going to be even more uncertainty. Analysts clear with this situation by using uncertainty analysis and sensitivity analysis basest upon something caI1ect Latin hypercube sampling, which is a type of stratified Monte CarIo sampling. Essentially all you need to know about this technique is that parameter-estimation uncertainty is being translated into prediction-estimation uncertainty. Dr. Blower clescribect some applications of these techniques in her group's work on HIV infection. These inciucte ctetermining the probability and size of outbreak, the prevalence of disease and of c/rug-resistant cases, and the perversity point of a live vaccine uncler different circumstances. She summarized her talk tersely: "Even in an age of uncertainty, ~ believe prediction is possible." 211

212 Simon Levin "Remarks on Detection and Epidemiology of Bioterrorist Attacks Transcript of Presentation Summary of Presentation Video Presentation Simon A. Levin is the George M. Moffett Professor of Biology and was the founding director of the Princeton Environmental Institute at Princeton University. Dr. Levin is a member of the National Academy of Sciences and the American Philosophical Society and a fellow of the American Academy of Arts and Sciences and the American Association for the Advancement of Science. Dr. Levin has also served as president of the Ecological Society of America and has won its MacArthur Award and Distinguished Service Citation. He was the founding editor of the journal Ecological Applications and has edited numerous journals and book series. He is past chair of the board of the Beijer International Institute of Ecological Economics and is also a member of the science board of the Santa Fe Institute and the vice-chair for mathematics of the Committee of Concerned Scientists. ... ;; ~ ~ : :::: L5:.:...:: .'- -. 4. 5.~ :::: -., ~ ~~ 1 - i..; ...... 212

213 MR. LEVIN: I will say, Sally, that the one thing that I was amused by was the notion that user friendly was something my kids could do on the computer because that was by far no guarantee that it would be something that I could do. What I would like to do, if we can get the slides on, is to first of all touch on some of the points we heard, review them and put them into context, maybe try to identify some challenges, including some topics that haven't been hit and finishing up with sort of a pet hobby horse of mine, which has to do with system level aspects, indeed, the notion of developing a system level immune system. As we have heard, especially in the last two lectures, there is a large classical literature in epidemiology and, indeed, in much of mathematical modeling and infectious diseases, the focus has been on the sorts of things that Sally just talked about and we have talked about before that; namely, developing predictive models that will help us to understand what the spread of an infection will be once introduced into the population. There are some model aspects of this part of the problem that are associated with bioterrorism. You have also heard -- this is sort of the classic framework, the 213

214 compartmental approach, in which the population is broken into classes, such as susceptible, infected, recovered and then maybe into age groups or risk groups. Some compartmental analysis, but as you also heard, one can develop individual-based models. The unique aspects are that in the case of bioterrorism, one may not be dealing with a single introduction. Typically, we would ask what happens if a disease is introduced at low levels in a particular environment. What is the potential for spread, et cetera? But if bioterrorism could involve multiple introductions several different places, it could also involve the introduction of multiple agents. Diseases that might interact with each other might affect our response capabilities by moving resources one place and then hitting the system with resources somewhere else. So, one is going to have to extend these approaches to the modeling of multiple agents. Indeed, they don't even have to all be infectious disease agents. It could be different kinds of terrorist attacks coupled with bioterrorism. So, these are different sorts of notions that is in the same framework, but involves our understanding much more complex systems. But as we have been hearing, especially in this afternoon session, prediction is not the only thing that we 214

215 need to be concerned about. There are all sorts of challenges for mathematical models, from dealing with either chemical or biological threats, ranging from to what we do we respond? ~_, ~ ___ at least four different categories dealing with infectious disease modeling and I am sure there are others, most of which we have heard about, but they relate to how do you detect the surveillance systems about which we heard this morning, prediction. The response systems once a disease appears in the population and tracing, that is, how do you trace back to identify where the source is, something which we have seen doesn't work very well in the case of the anthrax attacks, but has been used, for example, in the foot and mouth disease in the U.K. to trace back to what were the . . Original sources. That introduces inverse problems, sometimes very simplistic inverse problems, but imagine anyway that you have a model for the spread of a disease, how do we run it backwards to try to identify the sources or can we do that? Is that a much harder problem? Now, all of these things and I will focus here briefly on prediction, at least two different aspects, what ___ __ _ _ __~_ or n~o~og~ca. threats prevention to detection, from prediction after an introduction, remediation. How do do. I will identiEv 215

216 I will call a priori and a post priori, namely, there are things that you could do in advance to prepare you before there has even been an attack and these would include running scenarios using mathematical models. A very important use of mathematical models is in exercises like Tupoff(?) or Dork(?) Witter(?), which have been exercises in which one uses a model to drive a theoretical epidemic and allow people who in response systems to do what they would do if there were a real attack. It is like a flight simulator. It is a learning experience by which you learn about where the weaknesses are in the system and you give people experience with responding to it. So, models not only are going to be useful when there has actually been an attack, but they can be extremely useful in training people or helping you to identify where you ought to be putting the resources and what is the best way to set up response systems. There have been a few exercises of that sort, but not a lot and typically the models that have been used have been fairly unsophisticated in driving the epidemics in those systems. So, detailed models such as Steve told us about, for example, could be extremely useful in these training exercises. Then there are ones after the fact. The 216

217 difference between single threads and multiple threads or single focus and multiple ones, I have already mentioned. I will just shove these in after listening to Sally because first of all, I wanted to emphasize what is sort of a central concept of the use of models of this sort and then to identify some of the particular new problems that we may need to be thinking about. Sally talked about rO. That is the number of secondary cases per primary case in a naive population. Notice I called it r-naught. I speak British. So, the criterion usually here is that rO is greater than 1, then the introduction will spread and, therefore, to control it, as she showed us, you are going to have to reduce rO below 1. In the sorts of models that she talked about, rO typically can be broken up into the mean, infectious time per individual, multiplied times the average number of secondary infections per unit time while an individual is infectious. So, one can look at this. Beta would be the probability of transmission per individual and it is the population size. Nu(?) would be with death rate and gamma would be the recovery rate and, therefore, understand that there are various ways to control the disease by focusing either on improving the recovery rate -- you could also increase the death rate, 217

218 but that is probably not a good way to do it, but some sort of removal rate that would reduce the infectious time per individual. Isolation would be one way to do it. Secondly, to reduce the mean infectious transfer per individual. For example, we do that with sexually transmitted diseases by the use of condoms and other ways by isolating individuals who are infectious and in itself is important, that is, the population size is important and one way we get at that is through vaccination to reduce the size of potential infectious individuals. But this very useful concept has some problems associated with it. It is a deterministic concept. So, for stochastic models, it is only an approximate threshold. If there are heterogeneities in the population, it may be only an estimate and in particular if there is heterogeneity in terms of the number of infectious transfers per individual that -- if some individuals are more sexually active than others or more likely to transmit the disease, then every individual in a sense has her own rO. Some individuals are much more likely to spread the disease and, therefore, where you introduce the disease becomes important. This is an average quantity and how to deal with that in a heterogeneous population, such as Steve told us 218

219 about, is a challenge. Another challenge in the individual-based models, which relates to this question is how do you reduce the dimensionality of the system? Can you take that large individual-based model and collapse it through some sort of moment closure or other techniques, through a system of equations that can be dealt with analytically. So, that is another mathematical challenge, which is to take these large scale, individual-based models and try to develop essentially a statistical mechanics that would allow us to do some analysis. MS. CHAYES: Are you looking at non-stationary, as well as stationary rO there? You said that rO need not be constant, that it can vary from one individual to the next. MR. LEVIN: Absolutely. It varies spatially and it varies over time. That is certainly the case. MS. CHAYES: So, you study non-stationary models. MR. LEVIN: Non-stationary models are studied and are -- the people who do this sort of modeling study -- but certainly the predominant literature focuses on the computation of one rO, but you are absolutely right. There are many examples of diseases which have been introduced and didn't take, depending on the season. So, rO is very 219

220 much a function of when the introduction takes place. In the case of vector diseases, for example, ones that are transmitted by mosquitoes, rO depends a great deal on the season of the year and whether the mosquito population is at high density. The classic example or a classic example is the introduction of mix(?) mitosis into Australia to control rabbit populations, which didn't take for a number of years and then suddenly after a number of wet years, it just took off and spread and temporarily wiped out the rabbit population. So, rO, certainly can be temporarily -- Now, in addition to the spread of infection over time, there is also the problem of spatial spread and there are two dimensions to that. One is the spread of the agent. For example, in the case of anthrax, there are lots of models that have been used, essentially variance on calcium plume models for how an agent with particular characteristics could be spread. But secondly there is the sort of spread that we have heard about today, which has to do with the infectious transfer of the disease, not applicable to anthrax, but to many other diseases, from one individual to the next. Most of them, near as I can tell, most of the modeling of spatial spread that is in use in our response agencies has 220

221 to do with the first element here, the spread of agents, the transport of materials in the environment. But there is obviously great potential for understanding the temporal and spatial spread due to infectious transfers. There are a couple of different dimensions to this. For example, in the case of foot and mouth disease, which has been a current hot topic for the last year and a half in the U.K. Issues of spread relate both to the spread of the agent, the possibility that the agent itself could be spread by airborne methods over very large distances and, secondly the spread due to the movement of individuals, due to the movement of cattle or sheep around the country, therefore, cutting down -- therefore, spreading the disease. The first thing that goes into effect when a new case is discovered is or would be shutting down movement of animals around the country because it is that rapid movement that spreads it. So, understanding spatial spread is crucial. That has multiple scales to it and the same explanations may not be true on all scales. For example, influenza typically could be approximated by a kind of diffusive spread once it is introduced into a particular community, but as the Russian Rovotchev(?) and Irolin Jenie(?) and others showed some 221

222 years ago, entirely different models would be needed in order to understand the spread among cities and that would relate on the airline traffic. Once one built that model, it might be useful for other sorts of diseases that would spread. You have got to keep updating that model as airline schedules change, however. So, there are multiple scales of transmission. That was prediction. We heard a good deal about detection methods, largely statistical methods in which they are involved, surveillance and how to discriminate a signal from the background. The use of inverse methods, tracing, how do you run models backwards, as I said, in order to identify, for example, what the sources are or in order to identify, as we have heard in the talks this afternoon, there is also the problem of identifying what the parameters are, which brings me to response and some of the -- as I said, the hobby horses I have. Again, these are responses that can be done before the fact or after the fact, but, in particular, in dealing with the responses to terrorist attacks, we need to recall that we are not dealing with the same sorts of problems as we are with accidental introductions. We are first of all dealing with a game theoretic problem in which 222

223 the terrorists are trying to outsmart us and understanding what our strategies of a response will be. Obviously, even work such as this that we are talking about that gets done informs potentially terrorists. So, it is really a game theoretic situation in which you are dealing potentially with multiple agents about which -- whose introduction may be unpredictable and, therefore, it is crucial to develop responses to have systems that are adaptive, that have the capability to respond to unpredictable attacks. In other words, to develop essentially an immune system for the whole system. So there are a whole class of operations research questions that relate among other things to how do you deploy individuals, how do you respond to the system, how do you seal it off. For example, modularity is an important part of the transmission system. We heard about small world networks. The particular typology of interactions has a great deal to say about the potential spread. How do we reduce modularity? Could we do targeted vaccinations, et cetera? Can we reduce contacts in certain places in order to make the system less susceptible to propagate an unexpected attack? Why don't I stop there. 223

224 MS. BROOME: Thank you. I think we are trying to stick more or less close to time so there will be some opportunity for further discussion. 224

225 Remarks on Detection and Epidemiology of Bioterrorist Attacks Simon Levin There are many challenges for mocteiing disease prevention, ctetection, prediction, and response in the case of bioterrorism. Typically, we wouIct ask what happens if a disease is introclucect at low levels in a particular environment. But in the case of bioterrorism, one may not be clearing with a single introduction. Bioterrorism couIct involve multiple introductions in several different pi aces , and it couict al so involve the introduction of multiple agents. Incleect, they don't even have to all be infectious-ctisease agents- ctifferent kinds of terrorist attacks couict be coupiect with bioterrorism. These diverse possibilities are all more or less in the same framework, but they require an uncterstancting of much more complex systems. One important too] to prepare for an attack is running scenarios using the mathematical mocteis. In this way, they are like flight simulators. These are experiences by which you learn about where the weaknesses are in the system- where to put resources and they can be extremely useful in training people. Particular areas that shouict be stuctiect inciucte these: 1. Spatial spread, related to the spread of disease over time. Most mocteiing of spatial spread involves mocleling the spread of the agent, although analyses and actions relating to the mobility of inctivicluais have proven effective in such epidemics as foot and mouth disease Outbreaks related to terrorist attacks as opposed to acciclental introductions. These are game-theoretic problems, in which the terrorists are trying to outsmart us. 225

226 Arthur Reingold "Remarks on Detection and Epidemiology of Bioterrorist Attacks" Transcript of Presentation Summary of Presentation Video Presentation Arthur Reingold, M.D., M.P.H., is professor of epidemiology and head of the Division of Epidemiology at the School of Public Health, University of California, Berkeley (UCB). He holds concurrent appointments in the Departments of Medicine and Epidemiology and Biostatistics at the University of California, San Francisco (UCSF). He has devoted the past 20-plus years to the study and prevention of infectious diseases in the United States and in various countries in Africa, Asia, and Latin America, initially at the Centers for Disease Control and Prevention (CDC) for 8 years and at UCB since 1987. Current activities include directing the National Institutes of Health-funded UCB/UCSF Fogarty International AIDS Training Program, now in its 14th year, and co-directing the CDC-funded California Emerging Infections Program, now in its 8th year. Dr. Reingold's current research interests include prevention of transmission of HIV in developing countries; the intersection of the HIV/AIDS and tuberculosis epidemics; malaria in Uganda; emerging and reemerging infections in the United States and globally; sexual transmission of hepatitis C virus; vaccine-preventable diseases; and respiratory infections in childhood. 226

227 DR . RE INGOLD: Thank you. We are going to go from power point to slides, back to overheads. I am going to actually focus my comments primarily around the context in which Ken is doing the work he is doing about detection of bioterrorist events and outbreaks. This is a topic that I first had the opportunity to think about and work on when I was at CDC working with Claire and we were thinking about how to detect meningitis outbreaks in the meningitis belt as soon as possible in order to rush in with vaccines and reduce morbidity and mortality. The first point I would make, I think, is that detection and rapid response to outbreaks is more or less the same whether or not the outbreaks are naturally occurring or intentionally introduced. At least from an epidemiologic perspective, most of the issues are the same. What I would like to do in the few minutes I have is try and give you some context for the complexity of what Ken is trying to do in Massachusetts and what others are trying to do. You may have seen the picture of George Bush looking under a microscope in Pittsburgh a couple of weeks ago, looking for outbreaks in Pittsburgh. This is a challenging thing to do for a variety of reasons. 227

228 So, let me just first of all give you a quick definition of surveillance in terms of what we are trying to do in terms of detecting diseases or illnesses. Those of us in public health generally refer to surveillance as a systematic ongoing collection analysis and dissemination of health data with findings linked to actions in the decision-making process. The implication here fundamentally in terms of bioterrorism is we would like to have a system in place that would permit us to detect illnesses quickly and then be in a position to respond rapidly to keep morbidity and mortality to a minimum. Now, in terms of surveillance needs for bioterrorism, there really are two phases, I think, one can look at. One is pre-event and the other is post-event or release. In the case of pre-release or pre- event, obviously, what we are talking about is can we detect the event and can we detect it earlier than we would have in the absence of our system. Then, of course, in terms of post-release, there are some important issues around being able to map the event and guide the response, make projections about the numbers that will be affected and other things of that kind. Now, the reality is most of the emphasis has been on the first phase of this, the detection of the event. I am 228

229 going to spend most of my time talking about that, rather than about the second. Now, I think it is worth pointing out since many of you are not physicians and not working in public health, how it is we in public health generally do detect outbreaks in the absence of fancy systems and computers and software and hardware. The reality is that generally we detect many outbreaks because notices that, gee, an astute patient or family member aren't a lot of us who ate at the church supper or who all went to the same wedding all vomiting more or less at the same time. It doesn't take an enormous lot of smarts to figure that out and frequently the health department learns because someone calls and says I think we have an outbreak here among people who were in such and such a place or participate in such and such an event. Similarly, we learned a lot of outbreaks from an astute clinicians, who recognize the one or more cases of an illness are more than they should be seeing and they are smart enough to let the health department know about that. If you could say, there is an intrinsic comparison of what the expected is, but it is primarily because that individual or that individual and someone they talked to on the elevator come to the conclusion that they 229

230 are seeing too much of something that this whole process has set in motion. Sometimes it happens because an astute laboratorian looking at specimens in a clinical microbiology laboratory notices the same thing. On rare occasions, it is actually those of us in public health who are collecting and analyzing surveillance reports or laboratory reports, who detect outbreaks that otherwise had not been detected yet or might have gone undetected. Clearly, what we would like to do with these fancier systems we are developing is see if we can be a larger part of this detection process, as opposed to waiting for clinicians and patients to tell us about the problem. Now, I want to point out, this is the epidemic curve. These are the cases. The X axis here is the date. The Y axis is number of cases and for epidemiologists this is what is called an epidemic curve, showing the cases of anthrax from last fall. What I would like to point out is a couple of things about this actual bioterrorist event that we now have experience with. The first is that these cases were primarily brought to our attention by astute clinicians; that is to say, a doctor was able to look at a patients and say I wonder if this person has anthrax based on the 230

231 clinical diagnosis and then report that to someone and collect appropriate specimens. I am sorry to say that I think the systems that many people are building to try and detect various syndromes would never have detected this particular outbreak because there was one case here and one case there, because in this case, the cases were related to mailing of letters and the anthrax spores ended up in various places. So, if you are looking to detect increases in patients with febrile illnesses, this outbreak probably would never have been detected by that kind of system. I think it is fair to say that a lot of the activity around detection of outbreaks in terms of anthrax in particular is assuming that some plane will fly over New York or Washington, release a cloud of spores or an individual with small pox will walk through Washington Dulles Airport and infect various people and then a whole thing will unfold and large numbers of people will become ill. This particular event obviously had different features, which would not have been readily detectible by the systems that many people are building. So, what are the various approaches people are taking to improving surveillance to detect bioterrorist events? Well, 231

232 obviously, as Ken has already said, one approach to this is trying to monitor visits to health care providers and while we do a lot of that already through hospital admissions for various diseases, many people are trying to add a component that looks at outpatient visits and intelligently doing it in places like HMOs, where people already collect and enter these data, where the hardware and software things you need to add to the system are minimal or non-existent. People are interested in looking at energy department visits, clinical microbiology laboratories, indicators, things such as 911 calls, over-the-counter drug sales, absenteeism at work and at school. Some people are even considering direct monitoring of samples of the population through Nielson rating type setups, in which people are routinely answering questionnaires over the Internet, a whole host of different, creative approaches. I think we need to give some attention to the question of how likely is it that these kinds of systems will actually improve our early detection of bioterrorist events. Well, Ken has already referred to the fact that one of his problems is influenza and these are the kind of monitoring data we have on influenza and have had for a number of years. This just simply looks at the number of - - the proportion of all deaths due to pneumonia and 232

233 influenza in the 121 largest cities in the United States and you can see here that that fluctuates with season because influenza is a seasonal disease. Now, one thing I would take issue with Ken a little bit about is influenza is not our only problem in this regard. There are many other viruses that can cause similar illnesses, parainfluenza, adenoviruses, microplasma, which is a bacterium. So, problem with influenza, but there perturbing the background that we certainly will make it more challenging like inhalational anthrax at an early before the rash erupts. This is also just to point out that, in fact, we currently do have surveillance systems in place and have had for a long time for influenza, not just on what proportion of deaths are due to pneumonia, but an outpatient visits to sentinel physicians and to a number of other indicators that all pretty much tell us that influenza is a seasonal disease and that all tend to peak at approximately the same time. I would really like to point out and you may not be able to read this very well is people in France a number of years ago were interested in the question of how well do it is not just the are many things can monitor, but to detect something stage or small pox 233

234 these various indicators allow us to say when an outbreak of a febrile respiratory illness is occurring. So, what they looked at here, what we epidemiologists call the sensitivity, the specificity and the predictive value of a positive of these various indicators to say when has an influenza epidemic occurred. They were looking at everything from emergency visits to sick leave reported to the National Health Service, sick leave reported by companies, visits to general practitioners, hospital fatalities, a whole host of drug consumption, many of the things that we are considering looking at now and measuring how sensitive, specific or the positive predicted value. By sensitivity we mean of all the outbreaks that occur, what proportion will be predicted by this indicator. By specificity we mean of all the times there is not an outbreak, what proportion of the time will the system tell us there is not an outbreak and most importantly, the predicted value of a positive of all the times the system tells us there is an outbreak, what proportion of the time is it right and there really is an outbreak. That is important because if we envision rushing in with vaccines or very labor and time and other intensive 234

235 things in response to some indicator, we would like to be doing it in response to a real problem most of the time, rather than response to a false alarm. You can see that, in fact, some of these various indicators have reasonably good sensitivity and specificity and predicted value positives and others don't. I think these are the kind of indicators that we need to look at as we are trying to judge how good a job our detection methods are doing. So, when we think about detecting outbreaks, these, it seems to me, are the things we need to think about. I have mentioned three of them. The sensitivity of the system to detect a bioterrorist event, the specificity in predicting a bioterrorist event, the predictive value of a positive, what kind of response will we make to a false alarm. What I would point out to you is if you set up these systems in many counties or many parts of the country, in any given county, in any given week, there will be zero bioterrorist attacks. Therefore, every positive -- everything above the baseline that stimulates a response, if all you are looking for is bioterrorist events, a hundred percent of the events you detect will be false positives in most counties most of the time. 235

236 So, response you are going to make requires a lot of thought because most of the time it will be response to something that is not the bioterrorist event. The timeliness is obviously critical because we want something that will tell us about something sooner than we would learn about it otherwise and I think that is the real challenge in this. Certainly cost used to be an issue. Maybe it is not so much anymore. So, let me just end -- I think this is the last one -- there are a lot of things we need to think about in figuring out how to affect bioterrorist events. Ken has referred to monitoring people who have a febrile respiratory illness and that is how things like anthrax and small pox will present initially. But botulism and a host of other things would present with different things. So, we need to consider what syndromes to monitor, not just febrile respiratory illness, but possibly other syndromes. What kind of case definitions to use, in order to determine who does and doesn't have the illness, whether we are looking at inpatient or outpatient or other kinds of data. Who will enter the data may be less of an issue in an HMO, but it is a big issue in other health care provider settings? How will the data be transmitted to people like Ken in order to analyze them? How often will the data be 236

237 examined? What statistical parameters will be used to signal a possible event and what actions will be taken in response to that signal? So, I think these are all some of the complexities that we need to, those of us working in public health need to think about and have to have some pretty concrete answers for it. So, let me stop there so we can proceed to general questions and discussion. MS. BROOME: Thank you. Any clarification questions for Dr. Reingold? [There was no response.] We can then open the session for more general discussions. I would like to just remind folks that the overall session was on detection and epidemiology of bioterrorist attacks. So, even though Ken and Art have focused on the detection issue and I think there is a number of threads there that we could pursue, I do hope that we also pay attention to the epidemiology; that is, what is the distribution of an outbreak once it occurs and what is the strategy and effectiveness of the response to that outbreak. I think some of the work that Steve and Sally discussed are relevant in that area. So, I would like to direct folks thinking to both how can mathematics be useful on the detection side, but 237

238 also what are some of the contributions that we get from simulation and modeling in these other areas. So, let me see if we have some general comments or questions for the panel or among the panel. I know you all have been thinking about this a great deal. PARTICIPANT: The question is, I guess, for Sally. To what extent are the models that you are constructing -- what kind of data is there that can actually -- DR. BLOWER: For the HIV models -- PARTICIPANT: Yes. DR. BLOWER: For the HIV vaccine models, all we can do is make sure that they fit the current data because there are no vaccines out there and nothing has been done. So, those you can't test and those models are used more as what could happen and this was not to show whether it would be a good idea or not. For the HIV ones, we are actually just finally getting some data from San Francisco. They have been collecting it, but they haven't actually analyzed it or released it. So, we will actually be -- I am working with Rob Weiss at UC~A. He is, you probably know, in biostatistics and his thing is longitudinal data analysis. 238

239 So, we will actually be fitting the models to data and trying to do so using new statistical methods, not just, you know, minimize squares and things. So, we haven't yet done it because of the data. PARTICIPANT: Also a question for Sally. In your sensitivity analysis and uncertainty analysis, how large are your systems and how many parameters do -- DR. BLOWER: How large are the systems -- I am not quite -- the system, there are about five to eight, depending on which model we are using, ordinary differential equations and they have probably about 12 parameters. Obviously, the different models have different numbers of parameters. When we do a sensitivity analysis, we put all of the parameters in. What you generally find, though, are two or three parameters that are driving the whole system. PARTICIPANT: This is a question for Steve. In social network theory, one of the key difficulties they have always had with that literature is a goodness of fit test to determine whether or not the models that they have had for social network formation are, in fact, adequate descriptions of the data. It seems to me that what you have would be a true or empirical goodness of fit 239

240 evaluation that could be useful for them and then also useful for evaluating whether or not those sort of simple models can be applied to the -- structures that -- disease spread in populations. Would that be something fun to do? MR. EUBANK: Yes. Basically, I agree. It is not just epidemiology. As I mentioned we can take the implications of the network structure that we have, things like traffic or mobile communications or energy use, all these different areas and validate against them all. One slide I was going to show, but didn't was showing the results of the traffic simulation, which estimates traffic counts you would observe on different -- on particular streets in the city. We can verify those against collected counts or just do simple hypothesis testing. Does our traffic look like observed traffic? But it turns out that depending on what we constrain for, we can fit to -- we can have a better fit or a much looser fit. So, we could fit to traffic counts. Works fine, but we might not fit to demographics of the people who are traveling down those roads or it might not fit to the purpose of the trip, the data that is also collected. Part of the problem we have though is the observations are so difficult to come by. In the case of epidemiology, for example, I would love to work on flu and 240

241 get a typical background of flu season in the city, but the reporting requirements on flu are loose and then we need a model for who went to a doctor, who even let it be know that they had flu to anyone. It is kind of hard to convolve all that into the validation of these models. PARTICIPANT: I have a question both to you and to the panel in general. When you had your contact graph, you compared it to the random graph and to the social network graphs. We know how to -- general random graph. We also know how to generalize the social network graphs, using -- attachment. We don't really have good mathematical models, it seems, to generate this kind of degree distribution for your contact graph. So, my question is, one, are there ideas how one could generate those and, two, would it be of any use for antiterrorism to do so? MR. EUBANK: To answer your second question, yes, I think there would be a big use for antiterrorism to do so . How do you generate them? I think what I would need to do is understand what structural properties in the network are important. Maybe it is not even a degree distribution that is important, but something else. Then 241

242 turn the mathematicians and figure out to generate those kind of graphs. For instance, for the small world graphs, Strogatz and Watts decided to look at triangles, numbers of triangles in a graph compared to what you might expect in a random graph. Once you have done that, it is easy to understand how to generate particular kinds of small world structures. PARTICIPANT: So, what would be the structure? MR. EUBANK: I don't know yet. But that is an open question that I would hope mathematicians would be able to give me some advice on. MS. CHAYES: You would probably also be very interested in some type of vertex cover because if you had a vertex cover, then you could go in and knock out the disease at those points and wipe it out. Right? So, that would be a minimal -- the cheapest, most effective way of wiping it out. MR. EUBANK: These graphs -- because we have both people and locations, they are bipartite graphs. If you could find a cover of the locations, that would guarantee you caught every person in the city or every person who was infected, it would be wonderful. 242

243 MS. BROOME: Although I think one of the issues is also separating out the different modes of disease transmission. If you have got a common source outbreak where you have got contaminated food or contaminated water, you know, there are some diseases where a person-to-person spread and contact is important or respiratory droplet spread is important, but there are a number of others where that is not the issue. I have a question for Ken. I was interested -- you know, you are applying your aberration detection tool to influenza-like illness and there is an obvious reason for that and you also do have a lot of historical comparable data. But another approach that is taken for early detection is to try to pick markers that might be more specific. For example, somebody who shows up in an emergency room with a high fever and has a blood culture taken might select for more high probability, more specific likelihood that this is at least a severe disease. There are lots of other things that can present that way, but it might increase the yield. But obviously, you would have much less substantial background and I wondered how good your statistical approach was, you know, in trying to pick up other types of signals. 243

244 MR. KLEINMAN: Actually, I presented the talk as if all we were doing was anthrax surveillance. We also actually run a very similar system for upper respiratory complaints and upper and lower gastrointestinal complaints and we get a lot more visits for upper respiratory complaints than for the lower respiratory and a lot fewer for either of the gastrointestinal things. My sense is from looking at the way the models worked in the past six months when we have been running it, that it doesn't work as well for the rarer events. The events that you are talking about would be rarer still. We have some ideas about how we might do that kind of surveillance anyway, but they haven't been tested yet. So, I don't want to say anything more about them. MS. BROOME: But I mean I do think as a general statement the issue of what types of aberration detection approaches will be useful is going to be highly dependent on, you know, what you are looking for, the specificity, the kind of -- basically the historical data if you are doing time series or other sort of signal to noise ratio issues. So, you know, we have had a lot of success with detecting a particular kind of food borne disease due to salmonella, but in that case we actually have a bacterial 244

245 fingerprint of the particular outbreak organism and we digitize it and we can look across a national database to say there is an excess of that particular organism. That is sort of the opposite, where you are taking a highly specific signal, even though you are trying to pick it out from a huge background. MR. KLEINMAN: Yes. We made the decision to focus on less specific traits because the turnaround time is so much faster. We could wait for a lab analysis, assuming a lab analysis was going to be done on many people who showed up, which is not the case. But we could make the decision to wait for that, but then we have to wait for the lab to process the data and if they have a different data entry system, we have to wait for them to actually enter the data. Then it gets back to us. It is three days after the visit. By then, many, Phase 2 of anthrax. many more people are In So, we could tailor the system to work that way, to be more sensitive. MS. BROOME: But the test orders are in some ways also attractive because the timeliness may be there, but there are a lot of issues that are concealed behind all of the good data that you showed. Yes. 245

246 PARTICIPANT: I wanted to comment that the social networks graphs and the kind of distribution of connectivity that you show in your graphs is very closely related to the kind of linkage you see in the worldwide web pages and there is a heuristic that motivates that that applies to the same kind of situation you are trying to model. So, it is something to go on line to look at, I think. But there is a question I wanted to ask really was I work more in information security. So, when I was listening to this talk, these talks, I was trying to cast them in the light of how can I take what you guys are studying and abstract that away and apply it to the situation of spread of infection among computer systems. This has been attempted by several people in the literature and I see some differences of how some of the assumptions and some of the things you have to deal with simply don't carry over, like the geographic context. It doesn't really carry over in computer systems. Basically everything is connected to everything from our point of view, but there are lots of other things that do carry over and this is the thing, I think, that mathematicians are particularly adept at doing is trying to 246

247 abstract a way and see what commonality is there between these situations. But think about it. It is a very similar problem that is also based in probably homeland security. But, of course, the rate of infection spread is -- MS. BROOME: Yes. PARTICIPANT: I wanted to just take a step backwards and ask about the social organization of a concerted effort to work on some of the very difficult problems that I think arise in this context. We have been having meetings of this type for several months now. What is clear from these is that the epidemiological and surveillance questions pose serious methodological questions that have to be dealt with and I think that they largely have to be dealt with by interdisciplinary teams of researchers. At least that is my impression. And that to really get started, you have to think about how these teams are going to be put together and whether it is, indeed, feasible to have people who are doing very successful research on their own topics, have them break away from that for awhile and work seriously on this very important issue or do we think that perhaps going on doing business as usual with individual researchers 247

248 communicating sort of by e-mail and phone will be sufficient to meet the challenge. Where I am coming from is the fact that what I have seen is a lot of communication like this, but no urgency to move forward and really put something together fast if it hasn't happened right away. So, I just want to hear from the panel and from, indeed, the rest of the participants how we might begin thinking about getting something really going and what structures we might try to work on. DR. BLOWER: I mean, I think that is very interesting and I do think there are lessons from -- Simon was talking about foot and mouth that occurred in the U.K., that the British Government response was to get scientists involved and to have teams and different teams involved directly with the government and focus on that. Obviously, you don't want to do that once small pox is actually happening. MR. EUBANK: There is some problem with time scales. My group is computer scientists, mathematicians and whatever I am, some sort of applications engineer, I suppose. But the mathematicians cannot work on a 60 day turnaround. 248

249 MR. LEVIN: Well, in fact, there has been discussion of something of that sort. There is an Academy report that will be coming out in a week or two and there is likely to be some mention of the development of new structures for that. But I am not -- that is not going to have a very mathematical focus to it. But, I think, indeed, there is a need to take people out of their usual context and put them together for a Manhattan Project approach. MS. CHAYES: But there are some structures for that on a small scale, like the new Bampf Research Center has focused research groups, where people, you know -- something on the order of five or six people can come together for a couple of weeks and work together away from their institutions, if you can manage to get away from your labs and your students and your families, but, you know -- which is obviously a difficult thing, but there are structures in the research community now, I mean, there is the infrastructure to respond to that in a way that there wasn't a few years ago. So, maybe with the impetus of September Ilth -- MS. KELLER-McNULTY: Let me chime in here. I am Sally Keller-McNulty on the board and also chair of CATS. I want to follow directly up on what Tom said. Part of the 249

250 goal of this workshop is to get exactly at that question is that, you know, these are important problems and, you know, this morning we heard in the data mining session that there is a lot of proprietary work that is going on clearly and for good reason and things that are going on in different sectors and how do we share, but if we really want to energize the mathematical sciences community to try to help address the homeland security problems and vice-versa, we have to begin to address exactly what Tom said. How do we do this quickly? You know, how do we find and mobilize the people to try to do these problems and to pull them away from their really successful research? So, anyway, so hopefully, during the course of this workshop some good ideas will come out. MS. BROOME: I would like to suggest that the way the research -- the workshop is set up has the potential in terms of you brought in folks like me, who are not mathematicians, but I actually spend a lot of time thinking about what do we need to have an effective surveillance and response infrastructure. But I must say this is only going to work if we, I think, think about cross disciplinary teams but think about what are the most likely questions or challenges to set them where there is likely to be a payoff. I will pick 250

251 one where I tend to doubt it, the suggestion that maybe we do inverse modeling to find out the source of an outbreak. You know, to an epidemiologist, that is an intriguing thought, but I would much rather send out a team of epidemiologists to interview a bunch of people and find out what is going on. We do that and we generally find the answer. I am not saying that -- PARTICIPANT : [Comment off microphone.] MS . BROOME: Talk to the FBI . I don't think modeling is going to do that one either. On the other hand, I have heard some -- you know, there is clearly a whole area around detection that Ken is addressing, but indicate a much broader range laid out, which are going to alone as was alluded to in the data mining session, aberration, detection issues there is, as I was trying to of possible syndromes, as Art have different challenges, let you know, you have got to have the data to apply these to. So, there is some highly applied questions in having data available electronically so that these wonderful tools can be used. We are sort of working on that side by trying to get down to the nitty-gritty with clinical information technology systems to say what data can we get tomorrow 251

252 that then might be used for these kinds of -- to validate how useful these different approaches might be. I think, obviously, Sally has given a very concrete example of how policy decisions on vaccine usage could be approached with modeling. There is a very active debate on the use of small pox on the vaccinia vaccine or other new vaccines that benefit from that kind of a quantitative approach. So, those are just some things I would throw out as focus areas that are -- you know, would benefit from -- MR. TONDEUR: I want to return to the idea of the infrastructure for this and I want to say two avenues within the Division of Mathematical Science. One is focused research groups, which are specifically to address such issues and the other is an institute we plan to fund attached to the American Institute for Mathematics, which is specifically targeted to have group focused workshops, which actually do the work, where you can assemble teams from different disciplines. MS. CHAYES: I have got one more area, which kind of came up, I think, in Simon's talk, which is games theory. What I have seen happening in network research is that for awhile people were just looking at the structure of networks like the Internet or the worldwide web and now 252

253 they are overlaying game theory on that. So, some cost benefit analysis or some protocol analysis on top of that and if you want to try to implement some of the things that Sally is talking about, you might want, rather than just looking at the differential equations, to take one of Steve's networks and then, you know, put on a game theory functional that would give you some cost benefit analysis on top of that and see how that -- what the results of that are. I think that that is an area of mathematics that people are just starting to look at for networks for the Internet and the worldwide web and it would probably also be very useful in this context. MR. LEVIN: The fact that the networks are in some sense adaptive, that as you make interventions, the networks will change in some of the directions. So, for example, if you remove a focal vertex that in the case of a sexually transmitted disease or prostitute another note becomes crucial. MR . TONDEUR: When you say games theory, I sort of -- if I were a terrorist, I would do the following game theoretic approach. I would say how would I achieve the maximum damage. But then that means we should probably play that same game and say, okay, so if I were a 253

254 terrorist, how would I achieve the maximum damage, then how can we protect against. PARTICIPANT: They don't always think that way. Remember, the main thing, how do they achieve the maximum terror, not the maximum damage. There is a difference between those. PARTICIPANT: That could be a measure of damage. MS. CHAYES: It is a different function. MR. LEVIN: I just don't want people to think about this problem in just the way, you know, how many people are going to die. That is not the only way -- MS. CHAYES: Right. Well, the anthrax certainly didn't kill a lot of people, but it terrorized people, but that is just a different -- I mean, if you want to model, that is a different functional for your game theoretic. MR. LEVIN: That is right, but the point I was making is that for either side, the point would be how do I achieve the maximum damage or terror given that the response is likely to be this. Do I assume that -- or how do I, if I am interested in response system, create a response system based on the fact that the terrorists are going to respond to my response system. So, those are the -- what makes it a game theoretic problem. 254

255 DR. BLOWER: I think the thing -- if somebody said that they had released small pox and alerted all the news media that Washington and New York and San Francisco had now been contaminated and the cases were going to -- that could bluff the government into doing a mass vaccination campaign that would do more harm than good. So, they wouldn't actually have to do anything, just alert the media and the response could be worse than -- I think this needs to be thought about. MR. MC CURLEY: I would like to ask one other question on the detection, only speaking about detection at the level of people getting sick. There are more automated systems that are looking at more pathogenic agents. We can feed into the data mining thing. I don't know how practical it is. My understanding was at the Olympics that we did have some sort of detectors going for anthrax and other things. MR. is doing that MS. KLEINMAN: I know the Department of Defense sort of thing. They have sniffing machines. BROOME: I think, again, it comes back to the specificity and the predictive value of a positive. I mean, just during the Gulf War, there were enumerable alerts of gas that worked as false positives. You know, I think there are some fundamental parameters defining the 255

256 characteristics of these tools tremendous attention to. MR. MC CURLEY: -- way of merging the first session on data mining -- get rid of these delays. MS. BROOME: There is a lot of interest in that and I think a lot of research ongoing, but it is actually a real challenge. I mean, in many of these settings, as has been noted before, you basically have to have a specificity of a hundred percent to have a useful tool. MR. EUBANK: If I could make a comment about the practicalities of getting mathematicians involved in these research areas, we have a curiosity-driven research model. So, that means that my goal is trying to convince mathematicians that the problems we have are interesting for them to work and that they should be curious about them. But if there is some way to drive research based on our problems, I think it would be worth exploring because it is not -- I think the problems are interesting, but I don't always get agreement from the people I am trying to convince. MS. BROOME: Okay. We are over time. I am looking at folks in terms of -- take a couple more questions? Need to break? Quickly. that we have to pay 256

257 MR. MC CURLEY: I wanted to ask how many people here have a degree in mathematics? DR. BLOWER: Mine is in biology. MR. MC CURLEY: So, this is a question that you are really -- how do you get mathematicians to interact. This is actually very rare for mathematicians to interact this way with other scientists. MR. TONDEUR: [Comment off microphone.] [Multiple discussions MR. AGRAWAL: MS. CHAYES: the mathematics you want MR. AGRAWAL: MS. CHAYES: MR. AGRAWAL: MS. CHAYES: PARTICIPANT: MR. AGRAWAL: [Comment off microphone.] Those are the things you -- that is ; to look at. [Comment off microphone.] Collaboration of whom? [Comment off microphone.] Surveillance. [Comment off microphone.] [Comment off microphone.] MS. BROOME: I think one of the issues that Art and I could spend a lot of time talking about is the complexities and the varieties of surveillance. For traditional public health surveillance because it includes individual identities, it doesn't lend itself to wide open, although certainly the project that I am managing actually 257

258 is just getting us on the web for doing traditional surveillance and interfacing with hospitals. But there are also other applications where we do, for example, we are looking at doing kind of survey stuff we do over the Internet so that there is group participation. Then there is some fairly substantial data sets that are put together, for example, of pharmacy data that is de-identified. But most of these are -- you know, they rely on a number of collaborators. I don't think it is the sort of mass computing platform you are thinking about. MR. AGRAWAL: [Comment off microphone.] MS. BROOME: We have thought about trying to -- for example, for our web site, trying to also have two-way communication, where we can record incoming information from physicians or the public. So, you know, I think there is a lot of interest in seeing what could be done with that. But, again, there is complexities when you get down to individually identifiable data that we have to be very conscious of. MS. CHAYES: Do you want the ten minute break now? MS. BROOME: Okay. 258

259 MS. CHAYES is giving a Also, one other thing, everyone who presentation, please give us a transparencies or your power point presentations 259 copy of your .

260 Remarks on Detection and Epidemiology of Bioterrorist Attacks Arthur ReingoIc! Disease surveillance is a systematic ongoing collection, analysis, and dissemination of health data, with findings Iinkect to actions in the ctecision-making process. With regard to bioterrorism, Dr. ReingoIct and his colleagues wouIct like to have a system in place to permit them to ctetect illnesses quickly and then be in a position to respond rapidly to keep morbidity and mortality to a minimum. While the 2001 anthrax attacks generated a lot of concern about detection and rapist response to outbreaks, the pub~ic-heaith community comes to bioterrorism ctetection and response with a gooct clear of relevant experience. Generally, it detects outbreaks because an astute patient or family member notices them: "Gee, aren't a lot of us who ate at the church supper all vomiting at more or less the same time?" Similarly, the community learns a lot about outbreaks from astute clinicians who recognize they're seeing more cases of an illness than they shouIct be seeing and are smart enough to let the health department know about that. However, public health workers rarely ctetect outbreaks. Therefore, the goad is to see if they can be a larger part of this detection process, as opposed to waiting for clinicians and patients to tell them about the problem. People are working in various ways to improve surveillance for the ctetection of bioterror events. One approach is to monitor visits to health-care providers, particularly outpatient visits, emergency-ctepartment visits, ciinicaI-microbio~ogy laboratories, and indicators such as 91~ calls, over-the-counter drug sales, and absenteeism at work and at school. Some people are even considering direct monitoring of samples of the population through Nielsen rating-type setups, in which a large group of inclivicluais is routinely answering questionnaires over the Internet. Three criteria will help us judge a proposed system: Sensitivity. Of all the outbreaks that occur, what proportion of them will be predicted by this system? Specificity. Of all the times there is not an outbreak, what proportion of the time will the system tell us there is not an outbreak? The predictive value of a positive. Of all the times the system tells us there is an outbreak what proportion of the time is it right? 260

Next: Image Analysis and Voice Recognition »
The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop Get This Book
×
Buy Paperback | $147.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Mathematical sciences play a key role in many important areas of Homeland Security including data mining and image analysis and voice recognition for intelligence analysis, encryption and decryption for intelligence gathering and computer security, detection and epidemiology of bioterriost attacks to determine their scope, and data fusion to analyze information coming from simultaneously from several sources.

This report presents the results of a workshop focusing on mathematical methods and techniques for addressing these areas. The goal of the workshop is to help mathematical scientists and policy makers understand the connections between mathematical sciences research and these homeland security applications.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!