Invited Paper: BioWatch Program Overview: A Local Public Health Perspective for Enhancing Bioterrorism Preparedness and Response
This white paper was prepared for the September 18-19, 2017, workshop on Strategies for Effective Biological Detection Systems hosted by the National Academies of Sciences, Engineering, and Medicine does not necessarily represent the views of the National Academies, the Department of Homeland Security, or the U.S. government.
Author and Affiliation
Name: Paul A. Biedrzycki, M.P.H., M.B.A., C.I.H.
Title: Director, Disease Control and Environmental Health (retired)
Affiliation: City of Milwaukee Health Department
THE BIOWATCH PROGRAM OFFICE AND MISSION
The White House Office of Homeland Security and later the U.S. Department of Homeland Security (DHS) Science and Technology Directorate designed, initiated, and launched the BioWatch program in April of 2003 for purposes of enhancing outdoor aerosolized detection of select bioagents within several predesignated metropolitan areas throughout the country. The primary mission of the program was to mitigate the severity of public health impact associated with widespread airborne exposure to select bioagents in the context of both potential widespread human exposure and rapid deployment of medical countermeasures for population treatment and prophylaxis.
The BioWatch program was originally conceived and developed as an early warning and detection system consisting of a network of indoor and outdoor biological portable air sampling units (PSUs) strategically placed throughout a targeted geographic region and continuously operational throughout the year. Each PSU within a BioWatch jurisdictional network collects air samples at regular intervals that are then remotely analyzed by predesignated laboratories for select bioagent presence. Full confirmation of sampling results is typically available 10 to 36 hours after PSU sample collection. The BioWatch system as currently configured is not capable of discriminating between detection of naturally occurring microorganisms and those intentionally released as an act of bioterrorism.
The BioWatch program and system was developed in response to a growing and perceived risk of a bioterrorist attack within populated urban areas and/or toward critical infrastructure. It is important to note that the BioWatch program was implemented in the direct aftermath of the 2001 Amerithrax episode in which confirmed Bacillus anthracis was distributed through common mail delivery utilizing U.S. Postal Service–designated facilities and distribution modalities. The Amerithrax incident resulted in 22 confirmed cases of B. anthracis infections (inhalation and cutaneous), including 5 deaths, and in many ways underscored as well as validated the potential bioterrorism threat to the United States. BioWatch was also proposed during a similar period of time when the U.S. invasion of Iraq (a.k.a. Operation Iraqi Freedom) was being contemplated for purposes of mitigating the threat posed by weapons of mass destruction.
In a more contemporary context, the evolving nature of biothreats is widely recognized by several federal law enforcement, national security, military, and health agencies. The current biothreat dynamic is increasingly informed and influenced by the rapid growth of biotechnology, including the emergence of synthetic biology as a legitimate academic research pursuit, acceleration of economic opportunities fueling biotechnology-sector growth, and ease of information exchange via the Internet and other various electronic data repositories. The convergence of these factors has resulted in the scope and breadth of biothreats exponentially expanding over the past decade in terms of potential actors and array and ease of technical capabilities and acquisition.
The BioWatch Program Office is overseen and managed at the federal level by the DHS Office of Health Affairs (OHA) Health Threats Resilience Division. This was a direct result of the issuance of Homeland Security Presidential Directive 10 (Biodefense for the 21st Century, 2004). The BioWatch program is further supported by various other federal agencies such as the Centers for Disease Control and Prevention (CDC), the Department of Health and Human Services Office of the Assistant Secretary for Preparedness and Response (ASPR), U.S. Environmental Protection Agency (EPA), Department of Defense (DoD), and U.S. Federal Bureau of Investigation (FBI) to ensure alignment and continuity across the spectrum of planning and response conducted by the U.S. government. The BioWatch program is active in more than 30 jurisdictions in the United States as well as deployable during special venues or national security events in jurisdictions as requested by local authorities and designated or approved by the national BioWatch Program Office.
In addition to the role of consequence management that has been typically absorbed by local health departments (LHDs), the BioWatch program fundamentally consists of both field operations and laboratory analysis components that are closely coordinated and monitored by LHDs in coordination with jurisdictional coordinators (JCs) assigned by the national BioWatch Program Office. Each of these components is governed by extensive standard operating procedures and quality assurance and control measures that ensure, to the greatest extent feasible, reporting of timely, accurate, and reliable data necessary for rapid and strategic public health prevention and intervention actions. Currently, BioWatch operations and technology are limited to an estimated range of 10 to 36 hours between exposure and confirmation and response to laboratory-detected select bioagents.
Nationwide, BioWatch enlists 200 local, state, and federal partners in coordinated implementation of the BioWatch air-monitoring network and mission. Typical local stakeholders include emergency management agencies, local police and fire departments, emergency medical services, and other community first-responder organizations. This level and scope of collaboration is crucial to the success of the BioWatch endeavor, particularly in limiting severe morbidity and mortality as a result of a scenario involving unmitigated and widespread exposure to a large population as the result of an intentional indoor or outdoor aerosolized release of a bioagent. This level of regional partnership is often cited as a hallmark of the BioWatch program at the jurisdictional level as well as central to the success of local planning efforts.
BIOWATCH ADVISORY COMMITTEES (BACs)
BioWatch Advisory Committees (BACs) are integral to robust and meaningful response planning that engages appropriate community stakeholders, including key first responders such as emergency medical services (EMS), local law enforcement, fire department HazMat teams, hospitals, environmental agencies, and emergency management offices. BACs also are a visible representation
of BioWatch Program Office’s commitment to support and collaboration with local authorities in successful achievement of the program mission. LHDs typically serve as BAC chairs, organizing and leading meetings, developing appropriate agendas, and overseeing protocols and procedures necessary to successful planning and response. BioWatch program JCs provide technical support and consultation to BACs and are integral to ensuring the standardization of protocols and procedures necessary for effective communication and program efficiency between local stakeholders and the BioWatch Program Office. BACs serve as the regional planning and response point of contact for the BioWatch Program Office, ensuring communication and coordination throughout the span of system operation and evaluation. Finally, BACs interact with or create the framework for the requisite Incident Command System, which is potentially necessary should the BioWatch system detect and later confirm a select airborne bioagent of public health consequence.
THE BIOWATCH ACTIONABLE RESULT (BAR)
In most jurisdictions in which the BioWatch air-monitoring network is present, LHDs serve as the primary agencies responsible for pre-event planning and consequence management associated with a declaration of a BioWatch Actionable Result (BAR). A BAR declaration represents a laboratory-confirmed positive detection of one or more genetic markers of select bioagents on a single and/or multiple sampling unit filters collected within a jurisdiction’s BioWatch air-monitoring network. BARs are reported by designated BioWatch laboratory facilities, many of which exist within the CDC Laboratory Response Network (LRN), and have undergone specialized personnel training and proficiency testing. Filters from individual Biowatch PSUs located within a jurisdiction network are collected and transported or arranged for secure delivery to a BioWatch laboratory by trained environmental field staff observing appropriate chain-of-custody procedures.
BioWatch laboratories typically perform and report air monitoring unit filter analysis within 12 to 18 hours postfield PSU filter collection to DHS, CDC, and LHDs. Analytical methods used by BioWatch laboratories employ real-time polymerase chain reaction assay panels for both initial screening for microorganism identification and subsequent confirmatory analysis of microorganism DNA signature. BioWatch-designated LRN facilities report all individual jurisdiction-confirmed BARs immediately to the OHA desk at the DHS National Operations Center (NOC) upon validation by the BioWatch Laboratory Director and submission to the CDC Emergency Operations Center. Consequently, jurisdictional notification protocols are initiated prior to a formal BAR declaration and following optional BioWatch laboratory technical consultation.
A confirmed BAR, along with associated LRN laboratory analytics, does not validate that a bioterrorism attack has occurred or that there is an imminent risk to the public’s health. Furthermore, while a BioWatch laboratory analysis can provide limited microorganism subspeciation analysis, it does not typically
provide information on the viability of the organism, organism virulence, or any genetic modifications to an organism related to antimicrobial resistance or environmental persistence. A BAR can represent an environmental detection of the genetic material from a naturally occurring organism. All BARs reported since the launch of the BioWatch program fall into this category, including several BioWatch system detections of Francisella tularensis in the Denver, Colorado, region since 2014. Regardless, a report of a BAR is a recognized and defined “trigger” for some level of federal, state, and LHD action and is therefore treated as a response priority within a BioWatch jurisdiction.
The declaration of a BAR can have significant consequences for state and local health departments, who are generally responsible (and often legally mandated) to address prevention and control of communicable diseases within their respective jurisdictions. This potentially includes implementation of both non-pharmaceutical interventions such as enforcement of isolation and quarantine measures and well as distribution of pre- and postexposure prophylactic medical countermeasures. For credible large-population-exposure scenarios, this conceivably entails federal, state, and/or local declaration of a public health emergency and subsequent request for deployment of the Strategic National Stockpile (SNS) assets by the CDC. Receipt, storage, distribution, and dispensing of the SNS is a core objective of the CDC’s Cities Readiness Initiative (CRI) launched in 2005 with funding to state and local health departments to coordinate preparedness activities in this regard. The declaration of a BAR by the BioWatch program represents one of the most plausible scenarios in which the SNS would be deployed with associated state and LHD operational activation following the CRI response model.
BIOWATCH NATIONAL CONFERENCE CALL AND LOCAL TELECONFERENCES
An important protocol within BioWatch program management integral to a strategic, coordinated and successful response to a declared BAR for single or multiple air-monitoring collectors within a jurisdiction are BioWatch local and national teleconference calls. Each of these are held under a strictly defined timeframe following the declaration of a BAR for purposes of reviewing and assessing field, laboratory, epidemiologic, and environmental data as well as law enforcement intelligence that would suggest or corroborate a potential bioterrorism threat or risk.
The local conference call is conducted typically within 1 hour of a BAR declaration within a jurisdiction and is arranged and conducted by the BAC chair with support from the JC. The purpose is to initially review available data and prepare for the BioWatch National Conference Call (BWNCC) typically conducted with the DHS NOC, the appropriate BioWatch LRN Director, and designees from other federal agencies such as CDC, EPA, and FBI.
The BWNCC is instrumental to (1) providing a comprehensive overview of local situational awareness and available data related to declared BAR(s) and
for participating federal, state, and local partners and stakeholders; (2) assessing credibility and plausibility of an intentional aerosolized release of a bioagent; (3) reaching consensus on a key strategic response to the declared BAR(s) for the protection of public health, including the need for enhanced human and animal surveillance, initiating Phase I environmental sampling, providing public risk communication, and/or launching a request for the CDC’s SNS; and (4) requesting federal response resources. The BioWatch Program Manager and/or the jurisdictional BAC chair typically moderates the BWNCC with notes taken by the JC or designee and disseminated to participants afterward as well as posted on the BioWatch portal.
BIOWATCH PHASE I AND PHASE II ENVIRONMENTAL SAMPLING
Phase I and Phase II environmental sampling is a component of the BioWatch post-BAR declaration protocol that entails collection of field samples at and near PSU locations for purposes of further characterizing a bioagent release and dispersion trajectory. This sampling is initiated by request of the BAC and upon consensus with the BioWatch Program Office and as the result of strategic discussion and decision making immediately following the BWNCC. Both Phase I and Phase II environmental sampling can be used to inform event reconstruction modeling by National Laboratories at the request or direction of the BioWatch Program Office.
Phase I sampling refers to the collection of field environmental sample(s) at and around the actual PSU site. Phase II sampling refers to additional field environmental sampling within a prescribed perimeter and distance outside of the PSU site and only if one or more Phase I sampling results are confirmed to be positive and informed by air dispersion modeling data.
The EPA is the lead agency in supporting and/or conducting Phase I and Phase II environmental sampling in coordination with the jurisdictional BAC, BioWatch Program Office, and assigned BioWatch LRN. This includes delineating sample locations, personal protective equipment requirements, number and type of samples, assurance of chain of custody and evidence-preservation measures, and secure transportation and delivery to the BioWatch LRN. Phase I and Phase II environmental sampling is also used to inform and support event reconstruction modeling by National Laboratories, which is necessary to delineate the scope and breadth of bioagent release, geographic impact, as well as acute and chronic health risk to potentially exposed populations.
BIOWATCH COMMUNICATIONS STRATEGY, STAKEHOLDER WORKGROUPS, AND WEB PORTAL
The BioWatch program strongly emphasizes and routinely promotes information sharing between all partners and stakeholder groups both as part of preparedness planning and during response related activities. Furthermore, the integration of intelligence fusion practices at both state and local levels, along
with support of the National Biosurveillance Integration Center (NBIC), is central to the program mission. For the most part, specific details related to both strategic and operational practice under the BioWatch program remain in the realm of law enforcement–sensitive, For Official Use Only (FOUO) designation. However, this has not significantly inhibited the development of practices and guidance for local jurisdictions necessary to build or enhance communications strategies for policy makers, businesses, universities, and the general public.
The BioWatch Program Office has also created several stakeholder workgroups to further facilitate federal-local collaboration, comprehensive strategic planning, and cross-jurisdictional information exchange and protocol quality assurance as related to achieving program outcomes. These include the Public Health Engagement Network, the Public Information Officers’ Network, and the Military Civilian Working Group. Workgroup meeting frequency, agenda setting, and summary reports are coordinated by the BioWatch Program Office and available through the BioWatch Portal, a secure restricted-access website available to all federal, state, and local BioWatch stakeholders. The BioWatch Portal is a valuable stakeholder tool for dissemination and review of planning guidance, communication of protocol and procedural updates, posting of workgroup summaries and updates, announcements of training opportunities, inventory for BioWatch jurisdictional exercise after-action reports, and summary reports retrospectively of BAR occurrences nationwide. However, cybersecurity concerns have been raised concerning the BioWatch Web Portal in terms of outsider access and information availability that deserve further attention and scrutiny by OHA and the BioWatch Program Office.
BIOWATCH VALUE PROPOSITION FOR PUBLIC HEALTH EMERGENCY PREPAREDNESS (PHEP)
In its current form and level of operational efficiency, the BioWatch system provides a platform for development of airborne biological early warning and detection previously absent from the local public health department portfolio. It has laid the early framework for regional collaboration and interaction with a wide variety of stakeholders necessary to effectively respond to and mitigate impact from an intentional bioagent release across a metropolitan area. This is best exemplified through the use of BACs and federal coordination and consultation services provided by the BioWatch Program Office as well as JCs.
The BioWatch system has informed integration of other bio-counterterrorism planning activities conducted at the LHD level including those funded through CDC Bioterrorism Cooperative Grant Program, Cities Readiness Initiative, and Pandemic Flu Preparedness. Each of these programs is designed to enhance LHD capacity and capabilities associated with planning, response, and recovery to a credible bioterrorism threat and are calibrated to minimize the severity and scope of population health morbidity and mortality. Indeed, a reported BAR has increasingly been used by LHDs in training and exercise scenarios as a trigger for public health emergency preparedness (PHEP) activation
and tiered prevention and response such as mobilization of medical countermeasure distribution within an affected community.
The current BioWatch system model has also catalyzed significant dialogue and discussion concerning public risk messaging before, during, and after report of a credible BAR. Although specific BioWatch system operational logistics and geographic placement of the air-monitoring network are considered FOUO and law enforcement sensitive, the endorsement and encouragement by the BioWatch Program Office for accurate and transparent risk communication to the public has been recognized as essential to the program mission and success. The use of Joint Information Centers’ risk-messaging templates and engagement of public information officers within BioWatch jurisdictions in preplanning and product evaluation is evidence of this priority.
The BioWatch program as implemented at the local level represents environmental biosurveillance that engages an array of public health expertise that convenes human and animal epidemiology, laboratory analytical methodologies, and environmental health practice. This comprehensive approach, further strengthened by partnerships with EMS, private health care, law enforcement, and emergency management agencies along with interjurisdictional considerations, results in BioWatch inhabiting a unique niche within the traditional PHEP paradigm. The BioWatch program is value added in creating and reinforcing a regional, collaborative, and cross-disciplinary model for bioevent preparedness and response.
CURRENT BIOWATCH PROGRAM LIMITATIONS
The existing BioWatch air-monitoring network and mode of operation does have several constraints that limit its utility and optimization as an aerosolized bioagent early warning and detection system. Among these are the following:
- Limited array of microbial pathogen analytical capabilities by designated BioWatch LRN facilities. Current assays and laboratory analytics focus on select microbial pathogens of concern based on presumably previous national security agency risk-based intelligence. BioWatch does not incorporate chemical or radiological threats under the current operational portfolio.
- Limitations on autonomous pathogen detection that would enhance early warning and detection capabilities. The BioWatch program heavily relies on remote laboratory analysis of PSU air-monitoring filters, which is time consuming and presents challenges and/or vulnerabilities in terms of field collection, transportation, chain of custody, and evidence preservation.
- Lack of comprehensive population coverage within the current designated network configuration. The current BioWatch system is not configured to provide 100 percent population coverage in jurisdictions in which
it is deployed. Therefore, gaps exist in terms of identification of aerosolized pathogen release and accurate estimation of population exposure and health risk.
- Limited and nonuniform distribution of air-monitoring technology within indoor and outdoor environs including near or within critical infrastructure, private-sector, or high-risk target locations. PSUs are currently located primarily based on security and operational logistics (power availability). This limits the scope and breadth of strategic placement of sampling units and optimization of the air-monitoring network.
- Lack of near-real-time event reconstruction capabilities following reporting of a BAR to LHDs for environmental exposure analysis. A lack of immediate situational awareness to LHDs that would ideally include electronic fusion and sharing of environmental, clinical, epidemiologic, laboratory, and law enforcement data necessary to assess and address population health risk encumber the current system.
- Lack of adequate funding mechanism to LHDs and partners for sustaining BioWatch preparedness and response capabilities. LHDs continue to experience both state and federal funding cuts that will compromise PHEP capacity and capabilities, including meaningful participation in the BioWatch program.
- Lack of standards of practice for next-generation BioWatch technology and field-deployment metrics. The BioWatch program needs to continue to sufficiently engage state and local health departments in the design, planning review, piloting, and evaluation of new generations of system technology that ensure alignment with public health practices and community assessment and improvement planning.
The BioWatch system and network were not necessarily conceived to meet the expanded criteria listed above given the timeline for federal deployment and necessary field testing and validation. In addition, the changing landscape of state and local public health in the context of health care reform, the evolving health care business model, and the emergence of health care coalitions as the nexus for regional preparedness demand rethinking of next-generation biosurveillance systems. This will inherently require coalescence of clinical, environmental, and epidemiologic electronic data collection, analysis, and dissemination across the public and private health care enterprise within a community and vertically between government and medical-oriented business sectors.
BIOWATCH: FUTURE CONSIDERATIONS ON THE WAY FORWARD
State and local public health agencies, along with health care providers, are on the front line for preventing and controlling communicable disease transmission within communities and encompassing regions. Surveillance and epidemiologic follow-up for communicable diseases is a core competency of
public health departments to minimize population morbidity and mortality associated with unanticipated or unmitigated infectious disease transmission within a community.
Environmental biosurveillance systems such as BioWatch provide valuable insight as to the complexity of operational logistics and metrics inherent in collecting and providing meaningful information to public health agencies. Consideration and attention to ensuring the accuracy and reliability of real-time autonomous microbial detection at extremely low thresholds across a spectrum of pathogens is paramount.
Similarly, providing this information in a format and frequency that is compatible with current and emerging local public health communicable disease surveillance and epidemiologic investigation protocols and models is equally important. Providing local public health agencies with data that are ambiguous, uncertain, or unable to be validated creates inefficiencies in response and potentially undermines public health risk messaging credibility.
Although the current BioWatch system as launched in 2003 through the DHS OHA is imperfect with regard to the above criteria, it does provide a context if not a foundation for continuing state and local public health engagement regarding environmental bioagent detection. The BioWatch program has successfully engaged regional planning and preparedness through initiation of BACs. It has also further delineated the role of environmental health professionals in the planning and response to public health emergencies involving communicable diseases.
The BioWatch program has also catalyzed relationship building between law enforcement and public health officials in forwarding the value of “intelligence fusion” in the prevention of and/or response to an intentional (or unintentional) bioagent release or bioterrorism event. Timely information sharing across disciplines and levels of government and community has been shown time and time again to be instrumental in successful preparation, response, and recovery to emergency incidents that threaten community safety and health.
Below is a short list summarizing future BioWatch technology or similar system needs to attract and advance collaboration with state and local public health agencies.
- Future technologies must expand on existing and validated capabilities of the current system and provide a cost-effective and sustainable platform for routine public health biosurveillance as well as for emerging infectious diseases. This includes rapid and reliable disease indicator reporting across a defined geographic region that leverages and/or integrates within existing medical and public health infrastructure and systems. Ideally, the system must inform, coincide with, and be easily reconciled within the existing state/local public biosurveillance model reliant on reportable communicable disease, laboratory and syndromic surveillance data (e.g., Electronic Surveillance System for the Early Notification of Community-
based Epidemics [ESSENCE] and BioSense), as well as emerging electronic event-based systems and paradigms.
- Future technologies must improve the sensitivity and specificity of airborne organism detection and identification at low thresholds and include information on the stability and viability of organisms recovered via outdoor and/or indoor collection devices. This enhances public health decision making as to population risk associated with adverse exposure pathways and improves confidence in deploying necessary prevention and intervention measures to mitigate deleterious outcomes.
- Future technology must be vetted and deployed with consensus and at a timeline agreeable to state and local public health agencies in coordination with other local stakeholders. This includes reviewing field and laboratory technology sensitivity and specificity metrics as well as objective evaluation of pilot systems or models to be deployed at the local level. Other considerations related to the Health Insurance Portability and Accountability Act and cross-jurisdictional information exchange associated with technology design and implementation should be inherent and prominent in early strategic planning and activation of any system.
- Future technology must incorporate a sustainable funding mechanism to support state and local health departments’ meaningful participation through appropriate capacity and capability essential to authentic partnership and success of the program. Local health departments’ budgets are increasingly limited as well as strained even in meeting core competencies, duties, and responsibilities outlined by state and local statutes.
- Future effective technology must integrate within emerging LHD biosurveillance (emergency medical records) and regional public health emergency preparedness paradigms (health care coalitions). PHEP programs at the state and local levels are being redefined in terms of federal grant objectives and complexion of partnership and outreach to achieve meaningful community resiliency.
- Future technology must be valued added to state and local public health operations on a daily basis (e.g., inform daily public health practice around PHEP, CD, and EH). The technology must provide exceptional situational awareness and not conflict with existing public health networks, information-sharing models, or electronic data collection, analysis, interpretation, and dissemination methodologies.
- Future technology must be interoperable and unequivocally adopted, promoted, and supported across federal agencies involved in National Health Security and cogent biosurveillance missions (e.g., CDC, ASPR, EPA, DoD, DHS, and FBI). Technology should not compete with other federal agency programs but rather complement or enhance program outcomes at the state and local levels. The NBIC’s role and responsibility in the context of clarifying current biosurveillance activities should be re-calibrated, and the availability of technical assistance and information to
better meet state and local public health needs should be considered. Technology outcomes should be aligned with the 2012 National Strategy for Biosurveillance.
- Developing meaningful and reasonable systems of field and laboratory operational evaluation metrics in consultation and cooperation with state and local public health authorities prior to technology implementation is essential. Defining parameters that delineate successful outcomes of system deployment for policy makers and the general public is critical to fostering partnerships, enhancing risk communication, and improving community preparedness and resiliency.
Finally, the future success of any federally funded and managed environmental biosurveillance system requires development of a close, trusted, and sustainable relationship with local public health and first-responder communities. Surveillance technology is a tool and relies on a close human interface to ensure the decision making based on remote field pathogen detection is vetted for accuracy and reliability prior to implementation of prevention/intervention strategies.
Next-generation BioWatch system technologies must therefore inherently contain evaluation metrics that are acceptable and provide “meaningful use” to state and local public health agencies that are responsible for consequence management planning and response activation. Therefore, federal agencies like DHS OHA should develop strong relationships and communication pathways with local public health agencies to ensure close alignment between environmental biosurveillance initiatives and current local public health emergency preparedness models.
Anderson, J. A., C. N. Rozenweig, J. Roos, and B. Flores. 2015. The Global Biosurveillance Portal: Biosurveillance for the Department of Defense. Online Journal of Public Health Informatics 7(1):e63.
Brodsky, B. 2008. The Next Generation of Sensor Technology for the BioWatch Program. Washington, DC: Nuclear Threat Initiative. March.
Committee on Homeland Security, House of Representatives Subcommittee on Emergency Preparedness Response. 2014. Biowatch: Lessons Learned and the Path Forward. November 10.
DHS (Department of Homeland Security). 2016. The BioWatch Program. https://www.dhs.gov/biowatch-program (accessed October 24, 2016).
Gajewski, K. N., A. E. Peterson, R. A. Chitale, J. A. Pavlin, K. L. Russell, and J. P. Chretien. 2014. A review of evaluations of electronic event-based biosurveillance systems. PLoS ONE 9(10):e111222.
GAO (U.S. General Accountability Office). 2015. Biosurveillance: DHS should not pursue BioWatch Upgrades or Enhancements until System capabilities are established. Report to Congressional Requesters, GAO-16-99, October.
The Heritage Foundation. 2012. BioWatch: Enhancing Biological Threat Detection. Homeland Security Report. July 10.
Huff, A. G., N. Breit, T. Allen, K. Whiting, and C. Kiley. 2016. Evaluation and verification of the global rapid identification of threats system for infectious diseases in textual data sources. Interdisciplinary Perspectives on Infectious Diseases 2016:5080746.
IOM and NRC (Institute of Medicine and National Research Council). 2011. Biowatch and Public Health Surveillance: Evaluating Systems for the Early Detection of Biological Threats: Abbreviated Version. Committee on Effectiveness of National Biosurveillance Systems: Biowatch and the Public Health System. Washington, DC: The National Academies Press.
IOM and NRC. 2014. Technologies to Enable Autonomous Detection for BioWatch: Ensuring Timely and Accurate Information for Public Health Officials: Workshop Summary. Overview of the BioWatch Program. Washington, DC: The National Academies Press. doi: 10.17226/18495.
Jackson, H. 2017. Securing the BioWatch web portal. Journal of Bioterrorism and Biodefense 8:1, doi: 10.4172/2157-2526.1000153.
Kman, N. E., and D. J. Bachmann. 2012. Biosurveillance: A review and update. Advances in Preventive Medicine 2012:301408.
Shea, D., and S. Lister. 2003. The BioWatch Program: Detection of Bioterrorism. Congressional Research Service Report RL 32152, November 19.