|
||||||||||||||||
|
||||||||||||||||
The following HTML text is provided to enhance online readability. Many aspects of typography translate only awkwardly to HTML. Please use the page image as the authoritative form to ensure accuracy. 3
|
||||||||||||||||
 |
 |
Page 61 |
 |
 |
 |
|
| ||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 61
Biowatch and Public Health Surveillance: Evaluating Systems for the Early Detection of Biological Threats - Abbreviated Version
3
Evaluation of the BioWatch System
The committee was tasked with evaluating the relative merits and current and potential capabilities of the BioWatch monitoring system (Generation 2 and Generation 3) to detect bioterrorist attacks and other biothreats. This chapter provides (1) an evaluation of the current BioWatch system; (2) an assessment of plans for the next generation of environmental monitoring technology; and (3) recommendations for improving the program’s operation and processes for further development.
In its evaluation, the committee considered the entire BioWatch “system,” as introduced in Chapter 1—the technology to collect and test air samples, the associated laboratory assays, the additional information gathering needed to confirm and characterize an incident, operational guidance, interagency and risk communication, response planning, and the personnel to support these operations. It also looked at the BioWatch program—the programmatic activity managed and funded by the Office of Health Affairs (OHA) in the Department of Homeland Security (DHS) and carried out through work with other federal agencies and state and local partners.
EVALUATION OF THE CURRENT BIOWATCH SYSTEM
For the BioWatch system to effectively detect a terrorist attack and prevent illness and death, a variety of conditions must be met. Some of these conditions are clearly independent of the performance of the BioWatch system (e.g., whether people are exposed to the biological agent, or whether the agent is susceptible to medical countermeasures), but they are necessary conditions for prevention of illness and death:
OCR for page 62
Biowatch and Public Health Surveillance: Evaluating Systems for the Early Detection of Biological Threats - Abbreviated Version
An airborne bioterrorist attack must occur in a place where the BioWatch system is deployed and with a biological agent that BioWatch air samplers and laboratory test methods can detect.
Atmospheric conditions must disseminate the material in a way that exposes people to a viable pathogenic biological agent.
Atmospheric conditions must disseminate the material in the direction of the BioWatch air samplers, resulting in sufficient quantities reaching their filters to allow detection.
The samplers must function as anticipated; samples must be collected, managed, and tested appropriately to allow for detection of the genetic signature of the biological agent; and positive results must be reported to public health authorities in time to allow for any assessment that must be made before a decision regarding mass prophylaxis can be reached.
Public health and other authorities must collaborate effectively in conducting an assessment of the likelihood that the positive test indicates a bioterrorist attack and a population health risk, based on the mix of factors that may shape such assessments (see Chapter 2); and, taken together, the correct conclusion must be made that mass dispensing is indicated.
Infection with the biological agent can be successfully prevented or treated with the drugs or other medical countermeasures that could be used in the mass prophylaxis program.
State or local public health systems must be able to deliver mass prophylaxis in time for its benefits to be realized; depending on the disease-causing agent and the size of the exposed population, this may involve providing prophylaxis within 2–3 days to several million people.
As noted in Chapter 1, evaluation of the likelihood of an airborne biological attack that could be detected by the BioWatch system is beyond the committee’s scope, as is an evaluation of the ability of public health officials to deliver mass prophylaxis in a limited time period. Here, the committee evaluates elements of the BioWatch system that are necessary, if not sufficient, for the system to be effective, including the air samplers and their placement, the samplers’ capture of targeted organisms, the laboratory assays, information reporting, event characterization, and public health decision making.
PERFORMANCE OF BIOWATCH TECHNOLOGY
The committee considered information provided by DHS regarding the separate components of the BioWatch system, as well as information relevant to the operational capabilities of the components working together.
OCR for page 63
Biowatch and Public Health Surveillance: Evaluating Systems for the Early Detection of Biological Threats - Abbreviated Version
BioWatch Air Samplers
There are two types of BioWatch air samplers: the Portable Sampling Unit (PSU) and the Dry Filter Unit (DFU). Information about the performance of these devices is critical for a realistic assessment of system performance. The committee examined the available evidence provided by DHS, but some of the information that the committee would have liked to review has not been collected. The committee also considered information about the number and placement of air samplers in BioWatch jurisdictions and the analytic tool that was developed by the Los Alamos National Laboratory and used to guide their placement for the Generation 2 deployment. The committee considered the siting methodology applied in the Generation 2 deployment to be an improvement over the original approach, but it saw areas where better data and modeling approaches for complex urban environments would be valuable.
Analysis of BioWatch Samples
Since 2003, millions of BioWatch samples have been analyzed. Positive laboratory findings have led to BioWatch Actionable Results (BARs) being declared in a few dozen instances, none of which has been shown to be the result of the intentional release of a biological agent. DHS does not consider these BARs to be “false positives” because the target DNA was actually detected by the specified assays.1 The committee’s view is that from an operational perspective these detections can indeed be considered “BAR false positives,” because the detections were not the result of a bioterrorist attack.
The laboratory assays used to detect the presence of genetic material from biological agents of interest need to be both specific and sensitive: that is, the assay must have a high probability of detecting the biological agents that BioWatch targets and have a low probability of reporting the presence of genetic material from other organisms as “detections.” Several agents of concern, and their close genetic relatives, exist naturally in water or soil (Kuske, 2005; Kuske et al., 2006). Distinguishing natural background levels of endemic agents from a bioterrorism threat and separating genetic “near neighbors” from the target agent present technical challenges. In addition, the potential exists for newly emerging or “engineered” threats to be used in an airborne biological attack. Ideally, BioWatch probes and assays could
1
The probe for PCR screening confirms the presence of DNA matching the probe; however, a given DNA signature may be shared between species sharing the same branch on a phylogenetic tree.
OCR for page 64
Biowatch and Public Health Surveillance: Evaluating Systems for the Early Detection of Biological Threats - Abbreviated Version
be rapidly updated as newly emerging or engineered threats are recognized and as risk-management analyses indicate a need.
An assay that is not sufficiently sensitive can give false negative results, if, for example, an airborne threat were present but was undetected during analysis. This “missed detection” probability is a parameter that is essential for evaluating the effectiveness of the system. In addition, the PCR assay panels employed on the existing analysis platforms need to be adequately compared, standardized, and validated in pursuit of optimal performance across the BioWatch system. A close collaboration is needed between DHS and the Department of Health and Human Services (HHS) to do so.
Collection efficiency, efficiency of extraction, and subsequent sample dilution are all important factors in assessing a detection system’s sensitivity. Members of the committee had the opportunity to review information about the BioWatch Generation 2 sensitivity levels, but the committee did not have adequate test and validation data to evaluate the ostensible performance of the BioWatch technology. However, as discussed earlier, the experience available from BioWatch Generation 2’s deployment to date provides real-world data to consider: numerous BARs for certain agents, and no alarms (and no apparent misses) for other BioWatch target pathogens.
To the extent it is well understood for a particular biological agent, the consideration of infectious dose (i.e., the minimum amount of a biological agent necessary to cause infection in a host) may help in the interpretation of BioWatch results. However, uncertainties about infectious doses for some agents, differences among pathogens in the size of their infectious doses, and uncertainties about the concentrations and dispersion of an aerosolized agent used in an attack mean that the committee does not recommend that consideration of infectious dose be a factor in distinguishing between an attack and natural background. To the extent that the analysis of BioWatch samples can relate the amount of genetic material present in a sample to infectious dose, the information is likely to aid public health officials in planning a response. But BioWatch collectors and assays should be sensitive enough to detect small quantities of genetic material for a targeted pathogen, because the presence of small quantities at one collector may be related to the presence of much larger quantities elsewhere or, perhaps, a failed attack that requires further investigation.
System Testing and Evaluation
Test and evaluation (T&E) processes provide an opportunity to assess a system’s actual performance against the stated requirements and specifications, to assess technical maturity, and to provide assurance that the system will effectively perform its mission. Developmental T&E (DT&E) focuses on the technological and engineering aspects of the system, and operational
OCR for page 65
Biowatch and Public Health Surveillance: Evaluating Systems for the Early Detection of Biological Threats - Abbreviated Version
T&E (OT&E) evaluates a system’s ability to perform as intended in its operating environment, including consideration of end users’ capabilities and proficiencies. OT&E can identify critical issues in system operations (e.g., failure, inaccuracies, subsystem incompatibility, and cost) that need to be resolved to ensure that the system performs to stated capabilities. OT&E can also provide the opportunity to empirically and realistically compare the costs and benefits of next-generation technology against currently fielded systems. It provides the opportunity to consider otherwise-unforeseen barriers to accurate and reliable detection and to make corresponding system improvements before full-scale deployment.
At Dugway Proving Ground, it is possible for an entire sampling device or its components to be challenged with simulants2 or live agents in chambers or to undergo whole-system testing with simulants in open-air tunnels. DT&E at Dugway allows a designer to assess the performance of a system in a relatively “pristine” environment, without consideration of confounding factors (e.g., weather, pollution, humidity, buildings or other physical structures, and people and animals and pathogens associated with them) that may affect the performance of a system in an urban environment.3 The BASIS technology that was adapted for BioWatch underwent DT&E at Dugway, where it was challenged with live agents inside a sealed chamber, but the results from the tests were not provided to the committee.
With only limited information available from test and evaluation studies, the committee could not appropriately evaluate whether the BioWatch system meets user needs.
FINDING: The rapid deployment of BioWatch Generations 1 and 2 allowed limited opportunities for testing, validation, and evaluation of the system. Rigorous technical and operational testing and documentation of system performance are needed as a basis for risk-management decisions.
PUBLIC HEALTH RESPONSE TO A BAR
DHS has direct programmatic responsibility for the BioWatch air samplers, laboratory assays, and the reporting of assay results by its contractor staff to local public health officials. DHS’s role after the declaration of a BAR, along with that of HHS/CDC and other federal partners, is to support
2
Simulants to be used in such tests would be nonpathogenic or nontoxic surrogates for bioterrorism agents. Testing with well-selected simulants would provide useful information for evaluating the performance of a biological detection system (NRC, 2008a,c).
3
Dugway Proving Ground, located in Utah, has minimal background interferents. It is dry, isolated, and free of urban pollutants.
OCR for page 66
Biowatch and Public Health Surveillance: Evaluating Systems for the Early Detection of Biological Threats - Abbreviated Version
their state and local partners who are responsible for public health decision making. Having the information tools necessary to make post-BAR decisions as rapidly as possible is likely to be crucial in being able to maximize the possible benefit from the BioWatch system’s detection of a bioterrorism attack.
Little or no empirical data were available to help the committee in its evaluation of the response aspect of the BioWatch system. One after-action report was provided to the committee (Lindley, 2009), but the chief source of information was testimony by state and local public health officials during the committee’s information-gathering meetings. (See Appendix A for a list of participants at these meetings.)
In its information gathering, the committee explored aspects of the current BioWatch response capability, including the coordination and collaboration between DHS and public health officials as well as the resources needed for response. The committee presents its findings and recommendations for improvements to the BioWatch system in this chapter. These improvements can be implemented immediately to increase the potential effectiveness of the existing system and need not await development and implementation of the next generation of BioWatch technology.
Coordination and Collaboration
The BioWatch program appears to lack necessary coordination, communication, and collaboration among the several contributors at federal, state, and local levels that must be fully engaged for a functional system. The committee heard testimony about significant shortcomings in the earliest days of the BioWatch program in DHS’s inclusion of public health officials as partners and collaborators in developing plans and procedures for a response to BARs. At that time, the planning and concept of operations for the response to a BAR had not been well developed and did not appropriately take into account the central role of local officials. A lack of consensus about communication with local officials was illustrated by a 2005 instance in which tests of air samples indicated the presence on the National Mall in Washington, DC, of one of the biological agents monitored by the BioWatch system. The criteria for a BAR were not fully met, but local officials were informed of the detection 4 to 5 days after the event (Dvorak, 2005). Moreover, they learned of it from CDC rather than from DHS (Dvorak, 2005). Public health officials also told the committee that DHS had initially been reluctant to inform local and state officials about the locations of BioWatch air samplers within their own jurisdictions.
Further, DHS has not always provided technical information to public health officials about the sensitivity, specificity, and limits of detection of the laboratory assays that are the basis for declaring a BAR. As a result,
OCR for page 67
Biowatch and Public Health Surveillance: Evaluating Systems for the Early Detection of Biological Threats - Abbreviated Version
the laboratory directors in some jurisdictions, whose responsibility it is to report a BAR, have not had the means to evaluate the laboratory evidence concerning the presence of a biological agent (Personal communication, Scott Becker, Association of Public Health Laboratories, April 22, 2009). Although procedures for interpreting and responding to positive test results have improved, there remain serious gaps in effective communication and trust between the BioWatch program and public health personnel in BioWatch jurisdictions.
The seeming lack of trust by DHS in public health officials has been mirrored by an apparent lack of confidence in DHS and the BioWatch program by many local public health officials. They are cautious about the interpretation of BARs because of experience with the system to date, and some question the value of BioWatch.
The BioWatch system relies on prompt public health action, such as prophylaxis in response to a BAR, to be most effective in saving lives. Critical to the success of the BioWatch system is the confidence of local and state public health officials and decision makers in the system’s detection and assay technology, and in their ability to work as partners with DHS and other federal agencies in preparing for and possibly having to respond to a bioterrorist attack detected through BioWatch. Public health officials emphasized in their discussions with the committee that they are aware that they depend on the public’s trust to be able to motivate action and cooperation and avoid panic in life-threatening circumstances. These officials do not want to risk this trust by initiating potentially high-regret actions for a possible bioterrorist attack when a full-blown response is not warranted, as has been the case with the BARs to date. Some stated to the committee that they would be unlikely to administer prophylaxis on the basis of a BAR alone, waiting instead until clinical cases occur before taking that step.
The committee sees the need for DHS to strengthen its relationships with state and local health officials and other key responders through training, exercises, and information exchange. The annual BioWatch meeting provides one forum for information exchange between BioWatch jurisdictions and DHS and among the jurisdictions. However, this annual conference is not sufficient for such information sharing. In the past, DHS has provided limited opportunities for state and local input into the conference agenda or an active role in planning it. Going forward, DHS should use this conference as an opportunity to more actively involve local partners as part of forging a stronger partnership.
DHS has begun taking some important steps to test and validate BioWatch assays that should help build confidence in the assays. DHS can further improve upon this in continuing its efforts to strengthen relationships with its local and state partners. It needs to not only invite input but also respond to it, and it needs to facilitate more information sharing about
OCR for page 68
Biowatch and Public Health Surveillance: Evaluating Systems for the Early Detection of Biological Threats - Abbreviated Version
matters such as BARs so that the BioWatch jurisdictions can learn from others’ experiences. To the extent that such information sharing has been constrained by security concerns, DHS should help ensure that essential public health officials in each jurisdiction have appropriate security clearances. The training and exercises that DHS holds with BioWatch jurisdictions should also continue to be helpful in strengthening necessary skills and the relationships vital to effective action under critical circumstances following a BAR.
Information and Tools for Decision Making
Several critical decisions need to be made in the event of a BAR: (1) Does it indicate a real bioterrorist attack? (2) If so, or if an attack cannot be ruled out, should public health officials begin to mobilize to start prophylaxis, and what population should be targeted initially? (3) Is it safe to allow persons to stay in the area around the site of the BAR, or should recommendations be made to evacuate or shelter in place? and (4) What information should be provided to the public?
Decisions such as these will need to be made very quickly, even though important information is likely to be limited. At the time of a BAR, the available information may include the site(s) of the air sampler(s) that produced positive assay results, weather conditions, and the biological agent detected. It may take 1 to 2 days to obtain additional data from environmental sampling and surveillance inputs. Initial decisions may need to be revised as additional information becomes available.
Information from law enforcement and national security sources is not typically a factor in public health decision making, but it will be critical if bioterrorism is suspected. While it is quite likely that a biological attack could occur without any meaningful intelligence warning, all relevant foreign and domestic intelligence that might possibly reflect on the likelihood of an attack that threatens human health should be immediately delivered to public health decision makers, along with any insights on how to evaluate or interpret such findings. The decision makers will need to know of any information such as thefts or diversions of agent stocks or known efforts by potential terrorists to culture organisms or develop means of dissemination.
Incident Characterization and Event Reconstruction
Environmental Sampling
One critical gap in current tools and technology is a means to determine the extent of contamination. The 2001 anthrax attacks demonstrated seri-
OCR for page 69
Biowatch and Public Health Surveillance: Evaluating Systems for the Early Detection of Biological Threats - Abbreviated Version
ous weaknesses in the environmental sampling approaches taken by federal agencies to determine the presence or absence of a biological agent. Because validated methods for collection of environmental samples and analysis were not used, there could be little confidence in the results (GAO, 2007). Draft guidance on environmental sampling following a BAR has been developed by DHS (2008c), and other agencies have also offered relevant guidance (CDC, 2002; EPA, 2008). Nevertheless, no validated sampling strategy and collection methods are currently available for use in situations in which contamination from a biothreat agent is suspected. Without such validated methodologies, states and localities lack much-needed means to determine the extent of contaminated areas or identify populations most likely to be exposed.
Dispersion Modeling
Another potential tool for characterizing an event that may be a bioterrorist attack is the use of dispersion modeling to assess where exposures to airborne pathogens may be likely.4 According to GAO (2008b), field testing and evaluations show that plume models developed by federal agencies specifically for tracking the atmospheric release of CBRN (chemical, biological, radiological, or nuclear) materials in urban areas have severe limitations and require additional field evaluation. Moreover, the deficiencies in environmental sampling, noted above, mean that it is unlikely to be helpful in refining the results produced by dispersion models.
DHS has supported the development of tools intended to help jurisdictions assess a biological event. For example, the Bioagent Event Reconstruction Tool (BERT) employs modeling to simulate what might happen during an airborne biological agent attack (Brown et al., 2006). It uses as inputs measurements from biological agent detectors and data from wind sensors. While BERT can estimate the amount of material released and offer possible locations for a release for certain kinds of scenarios, it is not applicable to other conditions. In addition, its plume models have important limitations (Brown et al., 2006). During TOPOFF exercises led by DHS, contradictory results provided by competing plume models led to confusion by first responders and decision makers.
The DHS Interagency Modeling and Atmospheric Assessment Center (IMAAC) at Lawrence Livermore National Laboratory has been designated as the focal point for coordinating and disseminating plume modeling products. However, GAO (2008b) concluded that IMAAC needs better
4
For more information on the use of dispersion models for event reconstruction, see Keats et al. (2007); Rao (2007); Chow et al. (2008); Delle Monache et al. (2008); Senocak et al. (2008); and Yee (2008).
OCR for page 70
Biowatch and Public Health Surveillance: Evaluating Systems for the Early Detection of Biological Threats - Abbreviated Version
procedures for working with federal, state, and local agencies to deal with conflicting modeling results.
The available dispersion models can be used to generate hypotheses for testing and to inform analysis of release sites, but they are not designed to guide public health interventions. On the other hand, as made clear in other parts of this report, the use of epidemiological models, and models of responses to detection, alerts, and positive reports from syndromic surveillance, can and should be used to evaluate the current (and proposed) versions of BioWatch and related systems and technologies.
Decision Support Tools
Beginning in 2005, DHS sponsored development of a decision support tool called the Biological Warning and Incident Characterization (BWIC) system, which was pilot tested in three jurisdictions. The aim of BWIC is to integrate databases and several modeling programs into a common system for an emergency management team. A major component is the application of plume models to indicate where a release is likely to have occurred and what populations and areas are likely to have been exposed. It also includes a situational awareness tool to keep decision makers informed of estimates from analysts using the modeling components (Argonne National Laboratory, 2008). As tested, BWIC also supports monitoring routine operations, such as daily filter collection from BioWatch air samplers and reports from laboratory analysis of filters. It also incorporates scenarios to facilitate exercises and planning. Decision support and situational awareness tools such as BWIC may well aid state and local decision making following a BAR, but further development to address limitations identified during pilot testing is needed.
Developing New and Better Tools to Aid Decision Makers
The committee urges action to improve tools for environmental characterization of the nature and extent of aerosol releases of biological agents in order to inform public health decision makers following a BAR. DHS should work with HHS (CDC), EPA, and the states and localities to ensure that the public health and law enforcement authorities in BioWatch jurisdictions have the capabilities to rapidly confirm and characterize an event that results in a BAR (event reconstruction) via environmental testing and other methodologies (e.g., plume modeling) to rapidly determine the area affected in order to target control measures effectively. Support is needed for work to identify, develop, and validate methodologies for specimen collection, laboratory analyses, and interpretation of environmental data. Examples of potential approaches to improving tools for incident characterization are shown in Box 3-1.
OCR for page 71
Biowatch and Public Health Surveillance: Evaluating Systems for the Early Detection of Biological Threats - Abbreviated Version
BOX 3-1
Potential Approaches for Improving Incident Characterization
If release of a bioterrorism agent is suspected or confirmed, public health officials will need additional information to understand the scale of the release and the extent of the dispersion of the agent. Various approaches to producing such information could be developed and tested to determine their feasibility and effectiveness. Some examples include
Developing low-technology strategies for quick-look assessments of the scale of an incident (e.g., test subway train ventilation filters with PCR as a first check on transport through or dispersion out of a subway system or existing ventilation systems);
Developing national standards for indoor and outdoor environmental sampling strategies and collection methods and federally validated procedures for DNA extraction and analysis from environmental sampling;
Encouraging academic–public partnerships to design new data analysis strategies and methods that leverage preexisting regional meteorological and engineering expertise and experience to integrate environmental sampling data with meteorological data, and find ways that allow these new partners to assist both routinely and during emergencies;
Building an interactive database for some or all BioWatch jurisdictions that contains simulations for several thousand releases of bioterror agents and that reflects a comprehensive range of meteorological conditions anticipated through the year and judgments by local FBI or police of higher-risk targets and potential release points. This database could be used for planning and training purposes and also during an actual event, when the database could be queried for releases yielding the precise number and location of positive BioWatch detectors during similar meteorological conditions. This information could help generate new hypotheses that could be tested with additional environmental sampling following an event; and
Integrating data from sources such as atmospheric aerosol lidars or other monitoring systems to help corroborate positive results from analysis of BioWatch samples and suggest source locations and dispersion trajectories.
OCR for page 85
Biowatch and Public Health Surveillance: Evaluating Systems for the Early Detection of Biological Threats - Abbreviated Version
a detector to function without being serviced for extended periods of time put a great engineering burden on the device.
Coverage and cost trade-offs: The BioWatch program is emphasizing coverage of major metropolitan areas and within them, locations and special events that bring large numbers of people together. But within jurisdictions, population densities and distributions vary between workday and evening and workday and weekend. Achieving desired population coverage will require fielding many samplers and thus the costs of a system’s capital equipment and operations must be extremely low to be affordable.
Uncertainty about the characteristics of airborne biological threats: The BioWatch program currently focuses on certain specific pathogens, but the scope of potential aerosol biothreats is great. Future bioterrorism threats may include engineered agents, and recent experience has demonstrated the threat from emerging respiratory pathogens (e.g., Legionnaires’ disease). Ideally, a biodetector should be readily upgradable to address novel naturally occurring and engineered biothreats.
These challenges to development of the Generation 3 BioWatch detector underscore the need for significant investments in early applied research leading to transformational innovations. Significant advances in several diverse scientific areas hold promise of converging to achieve development of the required biodetector. For example, developments in engineering research may permit creation of more robust and inexpensive detector platforms, and new communication systems and network architectures will support “smart” networks.
Opportunities also exist to refine the analysis of BioWatch air samples, but an important prerequisite is the characterization of endemic environmental organisms in and near BioWatch jurisdictions to establish background levels of both the pathogens of concern and near relatives that might cross-react in assays. With high-throughput sequencing technology rapidly filling in the genomic microbial tree of life, it is possible to refine the nucleic acid assays used in the BioWatch system to optimize their sensitivity and specificity for detection of the particular pathogens of interest. In addition, studies supported by several federal agencies are rapidly establishing relationships between organisms’ molecular information (e.g., genomics, proteomics) and their pathogenicity. With this knowledge, chemical analysis of aerosol samples may provide a better basis for judging the health hazard associated with a BAR.
Operational Challenges
Significant logistical challenges need to be resolved before full fielding of the Generation 3 system. Currently, the BioWatch program has a few
OCR for page 86
Biowatch and Public Health Surveillance: Evaluating Systems for the Early Detection of Biological Threats - Abbreviated Version
hundred samplers deployed (DHS, 2008a), but intends to deploy a greater number for Generation 3. The BioWatch program, in close coordination with HHS, has to ensure that the entire BioWatch system has sufficient resources. This includes not only the air sampling equipment and laboratory analysis, but resources for all associated maintenance and upgrade capabilities, doctrine and technical procedures, the initial and sustainment training base, laboratory facilities (and their safety systems), courier services, communications and computer systems, trained staff and leadership, and a fully resourced public health response capability. Even though it may be reasonable to assume that the BioWatch Generation 3 system will alleviate some of the current analytical and logistical burdens, it is equally reasonable to assume that its complex, automated detectors will require new, specialized technical and maintenance support and personnel with appropriate training to interpret the results it produces. The total system architecture must account for all of these system components, and program budgeting must be adequate to support all aspects of the system.
Programmatic Challenges
The availability of a Generation 3 autonomous detector has the potential to provide a major advance in capability and logistics of the BioWatch system. But DHS should heed the scale of the challenge, as signaled by the considerable investments by DoD and others—with only limited progress—toward the goal of an automated detector for bioaerosol threats. Further progress in the development and deployment of the Generation 3 system for BioWatch should benefit from a strong and effective collaboration within DHS between OHA and S&T. OHA should ensure that operational needs and concerns from BioWatch jurisdictions and federal BioWatch partners are brought to the development effort, while S&T should ensure that Generation 3 development responds to both technological opportunities and hazards.
RECOMMENDATION 4: DHS should improve the level of cooperation and collaboration between its Office of Health Affairs and its Science and Technology Directorate to promote effective research and technology development in support of the BioWatch program.
Opportunities to Advance Future Biodetection Systems
The instruments and knowledge currently used in the BioWatch system have origins in and are built upon previous research in academic, industrial, and governmental institutions, supported by a wide range of agencies. For example, the APDS system that was originally proposed for use in a Genera-
OCR for page 87
Biowatch and Public Health Surveillance: Evaluating Systems for the Early Detection of Biological Threats - Abbreviated Version
tion 2.5 deployment contains components and concepts from flow injection and segmented flow analysis introduced originally for high-speed analysis of large sample populations (Bergamin et al., 1978; Ruzicka and Hansen, 1978, 1988; Ruzicka and Guebeli, 1991); microfluidic analysis system concepts introduced for multiplex analysis based on mixtures of functionalized polymer beads, each containing unique capture/receptor chemistry (Manz et al., 1992; Harrison et al., 1993; Jacobson et al., 1994a,b); fluorescence labeling to detect unique hybridization reactions which was a part of the genome sequencing chemistry (Hunkapiller et al., 1991); information technology systems; and nucleic acid amplification chemistry (Saiki et al., 1985). A foundation of support for research and the resulting advances in knowledge and technology are crucial to maintain the essential science base that will support development of next generation biothreat reduction capabilities.
A sustained research and development effort is needed to provide the scientific and technological knowledge required for effective and sustainable outdoor and indoor environmental biosurveillance in urban environments. The committee encourages DHS to work with other research funding agencies (e.g., the National Science Foundation, DoD, EPA, and HHS) to coordinate and leverage investments that will contribute to the development of less expensive and more capable environmental biosurveillance systems, including contributing to the knowledge and innovation required to engineer an autonomous environmental biodetection system.
In particular, DHS should collaborate with DoD and the National Institutes of Health to establish an applied research program to advance the state of science needed for development of an autonomous, field-deployable detector with capabilities to meet operational requirements for BioWatch Generation 3 and beyond. Research is also needed to improve the knowledge base for interpreting surveillance results, developing techniques for addressing unknown threats (i.e., emerging, re-emerging, and engineered biothreats), and applying environmental monitoring systems to surveillance for natural disease (e.g., monitoring for Legionella) as well as biodefense to make them more cost-effective.
DHS should also consider participation in the work being done by DoD, the National Institute of Allergy and Infectious Diseases in HHS, and the World Health Organization on host–pathogen interactions, surveillance, and epidemiologic research investigations and, as part of that effort, establish shared databases to consolidate information of value to the work of all the participants. Through research on host–pathogen interactions it may prove possible to identify genetic or molecular markers that signal virulence or antimicrobial resistance in pathogens or increased vulnerability in human hosts. This kind of information may make it possible to devise more informative detection techniques that would help public health officials maximize the effectiveness of their response plans.
OCR for page 88
Biowatch and Public Health Surveillance: Evaluating Systems for the Early Detection of Biological Threats - Abbreviated Version
A research program could consider including the BioWatch jurisdictions as study sites in a program to characterize the microbial ecology in urban areas. Objectives would include characterization of the geographic and temporal distribution of pathogen and near-neighbor populations in air and natural reservoirs (e.g., soil, lakes, and sanitary waste facilities). The BioWatch air samplers currently in use have capabilities that may be useful for such research. One priority should be full characterization of samples from BARs, but examination of the large number of other BioWatch samples may also be informative.
Results of the ongoing characterization effort and BAR follow-on studies would be included in a central database for use in the design of bioassay signatures and interpretation of BAR results. This central database could be shared with bioforensic databases, thereby bringing together knowledge derived from biosurveillance and bioforensic programs. Seroprevalence studies could help to understand current human exposures to these endemic organisms.8 To the extent that the currently deployed BioWatch system can contribute to such research efforts, it should be seen as a resource to inform and enlarge the science and technology base from which new generations of BioWatch technologies will be derived.
As with other enhancements to BioWatch proposed by the committee, the costs and benefits of various research and development activities have to be evaluated and prioritized against the range of demands on DHS and BioWatch program resources.
RECOMMENDATION 5: As part of its response to the technical and operational challenges posed by the development and launch of Generation 3 BioWatch, DHS should collaborate with HHS, DoD, EPA, the National Science Foundation, and other agencies doing relevant work to develop and execute an aggressive research and development plan focused on (1) shorter-term goals to improve the capabilities and cost-effectiveness of the environmental monitoring for airborne biological threats performed by the BioWatch system, and (2) longer-term goals to improve the knowledge base needed to support transformational innovations in environmental biosurveillance. Work in support of shorter-term goals should focus on
Advancement of the state of science needed for the development of an autonomous field-deployable detector with capabilities to meet Generation 3 BioWatch operational requirements and beyond.
8
Seroprevalence is an indication of the proportion or number of people in a given population with antibodies in their blood indicating exposure to a particular organism.
OCR for page 89
Biowatch and Public Health Surveillance: Evaluating Systems for the Early Detection of Biological Threats - Abbreviated Version
Work supporting longer-term goals should include
Temporal and spatial characterization of pathogen and near-neighbor populations in air and natural reservoirs in urban areas, including those near BioWatch sites; and
Participation in the work by others on host–pathogen interactions, surveillance, and epidemiologic research investigations and establishment of shared databases.
BIOWATCH PROGRAM PLANNING AND MANAGEMENT
The preceding sections have described a series of needs and challenges facing DHS and the BioWatch program as it moves forward. In doing so, the program should be guided by strategic planning based upon program missions, goals, and objectives. The committee was very interested in whether DHS has documented overall programmatic goals, such as the number of cities (or people) the BioWatch program is intended to protect, the number of agents to detect, or the desired probabilities of detection of various kinds of aerosol releases. In response to a committee request, DHS (2008d) stated the goal noted in Chapter 2: “to establish and operate a bioaerosol monitoring capability to accurately detect the release of biological threat agents of greatest concern to the nation in locations that are at greatest risk of catastrophic consequences and to enable timely response and mitigation.” Although this goal provides a high-level focus for the program, it does not provide the detail needed, or identify the spectrum of capacities required, for development, operation, and continuous improvement of such a complex activity involving diverse stakeholder groups.
The relatively weak documentation and statement of the program’s goals—and associated performance metrics—may well be the result of the short time frame for the initial deployment of the BioWatch system and the infancy of DHS as a federal agency. Now, with several years of operational experience, a more mature program in a more mature agency requires more rigor in program planning and a better understanding of the goals for not only technological performance but overall system performance, including effective engagement of key partners in states and localities where the BioWatch system is deployed.
For a system to meet user needs, the users should participate in determining the goals and performance metrics. The BioWatch program management has apparently placed more emphasis on the “early detection” aspect of a bioaerosol monitoring capability than on “enabling timely response and mitigation.” Facilitating response and mitigation requires cooperation among DHS, public health, and other federal partners and the integration and assessment of many different types of information. The concerns of local
OCR for page 90
Biowatch and Public Health Surveillance: Evaluating Systems for the Early Detection of Biological Threats - Abbreviated Version
decision makers, who will have to act in response to a BAR, must be taken into account to validate performance measures and program scope, enhance confidence in BioWatch results, and achieve buy-in from key partners. State and local government officials, including public health representatives, should be involved in decisions regarding all stages of BioWatch planning, deployment, maintenance, and upgrades within their jurisdictions.
The committee recognizes that DHS does plan a more concerted effort to improve the relationship between the BioWatch program’s management and its federal, state, and local partners. DHS should produce a formal acknowledgement of this commitment and follow through with its implementation.
The BioWatch program needs both a stronger information base and better and more systematic input, analysis, and evaluation on an ongoing basis. The committee is encouraged by the more systematic and technically rigorous approach to planning and technical evaluation proposed for Generation 3. DHS should also take this opportunity to incorporate risk-assessment and risk-management approaches into its planning for the BioWatch program. Such an evaluation should include a reexamination of the agents to include in the monitoring program, the selection of jurisdictions, the location of air samplers or detectors within jurisdictions, and even the justification for the existence of the program. In the context of the program’s goals, analytical tools should be brought to bear to aid in decision making each time DHS considers a significant change regarding a technology or process.
In particular, DHS should make use of its biennial Bioterrorism Risk Assessment (BTRA) in assessing the BioWatch system. But the concerns about the 2006 BTRA that were noted in a recent NRC report should be resolved (NRC, 2008b). Using an improved BTRA, DHS should model scenarios and assess risks, and apply this information in its decision making about, and continuous evaluation of, the BioWatch program. Indeed, DHS appears to have applied these principles in its initial decision making in planning for the demonstration of the APDS (Hooks, 2008a).
Such an approach might include an examination of the most effective applications of the BioWatch system. DHS might, for example, consider whether special event monitoring would be more effective than constant monitoring, or whether focusing on transportation hubs would be better than attempting comprehensive coverage of localities. Any plans to add biological agents to the set to be monitored by the BioWatch system should include an assessment of whether analytical challenges will be encountered because of natural background levels of these biological agent signatures or the presence of other interferents. The committee did not receive evidence that DHS has factored such analyses into most decision making for the BioWatch program.
OCR for page 91
Biowatch and Public Health Surveillance: Evaluating Systems for the Early Detection of Biological Threats - Abbreviated Version
Finally, just as the committee advocates rigorous analysis of the costs and benefits of existing plans for BioWatch 2 and 3, it also advocates that any further work based on the recommendations in this report, or through other research and development efforts, be subjected to thorough scrutiny of the associated costs and benefits.
RECOMMENDATION 6: DHS should use its existing and periodically conducted bioterrorism risk assessment and other analyses to evaluate the overall effectiveness of the BioWatch system, examine the costs and benefits of the system’s current configuration and significant proposed changes, and articulate its program goals and associated performance metrics, using risk-assessment and risk-management principles. To accomplish this, DHS should
Actively solicit input from and collaborate closely on all aspects of the program with key partners and stakeholders at the federal, state, and local levels; and
Conduct comprehensive modeling and analysis to evaluate the potential contributions of the BioWatch system to public health decision making and outcomes using, where appropriate, a Bioterrorism Risk Assessment (BTRA) that has been modified according to the recommendations in a 2008 NRC report. Such analyses should be performed for all pathogens for which BioWatch will test and for both outdoor and indoor monitoring programs.
In the committee’s view, the BioWatch program has suffered from too little input from outside experts who can bring broader perspectives to bear. Obtaining advice from an independent panel, consisting of members with a mix of technical and operational expertise, would enhance the BioWatch program. Such an external advisory panel would provide specific analyses and recommendations on the BioWatch program’s technology upgrades, planning, and operations. Panel members could also receive, and relay back to their constituencies, information about pertinent research and development activities.
The advisory panel should bring to DHS expertise in epidemiology, environmental health, public health laboratory systems, meteorology, infectious diseases, biochemistry, genetics, law enforcement, emergency management, detection technology, systems engineering, decision and information science, and operations research. Among the members should be state and local officials responsible for responding to a BAR.
To further coordination and communication between DHS and HHS, the advisory panel should report jointly to both secretaries. The committee anticipates that the input from the advisory panel would lead to improved collabo-
OCR for page 92
Biowatch and Public Health Surveillance: Evaluating Systems for the Early Detection of Biological Threats - Abbreviated Version
ration among the state- or local-level constituents of the lead federal agencies, which would improve working relationships between those responsible for implementing environmental sampling technologies and those responsible for acting on the information that such sampling produces. It could also lead to better communication at all stages in the BioWatch implementation process, including selection of participating cities, identification of monitoring sites, ongoing program operation, and response planning and evaluation.
The BioWatch program is already benefiting from the work of some advisory groups. The BioWatch Technical Advisory Committee (BTAC), for example, consists of federal experts with backgrounds in microbiology and engineering from DHS (OHA and S&T) and CDC who provide DHS with technology input. But this group lacks sufficient breadth of expertise and the external perspectives that the BioWatch program needs. DHS also benefits from the work of the Stakeholder Panel on Agents for Detection Assays (SPADA), which is developing voluntary consensus standards for the collection and analysis of environmental samples of biothreat agents.9 DHS and HHS need such external expert input for all aspects of the BioWatch system.
RECOMMENDATION 7: DHS and HHS should jointly establish a formal mechanism for receiving ongoing external advice on all technical and operational issues related to the BioWatch system. The advisory panel should include both technical experts and state and local officials who have experience working with the BioWatch system and would have decision-making roles in the event of a BAR.
REFERENCES
AOAC (Association of Analytical Communities). 2008. Much accomplished, much to do: SPADA update. http://www.aoac.org/ILM/jan_feb_08/spada.htm (accessed April 8, 2009).
Argonne National Laboratory. 2008. Software package coordinates response to biological threats. http://www.dis.anl.gov/publications/fact_sheets/BWIC_FactSheet.pdf (accessed June 17, 2009).
Bergamin, H., E.A. Zagatto, F.J. Krug, and B.F. Reis. 1978. Merging zones in flow injection analysis: Part 1. Double proportional injector and reagent consumption. Analytica Chimica Acta 101(1):17–23.
Brown, M.J., M.D. Williams, G.E. Streit, M. Nelson, and S. Linger. 2006. An operational event reconstruction tool: Source inversion for biological agent detectors. 86th American Meteorological Society Annual Meeting. Atlanta, GA. http://ams.confex.com/ams/pdfpapers/126686.pdf (accessed July 9, 2009).
9
SPADA is an expert review panel organized by the Association of Analytical Communities at the request of DHS. Its members are drawn from the federal government, state governments, academia, and the private sector (AOAC, 2008).
OCR for page 93
Biowatch and Public Health Surveillance: Evaluating Systems for the Early Detection of Biological Threats - Abbreviated Version
CDC (Centers for Disease Control and Prevention). 2002. Comprehensive procedures for collecting environmental samples for culturing Bacillus anthracis. http://emergency.cdc.gov/agent/anthrax/environmental-sampling-apr2002.asp (accessed July 9, 2009).
CDC. 2004. Responding to detection of aerosolized Bacillus anthracis by autonomous detection systems in the workplace. Morbidity and Mortality Weekly Report 53(RR-7):1–11.
Chadwick, S.H. 2007. Defense acquisition: Overview, issues, and options for Congress. RL34026. Washington, DC: Congressional Research Service. https://www.policyarchive.org/bitstream/handle/10207/18699/RL34026_20070604.pdf?sequence=2 (accessed August 5, 2009).
Chow, F.K., B. Kosovič, and S. Chan. 2008. Source inversion for contaminant plume dispersion in urban environments using building-resolving simulations. Journal of Applied Meteorology and Climatology 47:1553–1572.
Delle Monache, L., J.K. Lundquist, B. Kosovič, G. Johannesson, K.M. Dyer, R.D. Aines, F.K. Chow, R.D. Belles, W.G. Hanley, S.C. Larsen, G.A. Loosmore, J.J. Nitao, G.A. Sugiyama, and P.J. Vogt. 2008. Bayesian inference and Markov Chain Monte Carlo sampling to reconstruct a contaminant source on a continental scale. Journal of Applied Meteorology and Climatology 47:2600–2613.
DHS (Department of Homeland Security). 2007a. Homeland security exercise and evaluation program. Vol. III: Exercise evaluation and improvement planning. Washington, DC: DHS. https://hseep.dhs.gov/support/VolumeIII.pdf (accessed July 1, 2009).
DHS. 2007b. Target capabilities list: A companion to the National Preparedness Guidelines. Washington, DC: DHS. https://www.llis.dhs.gov/docdetails/details.do?contentID=26724 (accessed July 1, 2009).
DHS. 2008a. Annual performance report, fiscal years 2007–2009. Washington, DC: DHS. http://www.dhs.gov/xlibrary/assets/cfo_apr_fy2007.pdf (accessed August 5, 2009).
DHS. 2008b. BioWatch Gen-3 autonomous detection system: Operational requirements document. December 11. Washington, DC.
DHS. 2008c. BioWatch outdoor program: Guidance documents for BioWatch jurisdictions. Washington, DC: DHS.
DHS. 2008d. NAS information request #2, DHS OHA written responses. Document submitted to the Committee on Effectiveness of National Biosurveillance Systems: BioWatch and the Public Health System, October 31, Washington, DC.
DHS. 2009a. BioWatch Gen-3 industry day. https://www.fbo.gov/download/18d/18d10ae7b9c2106327715c7a5dc53c43/BiowatchGen3_Industry-Day_Slides.pdf (accessed March 24, 2009).
DHS. 2009b. BioWatch Gen-3 request for proposal: HSHQDC-09-R-00045. https://www.fbo.gov/download/54e/54e9ae25a4a9d2d39a9e9e836955001f/HSHQDC-09-R-00045_(5-27-2009).pdf (accessed August 5, 2009).
DHS. 2009c. BioWatch Gen-3—Request for proposals: HSHQDC-09-R-00045. Attachments J.1–J.5 and J.7–J.9. Washington, DC: DHS. https://www.fbo.gov/download/c38/c38d6b04219a25945582398067211499/HSHQDC-09-R-00045_(Attachments_J_1__J_2__J_3__J_4__J_5__J_7__J_8__J_9).pdf (accessed August 5, 2009).
DHS. 2009d. Department of Homeland Security annual performance report, fiscal years 2008–2010. Washington, DC: DHS. http://www.dhs.gov/xlibrary/assets/cfo_apr_fy2008.pdf (accessed June 17, 2009).
DoD (Department of Defense). 2006. Defense acquisition guidebook. http://www.acq.osd.mil/sse/pg/guidance.html (accessed July 9, 2009).
Downes, F.P. 2008. Testimony before the Committee on Homeland Security, U.S. House of Representatives, July 16. http://homeland.house.gov/SiteDocuments/20080716143630-34598.pdf (accessed June 18, 2009).
OCR for page 94
Biowatch and Public Health Surveillance: Evaluating Systems for the Early Detection of Biological Threats - Abbreviated Version
Dvorak, P. 2005. Health officials vigilant for illness after sensors detect bacteria on Mall. The Washington Post, Sunday, October 2, C13. http://www.washingtonpost.com/wp-dyn/content/article/2005/10/01/AR2005100101209_pf.html (accessed April 21, 2009).
EPA (Environmental Protection Agency). 2008. Standardized analytical methods for environmental restoration following homeland security events. Revision 4.0. EPA/600/R-04/126D. Cincinnati, OH: EPA. http://www.epa.gov/nhsrc/pubs/600r04126d.pdf (accessed July 10, 2009).
GAO (Government Accountability Office). 2007. Anthrax detection: DHS cannot ensure that sampling activities will be validated. GAO-07-687T. Washington, DC: GAO. http://www.gao.gov/new.items/d07687t.pdf (accessed August 5, 2009).
GAO. 2008a. Biosurveillance: Preliminary observations on Department of Homeland Security’s biosurveillance initiatives. GAO-08-960T. Washington, DC: GAO. http://www.gao.gov/new.items/d08960t.pdf (accessed August 5, 2009).
GAO. 2008b. Homeland security: First responders’ ability to detect and model hazardous releases in urban areas is significantly limited. GAO-08-180. Washington, DC: GAO. http://www.gao.gov/products/GAO-08-180 (accessed July 9, 2009).
GAO. 2009. Defense acquisitions: Assessments of selected weapon programs. GAO-09-326SP. Washington, DC: GAO. http://www.gao.gov/new.items/d09326sp.pdf (accessed August 5, 2009).
Harrison, D.J., K. Fluri, K. Seiler, Z. Fan, C.S. Effenhauser, and A. Manz. 1993. Micro-machining a miniaturized capillary electrophoresis-based chemical analysis system on a chip. Science 261(5123):895–897.
Hooks, R. 2008a. APDS block-1 deployment strategy. Document submitted to the Committee on Effectiveness of National Biosurveillance Systems: BioWatch and the Public Health System, September 19, Washington, DC.
Hooks, R. 2008b. BioWatch program. Presentation to the Committee on Effectiveness of National Biosurveillance Systems: BioWatch and the Public Health System, Meeting 1, July 30–31. Washington, DC.
HSARPA (Homeland Security Advanced Research Projects Agency). 2003. Detection systems for biological and chemical countermeasures. RA 03-01. Washington, DC: Department of Homeland Security. https://baa.st.dhs.gov/Solicitations/HSARPA_RA-03-01_Body.pdf (accessed July 16, 2009).
Hunkapiller, T., R.J. Kaiser, B.F. Koop, and L. Hood. 1991. Large-scale and automated DNA sequence determination. Science 254(5028):59–67.
IEc (Industrial Economics, Incorporated). 2009. Revised estimated cost of BioWatch generations 2 and 3. Commissioned paper. Committee on Effectiveness of National Biosurveillance Systems: BioWatch and the Public Health System, Washington, DC.
Jacobson, S.C., R. Hergenrider, L.B. Koutny, R.J. Warmack, and J.M. Ramsey. 1994a. Effects of injection schemes and column geometry on the performance of microchip electrophoresis devices. Analytical Chemistry 66(7):1107–1113.
Jacobson, S.C., L.B. Koutny, R. Hergenroeder, A.W. Moore, and J.M. Ramsey. 1994b. Microchip capillary electrophoresis with an integrated postcolumn reactor. Analytical Chemistry 66(20):3472–3476.
Keats, A., E. Yee, and F.-S. Lien. 2007. Bayesian inference for source determination with applications to a complex urban environment. Atmospheric Environment 41(3):465–479.
Kuske, C. 2005. Analysis of pathogen backgrounds in soils and EPA aerosol samples. Report No. LA-UR-04-3191. Los Alamos, NM: Los Alamos National Laboratory.
Kuske, C.R., S.M. Barnes, C.C. Grow, L. Merrill, and J. Dunbar. 2006. Environmental survey for four pathogenic bacteria and closely related species using phylogenetic and functional genes. Journal of Forensic Science 51(3):548–558.
OCR for page 95
Biowatch and Public Health Surveillance: Evaluating Systems for the Early Detection of Biological Threats - Abbreviated Version
Lindley, C. 2009. BioWatch actionable result response: 2008 Democratic National Convention, August 28, 2008. Document submitted to the Committee on Effectiveness of National Biosurveillance Systems: BioWatch and the Public Health System, February 19, Washington, DC.
Manz, A., D.J. Harrison, E.M. Verpoorte, J.C. Fettinger, A. Paulus, H. Ludi, and H.M. Widmer. 1992. Capillary electrophoresis and sample injection systems integrated on a planar glass chip. Journal of Chromatography 593(1–2):253–258.
NRC (National Research Council). 2008a. A framework for assessing the health hazard posed by bioaerosols. Washington, DC: The National Academies Press.
NRC. 2008b. Department of Homeland Security Bioterrorism Risk Assessment: A call for change. Washington, DC: The National Academies Press.
NRC. 2008c. Test and evaluation of biological standoff detection systems. Washington, DC: The National Academies Press.
Rao, K.S. 2007. Source estimation methods for atmospheric dispersion. Atmospheric Environment 41:6964–6973.
Ruzicka, J., and T. Guebeli. 1991. Principles of stopped-flow sequential injection analysis and its application to the kinetic determination of traces of a proteolytic enzyme. Analytical Chemistry 63(17):1680–1685.
Ruzicka, J., and E.H. Hansen. 1978. Flow injection analysis: Part X. Theory, techniques and trends. Analytica Chimica Acta 99(1):37–76.
Ruzicka, J., and E.H. Hansen. 1988. Flow injection analysis, 2nd ed. New York: Wiley & Sons.
Saiki, R.K, S. Scharf, F. Faloona, K.B. Mullis, G.T. Horn, H.A. Erlich, and N. Arnheim. 1985. Enzymatic amplification of beta-globin genomic sequences and restriction site analysis for diagnosis of sickle cell anemia. Science 230(4732):1350–1354.
Senocak, I., N.W. Hengartner, M.B. Short, and W.B. Daniel. 2008. Stochastic event reconstruction of atmospheric contaminant dispersion using Bayesian inference. Atmospheric Environment 42:7718–7727.
Yee, E. 2008. Theory for reconstruction of an unknown number of contaminant sources using probabilistic inference. Boundary-Layer Meteorology 127:359–394.