The previous chapters have established the following main conclusions: First, the physical evidence associated with the anthrax spores in the attack letters did not reveal the source of the materials in the letters. Second, genetic analyses of the spores established that the anthrax belonged to a particular subtype known as the Ames strain. Third, while no genetic differences were observed between the predominant Bacillus anthracis genotype in the letters and the canonical Ames strain, FBI contractors discovered several subpopulations of B. anthracis cells in the letters that each produced distinct colony morphologies after growth on agar plates, and some of these subpopulations possessed specific genetic mutations that could be identified by specific sequence-based assays developed by the FBI and its contractors.
In this chapter, we describe the establishment of the FBI Repository (FBIR) of Ames strain samples, created by the FBI to look for the B. anthracis subpopulations found in the letters with the hope of being able to identify the source of the B. anthracis spores used in the attacks. The chapter also outlines the history of RMR-1029, the spore-containing flask at the U.S. Army Medical Research Institute for Infectious Diseases (USAMRIID) that became a focus of the FBI investigation, at least in part because it contained all of the subpopulation genotypes that were assayed.
The results obtained from screening the repository samples are described, including the evidence that appeared to implicate RMR-1029 as the source of the spores in the attack letters. Particular attention is given to the limitations of applying formal statistical methods to these results and of trying to quantify the strength of the relationship between the spores in the attack letters and those in RMR-1029. The committee then discusses evidence bearing on a disputed sample submitted by the suspect in this case as well as follow-up experiments performed by the FBI to determine whether that sample had come from its stated source, RMR-1029 (see Amerithrax Investigative Summary,
USDOJ, 2010, pp. 75-79, for a description of the circumstances surrounding the disputed sample). The chapter ends with the committee’s major findings regarding the genetic evidence relevant to the source of the B. anthracis spores used in the letter attacks.
At the time of the anthrax mailings, the federal government had not systematically collected information on which laboratories possessed anthrax. Although the “Select Agents” program was created in 1996 by the Antiterrorism and Effective Death Penalty Act (Public Law 104-132), this statute governed primarily the transfer of biological agents between research laboratories. The Act directed the Secretary of Health and Human Services (HHS) to issue regulations governing the transport of biological agents with the potential to pose severe threats to public health and safety through their use in bioterrorism, the so-called “select agents” (NRC, 2009). The authority to regulate select agents was delegated by the HHS Secretary to the Centers for Disease Control and Prevention (CDC). To ensure that these agents were transferred only between responsible parties, CDC required that any laboratories that might transfer select agents be registered and that transfers be reported to CDC and conducted under a permitting system. As long as the select agents were not transferred, specific information about the facilities that possessed these agents did not have to be reported (NRC, 2009b).
The determination that the Ames strain of B. anthracis was used in the attacks led to a process by which the FBI searched for and acquired samples of known and accessible derivative stocks of that strain for comparison. As noted in Chapter 2, the Ames strain had been widely distributed among laboratories around the world for research and vaccine trials, so the FBI first had to identify all laboratories that maintained stocks of the strain. Next, the Bureau had to obtain samples of these Ames strain derivatives, which would constitute the FBIR and be screened for the presence of the mutant genotypes found in the letters (see Chapter 5). To this end, the FBI prepared and issued a subpoena that included a protocol for the collection and submission of Ames strain samples (Box 6-1). This subpoena was sent in February 2002 to 16 laboratories or facilities in the United States that had been identified as possessing stocks of the Ames strain. (It was subsequently determined that one of these domestic laboratories did not possess the Ames strain1.) In addition to the subpoenas,
1 According to the FBI (discussion with committee, December 11, 2009), 16 U.S. laboratories were originally identified as candidates for having the Ames strain. According to the DOJ case closing summary, however, only 15 domestic laboratories were confirmed as repositories of the Ames strain (USDOJ, 2010, p. 17). CDC provided a listing of all laboratories registered to work with B. anthracis, and FBI investigators created their own list based on CDC select agent transfer records documenting every transfer of anthrax between 1997 and 2001 as well as anthrax inventory records
Purpose: Provide guidance in the preparation and shipment of Bacillus anthracis Ames strain to the United States Army Medical Research Institute of Infectious Diseases, Fort Detrick, Maryland. As per FBI request, submit two culture agar slants of each distinct B. anthracis Ames strain stock in your possession, which differs in source or in other parameters prescribed by the requesting Agency.
Tryptic Soy Agar (TSA) Slants (Remel catalog # 08932, or equivalent)
Sterile inoculating loops
Adhesive Labels, permanent, waterproof
STP-100 Infectious Substance Shipping Container (Saf-T-Pac), or equivalent
1. Collect each B. anthracis Ames strain stock as per your institutional inventory and personal knowledge.
2. Prepare a minimum of two TSA slant tubes per stock by prelabeling with permanent waterproof labels. Include the following information on the label: “B. anthracis Ames strain.” Other designators used by your laboratory, date, and your lab name. Additional information for each stock shall be provided separately.
3. A representative sample of each stock shall be used for inoculation of the TSA slants. If the stock is an agar culture, do not use a single colony, but rather use an inoculum taken across multiple colonies. Thawed frozen stocks or other liquid suspensions shall be well mixed prior to transfer of inoculum to the TSA.
4. Inoculate each TSA slants in a zig zag manner over the surface of the agar.
5. Incubate slants at 35-37C for 12-18 hours to confirm culture growth.
6. Individually wrap the slants in packing materials approved for the transport of infectious select agents in accordance with regulations for the shipment of such materials.
Source: FBI Documents, Preparing and Shipping TSA Slants for B. Anthracis Ames.
consent searches were conducted at USAMRIID in Maryland and at Dugway Proving Ground (DPG) in Utah, and a search warrant was executed at a private company in Ohio (Battelle Memorial Institute) in 2004 to ensure that samples were obtained from every stock derived from the Ames strain in those three facilities. Voluntary submissions were also requested from the three foreign
that were subpoenaed from over 100 Biosafety Level 3 (BSL-3) laboratories in the United States. “These records were supplemented by information culled from FBI interviews of scientists working at each of these labs, and through FBI reviews of relevant scientific publications mentioning the Ames strain” (USDOJ, 2010, p. 17).
laboratories (in the United Kingdom, Canada, and Sweden) known to possess the Ames strain, two of which provided samples.
The FBI described their approach to assessing Ames strains internationally in the following way: Complementing this initiative was a separate intensive effort to assess whether a foreign government or a terrorist organization may have gained access to the Ames strain and perpetrated the attacks. An exhaustive initiative, one that continued until the closing of the case, addressed this concern. As with the domestic investigation, it included reviews of published research and transfer records. In addition, it involved an assessment of available intelligence on foreign government and foreign actor capability, meetings with foreign experts, witness interviews, and the collection of Ames strain isolates from certain governments. (FBI/USDOJ, 2011).
Laboratories in possession of stocks derived from the B. anthracis Ames strain were directed to obtain samples from each of the stocks in both their institutional and personal inventories for submission to the FBIR. The subpoena asked for little information about the provenance of the samples being submitted, with no specific requirement that such information be provided. For this reason, information about the derivation of the various stocks from the original Ames strain was not standardized and varied greatly in its amount and specificity.
In general, stocks of B. anthracis are stored in different ways depending on the physiological state of the bacteria (spores or vegetative cells) as well as the procedures used in a particular laboratory. For example, some microbiological stocks are isolated from single colonies, cultured for only a few additional generations, and then stored in freezers; these stocks are expected to be genetically quite homogeneous. Other stocks exist as confluent lawns of cells on slants or as multiple colonies on agar plates; such stocks have the potential to accumulate genetic diversity if they are kept in these states for long periods. Still other stocks, including the one designated RMR-1029, are mixtures of several independent large-scale culture preparations; such mixtures may also harbor genetic diversity. The sampling protocol in the subpoena directed that a “representative sample” of each stock be provided. It specified different methods for obtaining the sample, depending on how the stock was kept in the laboratory. For agar-based stocks, the submitters were told to take an inoculum from multiple colonies (“do not use a single colony”, see Box 6-1), although a precise number was not specified. Samples taken from thawed frozen stocks or liquid suspensions were required to be “well mixed” prior to inoculating the slant that was to be delivered to the FBIR. The submitters were apparently not asked, however, to provide information about which methods they had used to make the inocula for submission nor in what form the stocks were stored. In addition, there was no effort to standardize the number of spores or cells that were submitted. These omissions limit the committee’s knowledge (and that of the investigators) about the quality of the samples submitted and, in particular, how well each submission met the requirement of being a “representative sample.”
In all, 20 laboratories (15 domestic, two foreign, plus USAMRIID, Dugway, and Battelle) submitted to the FBIR a total of 1,070 (USDOJ, 2010, p. 24) samples of stocks derived from the Ames strain of Bacillus anthracis. Of these, 1,059 samples were screened and results reported for the presence or absence of the four mutant genotypes—A1, A3, D, and E (FBI Documents, B2M10D2)—using the assays described in Chapter 5. The FBI told the committee that the other 11 samples were not viable, failed to grow the Ames strain of B. anthracis, or failed to grow B. anthracis at all. The results and interpretation of this screening are described below.
It is important, however, to recognize not only several inherent limitations of the FBIR collection that make it difficult to assess the evidence in any formal statistical sense, but also the effects of the decision to require growth of the samples before testing for the four mutant genotypes. In addition to issues of representativeness, there are issues with the independence of the samples.
First, statistical analyses typically assume that samples are taken at random from a defined population. The FBI aimed to create a comprehensive repository that encompassed the entire population of stocks derived from the Ames strain, rather than a representative fraction thereof. Given uncertainties in the extent of the entire population of Ames stocks worldwide (highlighted by concern about the possibility of clandestine stocks held by terrorist organizations—see section 3.4.3), the lack of specificity in the subpoena protocol, the uncertainties in compliance with the subpoena protocol, the incomplete information on transfers of Ames-derived stocks between laboratories, and the possibility that some stocks were produced but later destroyed, the repository was unlikely to have been comprehensive.2 DOJ states in its Summary, “The collection of Ames isolates from laboratories both from the United States and abroad that constitute the FBIR are a comprehensive representation of the Ames strain” (USDOJ, 2010, p. 28). Section 6.5 provides further discussion of this issue, along with the implications of the violation of this assumption of “representativeness” for the statistical inferences
Second, there were complex and varying degrees of genetic relationships among the stocks, reflecting their common descent from the original Ames isolate and the history of transfers, single-colony isolations, and mixtures of materials within and between institutions. As a consequence, some sets of related stocks were likely to be represented by many samples and others by few samples, rendering it impossible to assess the relevant frequencies of genotypes across the population of interest.
Third, statistical analyses are critically dependent on replication to provide measures of the sensitivity and specificity of assays, and such replication should
2 The issue of overseas samples discussion in Chapter 3, section 3.4.3 raises additional questions about the comprehensiveness of the FBIR: but as stated previously, these issues were beyond the scope of this committee.
be part of a protocol at every biological and technological level at which it is feasible. Although scientists were required to submit duplicate samples to the repository, only one sample was analyzed for the genotypes of interest while the other was retained in an archive. Subpoena recipients should have been required to submit three or more samples of each stock, so that replicate genotypic assays could have been performed. Such replication would have allowed a more rigorous assessment of the results of the genotypic assays.
Finally, even for replicate samples from the same stock, other sources of potential variation—such as in the density of cells or the spatial distribution of genotypes in the original stock—might have introduced statistically important dependence across the several assays performed on the samples. Thus, the results obtained from the several genotypic assays cannot be assumed to be independent.
Either during or after the development of the genetic assays, a decision was made that they should be performed on samples of B. anthracis Ames only after the cells in the sample were cultivated once again (in addition to the prior cultivation step by the recipients of the subpoena protocol before submitting the samples). According to the FBI, this further cultivation step was done in part to standardize the nature of the submitted samples prior to the testing (FBI, 2009). In addition, it would have expanded the amount of sample available for DNA preparation and subsequent testing by the different contractor laboratories involved in genetic assay development (FBI, 2009). However, the sensitivity of each of the molecular assays would not have necessarily required this additional cultivation step. Furthermore, the committee notes at least two problems with this additional cultivation step. First, the samples that were submitted to the repository but then failed to grow in culture could not be tested; however, they could have been tested if the genetic assays had been directly applied to the samples. Second, each cultivation step potentially created additional bias in the sample, because not all cell genotypes grow at the same rate under the same conditions (see also Section 5.5.2). Some variant cell types might grow more slowly and fall in abundance relative to the other cell types, such that they are no longer detectable, while others may grow more quickly and become more dominant. The committee was unable to assess the extent to which the benefits of sample cultivation outweighed these potential problems.
The presence of genetic variants in the anthrax letter samples (as described in Chapter 5) and in the spore population in RMR-1029 and in samples known
to have come from RMR-1029 (as described below) led the FBI to conclude that “RMR-1029 is the parent material of the evidentiary anthrax spore powder” (USDOJ, 2010, p. 28). Based on their investigation, FBI officials stated that RMR-1029 was produced in 1997 as “a conglomeration of 13 production runs of spores by Dugway, for USAMRIID, and an additional 22 production runs of spore preparations at USAMRIID that were all pooled in this mixture” (FBI, 2008c, p. 55). The resulting 164 liters of spore production were concentrated down to about a liter. Originally, the spore preparation was divided into two one-liter Erlenmeyer flasks, one of which was eventually depleted, so that only one flask remained for subsequent repeated sampling, as is discussed further below (FBI, 2008c, pp. 58-59).
This spore preparation was maintained at USAMRIID, where records indicate that it contained approximately 3 × 1010 spores per ml (USAMRIID, 1997). Between 1998 and 2003, aliquots were withdrawn from RMR-1029 for use in animal studies during the development of anthrax vaccines and therapeutics. The amount of RMR-1029 consumed in the course of these studies is documented in terms of transfers out of the flask (B3D14, B3D16), but given the concentrated nature of the preparation, a very small volume would have provided ample material for use as inocula to produce additional spore preparations such as those that might have been used in the letters.
The production history of the RMR-1029 spore population provides a probable explanation for the presence of subpopulations of genetic variants, including variants that could give rise to visually distinctive colonies (morphotypes). For example, colonies of genetic variants that are defective in spore formation typically have surface textures and coloration that distinguish them from the wild type (see Chapter 5). While it is standard practice in microbiology to prepare new cultures by starting from single-colony purified stocks, this procedure was deemed impractical (Martin, 2010) for the preparation of the large volumes of spores that were combined to produce the RMR-1029 stock. This spore suspension was intended for use as a reference stock, or population, that could be drawn upon for many studies, thereby allowing comparison of results across studies performed at different times. The large number of spores required for an extended set of studies could not, however, be prepared in a single batch (especially given the biosafety level required for work with a dangerous pathogen). Because the fermentors used for growing large batches of spores were not available at USAMRIID, the bulk of the material in RMR-1029 was generated at Dugway.
It is likely that some or all of the genetic variants (including especially those discovered on the basis of atypical colony morphologies) present in RMR-1029 were present in the material provided by Dugway, for the following reasons. First, the DPG material was prepared using inocula that had not been started from a single colony but instead came from stocks that had been obtained from USAMRIID. Bulk material from the stock was then used to inoculate blood-agar plates.
A photograph shown to the committee of a dense lawn of B. anthracis grown on such a plate at Dugway reveals the presence of many papillae, small outgrowths of bacteria indicative of mutants that are overgrowing their neighbors in the lawn or have some other distinctive feature (Martin, 2010). The scientist who repeatedly prepared these materials for the multiple production runs told the committee that the presence of numerous papillae was the typical outcome. The committee believes that the presence of these papillae can be taken as evidence that the agar slants already contained mutants, that growth of the bacterial population on the blood agar plates selected for mutants, or both. Because B. anthracis cells sporulate on blood agar once they reach high density, the papillae could have been outgrowths of sporulation-defective mutants that continued to grow for several generations after other nonmutant cells stopped growing and formed spores.
Second, according to one DPG scientist, the biological material (lawns, including papillae) scraped from these plates was then used directly to inoculate the fermentors at Dugway (Martin, 2010). This material was collected from the plates after the lawn population had largely converted to spores. Because spores must germinate before growth can resume, unsporulated cells (including from sporulation-defective mutants in the inoculum) would likely have had a growth advantage in the fermentor. Because material prepared at Dugway comprised the bulk of the material that was pooled in the RMR-1029 flask, and because the inocula used to prepare the spores had visible evidence of mutants that may have been defective in spore formation, the committee suggests that at least some of the morphotypes identified in RMR-1029 originated from Dugway. Various biological factors would have affected the resulting presence and abundance of the genetic variants, including their growth rates, germination rates, and sporulation efficiencies under the specific cultivation conditions used as well as the rate at which each variant arises by mutation.
The assays developed by the various contractors (Commonwealth Biotechnologies, Inc. [CBI], Midwest Research Institute [MRI], IIT Research Institute [IITRI], and the Institute for Genomic Research [TIGR]) and described in Chapter 5 were used to analyze the samples submitted to the FBIR under the subpoena protocol or gathered through the searches performed by the FBI. The purpose of these assays was to search for the presence of the genetic mutations—A1, A3, D, and E—among the FBIR samples to determine whether any of the samples had a genetic profile that matched that of the evidentiary material and could be a possible source of the material used in the letters.
It is important to reiterate here that the existence in laboratories of the Ames strain of B. anthracis dates back only to 1981 and that all samples of the Ames strain at laboratories outside USAMRIID had been derived either
directly or indirectly some time between 1981 and 2001 from USAMRIID’s stocks of the organism. This fact is of substantial significance and limits a quantitative statistical analysis of the results of the FBIR assays.
The committee reviewed the report provided to the FBI by a contractor who performed statistical analysis of the assay results for the 1,059 viable FBIR samples. The results of the assays were summarized by the contractor in a table (reproduced here as Table 6-1) in the Statistical Analysis Report.
The FBI did not seek formal statistical expertise until it had completed the genotype assays of the repository samples. Thus, the utility of the statistical analysis was limited in part by the fact that the experimental design was created without input from statisticians. The FBI contractor analyzed the data for
|No Bacillus DNA||17||17||—||—||15|
NOTES: MRI-D and IITRI-D refer to the two independent assays for the D mutation genotype developed and performed by the Midwest Research Institute (MRI) and IIT Research Institute (IITRI), respectively.
— indicates that no samples were reported in that category.
“Neg-u” = The meaning of this designation is not clear to the committee. “Neg-u” designation appears in the Statistical Analysis Report but it is not defined and does not appear in any of the final reports on D assays submitted to the FBI from MIR and IITRI. Subsequent analysis combined these samples from MRI with MRI’s 940 “negative” results.
Variant = sample similar to but not identical to the genotype.
Pending = sample analysis incomplete when Statistical Analysis Report submitted to FBI. Subsequent report from MRI indicates that MRI classified the sample as “inconclusive.”
Inconclusive (IITRI) = inconsistent results on replicates.
Inconclusive (MRI) = inconsistent results on replicates or no growth or no Bacillus DNA. Thus IITRI separated the three categories while MRI classified all three categories as “inconclusive.” (The final report from MRI [FBI Documents, B2M8D7] divides 35 “inconclusive” samples into nine “inconclusive analysis” (i.e., “failure to produce a clear reproducible positive indication for the presence of the Morph D deletion”), 19 insufficient DNA, and 7 no growth.
SOURCE: FBI Documents, B2M10D2.
947 of the samples that provided definitive results for all four genotypes. For genotype D, the IITRI data were dropped because they did not provide results in all categories, so analysis of the D genotype was based on the MRI data only. An additional 112 samples were omitted from further investigation because the results with these samples were recorded as “inconclusive,” “variant,” “no growth,” or “no DNA” for one or more of the four genotypes. Tables 6-2 and 6-3 summarize the data from the Statistical Analysis Report in two ways, in each case using only the 947 samples that produced definitive results. Table 6-2 shows the frequencies of positive and negative results for each of the four genetic assays, and Table 6-3 shows the frequency of samples that showed a given profile of positive and negative results across the set of assays.
SOURCE: FBI Documents, B2M10D2.
NOTE: — = negative result
SOURCE: FBI Documents, B2M10D2.
The Statistical Analysis Report prepared for the FBI (FBI Documents, B2M10D2) contains five conclusions that are summarized below.
(a) Frequency of 4-positive samples
The fraction of the 947 samples with positive results for all 4 genotypes was 8/947 = 0.00845. This fraction was offered as an estimate for the probability of the occurrence of a 4-positive (++++) sample. (“Retaining only positive and negative results of the four assays, in the 947 repository samples, eight showed simultaneous positive results ++++ for all four assays (i.e., 0.84 percent), with exact 95 percent confidence interval of 0.0037 to 0.0166 (i.e., from 1 in 270 to 1 in 60)” (FBI Documents, B2M10D2).
(b) Dependence among assays
The interdependence among the assay outcomes (co-occurrence of genotypes) was examined using various statistical tests and it was concluded that the occurrence of the four genotypes did not appear to be independent.
Briefly, as presented in the report, if the outcomes were independent, and noting that 27 of the 947 samples showed A1, 16 of the 947 samples showed A3, 51 of the 947 samples showed MRI-D, and 16 of the 947 samples showed E, then one would expect the fraction of samples testing positive for all four genotypes to be 0.4383 per million, or less than one in a million. Since there were 947 samples, one would have expected to see 947 times this fraction, or 0.0004 samples with all four mutations. That is, if the assay outcomes really were independent, one would not expect to have seen any samples with all four mutations.3 In fact, however, the FBIR contained eight samples that were positive for all four mutations, a number considerably larger than the expected 0.0004 samples based on the assumption of independence. Similarly, the report noted that the fraction of samples that would be expected to have only one of any of the four genotypes can be calculated as 0.1072; hence one would expect to see (0.1072 × 947) = 101.5 of the 947 samples with only one positive assay. The FBI observed 50 samples with one positive assay.
(c) Relationships among samples
Given the existence of historical relationships among the 947 repository samples, the Statistical Analysis Report described two methods to assess these relationships. The first was a “network analysis” applied to the distribution of combinations of assay results for the four genotypes among the FBIR samples. Table 6-3 provides the number of each of these 11 combinations among the 947
3 Under the assumption that the assay results are independent, the probability of joint outcomes equals the product of the probabilities of each separate outcome. The observed frequencies are estimates of the probabilities. Multiplying the four observed frequencies together yields 0.0285 × 0.0169 × 0.05385 × 0.0169 = 0.4383 × 10−6 for 4 positive genotypes, and 0.9715 × 0.9831 × 0.94615 × 0.9831 = 0.8884 for 4 negative genotypes.
FBIR samples that were considered in the Statistical Analysis Report. The network analysis provides a representation of the relationships among the genotypic configurations, informed by their frequencies and the mutational differences among them. In the Report, it was concluded from this representation that the eight samples that were positive for all four genotypes were most closely associated with the two samples that were positive for three genotypes (A1, A3, D) and with the six samples positive for two of these three genotypes. From the analysis, it appeared that the 16 samples positive for E were less closely related.
The second method in the Statistical Analysis Report to assess relationships among samples relied on a spreadsheet of dates (rows) and labs (columns) indicating the date when a sample “from” one institution was transferred “to” another facility. (The spreadsheet indicated that transfers occurred but it did not indicated the specific relationships among samples.) The FBI told the committee that the spreadsheet was generated from laboratory reports and included all recorded transfers of Ames samples between 1981 and 2001. The spreadsheet showed known direct or indirect relationships between Laboratory F (USAMRIID) and most (but not all) of the other nine laboratories or institutions that submitted samples that were positive for one or more of the four genotypes. Based on these two methods, the Statistical Analysis Report concluded that there were associations between the eight samples testing positive for four genotypes and the other samples that were positive for one or more of the genotypes.
(d) Sensitivity and specificity
The statement of work that preceded the submission of the Statistical Analysis Report indicated plans to provide estimates of sensitivities and specificities of the assays for the four genotypes. The committee was able to find only a limited analysis of this sort in the final report.
(e) Significance of “Laboratory F” as the source of seven of the eight samples found to be positive in all four assays
The Statistical Analysis Report concluded that the chance of finding eight 4-positive samples should be very low (i.e., less than 0.0166, or less than 1 in 60). The report also notes that seven of the eight 4-positive samples came from only one institution (“Laboratory F”), that the eighth sample came from another institution, and that “its occurrence in [the other laboratory] is explained by a recent sample transfer” (FBI Documents, B2M10D2).
Representativeness, Randomness, Independence
Many of the methods used in the Statistical Analysis Report rely on the assumption that the 947 FBIR samples were a representative and random
collection, independently sampled from some well-defined population of B. anthracis samples. In fact, there is no meaningful population beyond the repository itself, and the repository could not be a random collection of independent samples due to the relationships and sharing among laboratories. Further, the FBIR sometimes contained samples from the same source. The FBI confirmed (1/14/2011 meeting) one such instance: FBIR samples 006-002 (Statistical Analysis Report, p.28) and 067-001 (Statistical Analysis Report, p. 57) were duplicates made from the same submission (“the disputed sample”). Although the lack of independence among the genotype assays was properly acknowledged in the Statistical Analysis Report, statistical calculations were performed that relied on the assumption of independence among the 1,059 samples (e.g., chi-squared tests on p. 10, Statistical Analysis Report).
Committee concerns about each part of the Statistical Analysis Report are described below:
(a) Frequency of 4-positive samples
Because the eight 4-positive samples and the 947 overall samples were not independent samples, the proportion (8/947) as an estimate for the probability of the occurrence of a 4-positive (++++) sample is not meaningful nor is a calculated “95% confidence interval” for this probability.4
(b) Dependence among assays
Table 6-4 from the Statistical Analysis Report shows all possible ways of obtaining samples with no positive assays, and 1-, 2-, 3-, or all 4-positive assays. The Statistical Analysis Report includes a formal statistical test to assess the significance of the differences between these “observed frequencies” and the “expected frequencies.” However, the inference from this test is not valid because, as noted above, the 947 samples were not independent.
The committee identified two potential sources of dependence among repository samples in the results of the four genotype assays. The most obvious is the genealogical structure of descent from the original Ames ancestor in light of the history of transfer of B. anthracis samples and stocks among laboratories (see discussion below). Another possible source of dependence among the assay results stems from variation among samples in the process of preparation (under the guidance of the subpoena protocol) and in the amount of DNA, leading to sample-specific variation in probabilities of detection of the genotypes.
4 The confidence interval is computed from the binomial distribution (Snedecor and Cochran, 1989). This calculation is valid only if the 947 samples are independent. As an aside, even if it were appropriate to calculate a 95 percent confidence interval with these data, the calculation is misinterpreted in the Statistical Analysis Report. The calculated interval (0.0037, 0.0166) is actually a 95 percent confidence interval for the probability that a sample from the target population is a 4-positive sample, not for the occurrence of eight 4-positive samples in the target population, as stated in the Statistical Analysis Report (FBI Documents, B2M10D2; cited above in Section 6.1(a)).
|Number of + signatures||Observed frequency||Expected frequency|
SOURCE: FBI Documents, B2M10D2.
(c) Relationships among samples
Given the counts of the different genotypic configurations in Table 6-3, the result of the network analysis is inevitable and adds no new insights. This frequency-dependent analysis is dominated by the 876 samples showing none of the four genotypes, while the appearance of a more distant relationship of E-positive samples results both from their low frequency and the absence of E in the only 2 samples positive for three genotypes. Note, however, that only 16 of the 23 E-positive samples were included in this analysis, and that several of the 7 other E-positive samples excluded from the statistical analyses were also positive for other genotypes (Appendix C, Table C-1). Further, the network analysis is designed to display relationships among naturally evolving organisms. No conclusions can be drawn from its application to the FBIR samples, without knowing their complex history of transfer and mixing relationships.
(d) Sensitivity and specificity
As noted above, the statement of work that preceded the submission of the Statistical Analysis Report included a request for estimations of sensitivity and specificity of assays; however, the committee found only a limited analysis of this sort in the final report. While assay sensitivity and specificity were assessed by the assay development contractors in an idealized and artificial setting during the development of the assays, a more meaningful assessment in the context of the testing of actual repository samples would require replication of tests using FBIR samples, or the creation of multiple types of simulated stock samples.
To assess the sensitivity of the assays, dilution experiments were conducted on two samples, RMR-1029 and “SPS.266 Tube#5,” using three replicates at each of 10 dilution levels. Unfortunately, the analysis reveals substantial variation in the assay results. In RMR-1029, the genotypes were usually not detected beyond a dilution of 1:10. For SPS.266 Tube#5, one of the three replicates was positive for E out to a dilution of 1:100,000, yet two replicates
Sample 1: RMR-1029
Sample 2: SPS.266 Tube#5
NOTE:—denotes either negative or no growth
SOURCE: FBI Documents, B2M10D2.
were negative even at just a ten-fold dilution and one replicate for A3 was positive even at a 1:100,000,000 dilution, whereas most were negative at 1:100,000 to 1:10,000,000 dilutions. Table 6-5 provides the results from these dilution experiments summarized in the Statistical Analysis Report.
The lack of agreement between MRI and IITRI assays for the D genotype has already been noted (Table 6-1), and further illustrates the differing sensitivities and specificities of the assays. The results of the two assays agreed for 945 of the 1,059 samples, for a concordance rate of 92.1 percent, but disagreed for 30 samples.5 While concordance is informative, the 30 samples with discordant results provide a valuable opportunity to assess the validity of the assay and gain further insight into these samples; this opportunity was not fully exploited.
5 The decision to set aside the IITRI results seems somewhat arbitrary. The report states that IITRI obtained “no growth” with 13 samples but MRI obtained growth on all samples. However, the Final Report from MRI-D (FBI Documents, B2M8D17), dated 21 July 2006, notes “No growth” on 7 samples. The total number of samples yielding no information was similar for both MRI-D (35) and IITRI-D (37).
In its conclusions, the FBI paid particular attention to samples carrying three or four of the genotypes. However, the FBI did not address the issue of false negative results. In connection with this issue of sensitivity of the assays, a major concern regarding the Statistical Analysis Report is the restriction of its analyses to the 947 samples that contained no inconclusive or variant results. Additionally, no errors and uncertainties of detection, nor variation in sample preparation, are taken into account for the analysis. Four of the 112 disregarded samples scored positive for the remaining three genotype assays (see Appendix C, Table C-1).
The lack of replication in the assays of the FBIR samples makes it impossible to quantify the strength of any finding relating to the presence or absence of genotypes in the repository samples since some absences may be false negatives. Because samples were not retested and because the dilution experiments demonstrate the potential for different results on the same sample, one cannot quantify the strength of any finding related to the absence or presence of genotypes in the repository samples: thus, some test results of “negative” could well be false negatives (“present but unable to detect”). Consequently, the finding of all four genotypes in both RMR-1029 and seven samples from one laboratory clearly suggests a relationship between RMR-1029 and three of the four attack materials, but it is impossible to calculate any measure of “statistical strength” for this association.
(e) Significance of Laboratory F as the source of seven of the eight 4-positive samples
As suggested in the Statistical Analysis Report (p. 2), on first inspection it may seem noteworthy that seven of the eight samples scoring positive for all four genotypes came from “Laboratory F.” However, Laboratory F submitted almost two-thirds (63 percent) of the 947 samples. (Most laboratories submitted fewer than 15 samples; seven submitted 18 to 74 samples, and Laboratory F submitted 598 samples; see Table C-2 in Appendix C.) Consequently, Laboratory F could have submitted most or even all of the eight 4-positive (++++) samples merely by chance. More precisely, the probability that seven, or even all eight, of the 4-positive samples would end up among the Laboratory F 598 submissions, merely by chance, is 0.14223 (about 1 in 7).6
In late February 2002, Bruce Ivins and a technician who worked with him at USAMRIID prepared duplicate samples of four Ames-derived stocks from
6 This probability calculation arises from the hypergeometric distribution for the probability that 7 or 8 of the 8 samples arise in a subset of 598 samples from the total 947 samples; see Appendix C, Table C-2.
his laboratory for submission to the FBIR. The second copy of each sample was sent to Paul Keim’s laboratory at Northern Arizona University. According to the Department of Justice (USDOJ, 2010, p. 78), on or before March 28, 2002, the USAMRIID staff who had collected that laboratory’s Ames strain samples for submission to the FBI “advised Dr. Ivins and his laboratory technician that their submissions were not prepared according to the FBIR protocol.” The DOJ report continues: “Specifically, Dr. Ivins and his lab technician used homemade slants as opposed to the commercially available Remel slants specified by the protocol, so the four slants prepared on February 27, 2002 were rejected by the FBIR, and Dr. Ivins was told to resubmit his culture samples on the appropriate slants” (p. 78). The FBI disposed of its rejected samples but, importantly, the duplicate copies sent to Keim’s laboratory were retained, including one that had come from flask RMR-1029 and that would later be shown to contain all four genotypes. In April 2002, Ivins submitted four newly prepared samples, again in duplicate, and this time the samples were accepted. One copy of each sample entered the FBIR and the duplicates were again sent to Northern Arizona University.
In the course of the investigation, certain doubts were raised about whether Ivins had submitted samples of all relevant Ames stocks in his possession to the FBIR. In April 2004, the FBI secured RMR-1029 and other B. anthracis samples that had been under Ivins’s control, and these were transferred elsewhere for various tests. From these tests, the FBI concluded that: “Genetic analysis determined that … RMR-1029—the purest and most concentrated batch of Ames spores known to exist—was the parent to the evidentiary material used in the anthrax mailings” (USDOJ, 2010, p. 79). The committee addressed this conclusion (see discussion above) and also focused on secondary analyses performed in order to reconcile certain discrepancies in the multiple samples that were submitted by Ivins and purported to come from flask RMR-1029.
FBI investigators had observed that the second RMR-1029 sample submitted by Ivins in April 2002 did not score positive for any of the four genotypes discovered in the attack letters and in RMR-1029 as well as in samples derived from RMR-1029 that were submitted by other scientists. Based on this apparent discrepancy, in late 2006 the FBI obtained from Northern Arizona University the duplicate of the first RMR-1029 sample that Ivins had submitted in February 2002, which was then put into the FBIR and analyzed. This earlier RMR-1029 sample scored positive for all four of the genotypes that were assayed (A1, A3, D, and E), whereas the later sample had scored positive for none of them.
The FBI sought to determine the cause of this discrepancy between the earlier and later submissions by Ivins that both were supposed to have come from the same RMR-1029 flask. One possibility (the null hypothesis) was that repeated samples from RMR-1029 following the FBI protocol might produce variable results. This possibility could reflect, for example, insensitivity of the molecular tests at the genotypic frequencies present in that flask. To address
that issue, and on the recommendation of external science advisors (the “Red Team”; USDOJ, 2010, p. 79), the FBI directed that an experiment be performed in which the RMR-1029 flask was sampled in an identical manner 30 times following the subpoena instructions. These replicate samples were processed at the National Bioforensic Analysis Center and the resulting material analyzed by scientists at CBI, MRI, IITRI, and the University of Maryland for the presence of the same four genotypes. According to the DOJ summary of the case (USDOJ, 2010, p. 79), the results were as follows: “Occasionally, only three of the four genetic mutations were detected, and at no time were less than three detected. It followed that if Dr. Ivins prepared his submission to the repository in accordance with the protocol, that submission could not miss all four of the morphological variants present in RMR-1029.”7
Table 6-6 presents the results of these 30 analyses. These data demonstrate substantial variability among the 30 replicates. Sixteen of the 30 samples scored positive for all four mutations and another eight scored positive for three of the four mutations; however, five samples scored positive for only two mutations, and one sample (Sample 20) scored positive for just one of the four mutations. (Inconclusive results, as well as cases where the IITRI and MRI tests for the D genotype gave different results, were considered “negative” outcomes by the committee.) None of the 30 samples scored negative for all four mutations.
Given these assay results of the 30 replicates, what is the probability that analysis on an additional sample, taken from RMR-1029 in the same manner, would yield negative results for all four genotypes by chance alone? Since none of the 30 samples was negative for all four assays, the probability of obtaining a sample of four negatives, by chance alone, cannot be very high and, according to the binomial distribution8 it is unlikely to be higher than 0.095 (9.5 percent).
At the January 2011 meeting with the FBI, the committee was told that the second copy of this disputed submission was also analyzed and tested negative for all four genetic markers. That is, the second copy of Ivins’ disputed April 2002 submission, kept at NAU along with the second copies of all submissions, was moved into the FBIR, given a new identification number, retested, and found to be negative for all four genetic markers. If the assays on these two samples were as independent of one another as the assays performed on the 30 samples in Table 6-6, then the probability that both sets of assays would yield
7 On October 15, 2010, The DOJ issued an erratum stating: “Based on a ‘Red Team’ recommendation, experiments were prepared at the direction of the FBI Lab to address the FBIR submission process with regard to RMR-1029. RMR-1029 was sampled 30 times in accordance with the subpoena instructions. In a few instances, fewer than three markers were detected. However, in none of the 30 attempts were no markers detected. It followed that if Dr. Ivins prepared his submission to the repository in accordance with the protocol, that submission could not miss all four of the morphological variants present in RMR-1029.”
8 That is, if the probability of a sample having four negatives were greater than 0.095, the probability of obtaining no such samples among the 30 would be less than (1 − 0.095)30 = 5%.
|A1||A3||D IITRI||D MRI||E|
NOTES: += positive ; IITRI = IIT Research Institute; MRI = Midwest Research Institute
SOURCE: FBI Documents, B2M10D2.
negative results for all four genotypes would be (0.095 × 0.095) = 0.009025 (or 0.9 percent). While it is still possible that two such results could have occurred by chance alone, this chance is very small (less than one percent).
During the course of the committee’s review of the scientific evidence, the Department of Justice officially closed the investigation. The committee is not in a position to offer a judgment about the importance and strength of the scientific investigation relative to the importance and strength of the criminal investigative component of this case because it was not charged with (and lacked the expertise for) reviewing the latter. Our major finding is that:
It is not possible to reach a definitive conclusion about the origins of the B. anthracis in the mailings based on the available scientific evidence alone.
Finding 6.1: The FBI appropriately decided to establish a repository of samples of the Ames strain of B. anthracis then held in various laboratories around the world. The repository samples would be compared with the material found in the letters to determine whether they might be the source of the letter materials. However, for a variety of reasons, the repository was not optimal. For example, the instructions provided in the subpoena issued to laboratories for preparing samples (i.e., the “subpoena protocol”) were not precise enough to ensure that the laboratories would follow a consistent procedure for producing samples that would be most suitable for later comparisons. Such problems with the repository required additional investigation and limit the strength of the conclusions that can be drawn from comparisons of these samples and the letter material.
The FBI and contract scientists appropriately recognized that the mutations in the letter isolates provided information that might help identify the source of the B. anthracis used in the attacks, developed appropriate assays for four of these mutations, and created and screened a repository of Ames strain samples. Based on the results of that screening, FBI scientists appropriately concluded that the majority of repository samples contained none of the four mutations, although 50 of the samples contained one of the four mutations and 10 samples had three or all four mutations (the numbers with one or more mutations are higher if one includes samples that were excluded in the FBI’s statistical report). However, features of the repository including unknown sample provenance, and the history of sharing and mixing of stocks, presented investigative challenges.
The first challenge with the repository was the lack of independence among samples and an incomplete understanding of the provenance of samples due
to the known history of sharing. Sharing of samples between laboratories is an important part of scientific research and is critical to testing reproducibility and furthering scientific analysis. Prior to the attacks of 2001, several institutions shared samples of the Ames strain to different extents, resulting in variation in the numbers and kinds of samples they submitted to the FBIR. Also, this sharing extended not only to substrains but also to mixtures of several substrains that were grown in separate batches and then pooled. While such sharing and transfers are important in scientific research, they make it more difficult to identify a unique source of the mutations found in the attack materials. In recognition of this important issue, FBI scientists and investigators sought to determine the history of shipments among institutions and the genealogical relationships among samples in the repository, but they never obtained a complete record.
Another challenge with the repository was that, since the importance of the mutant genotypes was not fully understood when the subpoena protocol was written, the document was vague (e.g., “use an inoculum taken across multiple colonies”), and was not written in a way that would maximize the chance that variant genotypes in a mixed stock population would be submitted. Thus, if the four assayed genotypes had been present in a laboratory culture at low frequency, it is not clear whether they would have found their way into the sample of the culture submitted to the repository, since as few as two colonies would have satisfied the instructions provided in the subpoena protocol. After the importance of the mutant genotypes became known, there was no request for additional samples using a revised protocol that might have improved the sampling.
A final challenge was that the repository collection process was based on the integrity of the individuals asked to provide samples. If the motive for the repository was to identify the source of the letter material, standards of custody of evidence would dictate that agents of the FBI should have obtained the samples. In most instances, holders of the material were asked to provide samples and send them in. The sender could have been the instigator and may not have complied with instructions, as the FBI alleges with respect to Dr. Ivins.
Finding 6.2: The results of the genetic analyses of the repository samples were consistent with the finding that the spores in the attack letters were derived from RMR-1029, but the analyses did not definitively demonstrate such a relationship.
The scientific data alone do not support the strength of the government’s repeated assertions that “RMR-1029 was conclusively identified as the parent material to the anthrax powder used in the mailings” (USDOJ, 2010, p. 20), nor statements about the role of the scientific data in arriving at their conclusions, as in “the scientific analysis coordinated by the FBI Laboratory determined that RMR-1029, a spore-batch created and maintained at USAMRIID by Dr. Ivins,
was the parent material for the anthrax used in the mailings” (USDOJ, 2010, p. 8).
The committee agrees that the genetic evidence is consistent with and supports an association between the RMR-1029 flask and the B. anthracis used in the three attack letters that were tested; however, there are several important caveats. As discussed above, the nature of the repository collection, including the incompletely documented history of sharing and mixing of Ames strain stocks and the ambiguity in the subpoena protocol, makes it difficult to quantify the strength of the evidence linking RMR-1029 to the letters, because of the complex and ill-defined nature of the reference population. The materials from at least three of the four attack letters harbored a mixture of several variant genotypes. By contrast, the repository may include samples recently derived from single colonies, as is customary for microbiological research, but it is unlikely that these samples by virtue of this recent history would have more than one of the variant types. Moreover, owing to the practice of sharing samples among laboratories, many samples were essentially duplicates of one another. It is also possible that the sample repository was incomplete because the global distribution of Ames stocks was not known or because some stocks might have been destroyed prior to the subpoena. These limitations made it impossible for the committee to generate any meaningful estimate of the probability of a coincidental match between the B. anthracis genotypes discovered in the attack letters and those later found by screening samples from the RMR-1029 flask.
Finding 6.3: Some of the mutations identified in the spores of the attack letters and detected in RMR-1029 might have arisen by parallel evolution rather than by derivation from RMR-1029. This possible explanation of genetic similarity between spores in the letters and in RMR-1029 was not rigorously explored during the course of the investigation, further complicating the interpretation of the apparent association between the B. anthracis genotypes discovered in the attack letters and those found in RMR-1029.
Another challenge with determining the cause of the apparent association between some of the B. anthracis genotypes in the attack letters and those found in RMR-1029 stems from the possibility that the same mutations might have arisen repeatedly in other Ames strain populations. Colony variants with similar or even identical mutations might arise repeatedly in other populations for two reasons. First, we do not know the number and rate of possible mutations that could produce similar phenotypes. Research by Worsham and colleagues (see, for example, Worsham and Sowers, 1999) identified numerous oligosporogenous variants with phenotypes similar to those described in the letters. In response to questions raised by the committee, the FBI indicated that among the 296 Ames submissions to the FBIR by Worsham, only the Morph D genotype was detected, and it was in three samples. The A1, A3 and E muta-
tions were not detected in any of the Worsham samples submitted to the FBIR (FBI, 2010a). Second, under the conditions used to grow B. anthracis for the large-scale production of spores, there may well have been inadvertent selection that favored oligosporogenic mutants, which would cause the frequency of these mutants to be higher than expected for mutations that conferred no advantage to the cells, thereby increasing the likelihood of parallel evolution in replicate spore productions. The recent published work by Sastella and colleagues (2010) highlights the possible role of repeated passage of B. anthracis in the enrichment of sporulation-deficient mutants.
Finding 6.4: The genetic evidence that a disputed sample submitted by the suspect came from a source other than RMR-1029 was weaker than stated in the Department of Justice, Amerithrax Investigative Summary.
The committee reexamined the data that the FBI obtained following the discovery that one of the samples submitted by Bruce Ivins, which was supposed to have been taken from RMR-1029, did not test positive for any of the four assayed mutations (A1, A3, D, and E) in either of two copies analyzed. As discussed in Section 6.6, an experiment was performed in which 30 replicate samples were taken from RMR-1029 according to the FBI subpoena protocol and tested for the four mutations. Based on these results, the committee found that it is, in fact, possible that the disputed sample came from RMR-1029, and the probability of this outcome—that an actual sample from RMR-1029 would test negative for all four genotypes in two sets of assays—might be on the order of 1 percent. Hence, while the evidence is strongly suggestive that the disputed sample was not taken from RMR-1029, it is less certain than is indicated in the original version of the case-closing summary issued by the DOJ, which asserted that all 30 additional samples scored positive for at least three of the four genotypes, and concluded that “It followed that if Dr. Ivins prepared his submission to the repository in accordance with the protocol, that submission could not miss all four of the morphological variants present in RMR-1029” (USDOJ, 2010, p. 79).9
Finding 6.5: The scientific data generated by and on behalf of the FBI provided leads as to a possible source of the anthrax spores found in the attack letters, but these data alone did not rule out other sources.
The committee was not charged with reviewing, nor was it given access to, the findings from the criminal investigation component of this case. The committee therefore could not assess the potential value of additional scientific investigation with respect to better establishing the source of the B. anthracis
9 See footnote 7.
attack spores. Additional experiments might be of value were there a need to strengthen the scientific aspects of the case (in particular, if the case depended on the findings of the scientific investigation component as opposed to the criminal evidence).
Finding 6.6: Point mutations should have been used in the screening of evidentiary samples.
The FBI chose to study only a subset of the mutational variants of B. anthracis found in the attack letters. In particular, certain point mutations were not investigated, apparently because FBI scientists regarded them as less stable, more difficult to screen for their presence in the repository, or both. In the committee’s view, key evidentiary samples in the repository (along with appropriate controls) should have been screened for all of the mutations found in all of the letters. The stability of point mutations should not have been a concern, as most point mutations have extremely low rates of reversion and most of the methods and data used to track the spread of infections rest on this stability. Moreover, the forward rate of deletion mutations and both the forward and reverse rates of insertion mutations (the types of mutations screened for in the A1, A3, D, and E genotypic assays; see Chapter 5 and Table 5.2) are usually higher than the corresponding rates for point mutations. Thus, the possibility that identical mutations may arise independently (in parallel) is greater for insertions and deletions than for point mutations. With regard to the concern about the greater difficulty of screening for point mutations, it should be feasible to sequence directly the relevant genes in a large number of samples, including those that are genotypic mixtures such as RMR-1029 and the letter samples, using high-throughput “next-generation sequencing” methods (see Finding 6.8).
Finding 6.7: Biological material from all four letters should have been examined to determine whether they each contained all four genetic markers used in screening the repository samples.
The FBI observed morphological variants in the B. anthracis isolated from the attack letters and used those variants to identify mutations that were then used as genetic markers during the systematic screening of the repository samples. However, the FBI did not systematically examine the materials from the attack letters to determine whether all of these markers were present in each letter sample (possibly owing to concerns about the limited amount of biological material from some of the letters). And although genotypes A1, A3, and E were identified in all three letters that were examined (New York Post, Leahy, Daschle), the D genotype was found only in the New York Post letter. The material in the Brokaw letter was not examined for the presence of any
of the genotypes, so the FBI could only infer from other nongenetic evidence that the biological properties of the materials in this letter were the same as or similar to the materials in the New York Post, Leahy, and Daschle letters.
In the committee’s view, it would have been useful to determine whether all of the genotypes were present in all of the letters and, if so, at what relative abundances. This issue is important because similar morphotypes can and did arise from different mutations, and multiple mutations producing similar phenotypes were present in some of the letters. Thus, the presence of similar morphotypes in different letters does not mean that the same genotypes were present in all the letter samples. Given the conspicuous differences in the physical properties of some of the letter samples, it is even more important to establish their genetic similarities and, if relevant, consider the possible implications of any genetic differences that might have been found.
Finding 6.8: New scientific tools, methods, and insight relevant to this investigation became available during its later years. An important example is high-throughput “next-generation” DNA sequencing. The application of these tools, methods, and insight might clarify (strengthen or weaken) the inference of an association between RMR-1029 and the spores in the attack letters. Such approaches will be important for use in future cases.
When the committee began its deliberations the FBI’s anthrax letters investigation was still open, although the FBI had publicly declared its confidence in having identified the sole person responsible for the mailings. During the committee’s review of the scientific evidence, the DOJ officially closed the investigation. The committee is not in a position to offer a judgment about the importance and strength of the scientific investigation evidence relative to the importance and strength of the criminal investigation evidence, because it was not charged with (and lacked the expertise for) reviewing the latter.
Since 2001, important technological advances have occurred that would allow a more thorough and systematic analysis of genetic similarities and differences between key evidentiary samples, repository samples, and appropriate controls. When the investigation began, bacterial genomes were typically sequenced at an average coverage of fewer than 10 reads per nucleotide (see, for example, Read et al., 2002). Given the error rates inherent in sequencing technologies and the costs of whole-genome sequencing at that time, it was feasible to sequence only a few well-chosen clones as was done first to compare an isolate from the Florida victim with the Porton Down Ames strain (Read et al., 2002), and then search for mutations in several morphotypes (as described in Chapter 5).
It was also not easy in the early 2000s to sequence and interpret the data from mixtures of bacterial genomes. However, that changed during the course of the investigation and it is now possible to sequence bacterial genomes,
even including minority components of heterogeneous populations, to much greater depths of coverage much more quickly, allowing discovery of genetic polymorphisms (mutational variants) in bacterial populations without requiring phenotypic discrimination (Barrick and Lenski, 2009; Holt et al., 2009). Appropriate statistical methods must then be used to distinguish variations that reflect errors in sequencing from mutations actually present in the mixture. Importantly, any application of next-generation sequencing technologies to samples and evidentiary material from this case would have required that the methods be validated for use with samples and material of this type and for the intended purposes and questions at hand.
With such technologies and methods, one would likely have discovered additional polymorphisms in RMR-1029 and the letter samples. To the extent that any new polymorphisms were found to be concordant between these evidentiary materials (within statistical limits), that would strengthen the genetic association between the bacteria from the letters and those from the suspect flask. It is also conceivable, however, that such additional analyses might have revealed further mutations in the samples from the attack letters that were not present in RMR-1029, thereby weakening the link between the evidentiary material and RMR-1029. Thus, while it was not feasible at the start of the investigation, investigators should have subsequently examined the value from a forensic perspective of “deep sequencing” of key samples. Given the limitations of the existing repository, it is uncertain whether this further scientific investigation would have identified an altogether different source of the B. anthracis attack spores, but it could have provided additional information on the process used to generate the material in the attack letters.
More generally and looking forward, the committee anticipates that deep-sequencing methods (including, as appropriate, metagenomic analyses of environmental samples of diverse microbial communities) will be an important forensic tool in future investigations of any similar event involving microbial pathogens.
It should be noted that future biological attacks may pose even greater challenges than did this attack. For example, the biological agent may belong to a species with a more complex and less well understood population structure, it may be genetically modified in a manner that obscures its origin, or a direct sample of the attack material may not be available. This last possibility means that environmental and clinical samples, with their additional challenges, may have greater importance in a future investigation.
Finding 6.9: The FBI faced a difficult challenge in assembling and annotating the repository of B. anthracis Ames samples collected for genetic analysis.
Much of the challenge in assembling and annotating the repository was inherent to the types of materials involved, which included stocks that had been
derived, sampled (or combined), and then stored for different periods in several ways and for diverse scientific purposes in many laboratories. The FBI collected substantial information, or metadata, about many or all of these samples including details of historical derivation, mode of sampling and storage, and so forth. As was appropriate, the genetic screening of the samples in the collection was performed in a “blind” fashion so that this information would not influence the test results. If a future bioterrorism event requires the establishment of a sample repository, attention should be paid to the structure of the database and the inclusion of (or ways to link to) any and all relevant metadata.
Finding 6.10: The evidentiary material from this case is, and will be, immensely valuable, especially in the event of future work on either this case or other cases involving biological terrorism or warfare. It is critically important to continue to preserve all remaining evidentiary material and samples collected during the course of this (the anthrax letters investigation) and future investigations, including the overseas environmental samples, for possible additional studies.
Recent and future advances in scientific methods and insight may provide the means to extract additional valuable information from case-associated material and samples. In addition, in the event of a future biological attack, these materials and samples may prove useful for comparative analyses. Therefore, despite the closure of this case, all remaining case-associated materials and samples should be retained and preserved for possible further studies.