National Academies Press: OpenBook
« Previous: ACKNOWLEDGMENTS
Suggested Citation:"BACKGROUND AND INTRODUCTION." National Research Council. 2010. Continuing Assistance to the National Institutes of Health on Preparation of Additional Risk Assessments for the Boston University NEIDL, Phase 2. Washington, DC: The National Academies Press. doi: 10.17226/13054.
×

BACKGROUND AND INTRODUCTION

In 2003, the Boston University Medical Center (BUMC) was awarded a $128 million grant from the National Institutes of Health (NIH) to build one of two national high- and maximum-containment laboratory facilities for pathogen research. The National Emerging Infectious Diseases Laboratories (NEIDL) are meant to support the National Institute of Allergy and Infectious Diseases’ biodefense research agenda, conducting research to develop new approaches to treating, preventing, and diagnosing a variety of bacterial and viral diseases. Diseases and pathogens to be studied include viruses (e.g., Ebola, Marburg, dengue fever, Lassa fever, and highly pathogenic influenza) and bacteria (e.g., Shigella and plague) that occur naturally and cause infections or that could be used in deliberate attacks. The facility includes a biosafety level 4 (BSL-4) containment laboratory housed in a 192,000 square foot building. Although the NEIDL BSL-4 laboratory accounts for only 13 percent of the building’s total space, it has been the source of virtually all of the community concern surrounding this project. The location of the facility on Albany Street in Boston’s South End, which is an environmental justice community, (Boston Region Metropolitan Planning Organization, Journey to 2030; Loh, et al., 2002) has been controversial, and there have been numerous public meetings over the plans for the facility as well as three legal actions that challenge the project. Construction of the laboratory building is now finished although commissioning of the laboratory facilities has not been completed. A remaining issue is whether the BSL-4 component will become operational.

The building, including the BSL-4 laboratory, is part of the BioSquare Phase II project. Under the Massachusetts Environmental Policy Act (MEPA), the Secretary of the Commonwealth of Massachusetts’s Executive Office of Environmental Affairs issued a certificate stating that the BioSquare II project required the preparation of an Environmental Impact Report (EIR). Although the Massachusetts Secretary of Environmental Affairs in 2004 found that the final Environmental Impact Report adequately and properly complied with MEPA, this determination was challenged in court. In July 2006 the Superior Court of Massachusetts vacated Massachusetts’ certification of the EIR and remanded the matter to the Secretary of Environmental Affairs.

NIH prepared a document, “Draft Supplementary Risk Assessment and Site Suitability Analyses” (DSRASSA), regarding the siting and operation of the NEIDL in response to comments from the federal court presiding over another lawsuit under the National Environmental Policy Act (NEPA) and to supplement NIH’s previous assessments of the potential risks posed by the NEIDL at its current location in Boston.

At the request of the State of Massachusetts, in November 2007 the NRC committee authoring the current report released the first in a series of letter reports assessing the DRASSA.4 The committee’s assessment was critical of the DSRASSA, finding that it was not sound and credible, did not adequately identify and thoroughly develop worst-case scenarios, and did not contain the appropriate level of information to compare the risks associated with alternative locations. The report also raised specific concerns about agent selection, scenario development, modeling methodology, environmental justice issues, and risk communication.

In March 2008, NIH established its Blue Ribbon Panel (BRP) to provide scientific and technical advice to the NIH Director through recommendations made to the Advisory Committee

4

NRC. Technical Input on the National Institutes of Health’s Draft Supplemental Risk Assessments and Site Suitability Analyses for the National Emerging Infectious Diseases Laboratory, Boston University: A Letter Report (2007). Available at: http://www.nap.edu/catalog/12073.html.

Suggested Citation:"BACKGROUND AND INTRODUCTION." National Research Council. 2010. Continuing Assistance to the National Institutes of Health on Preparation of Additional Risk Assessments for the Boston University NEIDL, Phase 2. Washington, DC: The National Academies Press. doi: 10.17226/13054.
×

to the Director. The panel members were charged with providing ongoing, expert input to guide the development of any necessary additional risk assessment analyses. Also in 2008, the same NRC committee reconvened at the request of NIH. The NRC committee has been meeting with the BRP periodically as milestones were reached in the preparation of additional risk assessment materials. The NRC released its second letter report in April 2008.5 The committee restricted its comments in that report to suggestions based only on its previous review of the DSRASSA and improving the risk assessments presented therein as input to any additional studies that may be needed to assess risk associated with the siting and operation of the NEIDL. As noted in its 2007 report, the committee acknowledged and emphasized the need for biocontainment laboratories, including BSL-4 laboratories. However, the committee’s view remained that the selection of sites for high-containment laboratories should be supported by detailed analyses and transparent communication of the available scientific information regarding possible risks.

In its 2008 report, the committee refrained from prescribing specific methods and other details, electing instead to structure its suggestions to the NIH BRP around the following overarching questions that should be addressed in future reports about the risks associated with operating the NEIDL:

  • What Could Go Wrong?

    • Release scenarios for infectious agents

    • Agents to consider for risk assessment

  • What are the probabilities that these scenarios will occur?

  • What would be the consequences if they did occur?

The committee also recommended that NIH make greater use of the accumulated wisdom in the published literature on how to achieve effective risk communication.

In 2009 NIH asked the NRC to convene the committee again to provide input at key milestones in the development of the supplementary risk assessment through a series of letter reports (see full Statement of Task, below). The first milestone for which input from the NRC was requested was the development of plans for the supplemental risk assessment. On March 19, 2010 at a joint meeting of the NIH BRP and the NRC committee, the two contractor groups selected by NIH to complete the supplemental risk assessment—Tetra Tech and its subcontractors from the University of Utah—made presentations on the proposed plans for the supplemental risk assessment. At NIH’s request, the NRC committee focused its discussions of the proposed approaches on the following questions:

  1. Is the range of agents being studied appropriate?

  2. Is the approach to event sequence analysis appropriate?

    • Will the method result in an adequate range of scenarios being considered and selected for analysis?

    • Are the plans for analysis and expression of results appropriate?

  1. Is the modeling approach appropriate?

    • Is the approach to initial infection sound?

    • Are the criteria for and selection of models sound?

    • Are the uses of the hybrid branching-compartment models and the extreme values analysis sound?

5

NRC. Technical Input on Any Additional Studies to Assess Risk Associated with Operation of the National Emerging Infectious Diseases Laboratory, Boston University: A Letter Report (2008). Available at: http://www.nap.edu/catalog/12208.html.

Suggested Citation:"BACKGROUND AND INTRODUCTION." National Research Council. 2010. Continuing Assistance to the National Institutes of Health on Preparation of Additional Risk Assessments for the Boston University NEIDL, Phase 2. Washington, DC: The National Academies Press. doi: 10.17226/13054.
×

On the basis of this meeting, in April 2010 the NRC committee delivered its third letter report.6 In that report, the committee noted that it had heard about plans, but not yet results. In general, the NRC committee found the proposed approaches to conducting the risk assessment suitable and well planned. The agents selected for analysis were appropriate and comprehensive, and the expertise available on and to the assessment team seemed strong. NIH and Tetra Tech appeared to recognize data limitations and the need for flexibility in study design. The committee encouraged NIH and Tetra Tech to develop qualitative analyses (an explanation of the safety and risk profile) of all 13 pathogens on the list in a manner that is clear and accessible to the public. The committee also suggested that the qualitative analyses in the body of the assessment be supplemented with results of quantitative modeling planned for five pathogens, with details provided in appendices. Further, the committee encouraged NIH and Tetra Tech to rely on data that are available from existing case studies, public health surveillance of the surrounding communities, and release incidents, not only to support its models but also to provide a complete and understandable picture for the public. The NRC committee again emphasized that the final risk assessment be able to serve as an effective risk communication tool.

Statement of Task for This Letter Report

This report is also based on a meeting between the BRP and the reconvened NRC committee. As with the committee’s April 2010 report, the statement of task for this letter report is as follows:

The NIH will engage the Committee on Technical Input on the NIH’s DSRASSA for the Boston University NEIDL at key milestones during the development of a draft supplementary risk assessment. The NRC and the NIH Blue Ribbon Panel (BRP) will meet together in public to discuss the developing draft report. Information contained in the draft risk assessment may include data on agents, models, and scenarios; preliminary modeling results; and quantitative and qualitative assessments. Documents reviewed and discussed at these meetings will be made available to the public. Following each meeting with the BRP, the NRC Committee in closed session will prepare brief letter reports on the preliminary results of the supplementary risk analyses, focusing on whether the analyses are scientifically and technically sound in general and whether they address the public health concerns previously raised by the NRC in its review of the July 2007 DSRASSA. These letter reports will be made available to the public. The committee will also provide written comments on the draft supplementary risk assessment when that document is made available for formal public comment. The Committee will submit its findings in the form of a final letter report that will also be made available to the public.

The NRC committee and the BRP met in person September 22, 2010 to discuss the developing draft risk assessment. Tetra Tech and its subcontractors from the University of Utah again made presentations, this time addressing some early results of quantitative modeling for the supplemental risk assessment. Presentations were made on:

6

NRC. Continuing Assistance to the National Institutes of Health on Preparation of Additional Risk Assessments for the Boston University NEIDL, Phase 1: A Letter Report (2010). Available at: http://www.nap.edu/openbook.php?record_id=12902&page=2.

Suggested Citation:"BACKGROUND AND INTRODUCTION." National Research Council. 2010. Continuing Assistance to the National Institutes of Health on Preparation of Additional Risk Assessments for the Boston University NEIDL, Phase 2. Washington, DC: The National Academies Press. doi: 10.17226/13054.
×
  • Release event selection and analysis

  • Probability of initial infections

  • Modeling secondary transmission, and

  • Approach to risk characterization

At NIH’s request, the NRC committee focused its analysis on the data presented as examples of the technical approaches being used that have produced some preliminary, but still incomplete, results.

COMMITTEE RESPONSE AND FINDINGS

The committee reviewed the material presented by the NIH contractors on September 22 and concluded that at this point in time it cannot endorse the illustrative analyses presented as scientifically and technically sound or likely to lead to a thorough assessment of the public health concerns previously raised by the NRC. The committee understands that the analytical results discussed were incomplete and work on additional analyses was still ongoing. For this reason, the committee had to base its review on the presentations, without documentation of the scientific rationale for the analyses.

Based on the limited information provided, the committee has a few overarching concerns. First, it appears that the contractor has not yet been responsive to the committee’s recommendation that qualitative analyses addressing the three questions raised in its 2008 letter report be prepared first and that these qualitative analyses then be supplemented by quantitative analysis through modeling using available data on the agents in question. The results of modeling are only as good as the quality of the modeling inputs, and the problem of limited data should be addressed in narrative, with supporting scientific rationale for its interpretation, as part of a comprehensive qualitative analysis for the 13 pathogens. Instead, NIH and its contractors used a modified Delphi process to gather expert opinions that were then used as a substitute for data for modeling. Circumventing the absence of data with a Delphi process is a tactical error. Parameter values acquired in this way may be misleading without validation (Kaplan, 1992). Had the NRC’s previous recommendations been heeded, it should have been clear that the parameter values acquired in this way were unnecessary. When data are not available in the literature, the contractors should turn to relevant case studies and argue by analogy.

In addition, it is important that modeling be used in a context that reflects scientific knowledge and experience. For example, the University of Utah analysts presented an extensive modeling analysis for plume dispersal of aerosolized Rift Valley Fever virus (RVFv). But because RVFv is transmitted primarily by a vector, it is not an appropriate candidate for aerosol dispersal modeling. The committee reiterates the need to include actual data based on published results in the models where possible, for example, for modeling the speed of secondary transmission of SARS. Again, the models must be transparent and couched in the context of the risk assessment and address appropriate uncertainties.

While it is possible that such analyses will be provided in the final assessment, the committee strongly believes that a mid-course correction by Tetra Tech and its subcontractors will be necessary to reach that point. In short, while the committee commends Tetra Tech and its subcontractors for carrying out some extensive illustrative quantitative risk calculations, much work needs to be done before risks are adequately assessed.

Suggested Citation:"BACKGROUND AND INTRODUCTION." National Research Council. 2010. Continuing Assistance to the National Institutes of Health on Preparation of Additional Risk Assessments for the Boston University NEIDL, Phase 2. Washington, DC: The National Academies Press. doi: 10.17226/13054.
×

In summary, the results presented on September 22 are insufficient for the committee to find that the analyses presented thus far will lead to a scientifically and technically sound risk assessment. The committee had endorsed the approaches presented at the last meeting in its third letter report, but noted that it had not yet seen results. The illustrative results presented to date are not yet sufficiently documented and supported to convince the committee that the contractors are on track to completing a comprehensive assessment of risk for the NEIDL facility.

The committee offers the following more specific comments on: the process used to generate dose-response models and the dose metrics used; the uncertainty analyses used in the modeling; other issues concerning modeling; the need for case studies based on actual data; and the method used for identification of vulnerable and susceptible populations.

Use of a Delphi Process to Generate Dose-Response Relationships

The committee is very concerned about the method by which dose-response assessment—a critical element of risk assessment for prediction of human infection, morbidity, and mortality—is being handled in the contractor analyses. The committee was informed that NIH elected to use a “modified Delphi method” to generate dose response estimates due to the absence of human data for predicting infections. This process involved soliciting opinions on human infective doses (HIDs) from an expert panel of biodefense specialists and laboratory researchers via questionnaires. Opinions were sought on values for HID10, HID50, and HID90,or the levels of inhalation exposure at which 10 percent, 50 percent, and 90 percent of an exposed human population might become infected with aerosolized pathogenic agents, for 13 pathogens. Although the report on the Delphi process was not presented to the committee at the September 22 meeting with the BRP, the committee subsequently asked for and was given a copy of the draft process report. Although NIH did not ask the committee to comment on the Delphi process report, the committee’s review of this report raised additional concerns, which are presented in an appendix (Attachment C). Only the major concerns are summarized here.

The scientific basis for the opinions of the experts involved in the process is not clearly explained in the report, especially regarding scaling animal data to humans, translating routes and endpoints, addressing low-dose issues, and other uncertainties. There is no detailed documentation of how the experts arrived at their individual opinions and judgments or of how interaction among them might have modified their opinions and judgments. Rather, the appendices to the Delphi report provide only numerical scores from “voting.” The scientific bases for the estimates derived by the experts, including citations to the published literature, case studies of human outbreaks, and knowledge of laboratory-acquired infections (LAIs), should have been integrated into the project team’s reports and presentations. The committee does not find convincing the claim that the experts’ conclusions regarding human infectivity from pathogen particles (models fitted to three median point estimates from 8 experts for 13 pathogens) “tracked the literature,” because the tabulated studies represent an incomplete accounting of the available literature (see Attachment C). To the committee’s knowledge, there has been no outside peer review of the results of the Delphi exercise.

The committee is not in a position to conduct a peer review of the risk assessment results presented because documentation was not provided. However, given the limited documentation provided, the committee is unable to endorse the use of its results in the risk assessment. The analysis presented does not appear to reflect sound scientific judgment or robust risk assessment practices.

Suggested Citation:"BACKGROUND AND INTRODUCTION." National Research Council. 2010. Continuing Assistance to the National Institutes of Health on Preparation of Additional Risk Assessments for the Boston University NEIDL, Phase 2. Washington, DC: The National Academies Press. doi: 10.17226/13054.
×

Uncertainty and Sensitivity Analyses

Good practice in risk assessment requires transparency and the development of a sensitivity analysis that addresses the effects of variances in the model inputs on results. Methods for deriving sensitivity analyses, typically via variations on Monte Carlo simulations, also usually provide a range of results rather than point estimates. Ranges convey the potential variability of the results better than single point estimates do. Good practice also includes the use of qualitative uncertainty analysis (NRC, 1994; Morgan and Henrion,1990). This is a frank discussion of the variables that are the least well understood and thus contribute most to the overall scientific uncertainty of the results. Inputs that may be highly variable but based on reliable data with little scientific uncertainty, such as human inhalation volume, contribute largely to the sensitivity analysis. Other input variables, such as pathogen dose-response, may be highly uncertain due to a lack of scientific data as well as variability within and between hosts. An input also may be highly uncertain and have low variability. Input variables of the latter type would have little impact in the sensitivity analyses, but might drive the total uncertainty of the results. The Tetra Tech team discussed the use of Latin Hypercube Sampling (LHS) as their basis for uncertainty analysis, but it was not clear whether these LHS analyses really addressed sensitivity, rather than uncertainty, analysis. The Tetra Tech team should exercise greater care in presenting these aspects of the data used for modeling and assure that both uncertainty and sensitivity analyses are adequately developed.

In addition, the committee is not persuaded that the “uncertainty analysis” provided has much useful content in its current form. As one example already mentioned above, it is inadequate to consider that the uncertainty in the dose-response relationship could be represented as a question of which of the eight Delphi process experts is correct, when another possibility is that none of the curves developed from the three points elicited from each expert provide the true human dose-response curve. Nor is it adequate to state that the results of fitting models to three median point estimates from 8 experts for 13 pathogens “tracked the literature” when the referenced studies represent an incomplete understanding of the available literature.

Incompleteness of the Analyses and Lack of Documentation for Assumptions

The Tetra Tech analyses presented to the committee contain undocumented or poorly documented assumptions. For example, the aerosol release from a centrifuge accident was described as a “10 ml leak from container into rotor, but only a small fraction is aerosolized.” How was the judgment made on how much of the 10ml was aerosolized? Is this fraction an important uncertainty? What was the quantity of pathogen contained in the release? No bounding calculation is given, such as the result if all 10 ml are aerosolized. This uncertainty was apparently not a component of the uncertainty analysis described, which is restricted to only a few parameters. While there are many acceptable ways to conduct a valid risk assessment and the choices made by the contractor team may be defensible, provision of the details of the many unspecified assumptions for the calculations behind the numerical results and risk matrices provided on September 22 would greatly improve the committee’s understanding of the strengths and limitations of the work conducted.

Risk assessment should provide insight on what events and processes give rise to risk, and then allow the acceptability of the risk—and the potential risk reduction from improved equipment and procedures—to be evaluated (NRC, 1996). Given the lack of cohesive knowledge of the dose-response relationships for the various pathogens in humans and animals, and the two disease transmission examples (RVF and SARS CoV), the presentations may have merit as

Suggested Citation:"BACKGROUND AND INTRODUCTION." National Research Council. 2010. Continuing Assistance to the National Institutes of Health on Preparation of Additional Risk Assessments for the Boston University NEIDL, Phase 2. Washington, DC: The National Academies Press. doi: 10.17226/13054.
×

illustrating the types of calculations needed for a risk assessment. But the committee is not persuaded that the project team has yet made progress in exploring and documenting the important issues for the NEIDL.

Consideration of the available case studies (such as the SARS case described below) suggests the possibility that transfer of a pathogen outside the laboratory by an infected worker is an important class of risk events. Such transfers can lead to diverse outcomes, for example, no secondary transfer in a recent case of tularemia in Maryland, but transfers producing secondary infections from laboratories studying more highly contagious pathogens. Scenarios for infection outside leading to secondary transmissions should be considered in addition to the centrifuge example, particularly where there are documented case studies of LAIs. As noted above, the degree of documentation of the details—such as for the centrifuge accident and the dose-response relationships for pathogens—is too sparse to be persuasive.

Modeling

The committee continues to believe that the use of branching process models and compartmental models is appropriate, rational and straightforward. The committee was pleased with the progress made with the two branching process models described in the presentation. However, based on what was presented, the committee has serious concerns about the modeling context and, in at least one instance, with the manner in which the models were implemented.

First, as noted above, the committee was disappointed with the extent to which qualitative and quantitative approaches were integrated in the September 22 presentations. To be clear, the committee believes that presenting the modeling results without also describing the natural history of a pathogen, the characteristics of the disease outbreaks with which the pathogen has been associated, and experience with the pathogen in the context of known releases from laboratories diminishes the credibility of these results, which cannot be expected to stand alone without an appropriate context and explication. It would boost confidence in the model if the insights it provides generally match the expectations engendered by field experience. If the insights appear to contradict the expectations of experienced researchers, then the specific circumstances that make such results plausible must be explained. For example, the results of the SARS modeling presented to the committee appear to be counterintuitive, yet no credible explanation was provided of why the risks of secondary infection transmission should be the same for urban, suburban, and rural sites with their significant differences in population density.

On the contrary, the committee notes that this counterintuitive outcome may have much to do with the fact that internal restrictions in the MACCS2 model do not allow modeling of a pathogen release within 100 meters of the building release site. The Tetra Tech team’s results “showed” that the concentrations of aerosol at the rural and suburban sites were “two- to four-times higher” than at the urban site due to the increased turbulent mixing of the released puff in the rougher urban topography. Modeling a plume within the first 100 meters of release in an urban zone is admittedly difficult. However, if the puff at exit from the building is assumed to be the same concentration for all locations, the higher population density in the urban location could be associated with higher risk than the suburban and rural sites, which have lower population densities. Thus, it may not be true, as Tetra Tech concluded, that risks are higher in the suburban and rural locations. This 100-meter zone gap in the plume modeling must be more satisfactorily addressed and cannot be ignored as it appeared to have been in the presentations. Perhaps upper bound risk conditions, such as low wind speed, atmospheric stability, and an early morning

Suggested Citation:"BACKGROUND AND INTRODUCTION." National Research Council. 2010. Continuing Assistance to the National Institutes of Health on Preparation of Additional Risk Assessments for the Boston University NEIDL, Phase 2. Washington, DC: The National Academies Press. doi: 10.17226/13054.
×

inversion where mixing complexities may be minimal and urban canyon channeling a major factor, could be considered.

In recent years, a great deal of research in modeling the near field, less than 100 meter urban zone, has been completed due to societal and government concerns over the potential impacts of release of chemical or biological agents by malevolent or other action. Alternatives to MACCS2 exist that Tetra Tech could consider. Granted, parameters for and methods of handling building downwash, building upwash, urban canyon channeling, building wake turbulence and other factors of the urban topography remain difficult to handle with certainty. These can be an issue in a suburban and rural near field as well. (Some publications that further discuss the problems and approaches to modeling include Belcher, 2005; Pullen, et al., 2005; Olvera and Choudhuri, 2006; Burrows, et al., 2007; Neofytou, et al., 2008; Singh, et al., 2008; and Brixey, et al., 2009.

The committee also has a specific concern with the implementation of the SARS modeling related to the way in which mitigation strategies were represented by an “instantaneous” reduction in the value of the reproduction number. This is not supportable. There are several sources that document gradual decreases in the reproduction number for a range of diseases. In almost all cases, the decline occurs over a period of weeks, not instantaneously. In the case of SARS, the decline in the estimated reproduction number can take between 5 and 25 days even in a hospital environment with an already recognized problem (Cooper. et al., 2009).

Finally, the committee notes that the intensive effort put into plume modeling and the earthquake scenario (stated as the worst case) places a great overemphasis on risk pathways that are not particularly relevant to Rift Valley Fever (primarily transmitted by mosquitoes) and SARS (highly transmissible person-to-person and a major concern for an infected laboratory worker).

Vulnerable Populations

The September 2010 presentations indicated that the Tetra Tech team had identified the vulnerable or sensitive groups as “those 5 years of age or younger, those 65 years of age or older, those with diabetes mellitus, those with HIV/AIDS, and those who are pregnant.” The report from the modified Delphi process with the expert panel listed the median percentage increases in vulnerability to disease and death among these five groups. These percentage increases were then used to calculate the increased risk of infection at each site (if any) based on the percentage of population falling into these groups. Although the committee believes that the contractors’ approach and its presentations at the September meeting contributed to a meaningful discussion of the issues surrounding vulnerability and sensitive subpopulations, to meet the risk assessment goals that this committee set out in its previous reports, the contractors should recast their vulnerability analysis and shift direction, as explained below.

Refining, Re-evaluating and Making Key Assumptions Transparent

The committee recommends that the contractors consider re-evaluating and refining several of their key assumptions regarding vulnerable and sensitive subpopulations. First, it is not clear how these categories of vulnerable groups were determined, but the committee gathers that they were presented to the expert group in the Delphi exercise, not developed as a result of that exercise. Appendix A, Part C of the Delphi process report states that “the groups have been dictated in part by the level of data available for the sites being evaluated.” Because this report was made available after the September 22 presentation, the committee did not have the

Suggested Citation:"BACKGROUND AND INTRODUCTION." National Research Council. 2010. Continuing Assistance to the National Institutes of Health on Preparation of Additional Risk Assessments for the Boston University NEIDL, Phase 2. Washington, DC: The National Academies Press. doi: 10.17226/13054.
×

opportunity to discuss this statement, or the vulnerability group criteria, with the Tetra Tech team. Based solely on its reading of this language, it appears that the published literature and available data were the key criteria for selection of vulnerable groups. As discussed below, the committee presents a different approach for consideration. Second, the contractors used these vulnerability categories only to estimate additional infection rates, though the experts addressed percentage increases in vulnerability to disease and death. The committee believes that the contractors should address both increased risk for mortality and more serious health outcomes (severe morbidity). This issue was not addressed in the presentations even though the Delphi report specifically asked experts to assign probabilities for both increased disease (morbidity) and mortality. Third, the modeling carried out by the Tetra Tech team assumed that no member of groups at risk for primary infection was a member of a vulnerable subpopulation. This assumption does not seem realistic. For modeling purposes, it would seem more prudent to assume that some proportion of individuals at risk for primary infection could also be a member of a vulnerable group (e.g., diabetics).

Refining and Re-evaluating the Concept of Vulnerable and Sensitive Subpopulations

As explained previously, the committee cannot comment in detail about the methodology used to determine the vulnerable categories used by the Tetra Tech team. If this methodology were made transparent, the committee would be in a better position to offer a critique and suggestions for next steps. Nevertheless, based on the September presentations, the committee believes that the Tetra Tech team could make some improvements in determining these categories. While it might be possible to include previously gathered data into this new evaluation, a different methodology will better serve the risk assessment and the decision makers who will use it.

EPA ’s National Environmental Justice Advisory Committee (NEJAC) set out a conceptual model that the committee believes is useful and should be considered by the Tetra Tech team (NEJAC, 2004). The model defines several key concepts, such as stressors, and conceives vulnerability broadly. More particularly, EPA ’s NEJAC defines four important characteristics of vulnerable populations:

  1. More susceptible or sensitive to disease outcomes

  2. Differentially exposed to environmental conditions that could render these populations more vulnerable

  3. Differentially prepared to address deleterious conditions, such as exposures to infectious diseases, and

  4. Differential responses to the same level of infection or exposure (as non-vulnerable populations) that may worsen response.

The South Boston and Roxbury neighborhoods that are potentially affected by the NEIDL facility have been identified as environmental justice communities and a vulnerable population analysis should take that into consideration. These communities suffer from higher rates of several chronic diseases, including asthma, which is not one of the higher risk factors presented, and have a much greater population density (see discussion below on dispersal modeling). The rural and suburban communities to which they are being compared are not environmental justice communities, to the best of the committee’s knowledge.

In recasting its vulnerability analysis, the committee urges the Tetra Tech team consider the following steps:

Suggested Citation:"BACKGROUND AND INTRODUCTION." National Research Council. 2010. Continuing Assistance to the National Institutes of Health on Preparation of Additional Risk Assessments for the Boston University NEIDL, Phase 2. Washington, DC: The National Academies Press. doi: 10.17226/13054.
×
  • Develop a robust methodology for determining categories of sensitive or vulnerable individuals. In doing so, the committee recommends that Tetra Tech adopt the EPA NEJAC conceptual framework as a starting point, and consult with community leaders, public health professionals familiar with the South Boston and Roxbury areas, and review the published public health literature and health surveillance data, if available.

  • Evaluate not only increased infectivity or morbidity, but also increased disease severity for vulnerable and sensitive subpopulations. This analysis should include endpoints such as mortality and, if possible, those that represent predicted differential morbidity outcomes.

  • Model primary infection assuming that some of the individuals at risk for such infection might also be members of vulnerable groups.

Use of Case Studies

In its April 20, 2010 letter report, the committee made a specific recommendation in the modeling subsection that “modeling should be augmented by case studies based on actual occurrences of laboratory or natural infections.” The committee believes that case studies can be used not only to provide information on how and whether LAIs may or may not be transmitted to the general population by infected workers. They may also provide ground truth examples for how potential time of exposure to a pathogen compares to time of recognition of an LAI, secondary transmission of the disease, and the effectiveness of treatment.

There are a number of well-documented accounts of recent LAIs that could be used in the development of brief case studies to illustrate more clearly the effect of infected laboratory workers on community health. Examples include, but are not limited to, infection with Brucella spp, SARS virus (see Box as an example), Francisella tularensis, and Burkholderia mallei. In addition, case studies describing naturally occurring illness can be developed for those agents potentially studied at NEIDL for which well documented or recent LAI have not been reported. A few examples include infection with Yersina pestis (tourists in New York), Monkey pox virus (contracted from exotic pets), Bacillus anthracis (contaminated drum hides), Marburg virus (a tourist who visited Uganda), and U.S. experience involving the recent H1N1 influenza virus pandemic. The committee is not advocating that these specific examples be developed as case studies, but believes that available information on a variety of the agents to be studied at the NEIDL could be used to provide context and a basis for reality to the qualitative aspect of the risk posed to the local community by an infected laboratory worker. A 2010 NRC report, Evaluation of the Health and Safety Risks of the New USAMRIID High-Containment Facilities at Fort Detrick, Maryland, provides a list of laboratory incidents that have occurred at USAMRIID’s laboratories that might provide useful examples from which to develop case studies. In the box below, the committee provides an example to illustrate what it means.

Sample Case Study: SARS/CoV

In China, SARS/CoV was grown in a BSL-3 laboratory by a worker who apparently had worn inappropriate personal protective equipment (PPE) and then treated the sample to inactivate the virus before removing it to a BSL-1 laboratory for further work on the open bench. The worker failed to verify the complete inactivation of the virus and subsequently became ill and was admitted to a fever hospital. The laboratory was not notified of this development and the worker later returned to the laboratory. A second worker who handled the “inactivated” sample also became ill. A graduate student who observed the laboratory procedure later traveled by train to

Suggested Citation:"BACKGROUND AND INTRODUCTION." National Research Council. 2010. Continuing Assistance to the National Institutes of Health on Preparation of Additional Risk Assessments for the Boston University NEIDL, Phase 2. Washington, DC: The National Academies Press. doi: 10.17226/13054.
×

her home several hundred miles away. After returning to the laboratory she became ill and once again traveled to her home by train where her mother, a physician, admitted her to a hospital and treated her. The student was asked if she worked with SARS/CoV (she said no because her research involved another virus). It was not until the mother became ill and died that SARS/CoV was identified. Other laboratory workers also became ill and other hospital personnel died. This case study illustrates several important points: people make mistakes (improper PPE); not everyone follows procedures (failure to test sample for inactivity); people may die if not properly diagnosed and treated.

Metrics

The committee believes that Tetra Tech should carefully consider what might be the appropriate metric(s) when evaluating the transmission of pathogens in heterogeneous human populations. The metric presented to the committee was the probability that a release would not lead to secondary transmission (probability of > 0 transmissions). This probability arises naturally and easily out of multiple stochastic simulations—and may be of interest to a policymaker concerned with the health and well being of the population as a whole—but it will be of no interest to those groups at particular risk of infection. Such at-risk groups will want to know what might happen to them should an introduction in fact lead to a secondary transmission.

SUMMARY

NIH and its contractors have not yet been responsive to the committee’s recommendation for tiered qualitative and quantitative analyses. The committee strongly recommended that qualitative analyses address the three questions raised in its 2008 letter report —What could go wrong? What are the probabilities? What are the consequences?—be prepared first for all 13 agents of concern using available data and case studies to scope or bound the problem broadly. The committee intended that this first tier of qualitative analyses would then be supplemented by second tier quantitative analysis for a subset of agents as necessary for decision making.

However, instead of beginning with a first tier qualitative analysis of both direct and indirect scientific evidence to bound the analysis for the 13 pathogens, a modified Delphi process was used to gather expert opinions on multiple unknown parameters. The results were then used as a substitute for data for all 13 agents. The median opinions for three infectious doses were fitted to empirical models intended to predict human infectivity for exposures generated by plume/puff models that also relied on opinions for influential parameters. Circumventing the absence of data with a Delphi process and then conducting modeling based on opinions was a tactical error. The committee considers the approach flawed in large measure because reliance on opinions for quantitative second tier modeling was unnecessary. In addition, the approach is problematic because the elicited opinions about possible parameter values, and the models fitted to them, may be incorrect and misleading without validation.

To expedite completion of a robust risk assessment, the committee strongly urges a mid-course correction to the use of tiered qualitative and quantitative analyses. The first tier should use narrative descriptions based on case studies and actual data reporting, with supporting scientific rationales provided for interpretation. Then, a more targeted second tier quantitative analysis should be developed where the existence of quantitative data allows. The quantitative analyses should reflect what is known from case studies and real world experience about transmission modes and other critical parameters.

Suggested Citation:"BACKGROUND AND INTRODUCTION." National Research Council. 2010. Continuing Assistance to the National Institutes of Health on Preparation of Additional Risk Assessments for the Boston University NEIDL, Phase 2. Washington, DC: The National Academies Press. doi: 10.17226/13054.
×
Page 5
Suggested Citation:"BACKGROUND AND INTRODUCTION." National Research Council. 2010. Continuing Assistance to the National Institutes of Health on Preparation of Additional Risk Assessments for the Boston University NEIDL, Phase 2. Washington, DC: The National Academies Press. doi: 10.17226/13054.
×
Page 6
Suggested Citation:"BACKGROUND AND INTRODUCTION." National Research Council. 2010. Continuing Assistance to the National Institutes of Health on Preparation of Additional Risk Assessments for the Boston University NEIDL, Phase 2. Washington, DC: The National Academies Press. doi: 10.17226/13054.
×
Page 7
Suggested Citation:"BACKGROUND AND INTRODUCTION." National Research Council. 2010. Continuing Assistance to the National Institutes of Health on Preparation of Additional Risk Assessments for the Boston University NEIDL, Phase 2. Washington, DC: The National Academies Press. doi: 10.17226/13054.
×
Page 8
Suggested Citation:"BACKGROUND AND INTRODUCTION." National Research Council. 2010. Continuing Assistance to the National Institutes of Health on Preparation of Additional Risk Assessments for the Boston University NEIDL, Phase 2. Washington, DC: The National Academies Press. doi: 10.17226/13054.
×
Page 9
Suggested Citation:"BACKGROUND AND INTRODUCTION." National Research Council. 2010. Continuing Assistance to the National Institutes of Health on Preparation of Additional Risk Assessments for the Boston University NEIDL, Phase 2. Washington, DC: The National Academies Press. doi: 10.17226/13054.
×
Page 10
Suggested Citation:"BACKGROUND AND INTRODUCTION." National Research Council. 2010. Continuing Assistance to the National Institutes of Health on Preparation of Additional Risk Assessments for the Boston University NEIDL, Phase 2. Washington, DC: The National Academies Press. doi: 10.17226/13054.
×
Page 11
Suggested Citation:"BACKGROUND AND INTRODUCTION." National Research Council. 2010. Continuing Assistance to the National Institutes of Health on Preparation of Additional Risk Assessments for the Boston University NEIDL, Phase 2. Washington, DC: The National Academies Press. doi: 10.17226/13054.
×
Page 12
Suggested Citation:"BACKGROUND AND INTRODUCTION." National Research Council. 2010. Continuing Assistance to the National Institutes of Health on Preparation of Additional Risk Assessments for the Boston University NEIDL, Phase 2. Washington, DC: The National Academies Press. doi: 10.17226/13054.
×
Page 13
Suggested Citation:"BACKGROUND AND INTRODUCTION." National Research Council. 2010. Continuing Assistance to the National Institutes of Health on Preparation of Additional Risk Assessments for the Boston University NEIDL, Phase 2. Washington, DC: The National Academies Press. doi: 10.17226/13054.
×
Page 14
Suggested Citation:"BACKGROUND AND INTRODUCTION." National Research Council. 2010. Continuing Assistance to the National Institutes of Health on Preparation of Additional Risk Assessments for the Boston University NEIDL, Phase 2. Washington, DC: The National Academies Press. doi: 10.17226/13054.
×
Page 15
Next: REFERENCES »
Continuing Assistance to the National Institutes of Health on Preparation of Additional Risk Assessments for the Boston University NEIDL, Phase 2 Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The National Research Council (NRC) reconvened its Committee on Technical Input on Any Additional Studies to Assess Risk Associated with Operation of the National Emerging Infectious Diseases Laboratory (NEIDL), Boston University to provide further technical input on the scope and design of any additional studies that may be needed to assess the risks associated with the siting and operation of the NEIDL.

To this end, the NRC committee met in open session with the NIH Blue Ribbon Panel on September 22, 2010 to hear presentations by NIH's contractors on the approaches they are taking to conduct the risk assessment. Following the open meeting, the NRC committee met in closed session to begin preparing this brief letter report, focusing on whether the analyses presented at that meeting are scientifically and technically sound in general and whether they address the concerns raised by the NRC in its first three letter reports.

The committee reviewed the material presented by the NIH contractors on September 22 and concluded that it cannot endorse as scientifically and technically sound the illustrative analyses presented. These analyses do not, so far, represent a thorough assessment of the public health concerns raised by the committee in its previous reports. The committee understands that the analytical results discussed were incomplete and that work on additional analyses is still ongoing. Therefore, the comments provided in this letter report will be helpful to the Blue Ribbon Panel as they consider how the remainder of the work is carried out.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!