Summary

Between 1942 and 1975, the U.S. Army conducted tests on human subjects to study the effects of a variety of agents, including chemical warfare agents, biological agents, medications, vaccines, and other substances. The tests investigated the immediate or short-term health effects from acute exposure to understand vulnerabilities to attack. Whether the exposures could have resulted in long-term health consequences to the test subjects has been assessed periodically, and the Army is required to notify subjects of information relating to potential health effects associated with exposure to the test agents. Most recently, a 2016 court injunction (Ninth Circuit Court of Appeals, D.C. No. 4:09-cv-00037-CW) directed the Army to provide test subjects with new information about potential long-term health effects associated with their exposures, and to provide medical care if an injury or illness could be attributed to their participation in an Army chemical or biological testing program. In support of the first requirement, the Army contracted a report, Assessment of Potential Long-Term Health Effects on Army Human Test Subjects of Relevant Biological and Chemical Agents, Drugs, Medications and Substances: Literature Review and Analysis (the Report), to determine whether new information published since 2006 should be provided to the veterans. The Report covers more than 100 test agents and compounds and provides an assessment as to which of these agents have potential long-term health effects associated with exposure. To address the second requirement of providing medical care, the Army developed an approach for assessing agent- and outcome-specific associations to assist with making determinations about causality.

At the request of the Army, the National Academies of Sciences, Engineering, and Medicine formed an ad hoc committee that was tasked with conducting an independent review of the Report. The committee was to assess whether the Report appropriately identified potential long-term health effects that could have resulted from test exposures using an adequate weight-of-evidence approach. The general approach for evaluating agent- and outcome-specific associations as outlined in an Army Memorandum was also to be reviewed. The committee was asked to prepare an interim report of its overarching findings and their supporting evidence and then prepare a final report. The interim report was provided to the Army in February 2018; this final report provides additional detail about the basis of the committee’s findings and recommendations. No new findings or recommendations have been added to this report.

REVIEW OF THE REPORT

The Report is a literature review of potential long-term health effects associated with exposures to more than 100 test agents. It focuses on studies published after June 30, 2006, the last date when test subjects were provided information about potential long-term health effects related to the testing. Brief descriptions of studies on each agent were provided and 18 agents were identified as having potential long-term health effects. The committee found that the Report does not constitute a hazard-identification assessment because it does not contain adequate support for the conclusions drawn about which test agents have evidence of long-term health effects. Best practices in literature-based reviews were not followed. Limitations of the report include failure to provide the literature search strategies, lack of critical analysis of the evidence, unclear criteria on which strength-of-evidence determinations were made, and no information on specific exposures to test subjects.

Best practices in conducting literature-based reviews include measures to ensure that assessments are performed in a credible and transparent manner. This includes specifying the literature search strategy, specifying what criteria were used to screen the results of the literature search to identify relevant studies, conducting a critical analysis of individual studies, synthesizing the evidence, and describing the weight-of-evidence approach for classifying the strength of evidence for causal determinations about agents. The only information provided about the literature searches and screening in the Report was the date range of the searches, so the committee was unable to determine whether the scope of the searches was adequately comprehensive or whether the most relevant references were selected for the analyses. The Report provided short summaries of the evidence without critical evaluation of the individual studies or any substantive synthesis of the collective evidence to explain how an agent was judged to have adequate or inadequate evidence of long-term health effects. For the agents found to have potential long-term health effects, the strength of association was classified as high, medium, or low, but the criteria used to make these classifications were inadequately explained. Some military-specific information on certain testing programs and past reviews of potential health effects from the testing was presented, but it was unclear whether or how the information was used in the Report. No information about specific exposures to test subjects was provided beyond generalizations. The committee recommends that the Army develop a streamlined, scientifically rigorous approach to categorize health hazards and, given the number of agents to be reviewed, a strategy to prioritize the evaluations. A proposed strategy is presented later in this Summary.

REVIEW OF THE ARMY’S CAUSATION METHODOLOGY

The Army developed an approach to assist a Benefits Application Panel (BAP) in making determinations about providing medical care. This approach was

outlined in a Memorandum to the committee dated February 14, 2018. The committee focused on the Causation Methodology described in the Memorandum because that section was most directly relevant to its charge of evaluating the process that will be used to characterize agent- and outcome-specific associations. The Army will use a two-step process that includes making determinations about “general causation” and “specific causation.” General causation involves establishing an association between an agent and a health effect, and specific causation involves characterizing the likelihood that an individual’s exposure to a specific test agent or a substance caused the specific health effect.

General Causation

General causation is to be determined by evaluating the evidence on a particular test agent and a specific health effect, and using a weight-of-evidence approach to draw conclusions about causality. The weight-of-evidence evaluations will follow the framework used in the Institute of Medicine (IOM) report Adverse Effects of Vaccines: Evidence and Causality. The IOM Vaccine Approach applies systematic review principles to identify and evaluate the evidence from epidemiological and mechanistic studies. The evaluations of the two lines of evidence are performed separately, and subsequently they are integrated to make determinations about a causal relationship. The evidence is classified into one of four categories of causality: (1) evidence convincingly supports a causal relationship, (2) evidence favors acceptance of a causal relationship, (3) evidence is inadequate to accept or reject a causal relationship, or (4) evidence favors rejection of a causal relationship. The committee agrees that this classification terminology may be appropriate for the assessment of potential long-term effects on the Army test subjects. However, the approach as designed is most appropriate when a large amount of human evidence-based data is available, as is the case with vaccines, which could limit its applicability for assessing the potential long-term health effects from the different types of test agents used in the Army testing programs. Therefore, the committee questions whether the IOM Vaccine Approach, beyond the categories of causality, is the most suitable for the Army’s purposes. It is likely that animal and mechanistic data will be important in hazard evaluations for many test agents and substances; therefore, other weight-of-evidence frameworks, such as the approaches used by the IOM committees on Veterans and Agent Orange and Gulf War and Health, may also be used as guides to establish causality. These IOM approaches were developed to assist the Department of Veterans Affairs with making determinations about medical care and disability related to exposures experienced by veterans during the Vietnam and Gulf Wars. The Gulf War and Health approach is particularly relevant as it has been applied to a variety of agents, including chemical warfare agents, environmental chemicals, vaccines, and infectious agents. These weight-of-evidence approaches are more flexible than the one in the IOM Vaccine Approach, which follows a formal classification scheme. The committee recommends that the Army use

these other IOM approaches to guide its modification of the IOM Vaccine Approach. Some key elements to consider include

• Establishing protocols and criteria for identifying and selecting studies that are relevant to the test agents under consideration.
• Using a weight-of-evidence approach that has flexibility to be applied to agents that have a spectrum of available data sets. For example, a formal method for classifying and integrating epidemiological and mechanistic data would be suitable for agents that have substantial human evidence whereas a more qualitative integration of human, animal, and mechanistic evidence would be more appropriate for agents with less robust human data sets.
• Providing context for how new information affects conclusions drawn in earlier reviews.
• Documenting how causal determinations were made.

Given the timeframes under which the Army is working, modifications will likely include streamlined approaches to conducting the assessments, as the IOM approaches are rigorous processes that require significant resources and time to complete.

Specific Causation

Building on the general causation analysis, four additional factors are considered by the Army to establish specific causation for each test subject. These factors include whether the test subject’s exposure was of sufficient magnitude to produce the alleged medical condition, whether the exposure was temporally related to the onset of the alleged medical condition, whether alternate causes of the medical condition can be ruled out, and whether there is coherence and consistency in the evidence. The Army has noted that information on individual test subjects, such as exposures they experienced and potential confounders, are unlikely to be available, and this will limit the ability of the BAP to draw definitive conclusions with respect to specific causation. The committee agrees that it is unlikely that the exposures during the tests can be adequately characterized or that potential confounders can be ruled out for most outcomes; consequently, some presumptions about exposure and disease may have to be made.

Compensation Decisions

The Memorandum indicates that members of the BAP will be making determinations of both general causation and specific causation. The committee recommends that the Army consider uncoupling the two processes, so that general causation is evaluated and determined by a separate group. The committee judges this to be a preferable approach to ensure that appropriate subject-matter

expertise is used to make determinations about general causation and to preclude bias in the individual adjudication decisions. This approach would be similar to how the Department of Veterans Affairs makes determinations about compensation related to claims by veterans of the Vietnam and Gulf Wars, where the causation evaluation at the population level is conducted separately from the adjudication of individual medical and claim applications.

STRATEGY FOR EVALUATING POTENTIAL LONG-TERM HEALTH EFFECTS IN ARMY TEST SUBJECTS

The committee’s proposed strategy for evaluating potential long-term health effects in Army test subjects is outlined in Figures S-1 and S-2. This strategy is based on best practices in hazard identification and systematic review, which the Army can tailor to its needs. In Step 1, various options are available for prioritizing the agent evaluations. Although the applications received by the Army will likely dictate the priorities, the Army might consider prioritizing agents and health effects on the basis of the number of individuals who were exposed and previous reviews of the literature. For example, the Report critiqued above could be used for prioritizing agents and health effects. In Step 2, problem formulation, the scope of the evidence available to evaluate an agent- and outcome-specific association is determined. If a credible assessment on the topic has already been performed by an authoritative source, the Army could rely on that assessment. If an independent assessment is necessary, the analysis methods should be planned in advance of performing the assessment, and should include developing literature search and screening strategies (Step 3) to ensure that the searches are of an appropriate scope and that the most relevant set of literature is captured. The quality of the evidence is determined in Step 4 by evaluating the different lines of evidence separately. For the animal and human evidence, procedures are well established for evaluating individual studies and then considering the studies collectively to assess factors such as consistency across study designs, biases in data collection or analyses, evidence of dose-response relationships, and the magnitude of the effect. Mechanistic evidence is often more diverse and voluminous and the practices for how to incorporate it into cancer and noncancer evaluations are still evolving. The different lines of evidence are integrated in Step 5. For this step, a weight-of-evidence approach should be used that has the flexibility to be applied to a disparate set of test agents with varying sets of data. Finally, in Step 6, a conclusion is drawn about the strength of association between a test agent and a health effect.