3

Evaluation of the Army’s Causation Methodology

The Army has developed an approach to assist a Benefits Application Panel (BAP) in making determinations about providing medical care to former test subjects. The Memorandum that describes the approach is presented in Appendix B. The purpose of the Memorandum is to determine “if an individual applicant’s medical condition(s) is the proximate result of participation in Army, or Army sponsored, chemical or biological substance testing programs between 1942 and 1975 in order to determine eligibility for DoD provided health care for the a claimed conditions(s).” The committee considered proximate cause¹ to be a legal term and not a medical or scientific term. In scientific or medical evaluations, causation is evaluated by describing the strength of the association between an exposure and a health effect. Accordingly, the review provided in this chapter focuses on the scientific and technical aspects of the methods presented in the Memorandum. The sections most relevant to the committee’s charge of reviewing the approach to evaluating “agent- and outcome-specific associations” are presented in Sections 7, 8, and 9 of the Memorandum, which cover the Review and Adjudication Process, Causation Methodology, and Documentation, respectively.

REVIEW AND ADJUDICATION PROCESS

Section 7 of the Memorandum states that the BAP will review and adjudicate applications from veterans on the basis of previous literature reviews and case-specific reviews, including consideration of extrinsic medical evidence and records from the Army and the Department of Veterans Affairs. Previous reviews cited in the Memorandum include older reviews (reports from the 1980s and 1990s by the Committee on Toxicology and the Institute of Medicine [IOM]), as well as the 2016 Report (Ho-Chunk Technical Solutions, 2016) reviewed by the committee in Chapter 2. How these reviews will be supplemented by new or case-specific information is not described. The aforementioned literature reviews from

___________________

¹ “Any health condition having a sufficiently strong causal link such that a reasonable person could find that the injury or disease was caused by testing exposure or participation in research” (see Appendix C, Section 8a of the Memorandum).

the 1980s and 1990s must be updated with new published literature relevant for the causation assessment, paying particular attention to newly identified health effects relevant to the case or to updates that could affect weight-of-evidence evaluations. For the 2016 Report, supplemental evaluations will be necessary to correct the deficiencies described in Chapter 2. See Chapter 4 for the committee’s proposed strategy for conducting the necessary assessments.

CAUSATION METHODOLOGY

Section 8 of the Memorandum presents two major concepts. One pertains to “general causation,” a process of establishing an association between an agent and a health effect, and the other to “specific causation,” a process of characterizing the likelihood that an individual’s exposure to a specific test agent caused a specific health effect.

General Causation Analysis

The question to be answered by the Army’s general causation analysis is, “Has the chemical(s) in question been shown to cause the disease(s) in question in humans?” The Memorandum states that the strength-of-evidence evaluations to answer this question will follow the framework used in the IOM report Adverse Effects of Vaccines: Evidence and Causality (IOM, 2012a). The IOM Vaccine Approach applies systematic review principles to identify and evaluate the evidence. Separate strength-of-evidence analyses of the epidemiological literature and mechanistic literature are performed and then the two lines of evidence are integrated to make determinations about a causal relationship. Under the framework, the epidemiological evidence is weighed more heavily and “mechanistic information can only support causation” (IOM, 2012a, p. 49).

Because the IOM Vaccine Approach was developed for the specific purpose of evaluating adverse effects of vaccines, the Memorandum indicates that the methods will be modified to suit the different test agents that the Army has to consider. In determining the modifications, it will be important for the Army to determine what lines of evidence should be used to characterize causal associations. For vaccines, a robust set of epidemiological studies is available that include controlled observational studies, randomized trials, uncontrolled clinical studies, surveillance studies, and case reports. Such a strong base of human evidence is unlikely to be available for many of the Army test agents, so modifications to allow for less reliance on human data will be necessary. Relevant approaches developed by the IOM that could be useful in guiding modifications are discussed later in this chapter.

Specific Causation Analysis

The question to be answered by the Army’s specific causation analysis is “whether a known or alleged chemical exposure an individual may have received was the most likely cause of the injuries or diseases of the individual.” Building on the general causation analysis, four additional factors are considered to establish specific causation in an individual person: evidence of a dose-response relationship, evidence of a temporal relationship, the ruling out of other possible etiologies for the condition, and coherence and consistency in the evidence. The Memorandum states that “the BAP will likely not have complete information on confounding variables and thus will not be able to rule out alternative etiologies.” The committee agrees that it is unlikely that the exposures during the tests can be adequately characterized or that potential confounders can be ruled out for most health effects. Nevertheless, the committee highlights the following issues regarding the available information about an individual’s exposure.

The type of exposure experienced by the test subjects should be characterized to the extent possible on the basis of what is known about the test programs they participated in. Factors to consider include how the individual may have been exposed, where the exposure occurred, the duration of exposure, the route(s) of exposure, and exposure to multiple test agents.

Testing conditions ranged from controlled exposure of individuals (e.g., conducted in aerosol chambers, intravenous injections, and topical applications) to variable exposures of a population (e.g., during open field activity, or onboard a ship). One example is Project SHAD (Shipboard Hazard and Defense), which involved studies to evaluate the effectiveness of shipboard detection and protective measures against chemical and biological warfare agents. The extent to which crew members were exposed depended on their location on the ship (above or below deck), job function (stationary or roving activity), and whether they were wearing protective equipment (NASEM, 2016a).

To characterize an individual’s exposure, a number of questions should be considered with respect to the frequency of exposure. Were single exposures of a fixed concentration used? Were multiple exposures of various concentrations used? What was the duration of the exposure(s)? Were study subjects tested in more than one scenario? Were the chemical and biological agents the same in each exposure scenario? Were test agents also evaluated for effects due to degradation to a second chemical of concern? Factors to consider related to route of exposure include how the test agent was administered and whether secondary routes of exposure were possible. For example, for aerosolized agents the primary route of exposure would be inhalation, but could the skin also have been exposed?

A final consideration is whether exposure to multiple agents or mixtures of test agents occurred. The Report described several testing programs that involved tests with different types of agents. For example, Operation Whitecoat involved tests to evaluate biological warfare agents (e.g., Coxiella burnetii,

Francisella tularensis) and vaccines (e.g., against disease like plague and yellow fever), and Project SHAD involved tests with biological agents (e.g., C. burnetti, F. tularensis, and Staphylococcus aureus enterotoxin B) and chemical agents (e.g., sarin and VX). Were test subjects exposed to a single agent or multiple agents? Did test subjects participate in more than one test within a program? Were mixtures even considered in the testing programs?

If an individual’s exposure cannot be characterized sufficiently, presumptions about exposure and disease may have to be made. This would be analogous to presumptions that are made with respect to exposures to Agent Orange during the Vietnam War. Presumption in this context means that certain health effects in Vietnam veterans will be presumed to be related to exposure to Agent Orange in the absence of exposure information (e.g., NASEM, 2016b).

DOCUMENTATION

Section 9 of the Memorandum indicates that the BAP review will be recorded in memorandum format and the votes and decision rationale documented. It is unclear from this brief description what the level of detail will be and whether it includes the analysis documents that are to be reviewed by the BAP, as well as the decision-making procedures and results. As noted in Chapter 2, it is important to document the methods and the analysis results that led to conclusions about causal associations to ensure that evaluations are conducted in a consistent and transparent manner.

REVIEW OF THE IOM RECOMMENDED APPROACHES TO CAUSALITY IN THE CONTEXT OF MAKING DETERMINATIONS ABOUT PROVIDING MEDICAL CARE TO FORMER TEST SUBJECTS

As discussed in Chapter 2, assessing the potential for a causal relationship between an exposure and a health effect requires a comprehensive and rigorous analysis of the available human, animal, and mechanistic literature for specific exposures of interest. Example frameworks for conducting these types of analyses have been developed by public health organizations such as the National Toxicology Program (NTP, 2015), the U.S. Environmental Protection Agency (e.g., EPA, 2005), and the International Agency for Research on Cancer (IARC, 2006). Similar types of frameworks have also been developed for the purpose of informing decisions related to compensation for adverse health effects associated with specific exposures, which have applicability to the Army’s need to respond to medical claims from former test subjects. These include three IOM approaches.

The IOM Vaccine Approach (IOM, 2012a) was created to address concerns about the safety of childhood vaccines. The resulting causality determinations were meant to assist in making decisions about compensating individuals that may have been injured by vaccines. The IOM has also developed approaches to assist the

Department of Veterans Affairs with making determinations about what health effects may be associated with deployment exposures experienced by veterans of the Vietnam War and the 1990-1991 Gulf War. These approaches were originally developed to determine associations between exposures to Agent Orange and other herbicides used in Vietnam and possible long-term adverse health effects (IOM Agent Orange Approach) (IOM, 1994, 1999, 2012b; NASEM, 2016b). The IOM Agent Orange Approach was slightly modified and applied to assess potential health effects from exposures to a variety of agents used during the 1990-1991 Gulf War (IOM Gulf War Approach) (IOM, 2000, 2003, 2004, 2007; NASEM, 2016c).

The Army Memorandum specifies that the IOM Vaccine Approach will be used to guide causality determinations. The approach classifies the evidence into one of four causality categories: (1) evidence convincingly supports a causal relationship, (2) evidence favors acceptance of a causal relationship, (3) evidence is inadequate to accept or reject a causal relationship, or (4) evidence favors rejection of a causal relationship. The committee agrees that this classification terminology may be appropriate for assessment of potential long-term effects of test agents on the Army test subjects. However, the approach as designed is most appropriate when a large amount of human data is available, as is the case with most vaccines. The Memorandum proposes using modifications to the IOM Vaccine Approach (IOM, 2012a) to determine general causality, but provides no description of what those modifications might be for agents with less robust human data.

The IOM Agent Orange Approach and the IOM Gulf War Approach, in particular, have been applied to a variety of agents, including chemical warfare agents, environmental chemicals, vaccines, and infectious agents (e.g., IOM, 2000, 2004, 2007). Thus, these weight-of-evidence approaches might be informative for guiding modifications to the IOM Vaccine Approach. The sections below compare elements of the IOM Agent Orange and Gulf War Approaches with the IOM Vaccine Approach.

Study Identification and Selection

The IOM Agent Orange Approach (e.g., NASEM, 2016b) and the IOM Gulf War Approach (e.g., NASEM, 2016c) follow preestablished procedures for conducting literature searches and for evaluating individual studies for relevance. Factors considered in evaluating relevance include, but are not limited to, study design characteristics (e.g., case studies as compared to prospective cohort studies, use of appropriate control groups), exposure and outcome measurements (e.g., environmental sampling, use of self-report or standardized methods), completeness of the reports (e.g., detailed description of statistical methods, adjustment for confounders such as smoking), and target populations of interest (e.g., veterans deployed versus nondeployed service members). The literature must provide sufficient methodological detail for complete evaluation of the study methods and

results. The IOM Vaccine Approach (IOM, 2012a) also followed preestablished procedures, with slightly different considerations with respect to relevance. Factors considered include study design characteristics (e.g., clinical trials as compared to observational studies), exposure and outcome measurement (e.g., flawed measurement of either vaccine administration or adverse event), and precision of the results (e.g., width of the 95% confidence intervals).

Evaluation of the Epidemiological Literature

In the IOM Agent Orange Approach (e.g., NASEM, 2016b) and the IOM Gulf War Approach (e.g., NASEM, 2016c), the epidemiological literature is thoroughly reviewed and considered in totality. The IOM reports that provide updates to previous reviews consider the new literature in context of the earlier assessments. The literature is examined and evaluated with regard to the observed direction and magnitude of the associations while considering bias (a distortion of the measure of association that results from flawed selection in the assembly of the study population or from an error in the measurement of studied characteristics), confounding (a distortion of the measure of association that can arise from factors related both to exposure and to the effect), the role of chance (the degree to which an estimated association might vary randomly among different samples of the population studied), and biological plausibility. For the approaches, exposure assessment methods are given special consideration because of the potential for bias resulting from misclassification of exposure and the need to evaluate the impact of that bias when considering the weight of evidence.

In evaluating the weight of evidence from epidemiological studies, no a priori assumption is made regarding the presence or absence of an association between exposure and any health effect in any of the IOM approaches. Furthermore, it is understood that “absence of evidence is not evidence of absence” (NASEM, 2016b, p. 4). The consistency of both the magnitude and the direction of the observed associations across all studies, while considering the methodological strengths and limitations of the individual studies, are evaluated to determine the final classification of the exposure of interest. Studies with similar strength of association and validity are given equal weight.

In the IOM Agent Orange and Gulf War Approaches, a qualitative analysis and scientific judgment is used to evaluate the evidence. In contrast, the IOM Vaccine Approach includes a method for categorizing the epidemiological literature into one of four weight-of-evidence categories (high, moderate, limited, or insufficient). Given the robust set of epidemiological studies and additional safety data available on human vaccines, this latter approach works well for classifying the available vaccine literature, particularly given that the exposures in these studies are well characterized. Such a robust set of epidemiological studies is unlikely to be available for many of the Army test agents. Furthermore, the availability of sufficient literature varies even in disciplines with an established history of study.

For example, despite the quantity and quality of the available literature used in the IOM (2012a) vaccine report, 85% of the relationships between a vaccine and an adverse event examined in the report were categorized as having inadequate evidence to be able to accept or reject a causal relationship.

Assessing Biological Plausibility

In the IOM Agent Orange Approach (e.g., NASEM, 2016b) and the IOM Gulf War Approach (e.g., NASEM, 2016c), animal and mechanistic studies are evaluated for the purpose of determining whether they suggest biological plausibility of agent- and outcome-specific associations identified in epidemiological studies. Furthermore, the animal and mechanistic studies inform whether an observed pattern of association in the epidemiological literature might be interpreted as the product of more than error, bias, confounding, and chance. The animal and mechanistic studies are not used as direct evidence to support or refute an association between the exposures of interest and health effects in humans. The IOM Vaccine Approach also considers mechanistic evidence in this manner, but uses a method for categorizing the mechanistic evidence into one of four weight-of-evidence categories (strong, intermediate, weak, or lacking evidence). This approach relies heavily on case reports (not animal or mechanistic data) to assess mechanistic plausibility given that a physiological response following the administration of a vaccine can provide evidence of an adverse effect. However, although case reports are helpful in identifying potential acute effects of an exposure, particularly when the dose is well described, they are typically not informative for evaluating the potential plausibility for long-term health effects from exposures where dose and timing of exposure may not be well characterized. For this reason, the IOM Agent Orange and Gulf War Approaches for assessing the long-term health effects of exposures have not used case reports in their evaluations, but rather rely more on animal studies.

Integration of Evidence

In the IOM Agent Orange Approach (e.g., NASEM, 2016b) and the IOM Gulf War Approach (e.g., NASEM, 2016c), human, animal, and mechanistic studies are evaluated to draw one of five possible conclusions: (1) sufficient evidence of a causal relationship, (2) sufficient evidence of an association, (3) limited/suggestive evidence of an association, (4) inadequate/insufficient evidence to determine whether an association exists, or (5) limited/suggestive evidence of no associations. The integration is described qualitatively to support a conclusion. This approach is preferred to a formulaic method because “adherence to a narrowly prescribed formula of what data would be required for each category of association or for a particular health outcome” (NASEM, 2016c, p. 28) would limit the ability to consider the nuances and methodological considerations needed to adequately integrate and assign causation categories for diverse test agents.

The IOM Vaccine Approach (IOM, 2012a) has a formal scheme by which to draw causality conclusions. Separate weight-of-evidence evaluations are performed for the epidemiological studies and mechanistic studies, and the evidence is classified into categories that describe the strength of the evidence. Those classifications are used in the schematic presented in Figure 3-1 to determine the causality conclusion.

Compensation Decisions

A critical component of all three IOM approaches is the separation of the causation evaluation at the population level from the individual adjudication of medical and claim applications (e.g., IOM, 1999, 2003, 2012b). The entity tasked with reviewing and determining causation categories has no role in the adjudication of claims or medical applications. This separation allows for an independent assessment of causality by subject-matter experts and precludes bias in the individual adjudication decisions.

FINDINGS AND RECOMMENDATIONS

The Army will use a two-step process for making determinations about agent- and outcome-specific associations that includes making determinations about general causation and then considering case-specific information to draw conclusions about specific causality. The committee recommends that the Army consider uncoupling the two processes, so that general causation is evaluated and determined by a separate group. The committee judges this to be a preferable approach to ensure that appropriate subject-matter expertise is used to make determinations about general causation and to preclude bias in the individual adjudication decisions. This approach would be similar to that used by the Department of Veterans Affairs when making determinations about treatment, disability, and compensation for veterans of the Vietnam and Gulf Wars, where the causation evaluation at the population level is conducted separately from the individual adjudication of medical and claim applications.

The Army has indicated that the IOM Vaccine Approach will be used to establish general causality. Although the committee agrees that the four categories of causality in this framework may be appropriate for the assessment of potential long-term health effects on the former test subjects, the framework is most appropriate when a large amount of human data is available, as is the case with most vaccines. This framework may not be as applicable for assessing the potential long-term health effects for the different types of agents used in the Army testing programs. Therefore, the committee questions whether the IOM Vaccine Approach, beyond the categories of causality, is the most suitable for the Army’s purposes. It is likely that animal and mechanistic data will be important in hazard evaluations for many test agents and substances; therefore, other

weight-of-evidence frameworks, such as the approaches used by the IOM committees on Veterans and Agent Orange and Gulf War and Health, may also be used as guides to assess causality. The committee recommends that the Army use these approaches to guide its modification of the IOM Vaccine Approach. Some key elements to consider include

• Establishing protocols and criteria for identifying and selecting studies that are relevant to the test agents under consideration.
• Using a weight-of-evidence approach that has flexibility to be applied to agents that have a spectrum of available data sets. For example, a formal method for classifying and integrating epidemiological and mechanistic data would be suitable for agents that have substantial human evidence whereas a more qualitative integration of human, animal, and mechanistic evidence would be more appropriate for agents with less robust human data sets.
• Providing context for how new information affects conclusions drawn in earlier reviews.
• Documenting how causal determinations were made.

Given the timeframes under which the Army is working, modifications will likely include streamlined approaches to conducting the assessment, as the IOM approaches are rigorous processes that require substantial resources and time to complete.

For determinations about specific causality, the Army has acknowledged that information about confounding variables will likely be incomplete. The committee agrees that it is unlikely that the exposures during the tests can be adequately characterized or that potential confounders can be ruled out for most outcomes; consequently, some presumptions about exposure and disease may have to be made.

REFERENCES

IOM (Institute of Medicine). 1994. Veterans and Agent Orange: Health Effects of Herbicides Used in Vietnam. Washington, DC: National Academy Press.

IOM. 1999. Veterans and Agent Orange: Update 1998. Washington, DC: National Academy Press.

IOM. 2000. Gulf War and Health, Volume 1: Depleted Uranium, Pyridostigmine Bromide, Sarin, Vaccines. Washington, DC: National Academy Press.

IOM. 2003. Gulf War and Health, Volume 2: Insecticides and Solvents. Washington, DC: The National Academies Press.

IOM. 2004. Gulf War and Health, Updated Literature Review of Sarin. Washington, DC: The National Academies Press.

IOM. 2007. Gulf War and Health, Volume 5: Infectious Diseases. Washington, DC: The National Academies Press.

IOM. 2012a. Adverse Effects of Vaccines: Evidence and Causality. Washington, DC: The National Academies Press.

IOM. 2012b. Veterans and Agent Orange: Update 2010. Washington, DC: The National Academies Press.

NASEM (National Academies of Sciences, Engineering, and Medicine). 2016a. Assessing Health Outcomes Among Veterans of Project SHAD (Shipboard Hazard and Defense). Washington, DC: The National Academies Press.

NASEM. 2016b. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press.

NASEM. 2016c. Gulf War and Health: Volume 10: Update of Health Effects of Serving in the Gulf War, 2016. Washington, DC: The National Academies Press.

NTP (National Toxicology Program). 2015. Handbook for Conducting a Literature-Base Health Assessment Using OHAT Approach for Systematic Review and Evidence Integration, January 9, 2015. Office of Health Assessment and Translation, Division of the National Toxicology Program, National Institute of Environmental Health Sciences [online]. Available: https://ntp.niehs.nih.gov/ntp/ohat/pubs/handbookjan2015_508.pdf [accessed December 28, 2017].