National Academies Press: OpenBook

Veterans and Agent Orange: Update 2014 (2016)

Chapter: 2 Evaluating the Evidence

« Previous: 1 Introduction
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×

2

Evaluating the Evidence

This chapter outlines the approach used by the Committee to Review the Health Effects in Vietnam Veterans of Exposure to Herbicides (Tenth Biennial Update) and its predecessors to evaluate the available scientific evidence. A more complete description is found in Chapter 5 of Veterans and Agent Orange: Health Effects of Herbicides Used in Vietnam,1 hereafter referred to as VAO (IOM, 1994).

OVERVIEW

Throughout its evolution, the committee has aimed to employ a methodologically sound evaluation process that is comprehensive, informative, and transparent. This has been achieved by the use of a systematic process of evaluation used over many years, one that has been consistent despite the fact the participating members of the committee have changed over time. This process of evaluation is depicted in Figure 2-1 below.

DEFINE THE EVALUATION PARAMETERS

This section describes the primary evaluation parameters used by the committee. This includes the choice and measurement of health outcomes of concern,

________________

1Despite loose usage of “Agent Orange” by many people, in numerous publications, and even in the title of this series, this committee uses “herbicides” to refer to the full range of herbicide exposures experienced in Vietnam, while “Agent Orange” is reserved for a specific one of the mixtures sprayed in Vietnam.

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
images
FIGURE 2-1 Process employed for evaluating the evidence.

the choice and measurement of chemicals of interest (COIs), and the target populations to be reviewed in the evaluation process.

Choice and Measurement of Health Outcomes

As discussed in Chapter 1, the committee was charged with summarizing the strength of the scientific evidence concerning associations between exposure to various herbicides and contaminants during service in the Vietnam War and individual diseases or other health outcomes. However, Public Law 102-4, which mandated the committee’s work, did not specify any particular health outcomes suspected of being associated with herbicide exposure. Such a list of outcomes was developed on the basis of diseases and conditions addressed in the scientific literature identified through the original VAO committee’s extensive literature searches. The list has been amended in the VAO updates to include additional outcomes reported upon in new publications, to requests from the US Department of Veterans Affairs (VA) and various veterans’ service organizations, and to concerns of Vietnam veterans and their families. Comments received at public hearings and in written submissions from veterans and other interested persons have been valuable in identifying issues to be pursued to greater depth in the scientific literature.

Although the increased risks of various psychological conditions, including but not limited to posttraumatic stress disorder (PTSD), among veterans of all US conflicts have always been of scientific and public health concern, such conditions have not been considered by VAO committees. In continuing this practice the current committee notes two underlying principles:

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
  • First, military service alone, including deployment and service in Vietnam, confers a range of potentially traumatic psychological exposures, which may be expected to increase the risk of developing PTSD and related psychological comorbidities. To illustrate, the prevalence of PTSD is more than twice as high for operational infantry units exposed to direct combat than it is in general population samples (Kok et al., 2012). Given the known relationship between combat exposure and an increased risk of mental health conditions, a synthesis of the literature would not provide the opportunity to identify any potential adverse effects on mental health outcomes caused by exposure to the COIs that may occur independently of psychological effects accrued through military service.
  • Second, from reviewing the vast toxicology literature related to the COIs, it is clear that there is a dearth of reports that address potential associations and mechanistic explanations of how exposure to the COIs experienced during military service in Vietnam could conceivably influence the risk of developing mental health conditions. This applies specifically to an overall absence of published evidence as to how dioxin/2,3,7,8-tetrachlorodibenzo-p-dioxin (TCDD) exposure could be etiologically implicated in the development of PTSD and related psychological comorbidities.

Thus, in aggregate, the health outcomes that the committee has focused on include cancers of all types, cardiovascular and metabolic outcomes, immune system disorders, neurological disorders (excluding psychological conditions), a range of other chronic health outcomes, and fertility and gestational effects. The primary focus of the evaluation was on adverse outcomes in the veterans themselves, but a targeted evaluation was also conducted to look for potential adverse health effects in offspring of Vietnam veterans.

Because any effect of Agent Orange in individuals or groups of veterans is evaluated in terms of disease or medical outcome, the committee paid particular attention to disease classification as it assembled pertinent data from various investigations related to a particular outcome in preparation for integrating the information. The researchers who conducted the studies that the committee reviewed faced the same challenge in interpreting the available documentation when assigning diagnostic labels to given subjects and then grouping the labels for analysis.

Pathologists, clinicians, and epidemiologists use several classification systems, including the International Classification of Diseases (ICD): the International Classification of Diseases, 9th Revision (ICD-9); the International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM); and the International Classification of Diseases for Oncology. The 10th revision of ICD (ICD-10) is currently used to classify mortality information. Most of the subjects investigated in the studies cited in the VAO updates were diagnosed under earlier systems, and most of the articles report results in accordance with

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×

ICD-9 if they use ICD codes at all, so VAO committees have retained the use of ICD-9. ICD codes are a hierarchic system for indicating type of disease and site. For example, ICD-9 162 specifies cancers of the lung, trachea, or bronchus; 162.2, cancer of a main bronchus; 162.3, cancer of an upper lobe; 162.4, cancer of a middle lobe; and 162.5, cancer of a lower lobe.

For a patient to receive a correct cancer diagnosis, careful determination of the extent of disease is necessary, and a biopsy of the tissue must be analyzed with microscopy, often with special immunohistochemical stains and more recently genetic testing, to confirm a clinical impression. Many of the epidemiologic studies reviewed by VAO committees have not used the ICD approach to classification of disease and have relied instead on clinical impression alone. Death-certificate diagnoses are notoriously inaccurate if the certificates are completed by medical officers who are not familiar with the decedents’ medical history (Smith Sehdev and Hutchins, 2001). Self-reported diagnoses, which are obtained from survey questionnaires, often are partially or completely inaccurate; for instance, a patient may report having been treated for stomach cancer when the correct diagnosis was gastric adenocarcinoma, gastric lymphoma, pancreatic cancer, large bowel cancer, or peritoneal cancer.

Many epidemiologic studies report disease outcome by organ system. For instance, the term “digestive system” may be used for conditions that are benign or malignant and that affect the esophagus, stomach, liver, pancreas, small bowel, large bowel, or rectum. Therefore, if a report indicated that a cohort has an increased incidence of digestive system cancers, then it would be unclear whether the association was attributable to excess cases of esophageal, gastric, hepatic, pancreatic, or intestinal cancers or to some combination thereof. Such generalization is complicated by the fact that the cause of cancer may differ between anatomic sites. For instance, there are strong associations between Helicobacter pylori infection and gastric cancer, between smoking and squamous cell carcinoma of the esophagus, and between chronic hepatitis B infection and hepatic cancer. Furthermore, a single site may experience a carcinogenic response to multiple agents, while the same agent may cause cancer at multiple sites.

The committee recognizes that outcome misclassification is a possibility when recording of a diagnosis with a specific ICD code is used as the means of entering an observation into an analysis, but this system has been refined over many decades and is virtually universally used and understood, in addition to being exhaustive and explicit. Therefore, this and previous VAO committees have opted to use the ICD system as an organizing tool. Although the groupings of cancer sites for which conclusions about association have been presented may correspond more closely with National Institute for Occupational Safety and Health or National Cancer Institute Surveillance Epidemiology and End Results categories (see Appendix C), the underlying ICD codes provide the most exactitude. In this report, ICD codes appear almost exclusively in the introductory sections of health-outcome discussions (particularly for cancers) to specify precisely what outcome

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×

the committee is addressing and, when possible, in the results table to indicate exactly what the primary researchers believed they were investigating. (See Appendix C for cancer groupings with corresponding ICD-9 and ICD-10 codes.)

Rare diseases, such as hairy-cell leukemia and tonsil cancer, are difficult to study because it is difficult to accumulate enough cases to permit analysis. Often, the result is that the observed cases are included in a broader, less specific category. Thus, epidemiologic data may not be available for assessing whether a particular rare disease is associated with Agent Orange exposure. In some instances, such as chronic lymphocytic leukemia and amyloid light-chain (AL) amyloidosis, VAO committees have reached conclusions on the basis of the data available and the etiology of the disease. Through systematic application of the hierarchic nature of the ICD coding system, committees intend to draw, for every type of cancer, an explicit conclusion about the adequacy of available evidence to support an association between herbicide exposure and that type of cancer. For nonmalignant conditions, the diversity of disease processes involved makes the use of broad ICD ranges less useful, but, because VAO committees could not possibly address every rare nonmalignant disease, they do not draw explicit conclusions about diseases that are not discussed. Thus, the category of “inadequate or insufficient evidence to determine an association” is the default or starting point for any health outcome; if a condition or outcome is not addressed specifically, then it will be in this category.

In general, VAO committees have not considered case reports, case series, or other published studies that lacked control or comparison groups. An exception was made, however, in the case of early-onset peripheral neuropathy, for which a considerable number of case reports associated with high dioxin exposure were available. These were accepted for review because the rapid appearance and frequently transient nature of the condition impose methodologic constraints that might have precluded the application of standard epidemiologic techniques. Clinical follow-up of many of these cases ultimately demonstrated that dioxin-associated acute peripheral neuropathy is not necessarily transient. A subsequent deviation from this rule was made in considering the toxicokinetic information gathered by monitoring Victor Yushchenko (President of Ukraine, 2005–2010) following his TCDD poisoning in 2004 while campaigning.

The committee is aware of the concerns of some veterans about the role of herbicide exposure in the occurrence of multiple health outcomes, such as multiple cancers, in a given person. Little research has been done to address whether the rate of concurrence is greater than would be expected by chance. Simultaneous analysis of multiple health outcomes could potentially provide more insight into whether the chemicals of interest cause multiple health effects, into competing risks among various health outcomes, and into the interactive effects of health outcomes. Addressing health conditions individually has remained challenging, however, and, at present, methods have not been developed to identify with any confidence patterns among multiple health outcomes associated with a single exposure.

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×

Choice and Measurement of COIs

Mixtures of 2,4-dichlorophenoxyacetic acid (2,4-D), 2,4,5-trichlorophen-oxyacetic acid (2,4,5-T), picloram, and cacodylic acid made up the bulk of the herbicides sprayed in Vietnam. At the time of the spraying, TCDD (a form of dioxin) was an unintended contaminant in the production of 2,4,5-T and so it was present in Agent Pink, Agent Green, Agent Purple, Agent Orange, and Agent Orange II, which all contained 2,4,5-T. It is important to note that TCDD and Agent Orange are not the same. Databases have been searched for the names of those compounds, their synonyms and abbreviations, and their Chemical Abstracts Service (CAS) numbers. The evidence indicates that a single protein, the aryl hydrocarbon receptor (AHR), mediates essentially all the toxicity of TCDD, so “aryl hydrocarbon receptor” also was used as a keyword.

It is well accepted that any TCDD or herbicide effect may be diluted somewhat in studies of Vietnam veterans because some of the veterans may not have been exposed or may have been exposed only at low concentrations. The problem is exacerbated in studies in which exposure is defined in terms of occupation (even on the basis of a full job history). Exploratory studies based on linking to a one-time statement of occupation (for example, on a death certificate or in a census) are thought to be of little use even when a job–exposure matrix is used to “convert” standardized job codes into specific exposures. Not only is there uncertainty about whether all members of a sample have been exposed to one of the COIs unless detailed personal monitoring and industrial-hygiene work have been performed but also for most occupational categories there is considerable certainty that the workers were exposed to many other potentially toxic agents as well. Thus, such studies may well minimize the effects of exposure to TCDD or the herbicides of interest while yielding misleading indications of health problems resulting from other exposures.

For cacodylic acid and picloram, the search terms were the chemical names, synonyms, and CAS numbers of these herbicides. It should be noted that cacodylic acid, or dimethyl arsenic acid of valence 5 (DMAV), is an organic form of arsenic. In addition to being synthesized as a herbicide, DMAV is a metabolite of inorganic arsenic exposure in humans. DMAV was long thought to be a biologically inactive metabolite, but accruing evidence suggests that methylated forms, such as monomethyl arsenic acid of valence 3 (MMAIII) (Aposhian et al., 2000) and perhaps DMAIII and DMAV (Cohen et al., 2006), are responsible for some of the adverse effects of inorganic arsenic. This committee considered the available toxicologic information on DMA very carefully in assessing its contribution to the biologic plausibility of an association of various health outcomes with exposure to the herbicides used in Vietnam. It could not, however, accept the hypothesis that direct exposure to DMAV would necessarily result in the same adverse health effects as would exposure to toxic concentrations of inorganic arsenic. Therefore, as in prior VAO reports, the epidemiologic literature on the

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×

health effects of inorganic arsenic was not considered relevant for the VAO task. Further details on the effects of inorganic arsenic can be found in recent reviews (IARC, 2012a; NRC, 2013).

With the structural representation at hand in Figure 2-2, one can readily see the basis of an assertion heard repeatedly from individual Vietnam veterans that “benzene is contained in TCDD.” Indeed, the two rings at the ends of the three-ring structure constituting the basic structure of dioxin compounds, to which chlorine molecules or other chemical radicals can be attached, do have the molecular structure of a single benzene molecule, and the “dibenzo-dioxin” in TCDD’s chemical name does mean that the molecule is a benzene-substituted dioxane. The benzene ring structure is a basic building block of a vast number of organic compounds, both industrial (such as polyaromatic hydrocarbons, the phenoxy herbicides, picloram, and polychlorinated biphenyls [PCBs]) and natural (such as estradiol, a hormone present in both men and women). However, the biologically active compound benzene does not emerge from dioxin, whose three-ring structure is extremely stable and resistant to metabolism.

Interaction or synergism among the COIs or in combination with other agents is another theoretical concern. The committee was not charged with attributing effects to specific COIs, and joint effects among them should be adequately identified by the committee’s approach. The combinations of the chemicals with other agents that might lead to problems are virtually infinite, and hence, not feasible for systematic and comprehensive evaluation. Real-life experience, as investigated with epidemiologic studies, effectively integrates any results of exposure to a target substance in combination with other substances that may be etiologically relevant.

Thus, in aggregate, the primary COIs evaluated by the committee with respect to potential associations with adverse health outcomes among Vietnam veterans are 2,4-D, 2,4,5-T, picloram, cacodylic acid, and TCDD. As explained above, inorganic arsenic and benzene were not considered as relevant service-related exposures among Vietnam veterans and thus were not evaluated in relation to their potential risk of adverse health outcomes.

Exposure Assessment of COIs

Much of the evidence that VAO committees have considered has been drawn from studies of populations that were not in Vietnam during the period when Agent Orange and other herbicides were used as defoliants. The most informative of those studies were well-documented investigations of occupational exposures to TCDD or specific herbicides, such as 2,4-D and 2,4,5-T. In many other studies, TCDD exposure was combined with exposures to an array of “dioxin-like” compounds, and the herbicides were often analyzed as members of a functional class; this is less informative for the committee’s purposes than individual results on a specific compound. In the real-world situations investigated in epidemiologic

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×

studies, exposure to multiple possibly toxic chemicals is the rule rather than the exception; for example, farmers and other agricultural populations are likely to be exposed to insecticides, fungicides, and herbicides. In its examination of these epidemiologic studies, the committee looked for evidence of health effects that are associated with the specific compounds in the defoliants used in Vietnam and sought consideration of and adjustment for other possibly confounding exposures.

The quality of exposure information in the scientific literature reviewed by this and previous VAO committees varies widely. Some studies relied on interviews or questionnaires to determine the extent and frequency of exposure. Such self-reported information, which has the potential for recall bias, generally carries less weight than do more objective measures of exposure. The strength of questionnaire-based information as evidence of exposure is enhanced to the extent that the information can be corroborated or validated by other sources. Written records of chemical purchase or production can provide one type of objective information. Even more useful are scientific measurements of exposure. In some occupational studies, for example, workers wear air-sampling instruments that measure the concentration of a contaminant in each worker’s breathing zone. Measurement of chemicals or their products in biologic specimens, such as blood and urine, can provide reliable indications of exposure for specific periods. Studies that categorize exposure from well-documented environmental sources of contaminants can be useful in the identification of exposed populations, but their results may be inaccurate if people with different magnitudes of exposure are assigned to the same general category of exposure. Studies that explore environmental exposure and disease frequency in regional populations (such as in states and counties) are known as ecologic studies. Most ecologic studies are considered preliminary or “hypothesis-generating” studies because they lack information on exposures and disease on an individual basis and are unable to address potential confounding factors.

Chapter 3 of this update addresses issues related to exposure estimation in more detail. The agent of interest may be assessed with various degrees of specificity. For instance, any of the four herbicides in question could be individually measured, and phenoxy herbicides would be a useful broader category for 2,4,5-T and 2,4-D; but a report of findings in terms simply of “herbicides” is only marginally informative, and results stated in terms of “pesticides” are too vague to be useful. For a given COI, the measure of exposure may be increasingly imprecise—for example, concentrations in target tissue, serum concentrations, cumulative exposure, possible exposure, and so on down to merely a report of service in a job or industry category. Those approaches can address complexities in specificity, duration, and intensity of exposure with various degrees of success. All may provide some information about an association with a COI, but this committee has determined that the investigation of associations between an exposure of concern and most health outcomes has reached the stage where some

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×

characterizations of exposure are too nonspecific to promote insight. For health outcomes with little evidence, a somewhat less stringent criterion would apply so that no possible signal of an association would be overlooked.

Identification of Target Study Populations

Because Vietnam veterans are the target population of the charge to the VAO committees, studies of these veterans (serving in any of the armed forces, American or otherwise) have always been accorded considerable weight in the committees’ deliberations, whether or not an estimation of exposure to herbicide-related substances has been attempted. The characterization of exposure in studies of the veterans was extremely uncommon at the time of the original VAO report, and the Vietnam veterans’ own ages were still below the ages at which many chronic illnesses are manifested. Consequently, the original committee made extensive efforts to consider several groups known or thought to have potentially higher and better-characterized exposure to TCDD or phenoxy herbicides than the Vietnam veterans themselves—both occupational exposure (for example, chemical-production, paper and pulp, sawmill, tannery, waste-incinerator, railroad, agricultural, and forestry workers) and environmental exposure (for example, residents of Quail Run, Seveso, Times Beach, and Vietnam).

Successive committees have been able to concentrate more on those studies that explicitly addressed the exposures specified by the charge. Some occupational and environmental cohorts that received exceptionally high exposures (such as the International Agency for Research on Cancer and Seveso cohorts) are now well characterized and have produced a stream of informative results. The Agricultural Health Study, a continuing prospective cohort study of agricultural populations with specific information on the COIs, has contributed a steady stream of information to the database. Most important, the Vietnam veterans themselves are advancing in age and, when studied, are capable of directly providing substantial information on chronic health conditions and, in some study populations, information related to serum TCDD concentrations. The committee for Update 2006 decided that doing exhaustive searches on job titles, occupations, or industries in order to identify additional study populations that had possible, but not specifically characterized, exposure to the COIs was no longer an efficient means of augmenting the evidence database in that the citations such searches would yield were more likely to be those with information about a health outcome at the expense of considerable uncertainty about exposure.

The previous and current committees followed the Update 2006 committee’s practice of performing more circumscribed searching. As the information in the database on populations that had established exposure to the COIs has grown, VAO committees have become less dependent on data from studies that had nonspecific exposure information and have been able to focus more on the findings of studies that had refined exposure specificity. In recognition of the more pivotal

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×

role that findings drawn directly from Vietnam veterans were able to play in its decisions, the committee for Update 2008 reordered its consideration of populations. For each health outcome, studies of Vietnam veterans, the target population of the VAO series, are addressed first and occupational and environmental studies are addressed second. The committee’s exact criteria concerning exposure specificity are presented at the end of Chapter 3.

Thus, study populations considered by the committee included Vietnam veterans (US and otherwise), who are presumed to have been exposed to all the COIs, plus cohorts believed to have been occupationally or environmentally exposed to at least one of the COIs. Although Vietnam veterans constitute the source population of interest, the committee has taken into account the potential for more precise quantification and evaluation of the risks of adverse health outcomes associated with the COIs in better characterized cohorts (i.e., occupational and environmental cohorts). As illustrated by Figure 1-1 in Chapter 1, when the rather limited collection of serum TCDD levels gathered from exposed populations are compared, it is found that the median TCDD levels in veterans who had worked in Operation Ranch Hand were higher than those measured in their own comparison group or in ground troops, both of which had median levels in the range of contemporaneous background (unitary ppt range), but about an order of magnitude less than herbicide production workers, who in turn had levels about two orders of magnitude less than individuals who resided near the Seveso industrial explosion. Including these more highly exposed populations had the additional advantage that epidemiologic studies of them were likely to have greater statistical power to detect any adverse effects that might occur.

Mechanistic and toxicology studies of the COIs are not core evidence in the overall evaluation of the potential associations between the COIs and health outcomes specific to Vietnam veterans, but they are considered for the potential insight they may provide into biologic plausibility.

COMPILATION OF EVIDENCE FOR EVALUATION

This section describes the manner in which potentially relevant literature was identified, how such literature was screened and processed, and how the new information that was identified has been integrated with previous information evaluated by the committee.

Identification of Potentially Relevant Literature

This report concentrates on the evidence published after the completion of work on Veterans and Agent Orange: Update 2012 (IOM, 2014). Relevant new contributions to the literature made during the period October 1, 2012–September 30, 2014, were sought. The information that the committee used was compiled through a comprehensive electronic search of public and

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×

commercial databases—biologic, medical, toxicologic, chemical, historical, and regulatory—that provide citations of the scientific literature. In addition, the reference lists of some review and research articles, books, and reports were examined for potentially relevant articles.

The specific search terms used by the committee directly parallel the target COIs defined above, but they were also purposely broad so as to be exhaustive in terms of identifying all relevant literature. The search strategy included the chemical names, synonyms, and CAS numbers of the specific COIs—2,4-D, 2,4,5-T, TCDD, cacodylic acid, and picloram (see Figure 2-2 for chemical structures and CAS numbers)—and the more generic terms involved with this project: Vietnam veteran, Agent Orange, aryl hydrocarbon receptor, dioxin, herbicide, and phenoxy. Results on other specific phenoxy herbicides are also of interest: 2-methyl-4-chlorophenoxyacetic acid (MCPA) and 2-(2-methyl-4-chlo-rophenoxy) propionic acid (MCPP or Mecoprop) are structurally similar to 2,4-D, while 2-(2,4,5-trichlorophenoxy) propionic acid (2,4,5-TP or Silvex) have structures analogous to 2,4,5-T (see Figure 2-2). Although the benzoate herbicide dicamba (2-methoxy-3,6-dichlorobenzoic acid) is not always categorized with the phenoxy herbicides, it has structural similarities with this class, and measures of its association with various adverse health outcomes have been factored into the evidence.

Because some PCBs and polychlorodibenzofurans (PCDFs) have dioxin-like biologic activity, studies of populations exposed to PCBs or PCDFs were reviewed when the results were presented in terms of toxic equivalents (TEQs). Findings related only to exposure to the diverse chemical families of pesticides were considered too nonspecific for inclusion in the evidence database that was used to draw conclusions about associations. (An ancillary analysis conducted during the preparation of Update 2008 determined that the term “pesticide” did not identify any relevant citations that were not picked up by more specific terms, so it was eliminated from the searches conducted since this reduced considerably the number of extraneous hits to be culled.)

Processing of Identified Publications

The search strategy was devised to ensure that abstracts of all potentially relevant articles were subjected to closer screening, but it also resulted in the identification of a large number of nonrelevant studies. The searches produced in excess of 7,600 “hits,” including some studies that were identified more than once. It was evident from the abstracts of most of the cited articles that they did not address health effects in association with exposure to the COIs; for example, many of the cited studies investigated the efficacy of herbicides in killing weeds. All studies that discussed health effects were considered if the search-related information (title, abstract, and keywords) indicated that any of the herbicides of interest (or any of their components) may have been investigated. For each of

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×

the more than 800 potentially relevant citations ultimately identified, a copy of the entire article was obtained online and reviewed more thoroughly by the committee for determination of whether it should be included in the report. For the present update, very few documents of interest had to be retrieved as hard copies from library sources.

In large part, the included reports are peer-reviewed journal articles, but generally available and formally published government studies (particularly those investigating health effects in Vietnam veterans) are also included under the assumption that they have been carefully reviewed. In practice, the articles are generally in English, but VAO committees have obtained translations for crucial ones that were not in English, as in the case of reports of a study of Korean veterans of the Vietnam War (Kim HA et al., 2003; Kim JS et al., 2003) when Update 2004 was produced.

TCDD, the 2,3,7,8-chlorinated congener of dioxin, is the most potent of the polychlorinated dibenzo-p-dioxins (PCDDs), dibenzofurans, and biphenyls, so it is presumed to be the most problematic of the dioxin-like chemicals contaminating the phenoxy herbicides used in Vietnam. However, our concern is not limited to that congener. In non-laboratory settings—for example, in epidemiologic studies—exposures occur not only to TCDD but also to mixtures of dioxins, dibenzofurans, and PCBs, which vary in their degree of chlorination. The concept of toxic equivalency has been developed primarily to permit an overarching estimation of oral exposure and risk from environmentally persistent chemicals that have structural similarities to PCDDs and PCDFs that bind the AHR, induce the same spectrum of effects, and bioaccumulate in the food chain (van den Berg et al., 2006). A toxicity equivalency factor (TEF) is an estimate of the dioxin-like potency of an individual congener relative to the toxicity of TCDD. TEQs are often used to estimate the cumulative toxic potency of mixtures as the sum of TEFs weighted by the concentrations of the corresponding congeners in the mixture; this total is denoted as the mixture’s TEQ in terms of dioxin-like activity. That approach is often taken in epidemiologic studies that focus on PCBs. Many epidemiologic studies of PCBs have been recovered in VAO literature searches although they were not specifically sought. Because dioxin-like and non–dioxin-like PCB congeners are found together in environmental mixtures and are known to mediate toxicity by various mechanisms, the relative contribution of dioxin-like PCBs to an individual health outcome can be difficult to determine. Therefore, evidence from epidemiologic studies of PCB exposure has been retained only for results that concerned specific dioxin-like congeners or that were reported in terms of TEQs. Although all studies reporting TEQs based on PCBs were reviewed, those studies that reported TEQs based only on mono-ortho PCBs (which are PCBs 105, 114, 118, 123, 156, 157, 167, and 189) were given very limited consideration because mono-ortho PCBs typically contribute less than 10 percent of the total TEQs, based on the World Health Organization’s revised TEFs of 2005 (La Rocca et al., 2008; van den Berg et al., 2006).

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×

The committee for Update 2008 investigated what pesticides are used in greenhouses and determined that greenhouse workers are not likely to be exposed to herbicides, particularly those of interest for VAO committee deliberations (Czarnota, 2004; Neal, 2006; University of Connecticut, 2006). Results on such populations (Abell et al., 2000, on fertility; Hansen et al., 1992, on cancers in female workers) were retroactively excluded from the evidence database considered in Update 2008, and no new citations of studies of such workers have been retained. Further consultation (email in Public Access File, November 12, 2012) with Helle Raun Andersen, an Associate Professor at the University of Southern Denmark and a researcher on a series of epidemiologic studies of reproductive effects among greenhouse workers, confirmed for the current committee the lack of herbicide exposure among such workers.

Integration of New Information

More than 70 articles on epidemiologic studies and several dozen toxicology studies contributed new information to the present update. New evidence on each health outcome was reviewed in detail. The committee’s conclusions, however, are based on the accumulated evidence, not just on recently published studies. In a considerable number of instances over the course of the VAO reports, single study populations have generated multiple entries for a given health outcome. Before Update 2010, the procedure had been to enter new results into the summary results tables in groups corresponding to the VAO update in which the study first appeared, so it has been difficult to recognize which findings are based on the experience of the same set of people.

The current committee has continued the revisions begun by the committee for Update 2010 in organizing the tables of cumulative results on health outcomes in such a way as to make the interrelationships more evident for its own deliberations and for the reader. For example, as part of its effort to achieve an integrated picture of how a health outcome is manifested in a given study population, the committee chose to move the citations to the far right of the findings so as to put less emphasis on individual publications. The reported findings on a given condition from a particular study population have been gathered and presented in reverse chronologic order so that the most mature set of statistics appears first. In many instances, that will also represent the most informative set of data—the set that has the greatest power to demonstrate an adverse effect in the population in question. For some health problems, particularly those common in old age, the toxic effect associated with an external factor may be to cause a disease to manifest sooner. In such situations, the evidence of an association with an exposure may consist of a wave of diagnoses in younger people, and the prevalence will equalize with that in the control group as the populations age. The committee therefore decided that it could not retain only the most recent findings when considering the experience of a given study population.

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×

The cohorts themselves have been ordered in the tables to reflect the overarching cohorts of which they are subgroups. The exposure of interest in each cohort is explicitly noted in the tables in order to facilitate judgments about when consistency might be expected among populations that experience the same exposure. This should minimize misapprehensions that there are inconsistencies if two excellent studies of groups exposed to different COIs have incongruent findings.

Primary findings, otherwise known as primary data analyses, are the specific type of evidence that the committee endeavors to integrate in drawing its conclusions. Reanalyses (without the incorporation of additional information), pooled analyses, reviews, and so on, may be discussed in conjunction with primary results or in synthesis sections on a given health outcome, but they are not themselves part of the evidence dataset. Nonetheless, well-conducted meta-analyses and literature syntheses can be informative to committee members in providing a broad framework in which to evaluate individual studies among a large body of literature.

CATEGORIZATION OF TYPES OF EVIDENCE

This section describes the manner in which the vast literature reviewed over many years has been categorized. With such a substantial literature base to review, a general categorization scheme was important for considering the relative contribution of individual studies. The committee’s general approach to the evaluation of scientific evidence corresponds closely to the approach developed by the original VAO committee as delineated in detail in Chapter 5 of VAO. The committee had three specific tasks: to determine whether there is a statistical association between exposure to the herbicides used in Vietnam and health outcomes, to determine the increase in risk of effects among Vietnam veterans, and to determine whether plausible biologic mechanisms provide support for a causal relationship with a given health outcome. Scientifically relevant associations between exposures to the COIs and specific health outcomes are determined through an analysis of available epidemiologic studies that is informed by an understanding of the toxicology of the chemicals and their exposure pathways.

In general, human studies conducted among Vietnam veterans were considered to be of greatest relevance to the committee’s charge. However, in many such studies, the characterization of exposure to the COIs was imprecise and was better characterized in human studies conducted among non-veterans, such as within occupational and environmental cohorts. Thus, in examining the literature in totality, there was continuous assessment of the specificity of the target population of interest (Vietnam veterans) as well as the specificity of the measurement of COIs. Animal and mechanistic studies were also considered as potentially relevant, but were reviewed in the context of biological plausibility, rather than as direct evidence for or against a statistical association between the COIs and health outcomes among Vietnam veterans. Similarly, in reaching their conclusions,

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×

VAO committees consider the nature of the exposures, the nature of the health outcomes, the populations exposed, and the quality of the evidence examined.

Human Studies Among Vietnam Veterans

The committee reviewed all available published studies of Vietnam veterans, who are all presumed to have been exposed to the COIs as a result of military service in Vietnam. Because studies of Vietnam veterans address the very population of concern to the legislation that mandated the present review, any demonstrations of increased incidence of particular health outcomes among them are of unquestionable pertinence in drawing conclusions.

It is difficult to quantify risk when exposures of a population have not been accurately measured. Relatively recent serum TCDD concentrations are available only for subgroups enrolled in the Air Force Health Study (AFHS) (the Ranch Hand veterans and Southeast Asia comparison subjects) and from VA’s study of deployed and non-deployed members of the Army Chemical Corps. Pharmacokinetic models, with their own set of assumptions, must be applied in order to extrapolate from contemporary readings and obtain presumably accurate estimates of original exposure of Vietnam-era veterans. The absence of reliable measures of Vietnam veterans’ exposure to the COIs limits the committee’s ability to quantify the risks of specific diseases in this population.

Although serum TCDD measurements are available in only a small portion of Vietnam-era veterans, the observed distributions of these most reliable measures of exposure make it clear that they cannot be used as a standard for partitioning veterans into discrete exposure groups, such as service on Vietnamese soil, service in the Blue Water Navy, and service elsewhere in Southeast Asia. For example, many TCDD values observed in the comparison group from the AFHS exceeded US background concentrations and overlapped considerably with those of the Ranch Hand subjects.

As explained in Chapter 1, the committee for Update 2006 decided to make a general statement about its continuing inability to precisely determine exposure to the COIs quantitatively, rather than to reiterate a disclaimer in the concluding section for every health outcome, and the present committee has retained that approach.

Human Studies Among Non-Veterans

The committee reviewed studies among non-veteran populations that might have been exposed to the COIs. These other populations that factored into the committee’s evaluation included cohorts of workers in chemical production and agriculture as well as populations residing near sites of environmental contamination. The committee believes that studies of such non-veteran subjects can help in the assessment of whether the COIs are associated with particular

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×

health outcomes. As noted above in describing the literature search, studies of non-veteran subjects were identified because one of the COIs was specified by the original researchers as presenting a possible toxic exposure rather than on the basis of occupational definitions. Some of the studies provide stronger evidence about health outcomes than do studies of veterans because the exposures were measured sooner after occurrence and were more thoroughly characterized than has been the case in most studies of veterans. Furthermore, in the studies of workers in chemical-production plants, the magnitude and duration of exposure to the chemicals were generally greater, so the likelihood that any possible health consequence would be manifested was greater. The studies were often large enough to examine health risks among groups of people that had different levels of exposure, so dose–response relationships could be investigated. The general practice of VAO committees in determining the strength and validity of findings has been to evaluate all studies, whether or not their subjects were Vietnam veterans, according to the same criteria.

Animal and Mechanistic Studies

Animal models used as surrogates for the study of a human disease must reproduce, with some degree of fidelity, the manifestations of the disease in humans. However, a given effect of an exposure in an animal species does not necessarily establish its occurrence in humans, nor does an apparent absence of a particular effect in animals mean that the effect could not occur in humans. In addition to possible species differences, many factors affect the ability to extrapolate results from animal studies to health effects in humans. For example, animals used in experimental studies are most often exposed to purified chemicals, not to mixtures. Even if herbicide formulations or mixtures are used, the conditions of exposure might not realistically reproduce the human exposures that occur in the field. Furthermore, Vietnam veterans were exposed to other agents—such as tobacco smoke, insecticides, therapeutics, drugs, diesel fumes, and alcohol—that may increase or decrease the ability of chemicals in herbicides to produce a particular adverse health outcome. Few, if any, studies either in humans or in experimental animals have examined those interactions. Thus, the results from animal studies are principally useful for evaluating biologic plausibility.

As discussed in Chapter 4, TCDD is thought to be responsible for many of the toxic effects of the herbicides used in Vietnam. Attempts to establish correlations between the effects of TCDD on experimental systems and their effects on humans are particularly difficult because there are well-known species-, sex-, and outcome-specific differences in susceptibility to TCDD toxicity. Some data indicate that humans might be more resistant than other species to TCDD’s toxic effects (Ema et al., 1994; Moriguchi et al., 2003); other data suggest that, for some outcomes, human sensitivity could be the same as or greater than that of some experimental animals (DeVito et al., 1995). Differences in vulnerability

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×

may also be affected by variations in the rate at which TCDD is eliminated from the body. (See Chapter 4 for details on the toxicokinetics of TCDD.) Although degree of susceptibility is generally thought to be an inherent biological response, it can be influenced by life stage, past history, co-exposures, etc.

It is important to account for TCDD’s mode of action in considering species and strain differences. There is a consensus that most of the toxic effects of TCDD involve interaction with the AHR, a protein that binds TCDD and some other aromatic hydrocarbons with high affinity, although it is now recognized that the AHR performs actions other than just those of a transcriptional enhancer, such as having a role in rapid signal transduction. The formation of an active complex that involves the intracellular receptor, the ligand (the TCDD molecule), and other proteins is followed by an interaction of the activated complex with specific sites on DNA. This specific interaction can alter the expression of genes involved in the regulation of cellular processes.

The development of mice that lack the AHR has helped to establish a definitive association between the AHR and TCDD-mediated toxicity. The affinity of TCDD for the AHR is species- and strain-specific, and responses to the binding of the receptor vary among cell types and developmental stages. In addition, genetically based differences in the properties of the AHR are known to exist in human populations (Zhou et al., 2009), as they are in laboratory animals, so there are genetically based differences in people’s responses to TCDD, leading to some people having an intrinsically greater risk of toxic effects from TCDD exposure and others having less risk.

Although studying AHR biology in transformed human cell lines minimizes the inherent error associated with species extrapolations, caution must be exercised because it is still not clear to what extent toxicity is affected by the transformation itself or by the conditions under which cell lines are cultured in vitro. Furthermore, humans have AHR with differing affinities for dioxin, so a single transformed human cell line will not accurately reflect the spectrum of responses observed in the entire human population.

DEFINE THE EVALUATION APPROACH

This section describes the manner in which the committee reviewed all of the evidence it compiled and synthesized in rendering its conclusions on the relationships between the COIs and health outcomes among Vietnam veterans. This includes the overarching evaluation principles that the committee employed and the role of judgment, the use of statistical association as the primary evaluation metric, and the methodological factors that were considered, such as multiple publications on the same study cohort, plausible biological mechanisms, and publication bias.

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×

Define Overarching Evaluation Principles

The VAO committees began their evaluation by assuming neither the presence nor the absence of an association between exposure and any particular health outcome. Over the series of reviews, the committee accrued evidence of various degrees of association, lack of association, or persistent indeterminacy concerning a wide array of disease states. However, for many conditions, particularly uncommon ones, associations with the COIs have remained unaddressed in the medical research literature; for these, the committee remains neutral, understanding that “absence of evidence is not evidence of absence.”

An issue related to evidence evaluation that was of concern for the Update 2006 committee was the evidence category of “no association.” That committee determined that a conclusion of no association would require substantive evidence of such a lack of effect for all of the COIs. Given the paucity of available information on cacodylic acid and picloram, either positive or negative for virtually all health outcomes, such a conclusion would seem suspect even if substantial evidence uniformly supported a finding of no association both with exposure to the phenoxy herbicides and with exposure to TCDD. Later committees have concurred in that determination and adopted a similar approach to the placement of health outcomes in this category.

When all the available epidemiologic evidence has been evaluated, it is presumed that Vietnam veterans are at increased risk for a specific health outcome if there is evidence of a positive association between one or more of the COIs and the outcome. The best measure of potency for the quantification of risk to veterans would be the rate of the outcome in exposed Vietnam veterans compared with the rate in non-exposed veterans, adjusted for the degree to which any other factors that differ between exposed and non-exposed veterans might influence those rates. Conley and Heerwig (2012) have noted, however, that selection of service members for deployment itself may introduce elements of bias due to consideration of additional factors and application of even more stringent criteria than necessary for enlistment, which translate into the deployed being somewhat healthier than or in some other way differing from those not selected. In any event, in the absence of actual measures of exposure, VAO committees have regarded comparisons between deployed and non-deployed Vietnam-era veterans as most relevant for their purposes. A dose–response relationship established in another human population that was suitably adjusted for such factors would be similarly suitable.

The committee concluded that it would be inappropriate to use certain quantitative techniques, such as meta-analysis, to combine individual study results into a single summary measure of statistical association. The committee reached that conclusion because of the many differences among studies in their definitions of exposure, health outcomes considered, criteria for defining study populations, corrections for confounding factors, and degree of detail in reporting the results. An appropriate use of meta-analysis requires more methodologic consistency

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×

among studies, especially in the definition of exposure, than is present in the literature that the committee reviewed (Egger et al., 2002; Petitti, 2000). A detailed discussion of the results of individual studies in appropriate categories (Vietnam-veteran, occupational, or environmental exposure; and exposure to Agent Orange or equivalent dioxin-contaminated phenoxy herbicides, to dioxin, to phenoxy herbicides without dioxin contamination, to cacodylic acid, or to picloram) with a thorough examination of each study’s strengths and weaknesses is fully informative without making unfounded assumptions of homogeneity.

Role of Judgment

This committee’s process for reaching conclusions about statistical associations involved more than a formulaic application of quantitative procedures to the assembled evidence. First, the committee had to assess the relevance and validity of individual reports. Then, it had to evaluate the possible influences of measurement error, selection bias, confounding, and chance on the reported results. Next, the committee integrated all the evidence within and among diverse fields of research. Finally, the committee based its conclusions on consensus reached within the committee. These aspects of the committee’s review required thoughtful consideration of alternative approaches at several points and could not be accomplished by adherence to a narrowly prescribed formula.

The realized approach, as described here, has been determined to a large extent by the nature of the exposures, of the health outcomes, and of the resulting evidence available for examination; for that reason, it has evolved during the course of the work of this and previous VAO committees. The quantitative and qualitative procedures underlying the present review have been made as explicit as possible, but ultimately the conclusions about association expressed in this report are based on the committee’s collective judgment. The committee has endeavored to express its judgments as clearly and precisely as the data allow.

In delivering the charge to the committee for Update 2010, VA’s representative requested that for health outcomes found to have some evidence supporting statistical association, the committee delineate how well each of the factors that make up the so-called Bradford Hill criteria for causality (Hill, 1965) had been satisfied. It was thought that having a scientific perspective on the extent to which those factors, in addition to biologic plausibility, were met would facilitate the Secretary of Veterans Affairs in making a policy decision concerning the presumptive relationship of any new health outcome to exposure to the herbicides used by the military in Vietnam.

The committee for Update 2010 was uniformly and strongly of the opinion that the execution of a checklist of the Hill criteria would not be an appropriate approach for fulfilling its charge, and the current committee is in complete agreement with the decision of that committee, as was the committee for Update 2012. The list of issues that Hill discussed is not a definitive set of factors to

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×

be addressed in evaluating whether a collection of evidence supports causality. The nine aspects of a statistical association noted by Hill (1965) to contribute to a finding of causality—strength, consistency, specificity, temporality, biologic gradient, plausibility, coherence, experiment, and analogy—vary in the importance that might be assigned to them, but none is sufficient by itself, and only temporality (that the cause precedes the effect) is necessary. Philosophers of science have established that a set of sufficient criteria for causality does not exist (Rothman and Greenland, 1998). Citing Weed and Gorelick (1996) and Holman et al. (2001), Rothman et al. (2008) noted that “epidemiologists have not agreed on a set of causal criteria or on how to apply them [emphasis in original]…. The typical use of causal criteria is to make a case for a position for or against causality that has been arrived at by other, unstated means.” The establishment of causality is not an absolute or discrete (or necessarily permanent) state. The Hill criteria have often been used as a point of reference in addressing the subject of causation in evaluating possible environmental harms, but even in theoretical and optimal circumstances scientists have not derived a definitive algorithm for establishing causality. The extent to which a relationship is judged to be causal entails many subjective elements involving the universe of information considered and the weight accorded to each evidentiary component considered. Furthermore, with regard to chronic diseases, causality is rarely limited to a single factor.

For those reasons, the committee for Update 2010 did not adopt the suggestion to perform what in effect would be a checklist approach to distilling the evidence concerning underlying causality for any observed statistical association between a human health effect and exposure to the components of the herbicides sprayed in Vietnam. The current committee also interprets its charge to be to summarize the scientific evidence for consideration by the Secretary, whose role is to make the policy decision of whether a contribution of herbicide exposure to the occurrence of an adverse health effect is likely enough to merit recognition as a presumptive condition.

Define Primary Metric (Statistical Association)

The primary metric used by the committee in synthesizing all its evidence and rendering its conclusions on the potential health effects of the COIs among Vietnam veterans is statistical association. The issues that arise in determining whether a valid statistical association exists are detailed in Chapter 5 of VAO. Since the work of that first committee, the primary relevant evidence for consideration has come from epidemiologic studies—investigations in which large groups of people are studied to identify an association between exposure to a COI and the occurrence of particular health outcomes. Subsequent committees have not modified the criteria set by the original VAO committee, but the committee for Update 2012 added the clarification that the object of its evaluation was to be “scientifically relevant association” in order to clarify that the strength

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×

of evidence evaluated, based on the quality of the scientific studies reviewed, was a fundamental component of the committee’s deliberations to address the imprecisely defined legislative target of “statistical association.”

Epidemiologists estimate associations between exposure and outcome in a specific population or group in terms of relative risk by using such measures as a standardized mortality ratio, odds ratio, rate ratio, or hazard ratio. Those measures indicate the magnitude of a difference in the rate of an outcome between two populations. For example, if the rate in an exposed population is twice the rate in a non-exposed population, then the relative risk is 2. Similarly, if the odds of a health outcome are 1:20 in an exposed population but 1:100 in a nonexposed population, the odds ratio is 5. In this report, both relative risk (also called risk ratio) and odds ratio are used to represent the association between exposure and adverse outcome. Both measures are often reported in prospective cohort studies. Case-control studies usually report odds ratios and cannot report relative risk because the sampling fraction of the population that is represented by the control group is usually not available in these studies. However, it is possible for case-control studies to provide unbiased estimates of relative risk. For rare diseases with low rates in both the exposed group and the control group, odds are approximately identical with the risk, so an odds ratio is approximately identical with a relative risk. That is,

images

so that when risk is close to zero, (1 − risk) is close to 1, and therefore, odds will be close to risk. An estimated relative risk or odds ratio greater than 1 indicates a positive association; in this case, it is more likely that the outcome will be seen in exposed people than in non-exposed people. A relative risk or odds ratio between zero and 1 indicates a negative or inverse association; the outcome is less likely in exposed people. A relative risk or odds ratio of 1 suggests the absence of association, which is usually the null hypothesis to be tested. A statistically significant association is one that would be unlikely to occur by chance—that is, if the null hypothesis is true. (Chapters 713 contain tables of results abstracted from the studies that provide evidence on individual health outcomes. Because the distinction between risk and odds is of little consequence in the deliberations of VAO committees, the column labeled “Estimated Risk” presents findings without specifying the precise nature of the reported statistic.)

Determining whether an estimated association between an exposure and an outcome represents a real relationship requires careful scrutiny because there can be more than one explanation for an estimate. Bias is a distortion of the measure of association that results from flawed selection in the assembly of the study population or from an error in the measurement of studied characteristics. Confounding is a distortion of the measure of association that can arise from factors related both to exposure and to outcome; awareness of this possibility permits

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×

collection of the ancillary data that could permit statistical adjustment to reduce the problem or to take such an influence into account when evaluating the import of data that have already been collected and analyzed. Chance is the degree to which an estimated association might vary randomly among different samples of the population studied. The width of a confidence interval is used to quantify the likely statistical variability of an exposure–disease association, but it does not incorporate the quantification of distortions that may arise from the systematic problems mentioned above. Even when a relative risk or standardized mortality ratio substantially exceeds a value of 1, a conclusion regarding increased risk must be qualified when the confidence interval is wide. In drawing conclusions, the committee examined the most thoroughly adjusted quantitative estimates of association, judged whether an adjustment for any crucial confounders was lacking, and evaluated the potential influences of bias and chance. In integrating the findings of various studies, the committee considered the degree of statistical significance associated with every estimated risk (a reflection of the magnitude of the observed effect and the power of the study designs) and took note of whether dose–response relationships were evident with increasing exposure rather than simply tallying the “significant” and “nonsignificant” outcomes as dichotomous items of evidence. The committee also considered whether controlled laboratory investigations provide information consistent with the chemicals of interest being associated with a given effect and perhaps causally linked to it.

In pursuing the question of statistical association, the committee recognized that an absolute conclusion about the absence of association is unattainable. As in science generally, studies of the health effects associated with herbicide exposure cannot demonstrate that a purported effect is impossible or could never occur, but only that it is statistically improbable. Any instrument of observation, even an excellent epidemiologic study, is limited in its resolving power. In a strict technical sense, therefore, the absence of an association between even one chemical and a health outcome cannot be proved. Convincingly demonstrating the lack of a particular effect of all five of the COIs simultaneously would be a daunting effort, especially in light of the paucity of information concerning picloram and cacodylic acid. The present committee therefore endorses the decision by the committee for Update 2006 to reclassify several types of cancer that had been classified since VAO (1994) as having “suggestive evidence of no association” with “exposure to herbicides” into this default category of inadequate or insufficient evidence for any conclusion about association to be drawn for a specific health outcome.

Define Factors to Consider in Weighing the Evidence

Totality of Evidence and Multiple Publications

VAO committees wanted to be clear in indicating what evidence is factored into their conclusions. The practice in the VAO reports has been to augment the

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×

results table for a given health outcome with any additional publications considered in the current update in the categories of Vietnam-veteran, occupational, or environmental studies. The inclusion of sequential sets of results from follow-ups of a study population has the potential to create the appearance of a greater weight of evidence than is warranted, so Update 2006 and Update 2008 used italicized citations in results tables to indicate that results had been superseded. The committee for Update 2010 did not want to convey the notion that earlier findings were of no importance. In an effort to compile a comprehensive and comprehensible picture of the history of each study population, the committee for Update 2010 decided to abandon the sequential entries by update that had been used in the results tables since Update 1996. The format adopted in Update 2010 for the results tables was a refinement of the cohort-based approach that had been introduced in Update 2006 for cardiovascular diseases. To make it easier for the reader to locate the discussion of the characteristics of particular study populations and the attributes of the publications based on them, the order of studies in the results tables corresponds to their presentation in Chapter 6. The main categorization of veteran, occupational, and environmental cohort studies and case-control studies has been retained in both instances. In an effort to provide a coherent picture of the occurrence over time of a specific health outcome in a given study population, the current committee has shifted its emphasis away from individual publications by moving the citation that was the source of a particular finding to the rightmost column in the results tables.

Plausible Biologic Mechanisms

Chapter 4, “Information Related to Biologic Plausibility,” previously called “Toxicology,” details the experimental basis of the assessment of biologic plausibility or the extent to which an observed statistical association in epidemiologic studies is consistent with other biologic or medical knowledge. Does the observation of a particular health effect make sense on the basis of what is known about how the chemicals in question act at the tissue, cellular, or molecular level? The relationship between a particular exposure and a specific human health outcome is addressed in the context of research on the effects of the chemicals on biologic systems and of evidence from animal studies.

Chapter 4 presents an integrated toxicity profile of each of the COIs without providing detailed commentary on each possibly relevant toxicology article published in the update period. Experimental information pertinent to a particular health outcome is now presented immediately after the epidemiologic evidence on that outcome in the “Biologic Plausibility” sections on individual health outcomes (see Chapters 713).

A positive statistical association between an exposure and an outcome does not necessarily mean that the exposure is the cause of that outcome. Data from toxicology studies may support or conflict with a hypothesis that a specific

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×

chemical can contribute to the occurrence of a particular disease. Many toxicology studies are conducted with laboratory animals so that variables, including the amount and duration of exposure, can be controlled precisely. Studies that use isolated cells in culture also can elucidate how a chemical alters cellular processes. The objectives of those toxicology studies are to determine what toxic effects are observed at different exposure levels and to identify the mechanisms by which the effects are produced. Ultimately, the results of the toxicology studies should be consistent with what is known about the human disease process if they are to support a conclusion that the development of the disease was influenced by an exposure.

Animal studies and in vitro studies with human cells and cell lines provide links that are important for understanding the underlying biochemical mechanisms associated with toxicity induced by xenobiotics (exogenous chemicals). In some cases, however, toxic effects that are not detected in humans are observed in animal studies. Many factors may contribute to differences between the results of controlled animal studies and the effects observed in humans. The following are among the most important:

  • Physiologic differences. Laboratory animals are not miniature humans. Depending on the biologic process under investigation, a particular test species may match the human system more closely and so be a better experimental model.
  • Magnitude of exposure. As is often the case for toxicologic studies of any chemical, TCDD exposure used for animal studies has been many orders of magnitude higher than Vietnam veterans are likely to have received during military service, although the ultimate body burdens may not be as divergent.
  • Duration of exposure. Although TCDD is a persistent organic pollutant, animal studies seldom examine the chronic low-level exposure that occurs over a period of years or even many months.
  • Timing of exposure. It is well known that many organ systems are highly susceptible to xenobiotic exposure during critical stages of development, such as gestation; the response of some systems (such as the immune system) may also depend on the timing of the exposure to antigens relative to the timing of the exposure to xenobiotics such as TCDD.
  • The route of exposure. The route of exposure by which an exogenous agent enters an organism may influence the nature of any toxic response elicited. The outcomes of animal studies may be perturbed by the delivery of treatment doses by “unnatural” routes of exposure such as a bolus by gavage or intraperitoneal injection, but the route of exposure does not seem to be a major reason that the results of epidemiology studies may not agree with the findings of controlled studies for the COIs considered in the VAO series.
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
  • Genetic constitution and expression. The etiologies of most diseases in humans and in animals are likely to be influenced by numerous genes and to involve complex gene–environment interactions, and preliminary evidence suggests that TCDD can induce epigenetic modifications of an organism’s DNA that may alter future expression of the genome.
  • Sex differences. There are well-known differences between male and female animals (including humans) in susceptibility to xenobiotic exposures, some of which are modified by sex steroids.
  • Prior and recurring exposures to multiple sources. Humans are exposed to xenobiotics from multiple sources throughout their lifetimes.
  • Complex mixtures. Most xenobiotic exposures occur in complex mixtures; the makeup of these mixtures can heavily influence the ultimate toxic effects. In addition to the dietary modulation of responses to other exposures of both humans and animals, human metabolism is perturbed by dietary supplements, prescription and over-the-counter pharmaceuticals, and other factors (such as cigarette smoking and ambient pollution).
  • Stress. Stress—of known or unknown origin—is a well-known modifier of human disease responses (such as immune responses); stress is an ever-present variable that is difficult to assess or control for in epidemiologic studies because there is substantial individual variation in response to it (Cohen et al., 2007a).

The absence of evidence of biologic plausibility from toxicology studies does not rule out the possibility of a biologic relationship. In fact, cases in which the epidemiologic evidence is strong, but toxicologic support is lacking, often drive new toxicology research. As noted in VAO, not only is information on biologic plausibility one of the primary elements in the oft-cited list of factors that have rather imprecisely become known as the Bradford Hill (1965) criteria for causality (previously discussed), but also insights about biologic processes inform whether an observed pattern of statistical association might be interpreted as the product of more than error, bias, confounding, and chance. The committee used toxicologic information in that fashion and placed the information before its synthesis and conclusion in order to provide readers with a more coherent argument for its ultimate conclusion about the adequacy of the available evidence to support the existence of a particular association.

Publication Bias

Depending on the results that have been obtained, some studies are more likely to be published than others. That is the concept of “publication bias,” which has been documented in biomedical research (Song et al., 2000; Stern and Simes, 1997). Most commonly, bias can be introduced when studies whose hypotheses are supported by statistically significant results or that are otherwise deemed

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×

favorable by their authors are selectively submitted for publication. In addition, papers with “interesting findings” may be of more interest to journal editors and reviewers and thus be more likely to be accepted for publication after submission. Conversely, “negative” studies, in which the hypotheses being tested are not supported by the study findings, often go unpublished. Investigators employed by industry may be inhibited from submitting findings that have potential legal or economic ramifications.

Thus, conclusions about the associations between exposure and outcome that are based solely on published results could be subject to bias. Despite that, the committee does not believe that its conclusions have been unduly affected by publication bias, for two reasons: The extensive publicity surrounding the possibility of health effects associated with the herbicides used in Vietnam has created considerable pressure to publish all findings on the subject, and the many published studies assembled and reviewed contain among their results the full range of possible statistical associations, from convincingly negative through indeterminate to strongly positive.

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×

This page intentionally left blank.

Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 37
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 38
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 39
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 40
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 41
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 42
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 43
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 44
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 45
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 46
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 47
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 48
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 49
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 50
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 51
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 52
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 53
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 54
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 55
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 56
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 57
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 58
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 59
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 60
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 61
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 62
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 63
Suggested Citation:"2 Evaluating the Evidence." National Academies of Sciences, Engineering, and Medicine. 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press. doi: 10.17226/21845.
×
Page 64
Next: 3 Exposure to the Herbicides Used in Vietnam »
Veterans and Agent Orange: Update 2014 Get This Book
×
 Veterans and Agent Orange: Update 2014
Buy Hardback | $124.00 Buy Ebook | $99.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

From 1962 to 1971, the US military sprayed herbicides over Vietnam to strip the thick jungle canopy that could conceal opposition forces, to destroy crops that those forces might depend on, and to clear tall grasses and bushes from the perimeters of US base camps and outlying fire-support bases. Mixtures of 2,4-dichlorophenoxyacetic acid (2,4-D), 2,4,5-trichlorophenoxyacetic acid (2,4,5-T), picloram, and cacodylic acid made up the bulk of the herbicides sprayed. The main chemical mixture sprayed was Agent Orange, a 50:50 mixture of 2,4-D and 2,4,5-T. At the time of the spraying, 2,3,7,8-tetrachlorodibenzo-p-dioxin (TCDD), the most toxic form of dioxin, was an unintended contaminant generated during the production of 2,4,5-T and so was present in Agent Orange and some other formulations sprayed in Vietnam.

Because of complaints from returning Vietnam veterans about their own health and that of their children combined with emerging toxicologic evidence of adverse effects of phenoxy herbicides and TCDD, the National Academy of Sciences was asked to perform a comprehensive evaluation of scientific and medical information regarding the health effects of exposure to Agent Orange, other herbicides used in Vietnam, and the various components of those herbicides, including TCDD. Updated evaluations were conducted every two years to review newly available literature and draw conclusions from the overall evidence. Veterans and Agent Orange: Update 2014 is a cumulative report of the series thus far.

READ FREE ONLINE

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!