2

Considerations in Evaluating the Evidence

This chapter outlines the approach that this and previous committees have used to evaluate the available scientific evidence. A more complete description of the committee's approach can be found in Chapter 5 of Veterans and Agent Orange: Health Effects of Herbicides Used in Vietnam (hereafter referred to as VAO; IOM, 1994).

CHOICE OF HEALTH OUTCOMES

As discussed in Chapter 1, the committee was charged with summarizing the strength of the scientific evidence concerning the association between herbicide exposure during Vietnam service and each of a set of diseases or conditions suspected to be associated with such exposure. The legislation (PL 102-4) that led to the committee's work, however, did not provide a specific list of diseases and conditions suspected of being associated with herbicide exposure. VAO included a list of diseases and conditions developed on the basis of what had been mentioned in the scientific literature or in other documents; the list has been supplemented in response to developments in the literature, requests made by the Department of Veterans Affairs (VA), and concerns of Vietnam veterans.

IDENTIFICATION OF RELEVANT LITERATURE

The information used by the committee was developed through a comprehensive search of public and commercial databases covering biologic, medical,



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 20
Veterans and Agent Orange: Update 2002 2 Considerations in Evaluating the Evidence This chapter outlines the approach that this and previous committees have used to evaluate the available scientific evidence. A more complete description of the committee's approach can be found in Chapter 5 of Veterans and Agent Orange: Health Effects of Herbicides Used in Vietnam (hereafter referred to as VAO; IOM, 1994). CHOICE OF HEALTH OUTCOMES As discussed in Chapter 1, the committee was charged with summarizing the strength of the scientific evidence concerning the association between herbicide exposure during Vietnam service and each of a set of diseases or conditions suspected to be associated with such exposure. The legislation (PL 102-4) that led to the committee's work, however, did not provide a specific list of diseases and conditions suspected of being associated with herbicide exposure. VAO included a list of diseases and conditions developed on the basis of what had been mentioned in the scientific literature or in other documents; the list has been supplemented in response to developments in the literature, requests made by the Department of Veterans Affairs (VA), and concerns of Vietnam veterans. IDENTIFICATION OF RELEVANT LITERATURE The information used by the committee was developed through a comprehensive search of public and commercial databases covering biologic, medical,

OCR for page 20
Veterans and Agent Orange: Update 2002 toxicologic, chemical, historical, and regulatory information. The majority of those databases were bibliographic, providing citations to scientific literature. The reference lists of major review and research articles, books, and reports were examined. Literature identification continued through July 1, 2002. More than 9,000 potentially relevant studies were identified in those searches, and more than 1,000 were reviewed. Suggestions received from veterans and other interested persons at public hearings and in written submissions were a valuable source of additional information. This report concentrates on the evidence published after the completion of work on Veterans and Agent Orange: Update 2000 (IOM, 2001) and Veterans and Agent Orange: Herbicide/Dioxin Exposure and Acute Myelogenous Leukemia in the Children of Vietnam Veterans (IOM, 2002). For each health outcome, the new evidence is reviewed in detail. Conclusions, however, are based on the totality of accumulated evidence, not just on recently published studies. That is, new evidence is interpreted not alone but in the context of evidence addressed in previous reports. The committee's judgments have both quantitative and qualitative aspects; they reflect both the evidence examined and the approach taken to evaluate it. In VAO, the committee delineated how it approached its task so that readers would be able to assess and interpret its findings. In offering that information, the committee wished to make the report useful to those seeking to update its conclusions as new information was obtained. The committees responsible for later reports have adopted the original committee's approach. As discussed in Chapter 3, cacodylic acid, or dimethylarsinic acid (DMA), in addition to being synthesized as a herbicide, is a metabolite of inorganic arsenic in humans. It is important, therefore, to consider the relationship between inorganic arsenic and DMA and the potential for similar adverse health effects after exposure to inorganic arsenic and to DMA. DMA was long thought to be a biologically inactive metabolite of inorganic arsenic, but evidence has been accumulating in recent years that one form of DMA (DMAV) is an active metabolite of inorganic arsenic and might be responsible for some of the adverse effects observed after exposure to inorganic arsenic. It has yet to be determined, however, whether human exposure to DMA results in the same effects as exposure to toxic concentrations of inorganic arsenic (skin, bladder, and lung cancer and cardiovascular effects). Although some experimental evidence indicates that DMA induces effects similar to those of inorganic arsenic (see Chapter 3), it is insufficient to support a conclusion that exposure to inorganic arsenic is directly relevant to exposure to cacodylic acid. Therefore, the literature on the effects of inorganic arsenic is not considered in this report. Further details on the effects of inorganic arsenic are in Arsenic in Drinking Water (NRC, 1999) and Arsenic in Drinking Water: 2001 Update (NRC, 2001).

OCR for page 20
Veterans and Agent Orange: Update 2002 COMMITTEE'S APPROACH TO ITS CHARGE As discussed in Chapter 1, the committee is charged with three specific tasks: determining whether a statistical association exists between exposure to the herbicides used in Vietnam and health outcomes, determining the increased risk of effects among Vietnam veterans, and determining whether a plausible biologic mechanism or other causal evidence of a given health outcome exists. This section discusses the committee's approach to each of those tasks. Determining Whether a Statistical Association Exists In trying to determine whether a statistical association exists between any of the herbicides used in Vietnam or the contaminant 2,3,7,8-tetrachlorodibenzo-p-dioxin (TCDD) and a health outcome, the committee found that the most helpful evidence came from epidemiologic studies—investigations in which large groups of people are studied to determine the association between the occurrence of particular diseases and exposure to the substances at issue. Epidemiologists estimate associations between an exposure and a disease in a defined population or group using measures such as relative risk, standardized mortality ratio, or odds ratio. Those terms describe the magnitude by which the risk or rate of disease is changed in a given population. For example, if the risk in an exposed population increases two-fold relative to an unexposed population, it can be said that the relative risk, or risk ratio, is 2.0. Similarly, if the odds of disease in one population are 1:20 and in another are 1:100, then the odds ratio is 5.0. Sometimes the use of terms such as relative risk, odds ratio, and estimate of relative risk is inconsistent, for instance when authors refer to an odds ratio as a relative risk. In this report relative risk refers to the results of cohort studies and odds ratio (an estimate of relative risk) refers to the results of case–control studies. An estimated relative risk greater than 1 could indicate a positive or direct association (that is, a harmful association), whereas values between zero and 1 could indicate a negative or inverse association (that is, a protective association). A “statistically significant” difference is one that, under the assumptions made in the study and the laws of probability, would be unlikely to occur if there were no true difference and no biases. Determining whether an observed association between an exposure and a health outcome is “real” requires additional scrutiny because there may be alternative explanations for the observed association. Those explanations include error in the design, conduct, or analysis of the investigation; bias, a systematic tendency to distort the measure of association so that it may not represent the true relation between exposure and outcome; confounding, distortion of the measure of association because of failure to recognize or account for another factor related to both exposure and outcome; and chance, the effect of random variation, which produces spurious associations that can, with a known probability, sometimes depart widely from the true relation. In deciding whether an association between

OCR for page 20
Veterans and Agent Orange: Update 2002 herbicide exposure and a particular outcome exists, the committee examined the quantitative estimates of risk and evaluated their likelihood of being due to error, bias, confounding, or chance or of representing true associations. In pursuing the question of statistical association, the committee recognized that an absolute conclusion about the absence of association may never be attained. As in science generally, studies of health outcomes after herbicide exposure cannot demonstrate that a purported effect is impossible or could never occur. Any instrument of observation, including epidemiologic studies, is limited in its resolving power. In a strict technical sense, therefore, the committee cannot prove the absolute absence of an association between a health outcome and exposure to the herbicides or TCDD. Determining Increased Risk in Vietnam Veterans Whether Vietnam veterans are at increased risk is relevant principally (but not exclusively) when there is evidence of a positive association between exposure and a health outcome. The best evidence for use in determining the risk is knowledge of the rate of occurrence of the outcome in Vietnam veterans who were exposed, the rate in those who were not exposed (the “background” rate in the population of Vietnam veterans), and the degree to which any other differences between exposed and unexposed veterans influence the difference in rates. When, as in most studies, exposure among Vietnam veterans has not been adequately determined, it is difficult to determine such an increased risk. Therefore, although the committees have found the available evidence (most of which is from studies of people exposed to dioxins or herbicides in occupational and environmental settings) sufficient for drawing conclusions about the association between herbicide exposure and a number of health outcomes, the lack of good data on Vietnam veterans, especially with regard to herbicide exposure, complicates the assessment of the increased risk of disease specifically among people exposed to herbicides during service in Vietnam. Evaluating the Evidence of a Biologic Mechanism Chapter 3 details the experimental evidence that provides the basis of the assessment of biologic plausibility, that is, the extent to which a statistical association is consistent with biologic or medical knowledge. As with the epidemiologic evidence, the chapter concentrates on studies published in 2000–2002 but considers all relevant studies in drawing conclusions. The issue of whether a relationship between a particular chemical exposure and a particular health outcome reflects a true association in humans is addressed in the context of research regarding the mechanism of interaction between the chemical and biologic systems, evidence from animal studies, and evidence of an association between exposure and the occurrence of a health outcome in humans, including evidence

OCR for page 20
Veterans and Agent Orange: Update 2002 from occupational and environmental chemical exposures. It must be recognized, however, that lack of data in support of a plausible biologic mechanism does not necessarily rule out the existence of an association. ISSUES IN EVALUATING THE EVIDENCE To assess whether a given human health effect is associated with any of the exposures of interest, the committees concentrated on reviewing and interpreting human epidemiologic studies and experimental investigations that might contribute to biologic plausibility, weighing the strengths and limitations of the available evidence. Their assessments have both quantitative and qualitative aspects and take into account the nature of the exposures, health outcomes, and populations exposed; the characteristics of the evidence examined; and the approach taken to evaluate that evidence. Some of the aspects the committees have considered in evaluating the evidence are addressed below. Toxicologic Studies A valid surrogate-animal model for the study of a human disease must reproduce, with some degree of fidelity, the manifestations of the disease in humans. Whole-animal studies or animal-based experimental systems continue to be used to study herbicide toxicity because they allow for rigid control of chemical exposures and close monitoring of health outcomes. Because many of the chemical exposures associated with diseases in humans have been confirmed in experimental studies, data derived from such studies are generally accepted as a valuable guide in the assessment of biologic plausibility. Whether a given effect occurs in an animal species, however, cannot always be used to establish whether it occurs in humans. As discussed in Chapter 3, TCDD, a contaminant of 2,4,5-trichlorophenoxyacetic acid (2,4,5-T), is thought to be responsible for many of the toxic effects of the herbicides used in Vietnam. Attempts to establish correlations between the effects of TCDD in experimental systems and in humans are particularly problematic because there are end-point, sex-, and species-specific differences in susceptibility to TCDD. Some data indicate that humans might be more resistant than other species to the toxic effects of this chemical (Dickson and Buzik, 1993), but other data suggest that, for some end points, humans may be at least as sensitive as some experimental animals (DeVito et al., 1995). Differences in susceptibility have a toxicokinetic component because elimination is slower in humans than in rodents (Geyer et al., 2002). It also important, however, to consider TCDD's mode of action when considering species and strain differences. There is a consensus that most of the toxic effects of TCDD involve interaction with the aryl hydrocarbon receptor (AhR), a protein that binds TCDD and other aromatic hydrocarbons with high affinity.

OCR for page 20
Veterans and Agent Orange: Update 2002 Formation of an active complex involving the receptor, ligand (the TCDD molecule), and other protein factors is followed by interaction of the activated complex with specific sites on DNA. That interaction results in DNA changes that alter the expression of genes involved in the regulation of cellular processes. The development of AhR-knockout mice (mice lacking the AhR) has helped to establish a definitive association between the AhR and TCDD-mediated toxicity. Toxicodynamic interactions are important because the affinity of TCDD for the AhR is species- and strain-specific (Lorenzen and Okey, 1991), and responses to occupancy of the receptor vary among cell types and developmental stages. The drug-metabolizing enzymes that are induced by TCDD in humans are different from those induced in rodents (Neubert, 1992); this suggests that the effect of different genetic backgrounds on AhR function is not completely understood. It is generally accepted that genetic susceptibility plays a key role in determining the adverse effects of environmental chemicals. Genetic susceptibility is central in the assessment of biologic plausibility because if polymorphisms of the gene encoding the AhR exist in humans as they do in laboratory animals, some people would be at greater risk or at lesser risk for the toxic and carcinogenic effects of TCDD. Ultimately, however, the challenge in the assessment of the biologic plausibility of the toxicity of herbicides and TCDD is not restricted to understanding receptor-mediated events. The dose–response relationships that arise from multiple toxicokinetic and toxicodynamic interactions must also be considered. Gene-regulation models described to date do not consider the intricacies of the many interactions between the AhR and other proteins. Future attempts to define the quantitative relationship between receptor occupancy and biologic response to TCDD must consider that multiple biochemical changes may influence the overall cellular response. In addition, although studying AhR biology in transformed human cell lines minimizes the inherent error associated with species extrapolations, caution must be exercised because the extent to which transformation itself influences toxicity outcomes has yet to be fully defined. Epidemiologic Studies To obtain information relevant to the evaluation of health effects of exposure to the chemicals of interest, the committee reviewed studies of cohorts of people other than Vietnam veterans potentially exposed to the herbicides used in Vietnam (2,4,5-T, 2,4-dichlorophenoxyacetic acid [2,4-D], cacodylic acid, and picloram), TCDD, phenoxy herbicides, chlorophenols, and other compounds. The cohorts include chemical-production and agricultural workers, people thought to be exposed to large amounts of herbicides or dioxins as a result of residing near the site of an accident or near areas used to dispose of toxic waste, and residents of Vietnam. The committees felt that considering studies of cohorts other than

OCR for page 20
Veterans and Agent Orange: Update 2002 veterans could help address the issue of whether those compounds might be associated with particular health outcomes, even though the results would have only an indirect bearing on the increased risk of disease in veterans themselves. Some of the studies, especially those of workers in chemical-production plants, provide stronger evidence about health effects than studies of veterans because exposure was generally more easily measured and often was determined. Furthermore, the general magnitudes and durations of exposure to the chemicals were greater, and the studies were large enough to examine the health risks among people with varied exposure. Because of the great differences among studies, the committee concluded that it was inappropriate to use a quantitative technique, such as meta-analysis, to combine individual results into a single summary measure of statistical association. Using such a summary measure would also inappropriately focus attention on one piece of information used by the committee, whereas, as discussed previously, many factors are important in evaluating the literature. Although its full potential has yet to be realized, the application of molecular and cellular end points to epidemiologic research promises to increase the understanding of the association between herbicide exposure and the occurrence of various health outcomes. Such information might provide an important advantage in the assessment of biologic plausibility because biologically based epidemiologic data allow more accurate identification and measurement of exposure. For instance, the analytic data available on people known to have been exposed to herbicides during the Vietnam War constitute a valuable resource for the study of TCDD-related disease; documented TCDD body burdens provide a quantitative bridge between experimental studies and epidemiology. Taken together, experimental studies and epidemiologic investigations provide complementary perspectives from which to view human health effects of exposure to herbicides. However, it must be recognized that the ultimate test of associations between exposure and effects lies in data on human populations. In recent years, the toxic equivalency factor (TEF) method of comparing the relative toxicity of dioxin-like chemicals has been used by several government agencies around the world. Although it is considered one of the best approaches to assessing the relative risk of complex mixtures of these contaminants, it is an interim approach, and it has several inherent uncertainties. TEFs are determined through inspection of the available congener-specific biologic and biochemical data on a compound and assignment of an order-of-magnitude estimate of relative toxicity in comparison with TCDD. TEF values are by no means precise; they are the result of scientific judgment and expert opinion taking into account all the available data form on the congeners. The scientific data on which they are based may vary considerably, often by several orders of magnitude depending on the different biologic end points chosen for a particular chemical. Therefore, considerable uncertainty exists about the use of these values, and it is often difficult to quantify the uncertainty. Although the recent World Health Organiza-

OCR for page 20
Veterans and Agent Orange: Update 2002 tion TEF values (Van den Berg et al., 1998) are most often cited and generally accepted, the values used can differ slightly among states, countries, and health organizations and with the classification scheme accepted by an agency. Nevertheless, most agencies in the United States, including the Environmental Protection Agency, support the basic approach as a “reasonable estimate” of relative toxicity. Furthermore, numerous countries and several international organizations have adopted it although, again, the accepted values may differ. The TEF concept is based on the premise that the toxic and biologic responses of all the chemicals in question are mediated through the AhR. Although all the available data support the concept, the set of data on each particular chemical considered to be dioxin-like is incomplete. One possible limitation of the approach is that it does not consider synergistic or antagonistic interactions among the chemicals. In addition, it does not consider possible actions or interactions of these chemicals that are not mediated by the AhR. Indeed, little research has been done on this. For a chemical mixture like PCBs, another limitation of the TEF method is that the risk posed by non-dioxin-like chemicals (that is, noncoplanar PCBs) is not assessed, and some noncoplanar PCBs can act as antagonists (Safe, 1997-1998). Furthermore, the kinetics and metabolism of each dioxin-like chemical differ considerably. Data are often available only on tissue concentrations at any given time and not necessarily on the original exposure of the organism. Sometimes, tissue concentrations are not available. Extrapolation to a meaningful dose may add considerable uncertainty to calculation of the TCDD toxicity equivalent (TEQ) to which a person was exposed. In vivo, there also is exposure to dietary flavonoids and other phytochemicals that are AhR antagonists that is not taken into account with the TEQ method (Ashida et al., 2000; Ciolino et al., 1999; Quadri et al., 2000). As discussed in Update 1998 (IOM, 1999), quantitative structure–activity relationship (QSAR) models have been used to estimate the binding affinity of multiple chemical classes. Prediction with these models has been largely unsuccessful because of a focus on minimal energy conformations to predict the activity of molecules. Some QSAR models have been useful across classes of halogenated aromatic compounds. With the exception of acute and subacute transient peripheral neuropathy, the committee did not specifically consider case studies or other published studies that lacked control or comparison groups. The committee elected to consider case histories when evaluating the association between exposure and those conditions because their transience precluded using case–control and other types of studies with comparison populations. Publication Bias It has been documented in biomedical research that studies with statistically significant findings are more likely to be published than studies with nonsignifi-

OCR for page 20
Veterans and Agent Orange: Update 2002 cant results (Song et al., 2000). Evaluations of disease–exposure associations that are based solely on the published literature, therefore, could be biased in favor of positive associations. In reviewing reports of overall associations with exposure, however, the committee did not consider the risk of publication bias to be high among studies of herbicide exposure and health risks; because numerous published studies show no positive association, it examined a substantial amount of unpublished material, and it felt that publicity surrounding exposure to herbicides, particularly of Vietnam veterans, has been so intense that studies that show no association would be unlikely to be viewed as unimportant by the investigators—that is, the pressure to publish such “negative” findings would be considerable. Role of Judgment The evaluation of evidence to reach conclusions about statistical associations goes beyond quantitative procedures at several stages: assessing the relevance and validity of individual reports; deciding on the possible influence of error, bias, confounding, or chance on the reported results; integrating the overall evidence within and between diverse fields of research; and formulating the conclusions themselves. Those aspects of the committee's review required thoughtful consideration of alternative approaches at several points and could not be accomplished by adherence to a narrowly prescribed formula. The approach described here therefore evolved throughout the committee process and was determined, to a large extent, by the nature of the evidence, exposures, and health outcomes examined. Although the quantitative and qualitative aspects of the process that could be made explicit were important to the overall review, ultimately the conclusions about association expressed in this report are based on the committee 's collective judgment. The committee has endeavored to express its judgments as clearly and precisely as the data allowed. REFERENCES Ashida H, Fakuda I, Yamashita T, Kanazawa K. 2000. Flavones and flavonols at dietary levels inhibit a transformation of aryl hydrocarbon receptor induced by dioxin. FEBS Letters 476:213– 217. Ciolino H, Daschner P, Yeh G. 1999. Dietary flavonols quercetin and kaempferol are ligands of the aryl hydrocarbon receptor that affect CYP1A1 transcription differentially Biochemical Journal 340:715–722. DeVito MJ, Birnbaum LS, Farland WH, Gasiewicz TA. 1995. Comparisons of estimated human body burdens of dioxin-like chemicals and TCDD body burdens in experimentally exposed animals. Environmental Health Perspectives 103(9):820–831. Dickson LC, Buzik SC. 1993. Health risks of “dioxins”: A review of environmental and toxicological considerations. Veterinary and Human Toxicology 35(1):68–77. Geyer HJ, Schramm KW, Feicht EA, Behechti A, Steinberg C, Bruggemann R, Poiger H, Henkelmann B, Kettrup A. 2002. Halflives of tetra-, penta-, hexa-, hepta-, and octachlorodibenzo-p-dioxin in rats, monkeys, and humans–A critical review. Chemosphere 48(6):631–644.

OCR for page 20
Veterans and Agent Orange: Update 2002 IOM (Institute of Medicine). 1994. Veterans and Agent Orange: Health Effects of Herbicides Used in Vietnam Washington, DC: National Academy Press. IOM. 1999. Veterans and Agent Orange: Update 1998. Washington, DC: National Academy Press. IOM. 2001. Veterans and Agent Orange: Update 2000. Washington, DC: National Academy Press. IOM. 2002. Veterans and Agent Orange: Herbicide/Dioxin Exposure and Acute Myelogenous Leukemia in the Children of Vietnam Veterans. Washington, DC: National Academy Press. Lorenzen A, Okey AB. 1991. Detection and characterization of Ah receptor in tissue and cells from human tonsils. Toxicology and Applied Pharmacology 107(2):203–214. NRC (National Research Council). 1999. Arsenic in Drinking Water. Washington, DC: National Academy Press. NRC. 2001. Arsenic in Drinking Water: 2001 Update. Washington, DC: National Academy Press. Neubert D. 1992. Evaluation of toxicity of TCDD in animals as a basis for human risk assessment. Toxic Substances Journal 12:237–276. Quadri S, Qadri A, Hahn M, Mann K, Sherr D. 2000. The bioflavonoid galangin blocks aryl hydrocarbon receptor activation and polycyclic aromatic hydrocarbon-induced pre-B cell apoptosis Molecular Pharmacology 58(3):515–525. Safe SH. 1997-1998. Limitations of the toxic factor approach for risk assessment of TCDD and related compounds. Teratogenesis, Carcinogenesis, and Mutagenesis 17(4-5):285–304. Song F, Eastwood AJ, Gilbody S, Duley L, Sutton AJ. 2000. Publication and related biases. Health Technology Assessment 4(10):1–115. Van den Berg M, Birnbaum L, Bosveld AT, Brunstrom B, Cook P, Feeley M, Giesy JP, Hanberg A, Hasegawa R, Kennedy SW, Kubiak T, Larsen JC, Van Leeuwen FX, Liem AK, Nolt C, Peterson RE, Poellinger L, Safe S, Schrenk D, Tillitt D, Tysklind M, Younes M, Waern F, Zacharewski T. 1998. Toxic equivalency factors (TEFs) for PCBs, PCDDs, PCDFs for humans and wildlife. Environmental Health Perspectives 106(12):775–792.