National Academies Press: OpenBook

Veterans and Agent Orange: Update 2008 (2009)

Chapter: 2 Evaluating the Evidence

« Previous: 1 Introduction
Suggested Citation:"2 Evaluating the Evidence." Institute of Medicine. 2009. Veterans and Agent Orange: Update 2008. Washington, DC: The National Academies Press. doi: 10.17226/12662.
×
Page 27
Suggested Citation:"2 Evaluating the Evidence." Institute of Medicine. 2009. Veterans and Agent Orange: Update 2008. Washington, DC: The National Academies Press. doi: 10.17226/12662.
×
Page 28
Suggested Citation:"2 Evaluating the Evidence." Institute of Medicine. 2009. Veterans and Agent Orange: Update 2008. Washington, DC: The National Academies Press. doi: 10.17226/12662.
×
Page 29
Suggested Citation:"2 Evaluating the Evidence." Institute of Medicine. 2009. Veterans and Agent Orange: Update 2008. Washington, DC: The National Academies Press. doi: 10.17226/12662.
×
Page 30
Suggested Citation:"2 Evaluating the Evidence." Institute of Medicine. 2009. Veterans and Agent Orange: Update 2008. Washington, DC: The National Academies Press. doi: 10.17226/12662.
×
Page 31
Suggested Citation:"2 Evaluating the Evidence." Institute of Medicine. 2009. Veterans and Agent Orange: Update 2008. Washington, DC: The National Academies Press. doi: 10.17226/12662.
×
Page 32
Suggested Citation:"2 Evaluating the Evidence." Institute of Medicine. 2009. Veterans and Agent Orange: Update 2008. Washington, DC: The National Academies Press. doi: 10.17226/12662.
×
Page 33
Suggested Citation:"2 Evaluating the Evidence." Institute of Medicine. 2009. Veterans and Agent Orange: Update 2008. Washington, DC: The National Academies Press. doi: 10.17226/12662.
×
Page 34
Suggested Citation:"2 Evaluating the Evidence." Institute of Medicine. 2009. Veterans and Agent Orange: Update 2008. Washington, DC: The National Academies Press. doi: 10.17226/12662.
×
Page 35
Suggested Citation:"2 Evaluating the Evidence." Institute of Medicine. 2009. Veterans and Agent Orange: Update 2008. Washington, DC: The National Academies Press. doi: 10.17226/12662.
×
Page 36
Suggested Citation:"2 Evaluating the Evidence." Institute of Medicine. 2009. Veterans and Agent Orange: Update 2008. Washington, DC: The National Academies Press. doi: 10.17226/12662.
×
Page 37
Suggested Citation:"2 Evaluating the Evidence." Institute of Medicine. 2009. Veterans and Agent Orange: Update 2008. Washington, DC: The National Academies Press. doi: 10.17226/12662.
×
Page 38
Suggested Citation:"2 Evaluating the Evidence." Institute of Medicine. 2009. Veterans and Agent Orange: Update 2008. Washington, DC: The National Academies Press. doi: 10.17226/12662.
×
Page 39
Suggested Citation:"2 Evaluating the Evidence." Institute of Medicine. 2009. Veterans and Agent Orange: Update 2008. Washington, DC: The National Academies Press. doi: 10.17226/12662.
×
Page 40
Suggested Citation:"2 Evaluating the Evidence." Institute of Medicine. 2009. Veterans and Agent Orange: Update 2008. Washington, DC: The National Academies Press. doi: 10.17226/12662.
×
Page 41
Suggested Citation:"2 Evaluating the Evidence." Institute of Medicine. 2009. Veterans and Agent Orange: Update 2008. Washington, DC: The National Academies Press. doi: 10.17226/12662.
×
Page 42
Suggested Citation:"2 Evaluating the Evidence." Institute of Medicine. 2009. Veterans and Agent Orange: Update 2008. Washington, DC: The National Academies Press. doi: 10.17226/12662.
×
Page 43
Suggested Citation:"2 Evaluating the Evidence." Institute of Medicine. 2009. Veterans and Agent Orange: Update 2008. Washington, DC: The National Academies Press. doi: 10.17226/12662.
×
Page 44
Suggested Citation:"2 Evaluating the Evidence." Institute of Medicine. 2009. Veterans and Agent Orange: Update 2008. Washington, DC: The National Academies Press. doi: 10.17226/12662.
×
Page 45

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

2 Evaluating the Evidence This chapter outlines the approach used by the Committee to Review the Health Effects in Vietnam Veterans of Exposure to Herbicides: Seventh Bien- nial Update and its predecessors to evaluate the available scientific evidence. A more complete description is found in Chapter 5 of Veterans and Agent Orange: Health Effects of Herbicides Used in Vietnam, hereafter referred to as VAO (IOM, 1994). CHOICE OF HEALTH OUTCOMES As discussed in Chapter 1, the committee was charged with summarizing the strength of the scientific evidence of associations between exposure to various herbicides and contaminants during service in the Vietnam War and individual diseases or other health outcomes. Public Law 102-4, which mandated the com- mittee’s work, however, did not specify particular health outcomes suspected of being associated with herbicide exposure. Such a list was developed on the basis of diseases and conditions addressed in the scientific literature identified through the original VAO’s extensive literature searches. The list has been amended in the VAO updates in response to new publications, to requests from the Depart- ment of Veterans Affairs (VA) and various veterans service organizations, and to concerns of Vietnam veterans and their families. Comments received at public hearings and in written submissions from veterans and other interested persons have been valuable in identifying issues to be pursued in greater depth in the scientific literature. The Veterans and Agent Orange committees began their evaluation by pre- suming neither the presence nor the absence of an association between exposure 27

28 VETERANS AND AGENT ORANGE: UPDATE 2008 and any particular health outcome. Over the series of reviews, evidence of vari- ous degrees of association, lack of association, or persistent indeterminacy with respect to a wide array of disease states has accrued. For many conditions, how- ever, particularly ones that are very uncommon, associations with the chemicals of interest have remained unaddressed in the medical research literature; for these, the committee remains neutral based on the understanding that “absence of evidence is not evidence of absence.” IDENTIFICATION OF RELEVANT LITERATURE Mixtures of 2,4-dichlorophenoxyacetic acid (2,4-D), 2,4,5-trichlorophenoxy­ acetic acid (2,4,5-T), picloram, and cacodylic acid made up the bulk of the herbicides sprayed in Vietnam. At the time of the spraying, 2,3,7,8-tetrachloro- dibenzo-p-dioxin (TCDD, one form of dioxin) was an unintended contaminant from the production of 2,4,5-T and was present in Agent Orange and some other herbicide formulations sprayed in Vietnam; it is important to note that TCDD and Agent Orange are not the same. Databases have been searched for the names of those compounds, their synonyms and abbreviations, and their Chemical Abstracts Service (CAS) numbers. The evidence indicates that a tissue protein, the aryl hydrocarbon receptor, mediates most of the toxicity of TCDD, so aryl hydrocarbon receptor also was used as a keyword, as were dioxin, Agent Orange, and Vietnam veteran. One of the herbicides used in Vietnam, cacodylic acid, is dimethyl arsenic acid (DMA), an organic form of arsenic. In addition to being synthesized as a herbicide, DMA is a metabolite of inorganic arsenic in humans. It was long thought to be a biologically inactive metabolite, but recent evidence suggests that one form—DMAIII—might be responsible for some of the adverse effects of inorganic arsenic. The committee carefully reconsidered that evidence but again determined that it does not support a conclusion that exposure to cacodylic acid would be expected to result in the same adverse health effects as would exposure to toxic concentrations of inorganic arsenic. Therefore, as in prior VAO reports, the literature on the health effects of inorganic arsenic was not considered here. Further details on the effects of inorganic arsenic can be found in Arsenic in Drinking Water (NRC, 1999) and Arsenic in Drinking Water: 2001 Update (NRC, 2001). For cacodylic acid and picloram, the search terms were the chemical names, synonyms, and CAS numbers of the herbicides. This report concentrates on the evidence published after the completion of work on Veterans and Agent Orange: Update 2006 (IOM, 2007). Relevant new contributions to the literature made during the period October 1, 2006–September 30, 2008, were sought. The information that the committee used was compiled from a comprehensive electronic search of public and commercial databases— biologic, medical, toxicologic, chemical, historical, and regulatory—that pro- vide citations of the scientific literature. In addition, the reference lists of some review and research articles, books, and reports were examined for potentially

EVALUATING THE EVIDENCE 29 relevant articles. As noted above, the terms used in the search strategy included the chemical names, synonyms, and CAS numbers of the specific chemicals of interest—2,4-D, 2,4,5-T, TCDD, cacocylic acid, and picloram—see Figure 2-1 for chemical structures and CAS numbers—and the more generic terms involved with this project (Vietnam veteran, Agent Orange, aryl hydrocarbon receptor, dioxin, herbicide, and phenoxy). By analogy, results on other specific phenoxy herbicides are also of interest: 2-methyl-4-chlorophenoxyacetic acid (MCPA) and 2-(2-methyl-4­-chlorophenoxy) propionic acid (MCPP or Mecoprop) for 2,4-D and 2-(2,4,5-trichlorophenoxy) propionic acid (2,4,5-TP or Silvex) for 2,4,5-T (see Figure 2-1 for chemical structures and CAS numbers); although the ben- zoate herbicide dicamba (2-methoxy-3,6-dichlororbenzoic acid) is not always categorized with the phenoxy herbicides, it shares structural similarities with this class, and measures of its association with various adverse health outcomes have been factored into the evidence. Because some polychlorinated biphenyls (PCBs) and polychlorodibenzofurans (PCDFs) have dioxin-like biologic activity, studies of populations exposed to PCBs or PCDFs were reviewed when results were presented in terms of toxicity equivalent quotients (TEQs). Findings related only to exposure to the diverse chemical families of pesticides were considered too nonspecific for inclusion in the evidence database used to draw conclusions about associations. However, pesticide was included among the search terms for VAO updates to ensure that all possible articles on herbicides (the specific tar- gets were only the phenoxy herbicides, cacodylic acid, and picloram) would be identified and subjected to the next phase of screening. (An ancillary analysis of the search results by this committee determined that the term “pesticide” did not identify any relevant citation that was not picked up by more specific terms, and so it will be eliminated from searches in future VAO updates, thereby reducing the number of extraneous hits to be culled.) Because they are the target population of the charge to the VAO commit- tees, studies of Vietnam veterans (serving in any of the armed forces, American or otherwise) have always been accorded considerable weight in the commit- tees’ deliberations, whether or not estimation of exposure to herbicide-related substances has been attempted. Characterization of exposure in studies of the veterans was extremely uncommon at the time of the original VAO report, and the Vietnam veterans’ own ages were still below the ages at which many chronic illnesses are manifested. Consequently, the original committee made extensive efforts to consider several groups known or thought to have potentially higher and better-characterized exposure to TCDD or phenoxy herbicides than Vietnam veterans themselves—both occupational exposure (as of chemical-production, paper and pulp, sawmill, tannery, waste-incinerator, railroad, agricultural, and forestry workers) and environmental exposure (as of residents of Seveso, Times Beach, Quail Run, and Vietnam). Successive committees have been able to concentrate more on studies that explicitly addressed the exposures specified in the charge. Some occupational and environmental cohorts that received exceptionally high exposures (such as the

30 VETERANS AND AGENT ORANGE: UPDATE 2008 Phenoxy Herbicides 2,4-D [94-75-7] 2,4,5-T [93-76-5] CI CI O O CI O O CI OH CI OH MCPA [94-74-6] Silvex [93-72-1] CI CI O O CI O O OH CI OH MCPP [93-65-2] Dicamba [1918-00-9] CI O O CI O O OH OH CI 2,3,7,8-TCDD [1746-01-6] CI O CI IC O IC Picloram [1918-02-1] Cacodylic Acid [75-60-5] CI NH 2 O HO CI As O N OH CI FIGURE 2-1  Chemical structures and CAS numbers for specific chemical of interest. Figure 2-1.eps

EVALUATING THE EVIDENCE 31 International Agency for Research on Cancer and Seveso cohorts) are now well characterized and producing a stream of informative results. The Agricultural Health Study, a continuing prospective cohort study of agricultural populations with specific information on the chemicals of interest, is also now contributing a steady stream of information to the database. Most important, the Vietnam veterans themselves are advancing in age and, when studied, capable of directly providing substantial information on chronic health conditions, often as related to serum TCDD concentrations. The committee for Update 2006 decided that exhaustive searches on job titles, occupations, or industries to identify additional study populations with possible, but not specifically characterized, exposure to the chemicals of interest were no longer an efficient means of augmenting the evidence database, because they are more likely to retrieve citations with infor- mation about a health outcome at the expense of considerable uncertainty about exposure. The current committee adopted the Update 2006 committee’s practice of more circumscribed searching. As the information in the database on populations with established exposure to the chemicals of interest has grown, VAO commit- tees have become less dependent on data from studies with nonspecific exposure information and have been able to focus more on findings of studies with refined exposure specificity. In recognition of the more pivotal role that findings drawn directly from Vietnam veterans are now able to play in its decisions, this commit- tee has reordered its consideration of populations. For each health outcome in this update, studies of Vietnam veterans are now addressed first followed by occupa- tional and environmental studies, rather than last as was the practice established when the information from studies on veterans was quite sparse. It is well accepted that any TCDD or herbicide effect may be diluted some- what in studies of Vietnam veterans because some of the veterans may not have been exposed or may have been exposed only at low concentrations. The problem is exacerbated in studies in which exposure is defined in terms of occupation (even on the basis of a full job history). Exploratory studies based on linking to a one-time statement of occupation (for example, on a death certificate or in a census) are thought to be of little usefulness even when a job–exposure matrix is used to “convert” standardized job codes to specific exposures. Not only is there uncertainty about whether all members of the sample have been exposed to one of the chemicals of interest unless detailed personal monitoring and industrial- hygiene work have been performed, but for most occupational categories there is considerable certainty that the workers have been exposed to many other poten- tially toxic agents. Thus, such studies may well minimize the effects of exposure to dioxin or the herbicides of interest while yielding misleading indications of health problems resulting from other exposures. The search strategy was devised to ensure that abstracts of all potentially rel- evant articles were subjected to closer screening, but it also resulted in the iden- tification of a large number of nonrelevant studies. The searches produced 7,000

32 VETERANS AND AGENT ORANGE: UPDATE 2008 “hits,” including some studies that were identified more than once. It was evident from the abstracts of most of the cited articles that they did not address health effects in association with exposure to the chemicals of interest; for example, many of the cited studies investigated the efficacy of herbicides in killing weeds. All studies that discussed health effects were considered if the search-related information (title, abstract, and keywords) indicated that any of the herbicides of interest (or any of their components) may have been investigated. For each of the more than 850 potentially relevant citations ultimately identified, a copy of the entire article was obtained online or retrieved from library sources and reviewed more thoroughly by the committee for inclusion in its report. In large part, included reports are peer-reviewed journal articles, but gener- ally available and formally published government studies (particularly those investigating health effects in Vietnam veterans) are also included under the presumption that they have been carefully reviewed. In practice, the articles are generally in English, but the committee would obtain translations for crucial ones that were not in English, as was done for reports of a study of Korean veterans of the Vietnam War (Kim HA et al., 2003; Kim JS et al., 2003) when Update 2004 was produced. TCDD, the 2,3,7,8-chlorinated congener of dioxin, is the most potent of the polychlorinated dibenzo-p-dioxins, dibenzofurans, and PCBs, so it is presumed to be most problematic. However, our concern is not limited to this single conge- ner. In non-laboratory settings—for example, epidemiologic studies—exposures occur not only to TCDD, but to mixtures of dioxins, dibenzofurans, and PCBs, which vary in their degree of chlorination. A toxicity equivalency factor (TEF) is an estimate of the dioxin-like potency of an individual congener relative to the toxicity of TCDD, as measured in assays of its AH receptor (AHR) activity. TEQs are often used to estimate the cumulative toxic potency of mixtures as the sum of TEFs weighted by the concentration of the corresponding congener in the mixture; this total is denoted as the mixture’s TEQ or toxicity equivalent quotient in terms of dioxin-like activity. That approach is often taken in epidemiologic studies focusing on PCBs. Many epidemiologic studies of PCBs were recov- ered in the literature search although they were not specifically sought. Because dioxin-like and non–dioxin-like PCB congeners are found together in environ- mental mixtures and are known to mediate toxicity by unique mechanisms, the relative contribution of dioxin-like PCBs to an individual health outcome can be difficult to determine. Therefore, evidence from epidemiologic studies of PCB exposure was retained only for results reported for specific dioxin-like congeners or in terms of TEQs. Investigation of the pesticides used in greenhouses determined that green- house workers are not likely to be exposed to herbicides, particularly those of interest for VAO committee deliberations (Czarnota, 2004; Neal, 2006; Univer- sity of Connecticut, 2006), so new citations for studies of such workers were not retained and previous results on such populations (Abell et al., 2000, on fertility;

EVALUATING THE EVIDENCE 33 Hansen et al., 1992, on cancer in female workers) were retroactively excluded from the evidence database. Roughly 300 citations contributed new information to this update. New evi- dence on each health outcome was reviewed in detail. The conclusions, however, are based on the accumulated evidence, not just on recently published studies. If statistics have been generated on the same study population over time (as noted in Chapter 5), multiple entries for a given health outcome in the summary results tables of Chapters 6–9 correspond to successive updates, but only the most com- prehensive version of the information on a given population is factored into the committee’s conclusion on that outcome. Primary findings are the components of the evidence that the committee endeavors to integrate in drawing its conclusions; reanalyses, pooled analyses, reviews, and so on may be discussed in conjunction with primary results or in synthesis sections for a given health outcome, but they are not themselves part of the evidence dataset. COMMITTEE’S APPROACH The committee’s general approach to the evaluation of scientific evidence is presented here. It corresponds very closely with the approach developed by the original committee as delineated in detail in Chapter 5 of VAO. The committee had three specific tasks: to determine whether there is a statistical association between exposure to the herbicides used in Vietnam and health outcomes, to de- termine the increase in risk of effects among Vietnam veterans, and to determine whether plausible biologic mechanisms provide support for a causal relationship with a given health outcome. This section discusses the committee’s approach to each task. Statistical Association The issues in determining whether a statistical association exists are detailed in Chapter 5 of VAO. The committee found that the most relevant evidence came from epidemiologic studies—investigations in which large groups of people are studied to identify an association between exposure to a chemical of interest and the occurrence of particular health outcomes. Epidemiologists estimate associations between exposure and outcome in a specific population or group by using such measures as relative risk, standardized mortality ratio, and odds ratio. Those measures indicate the magnitude of a dif- ference in the rate of an outcome between two populations. For example, if the rate in an exposed population is twice the rate in a nonexposed population, the relative risk, or rate ratio, is 2. Similarly, if the odds of a health outcome are 1:20 in an exposed population but 1:100 in a nonexposed population, the odds ratio is 5. In this report, relative risk refers to the results of cohort studies, and odds ratio (an estimate of relative risk) usually refers to the results of case–control studies. (The results of cohort studies sometimes are reported with odds ratios, again to

34 VETERANS AND AGENT ORANGE: UPDATE 2008 estimate relative risk.) An estimated relative risk greater than 1 indicates a posi- tive association (that is, it is more likely that the outcome will be seen in exposed people than in nonexposed people), whereas a relative risk between zero and 1 indicates a negative or inverse association (that is, the outcome is less likely in exposed people). A ratio of 1 suggests the absence of association. A statistically significant association is one that would be unlikely to occur by chance (that is, if the null hypothesis is true). Determining whether an estimated association between an exposure and an outcome represents a real relationship requires careful scrutiny because there can be more than one explanation for an estimate. Bias is a distortion of the measure of association that results from flawed selection in the assembly of the study pop- ulation or from error in measurement of studied characteristics. Confounding is a distortion of the measure of association that results from failure to recognize or account for some other factor related both to exposure and to outcome. Chance is the degree to which an estimated association might vary randomly among differ- ent samples of the population studied. The width of a confidence interval is used to quantify the likely variability of an exposure–disease association; even when a relative risk or standardized mortality ratio exceeds 1, a conclusion regarding increased risk must be qualified when the confidence interval is wide. In drawing its conclusions, the committee examined the quantitative estimates of associa- tion and evaluated the potential influences of bias, confounding, and chance. In integrating the findings of various studies, the committee considered the degree of statistical significance associated with every estimated risk (a reflection of the magnitude of the observed effect and the power of the study designs) rather than simply tallying the “significant” and “nonsignificant” outcomes as dichotomous items of evidence. The committee also considered whether controlled laboratory investigations provide information consistent with the compounds of interest be- ing associated with a given effect and perhaps causally linked to it. In pursuing the question of statistical association, the committee recognized that an absolute conclusion about the absence of association is unattainable. As in science generally, studies of health effects associated with herbicide exposure cannot demonstrate that a purported effect is impossible or could never occur, only that it is statistically improbable. Any instrument of observation, even the most excellent epidemiologic study, is limited in its resolving power. In a strict technical sense, therefore, the absence of an association between even one chemical and a health outcome cannot be proven. Convincingly demonstrating the lack of a particular effect for all five of the compounds of interest simultane- ously would be a daunting effort, especially in light of the paucity of information concerning picloram and cacodylic acid. This committee, therefore, endorses the decision by the committee for Update 2006 to reclassify several types of cancer that had been classified since VAO (1994) as having “suggestive evidence of no association” with “exposure to herbicides.” Interaction or synergism among the chemicals of interest or with other agents

EVALUATING THE EVIDENCE 35 is another theoretical concern. The committee was not charged with attributing effects to specific chemicals of interest, and joint effects among them should be adequately identified by the committee’s approach. The number of combinations of these chemicals with other agents that might be problematic is virtually infi- nite. Real-life experience, as investigated with epidemiologic studies, effectively integrates any results of exposure to a target substance over all other possibly detrimental or mitigating exposures that a population might have. It may not be possible to partition contributions of the chemicals of interest from those of all other factors quantitatively, but, to the extent that the possibility of confounding influences can be appraised, the committee will have achieved its objective. Increased Risk in Vietnam Veterans When all the available epidemiologic evidence has been evaluated, it is pre- sumed that Vietnam veterans are at increased risk for a specific health outcome if there is evidence of a positive association between one or more of the chemicals of interest and the outcome. The best measure of potency for the quantification of risk to veterans would be the rate of the outcome in exposed Vietnam veterans compared with the rate in nonexposed veterans, adjusted for the degree to which any other factors that differ between exposed and nonexposed veterans might influence those rates. A dose–response relationship established in another human population suitably adjusted for such factors would be similarly suitable. It is difficult to quantify risk when exposures of a population have not been measured accurately. Recent serum TCDD concentrations are available only on subgroups enrolled in the Air Force Health Study (AFHS) (the Ranch Hand and Southeast Asia comparison subjects) and from VA’s study of deployed and non- deployed members of the Army Chemical Corps. Pharmacokinetic models, with their own set of assumptions, must then be used to extrapolate back to obtain the most accurate estimates of original exposure available on Vietnam-era veterans. The absence of reliable measures of exposure to the chemicals of interest among Vietnam veterans limits the committee’s ability to quantify risks of specific dis- eases in this population. Although serum TCDD measurements are available for only a small por- tion of Vietnam-era veterans, the observed distributions of these most reliable measures of exposure make it clear that they cannot be used as a standard to partition veterans into discrete exposure groups, such as service on Vietnamese soil, service in the Blue Water Navy, and service elsewhere in Southeast Asia. For example, many TCDD values observed in the comparison group from the AFHS exceeded US background levels and overlapped considerably with those of the Ranch Hand subjects. As explained in Chapter 1, the committee for Update 2006 decided to make a general statement about its continuing inability to address that aspect of its charge

36 VETERANS AND AGENT ORANGE: UPDATE 2008 quantitatively rather than reiterate a disclaimer in the concluding section for every health outcome, and this committee has retained that approach. Plausible Biologic Mechanisms Chapter 4 details the experimental basis of assessment of biologic plausibil- ity or the extent to which an observed statistical association in epidemiologic studies is consistent with other biologic or medical knowledge. In other words, does the observation of a particular health effect make sense on the basis of what is known about how the chemicals in question act at the tissue, cellular, or molecular level? The relationship between a particular exposure and a specific human health outcome is addressed in the context of research on the effects of the chemicals on biologic systems and of evidence from animal studies. For this report, the committee reviewed toxicology studies that were pub- lished after Update 2006 (IOM, 2007) and considered them in combination with earlier studies in commenting on the biologic plausibility of individual health outcomes. In the current update, the practice of earlier reports in presenting the toxicologic evidence has been modified. Chapter 4 presents a more streamlined toxicity profile of each of the chemicals of interest without providing commentary on each possibly relevant article published in the update period. Experimental in- formation pertinent to a particular health outcome is now presented immediately after the epidemiologic evidence on that outcome in the “Biologic Plausibility” sections of the individual health outcomes (Chapters 6–9). A positive statistical association between an exposure and an outcome does not necessarily mean that the exposure is the cause of the outcome. Data from toxicology studies may support or conflict with a hypothesis that a specific com- pound can contribute to the occurrence of a particular disease. Many toxicology studies are conducted with laboratory animals so that variables, including the amount and duration of exposure, can be controlled precisely. Studies that use isolated cells in culture also can elucidate how a compound alters cellular pro- cesses. The objectives of those toxicology studies are to determine what toxic effects are observed at different exposure concentrations and to identify the mechanisms by which the effects are produced. Ultimately, the results of the toxicology studies should be consistent with what is known about the human disease process to support a conclusion that the development of the disease was influenced by an exposure. That approach is not without shortcomings; for example, the dose of a chemi- cal required to produce an effect in experimental animals is often many times higher than human exposures. (For TCDD, however, effects have been observed in animals whose body burdens are no more than 10-fold higher than the high end of those in the general population in the industrialized world [EPA, 2001].) Furthermore, animal and cell-culture models do not always accurately mimic hu- man responses. The absence of evidence of biologic plausibility from toxicology

EVALUATING THE EVIDENCE 37 studies, however, does not rule out the possibility that a biologic relationship ex- ists. In fact, cases in which the epidemiologic evidence is strong, but toxicologic support is lacking, often drive new toxicology research. As noted in VAO, not only is information on biologic plausibility one of the primary elements in the widely accepted Bradford Hill (1965) criteria for cau- sality but insights about biologic processes inform whether an observed pattern of statistical association might be interpreted as the product of more than error, bias, confounding, and chance. The committee used toxicologic information in that fashion and placed the information before its synthesis and conclusion to provide readers with a more coherent argument for its ultimate conclusion about the adequacy of the available evidence to support the existence of a particular association. EVALUATION OF THE EVIDENCE Associations between exposures to the chemicals of interest and specific health outcomes are determined through an analysis of available epidemiologic studies that is informed by an understanding of the toxicology of the chemicals and their exposure pathways. In reaching conclusions, Veterans and Agent Or- ange committees consider the nature of the exposures, the nature of the health outcomes, the populations exposed, and the quality of the evidence examined. Some specific issues that this and prior committees have considered are addressed below. Human Studies The committee reviewed studies of Vietnam veterans and of other popula- tions that might have been exposed to the chemicals of interest. In light of the dispute regarding whether veterans who served in the “Blue Water Navy” during the Vietnam War should be presumed to have been exposed to the herbicides sprayed in Vietnam, the committee notes that, like prior VAO committees, it has considered data on such individuals to be a part of the evidentiary database on Vietnam veterans. The other populations factored into the committee’s evaluation included cohorts of workers in chemical production and agriculture, populations that reside near sites of environmental contamination, and residents of Vietnam. The committee believes that studies of such nonveteran subjects can help in the assessment of whether the chemicals of interest are associated with particular health outcomes. As noted above in describing the literature search, studies of nonveteran subjects were identified because one of the chemicals of interest was specified by the original researchers as a possible toxic exposure, rather than on the basis of occupational definitions. Some of the studies, especially those of workers in chemical-production plants, provide stronger evidence about health outcomes than do studies of veterans because the industrial exposures were mea-

38 VETERANS AND AGENT ORANGE: UPDATE 2008 sured sooner after occurrence and were more thoroughly characterized than has been the case in most studies of veterans. Furthermore, in the studies of workers at chemical-production plants, the magnitude and duration of exposure to the chemicals were generally greater, so the likelihood that any possible health conse- quence would be manifested was greater. The studies were often large enough to examine health risks among groups of people with different levels of exposure, so dose–response relationships could be investigated. The general practice of VAO committees has been to evaluate all studies, whether or not their subjects were Vietnam veterans, according to the same criteria in determining the strength and validity of findings. Because the subjects of studies of Vietnam veterans are the concern of the legislation that mandated the present review, however, demonstra- tions of increased incidence of particular health outcomes among them are of unquestionable pertinence in drawing conclusions. The committee has concluded that it would be inappropriate to use quan- titative techniques, such as meta-analysis, to combine individual study results into a single summary measure of statistical association. The committee reached that conclusion because of the many differences among studies in definitions of exposure, health outcomes considered, criteria for defining study populations, correction for confounding factors, and degree of detail in reporting results. The appropriate use of meta-analysis requires more methodologic consistency among studies, especially in the definition of exposure, than is present in the lit- erature reviewed by the committee (Egger et al., 2002; Petitti, 2000). A detailed discussion of the results of individual studies in appropriate categories (Vietnam veterans, occupational, or environmental; exposure to Agent Orange or equiva- lent dioxin-contaminated phenoxy herbicides, to dioxin, to phenoxy herbicides without dioxin contamination, to cacodylic acid, or to picloram) with a thorough examination of each study’s strengths and weaknesses is fully informative with- out making unfounded assumptions of homogeneity. In general, the committee did not consider case reports, case series, or other published studies that lacked control or comparison groups. An exception was made, however, for early-onset transient peripheral neuropathy. Individual case reports were reviewed because the rapid appearance and transient nature of the condition impose methodologic constraints that might have precluded the appli- cation of standard epidemiologic techniques. Because any effect of Agent Orange in individuals or groups of veterans is evaluated in terms of disease or medical outcome, attention to disease classifi- cation was important to the committee in assembling pertinent data related to a particular outcome from various investigations before integrating the informa- tion. The researchers who conducted the studies reviewed by the committee faced the same challenge in interpreting the available documentation when assigning diagnostic labels to given subjects and then grouping the labels for analysis. Pathologists, clinicians, and epidemiologists use several classification sys- tems, including the International Classification of Diseases (ICD), the Interna-

EVALUATING THE EVIDENCE 39 tional Classification of Diseases, 9th Revision (ICD-9), Clinical Modification, and the International Classification of Diseases for Oncology. The 10th revision of ICD (ICD-10) is currently used to classify mortality information. Most of the subjects investigated in the studies cited in this update were diagnosed under earlier systems, and most of the articles report results in accordance with ICD-9 if they use ICD codes at all, so the committee has also used ICD-9. ICD codes are a hierarchic system for indicating type of disease and site. For example, ICD-9 162 specifies cancers of the lung, trachea, or bronchus; 162.2 specifies cancer of the main bronchus; 162.3, cancer of the upper lobe of the lung; and 162.4, cancer of the middle lobe of the lung. For a patient to receive a correct diagnosis, careful staging of the extent of disease is necessary, and a biopsy of the tissue must be analyzed with microscopy, often with special immunohistochemical stains, to confirm a clinical impression. Many of the epidemiologic studies reviewed by this committee did not use the ICD approach to classification of disease and relied instead on clinical impression alone. Death-certificate diagnoses are notoriously inaccurate if the certificates are completed by medical officers who are not familiar with the decedents’ medical history (Smith Sehdev and Hutchins, 2001). Self-reported diagnoses, which are obtained from survey questionnaires, often are partially or completely inaccu- rate; for instance, a patient may report having been treated for stomach cancer although the correct diagnosis was gastric adenocarcinoma, gastric lymphoma, pancreatic cancer, large bowel cancer, or peritoneal cancer. Many epidemiologic studies report disease outcome by organ system. For instance, the term digestive system may be used for conditions that are benign or malignant and that affect the esophagus, stomach, liver, pancreas, small bowel, large bowel, or rectum. Therefore, if a report indicated that a cohort has an increased incidence of digestive system cancer, it would be unclear whether the association was attributable to excess cases of esophageal, gastric, hepatic, pancreatic, or intestinal cancers or to some combination. Such generalization is complicated by the fact that the cause of cancer may differ at various anatomic sites. For instance, there are strong associations between gastric cancer and Helicobacter pylori infection, between smoking and squamous cell carcinoma of the esophagus, and between chronic hepatitis B infection and hepatic cancer. Furthermore, a single site may experience a carcinogenic response to multiple agents. The committee recognizes that outcome misclassification is a possibility when recording of a diagnosis with a specific ICD code is used as the means of entering an observation into an analysis, but this system has been refined over many decades and is virtually universally used and understood, in addition to being exhaustive and explicit. Therefore, this and previous VAO committees have opted to use the ICD system as an organizing tool. Although the groupings of cancer sites for which conclusions about association have been presented may correspond more closely to National Institute for Occupational Safety and Health

40 VETERANS AND AGENT ORANGE: UPDATE 2008 or National Cancer Institute Surveillance Epidemiology and End Results catego- ries (see Appendix B), the underlying ICD codes provide the most exactitude. In this report, ICD codes appear almost exclusively in the introductory sections of health-outcome discussions (particularly for cancers) to specify precisely what outcome the committee is addressing and, when possible, in the results table to indicate exactly what the primary researchers believed they were investigating. (See Appendix B for cancer groupings with corresponding ICD-9 and ICD-10 codes.) For Update 2006, VA made two specific requests. First, the committee was asked to consider whether the occurrence of hairy cell leukemia should be regarded as associated with exposure to the herbicides used by the military in Vietnam. Second, the committee was asked to comment on whether effects of veterans’ herbicide exposures might be manifested at later stages of their chil- dren’s development than have been systematically evaluated to date or in later generations and on the feasibility of assessing such effects. Those requests are addressed in Chapters 6 and 7, respectively. Rare diseases, such as hairy cell leukemia and tonsil cancer, are difficult to study because it is hard to accumulate enough cases to permit analysis. Often, the result is that observed cases are included in a broader, less specific category. Thus, epidemiologic data may not be available for assessing whether a particu- lar rare disease is associated with Agent Orange exposure. In some instances, as for chronic lymphocytic leukemia and AL amyloidosis, VAO committees have reached conclusions on the basis of the data available and the etiology of the disease. Through systematic application of the hierarchic nature of the ICD coding system, committees intend to draw an explicit conclusion for every type of cancer about the adequacy of available evidence to support an association between herbicide exposure and each type of cancer. For nonmalignant condi- tions, however, the diversity of disease processes involved makes the use of broad ICD ranges less useful, but, because VAO committees could not possibly address every rare nonmalignant disease, they do not draw explicit conclusions about diseases that are not discussed. Thus, the category of “inadequate or insuf- ficient evidence to determine an association” is the default or starting point for any health outcome; if a condition or outcome is not addressed specifically, it will be in this category. The committee is aware of the concerns of some veterans about the role of herbicide exposure in the occurrence of multiple health outcomes, such as multiple cancers, in a given person. Little research has been done to address whether the rate of concurrence is greater than would be expected by chance. Si- multaneous analysis of multiple health outcomes could potentially provide more insight into whether the chemicals of interest cause multiple health effects, into competing risks between various health outcomes, and into the interactive effects of some health outcomes on others; but addressing health conditions individually has remained challenging.

EVALUATING THE EVIDENCE 41 VAO committees wanted to be clear in indicating what evidence is factored into their conclusions. The practice in the VAO reports has been to augment the results table for a given health outcome with any additional publications consid- ered in the current update in the categories of Vietnam-veteran, occupational, or environmental studies. Inclusion of sequential sets of results from follow-ups of a study population has the potential to create the appearance of a greater weight of evidence than is warranted, so since Update 2006 an italicized citation in a table indicates that its results have been superseded. An issue related to evidence evaluation that was of concern for the Update 2006 committee was the evidence category of “no association.” That commit- tee determined that a conclusion of no association would require substantive evidence of such a lack of effect of each of the chemicals of interest. Given the paucity of information that has ever been found on cacodylic acid and picloram, that conclusion would seem suspect even if substantial evidence uniformly sup- ported a finding of no association both with exposure to the phenoxy herbicides and with exposure to TCDD. The current committee concurs with that determi- nation and has adopted a similar approach to the placement of health outcomes in this category. Exposure Assessment Much of the evidence that VAO committees have considered has been drawn from studies of populations that were not in Vietnam during the period when Agent Orange and other herbicides were used as defoliants. The most informative of those studies were well-documented investigations of occupational exposures to TCDD or specific herbicides, such as 2,4-D or 2,4,5-T. In many other stud- ies, TCDD exposure was combined with exposures to an array of “dioxin-like” compounds, and the herbicides were often analyzed as members of a functional class; this is less informative for the committee’s purpose than individual results on each specific compound. In the real-world situations investigated in epidemio- logic studies, exposure to multiple possibly toxic chemicals is the rule rather than the exception; for example, farmers or other agricultural populations are likely to be exposed to insecticides and fungicides and to herbicides. In such studies, the committee looked for evidence of health effects that are associated with the specific compounds in the defoliants used in Vietnam and also sought consider- ation of and adjustment for other possibly confounding exposures. The quality of exposure information in the scientific literature reviewed by this and previous committees spans a broad range. Some studies relied on inter- views or questionnaires to determine the extent and frequency of exposure. Such self-reported information generally carries less weight than would more objective measures of exposure. To the extent that questionnaire-based information can be corroborated or validated by other sources, its strength as evidence of exposure is enhanced. Written records of chemical purchase or production can provide one

42 VETERANS AND AGENT ORANGE: UPDATE 2008 type of objective information. Even more useful are scientific measurements of exposure. In some occupational studies, for example, workers wear air-sampling instruments that measure the concentration of a contaminant in each worker’s breathing zone. Measurement of chemicals or their products in biologic speci- mens, such as blood and urine, also can provide reliable indications of exposure for specific periods. Studies that categorize exposure from well-documented envi- ronmental sources of contaminants can be useful in the identification of exposed populations, but their results may be inaccurate if people with different levels of exposure are assigned to the same general category of exposure. Studies that explore environmental exposure and disease frequency in regional populations (such as states and counties) are known as ecologic studies. Most ecologic stud- ies are considered preliminary or “hypothesis-generating” studies because they lack information on exposure and disease on an individual basis and are unable to address potential confounding factors. Chapter 3 of this update addresses issues of exposure estimation in more detail. The agent of interest may be assessed with various degrees of specificity. For instance, any of the four herbicides in question could be individually mea- sured, and phenoxy herbicides would be a useful broader category for 2,4,5-T and 2,4-D; but a report of findings in terms simply of “herbicides” is on the margin of being informative, whereas results stated in terms of “pesticides” are too vague to be useful. For a given chemical of interest, the measure of exposure may be increasingly imprecise—for example, concentrations in target tissue, serum concentrations, cumulative exposure, possible exposure, and so on down to merely a report of service in a job or industry category. Those approaches can address complexities in specificity, duration, and intensity of exposure with vari- ous degrees of success. All may provide some information about association with a chemical of interest, but this committee has determined that investigation of associations between the exposure of concern and most health outcomes has ad- vanced to the stage where some characterizations of exposure are too nonspecific to promote insight. For health outcomes with very little evidence, a somewhat looser criterion would apply so that no possible signal of an association would be overlooked. Animal and Mechanistic Studies Animal models used as surrogates for the study of a human disease must reproduce, with some degree of fidelity, the manifestations of the disease in humans. However, a given effect of herbicide exposure in an animal species does not necessarily establish its occurrence in humans. In addition to possible species differences, many factors affect the ability to extrapolate from results of animal studies to health effects in humans. Animals used in experimental studies are most often exposed to purified chemicals, not to mixtures. Even if herbicide formulations or mixtures are used, the conditions of exposure might not realisti-

EVALUATING THE EVIDENCE 43 cally reproduce exposures that occur in the field. Furthermore, Vietnam veterans were probably exposed to other agents—such as tobacco smoke, insecticides, therapeutics, drugs, diesel fumes, and alcohol—that may increase or decrease the ability of chemicals in herbicides to produce a particular adverse health outcome. Few, if any, studies either in humans or in experimental animals have examined those interactions. As discussed in Chapter 4, TCDD, a contaminant of 2,4,5-T, is thought to be responsible for many of the toxic effects of the herbicides used in Vietnam. At- tempts to establish correlations in the effects of TCDD between experimental sys- tems and humans are particularly problematic because of known species-, sex-, and outcome-specific differences in susceptibility to TCDD toxicity. Some data indicate that humans might be more resistant than are other species to TCDD’s toxic effects (Ema et al., 1994; Moriguchi et al., 2003); other data suggest that, for some outcomes, human sensitivity could be the same as or greater than that of some experimental animals (DeVito et al., 1995). Differences in susceptibility may also be affected by variations in the rate at which TCDD is eliminated from the body (see Chapter 4 for details on the toxicokinetics of TCDD). It also is important to account for TCDD’s mode of action in considering species and strain differences. There is a consensus that most of or all the toxic effects of TCDD involve interaction with the aryl hydrocarbon receptor (AHR), a protein that binds TCDD and other aromatic hydrocarbons with high affinity. For- mation of an active complex involving the intracellular receptor, the ligand (the TCDD molecule), and other proteins is followed by interaction of the activated complex with specific sites on DNA. That interaction can alter the expression of genes involved in the regulation of cellular processes. The development of mice that lack the AHR has helped to establish a definitive association between the AHR and TCDD-mediated toxicity. The affinity of TCDD for the AHR is species- and strain-specific, and responses to binding of the receptor vary among cell types and developmental stages. In addition, genetic differences in the properties of the AHR are known in human populations, as they are in laboratory animals, so some people are at intrinsically greater or less risk for the toxic effects of TCDD. Although studying AHR biology in transformed human cell lines minimizes the inherent error associated with species extrapolations, caution must be exer- cised because it is still not clear to what extent toxicity is affected by the trans- formation itself or by the conditions under which cell lines are cultured in vitro. Publication Bias Some studies are more likely to be published than others. That is the concept of publication bias, which has been documented in biomedical research (Song et al., 2000; Stern and Simes, 1997). Most commonly, bias can be introduced when studies whose hypotheses are supported by statistically significant results or that are otherwise deemed favorable by their authors are selectively submitted

44 VETERANS AND AGENT ORANGE: UPDATE 2008 for publication. Conversely, “negative” studies, in which the hypotheses being tested are not supported by the study findings, often go unpublished. Therefore, conclusions about associations between exposure and outcome that are based solely on published results could be subject to bias. Despite that, the committee does not believe that its conclusions have been unduly affected by publication bias, for two reasons: the extensive publicity surrounding the possibility of health effects associated with the herbicides used in Vietnam has created considerable pressure to publish all findings on the subject, and the many published studies assembled and reviewed contain among their results the full range of possible statistical associations, from convincingly negative through indeterminate to strongly positive. Role of Judgment This committee’s process of reaching conclusions about statistical associa- tions involved more than a formulaic application of quantitative procedures to the assembled evidence. First, the committee had to assess the relevance and validity of individual reports. Then, it had to evaluate the possible influences of mea- surement error, selection bias, confounding, and chance on the reported results. Next, the committee integrated all the evidence within and among diverse fields of research. Finally, the conclusions drawn were based on consensus within the committee. Those aspects of the committee’s review required thoughtful consid- eration of alternative approaches at several points and could not be accomplished by adherence to a narrowly prescribed formula. The realized approach, as described here, has been determined to a large extent by the nature of the exposures, of the health outcomes, and of the result- ing evidence available for examination; therefore, it has evolved in the course of the work of this and previous VAO committees. The quantitative and qualitative procedures underlying this review have been made as explicit as possible, but ultimately the conclusions about association expressed in this report are based on the committee’s collective judgment. The committee has endeavored to express its judgments as clearly and precisely as the data allowed. REFERENCES Abell A, Juul S, Bonde JP. 2000. Time to pregnancy among female greenhouse workers. Scandinavian Journal of Work, Environment and Health 26(2):131–136. Czarnota MA. 2004. Weed Control in Greenhouses: Bulletin 1246. University of Georgia, College of Agricultural and Environmental Sciences, Cooperative Extension.   Throughout the report the same alphabetic indicator following year of publication is used con- sistently for the same article when there were multiple citations by the same first author in a given year. The convention of assigning the alphabetic indicator in order of citation in a given chapter is not followed.

EVALUATING THE EVIDENCE 45 DeVito MJ, Birnbaum LS, Farland WH, Gasiewicz TA. 1995. Comparisons of estimated human body burdens of dioxin-like chemicals and TCDD body burdens in experimentally exposed animals. Environmental Health Perspectives 103(9):820–831. Egger M, Ebrahim S, Smith GD. 2002. Where now for meta-analysis? International Journal of Epidemiology 31(1):1–5. Ema M, Ohe N, Suzuki M, Mimura J, Sogawa K, Ikawa S, Fujii-Kuriyama Y. 1994. Dioxin binding activities of polymorphic forms of mouse and human arylhydrocarbon receptors. Journal of Biological Chemistry 269(44):27337–27343. EPA (US Environmental Protection Agency). 2001. Dioxin: Summary of the Dioxin Reassessment Science (Information Sheet 1). Washington, DC: EPA, Office of Research and Development. Hansen ES, Hasle H, Lander F. 1992. A cohort study on cancer incidence among Danish gardeners. American Journal of Industrial Medicine 21(5):651–660. Hill AB. 1965. The environment and disease: Association or causation? Proceedings of the Royal Society of Medicine 58:295–300. IOM (Institute of Medicine). 1994. Veterans and Agent Orange: Health Effects of Herbicides Used in Vietnam. Washington, DC: National Academy Press. IOM. 2007. Veterans and Agent Orange: Update 2006. Washington, DC: The National Academies Press. Kim HA, Kim EM, Park YC, Yu JY, Hong SK, Jeon SH, Park KL, Hur SJ, Heo Y. 2003a. Immuno- toxicological effects of Agent Orange exposure to the Vietnam War Korean veterans. Industrial Health 41(3):158–166. Kim JS, Lim HS, Cho SI, Cheong HK, Lim MK. 2003b. Impact of Agent Orange exposure among Korean Vietnam veterans. Industrial Health 41(3):149–157. Moriguchi T, Motohashi H, Hosoya T, Nakajima O, Takahashi S, Ohsako S, Aoki Y, Nishimura N, Tohyama C, Fujii-Kuriyama Y, Yamamoto M. 2003. Distinct response to dioxin in an arylhy- drocarbon receptor (AHR)-humanized mouse. Proceedings of the National Academy of Sciences of the United States of America 100(10):5652–5657. Neal NC. 2006. Greenhouse Weed Control: Horticultural Information Leaflet 570. North Carolina State University. NRC (National Research Council). 1999. Arsenic in Drinking Water. Washington, DC: National Academy Press. NRC. 2001. Arsenic in Drinking Water: 2001 Update. Washington, DC: National Academy Press. Petitti DB. 2000. Meta-Analysis, Decision Analysis, and Cost-Effectiveness Analysis: Methods for Quantitative Synthesis in Medicine. New York: Oxford Press. Smith Sehdev AE, Hutchins GM. 2001. Problems with proper completion and accuracy of the cause- of-death statement. Archives of Internal Medicine 161(2):277–284. Song F, Eastwood AJ, Gilbody S, Duley L, Sutton AJ. 2000. Publication and related biases. Health Technology Assessment 4(10):1–115. Stern JM, Simes RJ. 1997. Publication bias: Evidence of delayed publication in a cohort study of clinical research projects. British Medical Journal 315(7109):640–645. University of Connecticut. 2006. Greenhouse Weed Control. University of Connecticut, Storrs, Co- operative Extension System.

Next: 3 Exposure to the Herbicides Used in Vietnam »
Veterans and Agent Orange: Update 2008 Get This Book
×
Buy Hardback | $140.00 Buy Ebook | $109.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

From 1962 to 1971, the U.S. military sprayed herbicides over Vietnam to strip the thick jungle canopy that could conceal opposition forces, to destroy crops that those forces might depend on, and to clear tall grasses and bushes from the perimeters of U.S. base camps and outlying fire-support bases.

In response to concerns and continuing uncertainty about the long-term health effects of the sprayed herbicides on Vietnam veterans, Veterans and Agent Orange provides a comprehensive evaluation of scientific and medical information regarding the health effects of exposure to Agent Orange and other herbicides used in Vietnam. The 2008 report is the eighth volume in this series of biennial updates. It will be of interest to policy makers and physicians in the federal government, veterans and their families, veterans' organizations, researchers, and health professionals.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!